There’s a genre of AI paper I’ve started to recognize.
Take a real industry debate.
Replace the strongest version of one side with a narrow toy abstraction.
Prove something true about the toy.
Then let everybody on the internet pretend the whole debate is settled.
That seems to be what’s happening with On the Reliability Limits of LLM-Based Multi-Agent Planning.
To be clear, the paper appears real, and the core theorem is not nonsense. In a closed multi-agent network where every agent has the same evidence and can only pass information through limited language channels, a centralized decision-maker with access to the same information will dominate the delegated network. In that regime, agents are not creating information. They are compressing and routing it.
Fine. Sure. Whatever.
The missing environment feedback
But that is not what actual agent systems look like in practice.
The experimental setup described in the paper uses relay-style prompting where downstream stages do not see the original question, only the compressed output of prior stages. In other words, they are measuring the failure modes of academic telephone and presenting it as a result about “multi-agent planning.”
That distinction matters.
Real agent systems derive value from things this framing strips away:
persistent state, shared artifacts, tool use, verifier access to source materials, environment feedback, heterogeneous roles, and fresh signals entering the system over time.
The paper itself effectively acknowledges this by treating tool outputs, database lookups, API responses, and environment observations as exogenous information that changes the analysis.
So what did they actually show?
Something narrow but useful:
if your multi-agent architecture is just multiple agents passing lossy summaries of the same context back and forth, then no, the architecture is not magic.
That’s a fair result.
But it is not a serious argument against modern agent orchestration. It is an argument against closed relay systems with shared evidence and bounded communication.
Those are not the same thing.
This is the pattern I wish more people would watch for:
🍎 sometimes a paper proves something true,
🍏 but only after amputating the part of the problem where the real-world value lives.
Remember children, a theorem can be correct, even when it's aimed at the wrong target.
Paper: