Saw this article making the rounds and wanted to offer a quick technical breakdown of why Salesforce’s LLM agents are struggling so predictably:
📎 https://www.theregister.com/2025/06/16/salesforce_llm_agents_benchmark/
First and foremost, Salesforce uses a tooled RAG system that comes with some very specific architectural problems.
💠 System Prompt Augmentation Overload
Salesforce agents are forced to reason through massive system prompts—roles, security, tool definitions, alignment logic—every single turn.
Most models can handle long system prompts, but system prompting is high-priority reasoning bandwidth, and it needs to be both well-structured and kept to a minimum, especially with GPT-based agents.
Think of system prompting in terms of math. You get X in reasoning bandwidth for every response, taken in total with every other response. Every additional piece of cyclomatic complexity you add in the system prompt is less cyclomatic complexity you have for ingestion later in your session or process.
(Quick aside: when I say "reasoning bandwidth," I don’t mean token count.
Rather, I'm talking about how much structured logic the model can meaningfully reason through per turn—how much weight it can carry before it starts dropping pieces. Token windows are wide. Bandwidth is narrow.)
From a modeling perspective, reasoning bandwidth acts like a bounded optimization layer inside the transformer’s attention manifold. Each turn is a constrained traversal across tasks: instruction resolution, memory reconciliation, tool abstraction, and output planning.
You don’t control the internal weights—but you absolutely shape the load. Add entropy, and the traversal fails. Add clarity, and the system routes clean.
🪙 Context Oversaturation
One of the reasons you would want to use a system like Salesforce Agents is because of how connected they are. Out of the box, they’re attached to your business systems, your pipelines, all of your enterprise-level enterprise stuff. And you can’t discount the value of connecting to these services. They are valuable.
Doing this right means getting valuable synthesis of documents, CRM data, and integrations. Doing it wrong means agents that are saturated in massive pools of things that are not contextually relevant to the tasks that are happening, which, again, every agent has to reason through at every turn in the process.
Cyclomatic complexity, in other words... how many yes/no on/off decisions can be made in a single turn, is how you measure this. Agents aren’t code, and should not be treated like code. Code is designed to give you complex, cyclomatically intense processes. LLMs are designed for tasking, execution, and probabilistic reasoning.
The trick here is understanding what the job entails, and what the right tool for it is. Typically, an agent is both code and a potentially long-running LLM process. Where does one stop and the other begin? The line can be blurry, but it does have to exist.
Dropping everything everywhere all at once into a context window and hoping for the best is not how you get there.
🥵 Role Confusion
Okay, so what are roles, exactly, and why are they important?
Roles are part of the protocol we use. There are a few different standards for that protocol. The most common is OpenAI, but other comparable API systems like Ollama do exist. Sometimes, you use both (that’s fun!), or neither. It’s going to depend on your project.
But whichever system you’re using as the glue between you and your system, your roles define who is talking to whom, and what is being said. User content will have a user role, tools get marked as tools, and custom roles can be created to give users, systems, and LLMs additional context for what’s going on, and how.
As an agent developer you understand, principally, that roles are sacred. You label things properly, with the correct role, every time. You keep your tasks segmented to dedicated roles, and you clearly label everything, because you know that not doing this will cause the machine to hallucinate in charming, but less than useful ways.
Want to see an LLM imagine it’s a terminal app that creates files?
Easy. Just confuse your tool role by assigning tool outputs to the assistant role.
Want to see your LLM hallucinate authority? Do the same thing with HTTP calls.
Interested in watching your LLM fall off the map completely? Just augment the user prompt with out-of-context memory data from elsewhere in the conversation, on every turn, in a way that can’t be followed rationally or deterministically—and call it a memory system!
Every role has free or low-cost reasoning weight associated with it that influences the results you get on every turn.
And that brings us to RAG systems. It hasn’t been that long, but RAG systems really do feel like relics of another era. One where tool calling didn’t exist, and you had to cram context in. What’s the safest obviously unsafe way to cram memory into an LLM system? Append or “augment” the user role with additional data.
Vector databases in and of themselves aren’t problematic. Done right, they’re incredibly helpful—even necessary for everyday tasks. In fact, there’s nothing stopping you from using vector database systems properly.
But when you’re using tool roles correctly, and retrieving the same vector databases without muddying your prompt and breaking the way your role-based protocols work… it’s not really a RAG anymore, is it?
By contrast, Salesforce agents are still built on a RAG architecture that appends retrieved context data and other information directly into the user prompt, without structure, tagging, or clear contextual awareness of what role that data is meant to play. Like I said, a throwback to simpler times and less capable systems.
This is how you get hallucinations. This is how you get missteps, degraded performance, and agents that can’t hold a coherent thread across turns.
Even if you clean up the retrieval pipeline or fine-tune the model to behave better under these conditions, you’re still working from the same broken foundation, where the fundamentals of task design are ignored in favor of solutions so pragmatic, they no longer make sense. Prompt contamination becomes a feature of the architecture itself.
The result is as predictable as it is frustrating: Stupid agents. Corrupted memory. Exhausted reasoning bandwidth. And maybe the occasional hallucinated art manifesto.
So how do you solve the problem?
Build something intentional
The first thing you need to do is understand how context and reasoning work within LLM systems. Remember that every piece of context you add to the context window gets re-litigated from beginning to end, every turn. But engagement slips as reasoning bandwidth is exhausted.
Reasoning bandwidth is different from attention through token context, which is what we’re usually talking about when we talk about context windows. A context window might hold ten trillion tokens before it truncates or attention slips. But that doesn’t necessarily mean the model itself is capable of reasoning through and ingesting the entire block coherently.
The differences between systems like Llama 4 Maverick and Gemini Pro 2.5 are stark for precisely this reason, even though they have similar context window limitations.
In other words, the only way to think about Salesforce here is as a cautionary tale. Rather than repeating their mistakes… build something intentional.
And by intentional, I mean:
🔹 Low-noise inputs, with everything the agent sees carrying a reason for being there.
🔹 Role and memory labeling that makes context legible—both to humans and machines.
Rule of thumb: if you couldn’t follow it, with infinitely more reasoning power than any AI, the machine probably can’t either.
🔹 Tool outputs that are treated as tools, not as user blur.
🔹 And above all, stop assuming your model understands “common sense.”
You have to tell the machine what “good” means.