I think what I need to do is break this down to kindergarten terms and ask, forget about good design, what's the easiest to design in a way that won't broach the complexity walls that I have to deal with until this system is done?
Let us ponder...
Dumb services that aren't aware of each other's states are the best way to leverage AI for code generation. So it makes sense that you would build code generating AI this way.
The dumber the service, the easier it is to write, the easier it is to write, the less likely you are to have to do it all again. That would mean that we want to write more, smaller, less complex modules. Cool.
Give budgeting back to the orchestrator
If we give back token budgeting to the Orchestrator, taking it out of the agent service, we get an orchestrator that can do the original job the orchestrator was supposed to do. But, we have to think about our concerns again, and keep the process light.
So, we give the Orchestrator the job of allocating the token budget, and thinking of keywords to track past context based on what's going on in the conversation, ticket, pr, or bug report.
The dispatcher
Next step is simplifying the agent service.
Thinking what we need here is a preliminary step that takes the token budget as well as the keywords, and then scours the context database, taking notes on past and current projects, building a context reference table for the task, and handing it off to the next leg of the journey.
We'll call this cute little thing, the "Dispatcher." And Dispatcher behaves differently, if the agent is, or is not already assigned to the task. After initial deployment, the Dispatcher no longer needs to modify the agent's state, except for incidental context things.
One switch we move from orchestrator to Dispatcher is, the question: "does this message need an agent response?"
From there, we have our agent, and the agent has two possible states. Assigned or Newly assigned.
The agent will receive context, instructions, context, and/or a randomly selected personality overlay from Dispatcher.
Agent responds, or does not, and that gives us everything other than the coding loop.
These get more complicated, progressively as we go down the chain.