One of the more interesting problems I’m working through in my coding agent app is scheduling.
Single-process agent apps are relatively easy to reason about. One shell, one queue, one local model backend, one apparent source of truth.
But that starts to break down the moment you open multiple instances on the same machine.
Now you have several agent windows, often across multiple projects, all sharing the same local inference backend. The GPU is the scarce resource. The local runtime is the scarce resource. But each instance only has a partial view of what’s happening.
That creates a subtle but important systems problem:
Each instance can behave as though it owns the backend, while only seeing its own local queue. So the status surfaces stop being fully truthful. One window thinks it’s next. Another thinks the model is idle. A third is trying to start or recover the backend. None of them have the whole picture.
For remote providers, this matters less. They already manage concurrency on their side.
For local inference, it matters a lot.
That’s why I think I need to implement a "broker."
The broker boundary
A user-global local inference broker would sit under the user config, own backend startup and health, maintain a shared queue, and provide truthful queue snapshots to every running instance of the agent app.
The important part is what it does not do:
It does not merge session contexts.
It does not merge project state.
It does not alter workflow law.
Each instance still owns its own sessions, tickets, related contexts, and internal isolated cognitive boundaries.
What becomes shared is the arbitration over scarce local compute.
My thinking is that I’m trying to build a system where separate layers of working contexts can coexist in bundles, while cleanly sharing a finite inference substrate in an honest and coordinated way.
In other words:
shared scheduling, over multiple actors.
That seems like the right long-term shape for local agent systems.