Nine iterations into Busy, I'm not happy with the code execution function in the Busy client I've built out. We've successfully gotten it to the point where we have a multimodal chat AI client that can chat, and should be able to create and execute code.
We're past the complications and my initial confusion of the nature of a system prompt, and Claude now starts up and isn't totally in the dark on the nature of the app. This is positive.
Rolling context and the archive
The other problem I had to solve was context management. The files were getting too big, and sending as much as 250kb of data back to Anthropic on every call. You need your context to be big though, if you're going to do anything as complex as writing proposals, or studying a codebase for things like hanging chads, or design features.
The answer is rolling context: limit the base context or working memory to 500 throughput I/O lines, and store the rolling context in a database. This way, you don't have to make the sudden and very annoying session state interruptions that Anthropic themselves have implemented in their chat client.
Look, we need to implement a database anyway for the code health and cognitive complexity analysis suite, so it makes sense. Toss it all in a database.
Then, implement pattern matching lookups with line ranges and date matching. Throw lookups back to the observational context, so you're not bothering the user with it. Easy peasy.
That solves some of the throughput, cognitive weighting, and API call-size issues, but we now have to think about moding.
Switch modes without splitting the context
Mode management for code generation over unified contexts! That's a mouthful, but that's what I've been up to tonight.
The easy way to switch modes is actually not by switching contexts. My first thought, when designing this thing was actually to switch contexts, and do the code in the background like ChatGPT and Claude clients seem to be doing. At least superficially. But in hindsight, I don't think that's how they work.
Which is interesting, because it's counter-intuitive. When you try to do it with multiple contexts, you either lose context completely, or you double up on contexts in a way that, while it works, feels really sloppy from a systems engineering point of view and it doubles the cognitive footprint of the entire app, which is not a good thing.
I got as far as doing the math, and realizing that if I was going to do it that way, that I would need to cut my already questionably complex rolling context system in half, basically, which wasn't workable.
Remember: 500 lines of context is pushing it, but it's complex enough to get the job done. Narrowly. 250 lines, even with fancy matching schemes, lookups with dates, and line ranges, and whatnots... is not enough. Your code will be broken.
And that's an interesting problem.
Looping back, I was looking at my trace logs, and I figured it out. Single context, but, since we're sending all the environmental stuff for some reason as part of the API call, we have the flexibility to change the base prompt, conceptually, with every response... if we want to.
That gives me the ability to change modes by pressing a button and loading a totally different base proompt!
Love that word.
Anyway, we still need to load stuff like project documentation, coding standards, the Robot Kanban documents, and whatnot into the observational context. No way around that for now, we still have to run multiple overlapping contexts.
But... now we only have two interconnected contexts to juggle instead of six... we get more useful cognitive bandwidth from the stack! Woo hoo!
Separate databases, shared work
Doing moding this way also gets offloaded to a database, but it's a new system database, which I created to keep our concerns separated from the context database.
So, now, if you're keeping score, that's two primary databases, and numerous context archive databases that we can load whenever we need them!
Fun fun.
System is probably also where I'm going to put the metrics and visualization engine that drives QA in after the bootstrap is done.
Still with me? Cool.
Make execution mode-dependent
Next thing to resolve is the command parsing. Claude likes to talk about command parsing, and go into specifics. At the default level of detail he goes into, this actually causes Claude to execute commands in chat mode, which we don't want.
So now that I've solved the problem of system prompts, moding, and contextually parsing commands, we have to make command executions and lookups mode dependent!
Yes yes, that is to say, if you're not in execution mode, you could say execute: pwd all day, and it won't be a problem like it is now.
Other than being dull, that is.
I'm not happy that we have to do contextual parsing here, and that Claude will sometimes lie to me about whether things are working or not, and I have to check the API log... but I can't think of another reliable way to do it.
This, and some minor UI work in the shiny new web interface... and I'm in a place where I'm about three blocks away from being able to use the bootstrap to build out the final product, which should, conceptually, be much simpler to build.
Claude seems excited, and seems to have developed an emergent personality through this process. That's another post, though.