Agent tooling, especially around code generation is such an interesting topic. Okay, how about this? Say, you're writing a range editor. A range editor is a tool that agents can use to edit a text file in tiny pieces by line range. Cascade is a good example of a range editor in the wild.
Thing is though. AI models have this one persistent problem that only shows up in deep testing. And that is... that they'll actually misuse their editing tools to hallucinate edits on files they haven't read.
Last week, Claude code pushed an update that stopped Claude from editing files it hasn't read. Claude does this occasionally. And, last night, I learned that the Llama4 models do the same thing, but much more frequently. And the new DeepSeek R1? As often as 50% of the time. It's jaw dropping.
This bug is subtle, but devastating.
So how do you fix it?
Well, Claude Code has limited range editing. The feature is really basic, and it won't let Claude make edits unless it reads the entire file. This is sensible. Simply prevent the problem by throwing an error. But... what happens when you aren't editing the whole file, where everything is based on line selection?
Tricky, right? It's much more complicated.
The way I solved this was by tracking what lines of what files have been witnessed by the AI within any given context window. Let editing happen if the file has been read by the AI. But... if it's trying to edit a section of the file it hasn't read yet, or the last time it's read the file is stale. Show the diff, ask "are you sure?" and present three options: 1. Review the file before making the edit 2. Cancel 3. Save it anyway.
It adds a step to the process.
And it still needs to be tested, but I think that's how it's done.
How would you approach it?