Skip to content

Teaching automated QA to reason about scope

New productivity milestone: My automated QA system started getting confused on scope and measuring project timelines. Previous to this, Claude's timeline estimates also seemed way off, but I ignored it as an artifact of the system.

But here, I was confused, because Mini (a QA process system built on ChatGPT) is designed to be thorough and detail-oriented, and she hadn't done that particular thing before. Remember, she's the sanity checkpoint.

Turns out, when I checked the line counts and cognitive complexity details that we're measuring now, I realized we're doing about a year's worth of code every week.

Figured out the code quality problem (as long as we're working in pythonic Go) and using the "robot Kanban" process I came up with.

It's interesting, because the way I've been managing context, is by making the documentation process absurdly pedantic and formal, while pulling back on unit testing until project milestones are hit. With humans, this would be impossibly stupid, but with AI, it's the only thing that makes sense.

The core idea here is that the documentation process is actually so formal, that in my 30 years of development, I have never seen any single human team do all of this in one place before. I don't think any single human ever could, honestly. It's a lot of a lot, but the audit trails are PERFECT.

This kind of production velocity presents some new practical challenges around architecture, focus, and organization that I need to figure out.

So I'm writing out a specialized tool to manage this. 59 files. Simplest thing I've built this way in a while.