Skip to content

The expensive part is the trial and error

I was watching this video by Wes Roth:

https://www.youtube.com/watch?v=eR167BCL-4g

And I have some thoughts, as an agent designer, that might not necessarily be obvious to engineers who don’t work closely with these systems.

I also spent a decade as a clean room engineer. Reverse engineering existing software is actually something I’m good at.

The thrust of the video is the Claw Code project, and the fact that it allegedly only took two hours to produce with a Codex wrapper. That’s totally believable, and it makes sense why people might be getting nervous about it.

After all, if anyone can produce something like Claude Code from scratch with Codex automation, why do you need developers at all?

Except it’s not quite that easy.

There is a massive gulf between reverse engineering a product that already exists and producing an original one.

Claude Code, in this case, is a system designed and implemented by Anthropic. The team has apparently done a major overhaul on it recently, if the source repo is to be believed, and it shows in the work. Love or hate Anthropic, that took real engineering grit, and months of testing to reach its current state.

Credit where credit is due.

Trial and error is the expensive part

The real issue is that last part. All the testing.

When you build agent systems, even straightforward harnesses like Claude Code, the biggest expense is the trial and error. Probabilistic agent systems are stochastic, but predictable once you know what to expect.

Unfortunately, given the nature of the beast, you do not know what to expect until something happens.

For example, if you look at this new Python fork for Claw Code, you'll notice that it falls short on issues like error handling, rate limiting, fallback, and exponential backoff. It is also weak on alternate tagging schemes, and the kind of harness weirdness that, say, a program like Opencode is not.

Why?

Because those are demons you only know you have to fight once you start dealing with smaller providers, unpredictable local model tool schemes, flaky infrastructure, and all the other ways real systems refuse to behave in practice, and outside of the lab.

Claw Code suffers from a number of completeness issues... but that is to be expected, given the age of the thing and where it came from.

What it does not represent, at least not yet, is the kind of thorough, thoughtful testing and design work that makes something production-ready.

So I guess what I’m saying is: your job is safe.

At least for now.