The nursing problem, infra agents, and their slow crystallization

As a small team, we always have a million things to focus on. In my mind, I sometimes think that we could expand our team in order to be able to brainstorm all these issues we’re trying to optimize: how can we have better oracles to query our codebases, how can we get our workers to start faster, how can we spend our budget more intelligently across tasks that we need to run, how can we cache artifact builds more efficiently across workers, etc.

A large company like Meta would have people focusing on these different parts of the stack, and coming up with clever algorithms to optimize things ad-infinitum. On the other hand, a big team has trouble moving fast, they’re constantly fighting against an aging codebase, and coworkers are fighting against other coworkers, and also nobody dares making changes too large as often nobody understands the whole codebase and is scared of breaking things.

Making important changes is fast and easy with a small team, we already knew that. Today, coding agents make this even more insane.

Wait, we have coding agents, why don’t we use these to solve all of these problems? Well, with coding agents we have another problem: the nursing problem.

Agents have to be nursed, we don’t trust them fully yet, they don’t seem to understand our tastes, where we want to go, and they’ll gladly do everything at the same time without caring too much about our metrics. Sure we can try to hardcode all of the stuff we want to optimize, and all of the stuff we don’t care about, but they still seem to come up with weird decisions that we don’t like. So for most of us, we don’t want to let them roam around and change code without supervising them.

The solution I’ve been toying with the idea of infra agents. Agents that have long, durable sessions, and live within the infrastructure, in place of code that would have attempted at deterministically figuring out what to do.

We’ve all sort of coming down to the conclusion that flows and orchestrations with a few edgecases should be hardcoded, and that the ones with a million edgecases should be given to the agents, so I guess I’m talking about the ones in the middle that should be hardcoded as well but we don’t have time to figure out how to do it and we don’t want to let crazy agents hardcode these rules forever.

So the solution is to just have an agent move and live in that part of the codebase, see the stuff we’re seeing, and make decisions based on that. When you see everything, the decision is much easier to make on the spot / just-in-time. Maybe with time they’ll come up with scripts to get there faster, and with more time these scripts will become so reliable that they will encode the full algorithm that we would have come up with in the first place had we had the time to do that.

I call this point crystallization. I’m not sure it ever happens, because our codebases are constantly shifting nowadays, but maybe we will see instances of crystallization happen around us, maybe you will witness some of that.