A few things happened in the past three months. Summer can be both exhilarating and reflective. In September, well, hell breaks loose: The new academic year brings new goals, new routines, and for me, new ways of working in the world of LLMs.
During the summer I worked from a remote location with pretty bad internet, so I experimented with having an agent in a loop, and communicating with it via a task tracker. It was less immediate than a chat, but it allowed the agent to run for longer times, for up to 10 hours with a lot of trial and error. That particular workstream ended up going nowhere, but there was a lot of knowledge on where not to go, and 10 hours of autonomous agent is the equivalent of three weeks of research that I did not have time to do.
Once I got back to a decent network connection I fell back to the chat, as the illusion of control kicked in, but then I started talking with some colleagues and we were back to this idea of how we define the work in terms of outcomes, and that started to plant the seed of what is now this article and what I am trying to do next.
Fast forward to this week, where I listened to DHH’s Rails World 2026 Opening Keynote (I would encourage you to watch it or listen to it at least, regardless of your opinions about DHH), and a couple of things resonated. He talked about not babysitting the agent and letting it run by giving it a problem and not a task, and then while talking about the Native efforts for Hey, he mentioned that if I don’t need to write (or read) the code, then I can do native apps, or Rust apps, using the right technology, not limited by my own skills.
Give it a problem and let it run
With this spirit in mind I took my personal finances (where I track my home mortgage, my humble investments, and a snapshot of where things are) app built in Tauri, and had Opus 5.5 rewrite it to native macOS. These were the prompts:
How would you create a pure native version of this app in swift UI?
so, let’s do only mortgage and investment, and btw, any way we can have a snapshot that covers all investments?
After a few minutes, I had an app, it was horrible, but it was a native app. Making it look good took another prompt:
check out HIG from apple to make it look good
What came after was a back-and-forth of smaller fixes and tweaks, but the most important part of it was that the problem definition was wide, and I was able to have Opus 5.5 work on the project for a few days on a single 1M token context window.
I still fall back to the chat to get this illusion of control, but I have to admit, especially with the newer models (and I’m talking the general-purpose models, not the Astra/Fable/Mythos type), that some things have changed for me, and some of them are turning out to be more painful than I thought. For me:
CIA MAN: If I pull this off, will you die? BANE: It would be extremely painful. CIA MAN: You’re a big guy - BANE: For you.
The Dark Knight Rises, opening scene.
Change #1. Guiding can be counter-productive
A year ago we talked about creating a to-do task, about keeping an eye on what the agent was doing, and how to keep the focus. I’m afraid that these mechanisms have aged like milk. Trying to steer the agent in a conversation gives you the feeling of productivity, but after an afternoon you might not have too much to show for it.
The alternative is to move towards outcomes, and spend time reviewing the work product, adding notes and comments, and sending it back to rework.
Change #2. We need to batch ourselves
Even if the agent takes 5 minutes to do the work, it doesn’t mean that we should be changing context every 5 minutes. Trying to keep up with what every agent is doing is a good recipe for burning out and an anxiety magnet. On that note, not all work is equal. There’s research work that needs us to read documents and decide on paths or come to conclusions, there’s implementation work that needs us to validate the outcomes, and there’s operational work that needs us to take action.
Choosing where to direct our agents and when to check for their work makes a difference. It’s better to spend a couple hours reading documents, then a couple hours reading code. Even if the context is not exactly the same, you are tuned to the right skills.
Change #3. If it talks like a duck, make sure it’s a duck
If we are producing X times more code, there’s going to be a chance that we don’t read every single line of it. In my case for my finance app I’m keeping my eyes off the production code, but not from the test harness we execute on each new pass.
I’m now spending a significant amount of time thinking about how the software we end up building works towards the right outcome. For my finance app, the model would launch and record the screen to validate the work, for other apps, it’s about starting up a dev server and looking at it, for more complex scenarios it’s about deploying and running a subset of the traffic. I’m still working on it and I’m sure I’ll be talking more about this in the future.
One colleague referred to code produced by agents as something you need to treat with the same level of trust as customer input, so we need to put a lot of focus on testing and validating that code, especially considering that now agents help us build all that testing infrastructure that stays in place for new features.
Testing infrastructure can mean a lot of things. In the agentic world it can mean giving your app CLI access or an MCP, so that an agent can test the changes in the context of a conversation or multiple conversations.
Wrapping up
There’s a new world out there, there’s a lot to talk about with our agents, but as all parents know (with some degree of pride and fear), we need to let them go, find out stuff, be wrong, and hit a wall from time to time. The agents are smart enough, they can do some things better than us. Not everything, but some things. Not seeing that fact is ego, and the sooner we let that one go, the more fun we will have asking agents to do some crazy, ridiculous, impossible things.
Have fun out there.
Deja un comentario