As a business that creates software for a living, the recent surge in the abilities of AI agents has been an intriguing thing to witness and explore — but, computer scientists at the Model Evaluation & Threat Research (METR), a non-profit research group have claimed:
“… we find that allowing AI actually increases completion time by 19 percent — AI tooling slowed developers down.”
The study involved 16 experienced developers who work on large, open source projects, and this limited sample size does at least warrant some scepticism.
Having used AI as an assistant for the last 5-6 months, I decided to build my own version of Google Maps, as a technical exercise, to explore — to a limited extent — the potential of AI.
How I built Viaje
I use Microsoft Visual Studio Code, known as an IDE, an integrated development environment that combines several different tools and services to aid in the process of software development.
Visual Studio Code allows me to chat with an AI agent, and there are several of them (known as “models”) to choose from — in this example, I switched between GPT-4o from Open AI, and Claude Sonnet 4 from Anthropic.
Each time I correspond with an agent, it’s considered a prompt, and like a conversation with a human, words have consequences. It’s possible to create new conversations with an agent, and then switch between them. If the task is complex, I use long-running conversations for the same reason we create an email thread, to retain the essential context of the conversation.
First, I explained to the agent what I wanted to build, supplemented with a list of features, the “stack” (the technologies I wanted to use), and used that as the basis of the continuing conversation with the agent.
Once I was confident it understood what I had in mind, I asked it to create a “README.md” file of what it understood. Here, a “README.md” is a file that contains essential information regarding a project, a common convention in software development.
Second, I asked it to create a plan of action: one each for the backend (the “business logic” of the application that resides on the host) and the frontend (the “presentation logic” of the application that runs in the web browser), which necessitated a bit of a discussion regarding the specifics of the two plans (some of the groups of tasks it had created were in the wrong order, while some were superfluous).
Third, I created an instructions document for the agent(s), most of which serving to add technical guard rails, nomenclature, and define specific versions of technologies.
Between the README, the plans, and the instructions, the agent had an excellent “context” from which to build. Put simple, each time I chatted with an agent, it would refer to the aforementioned documents to populate the context through which our conversation would be focused, refined, filtered and so on.
Fourth, we executed the plans. In the space of 3 weeks (non-contiguous time, that would have been about 4-5 days), I had a working product where I’d written less than 100 lines of code.
In addition to the usual spread of features (step-by-step navigation, local amenities, weather and so on), the agent and I added logic to detect if a destination was coastal using Ordnance Survey Boundary-Line data, and then the Environment Agency Tide Gauge API to find the actual tide times, throwing in tide prediction within a 24 hour window to supplement the route planning that allowed for choosing a date in the future.
Conclusions
Viaje was a success, but what did I learn?
- AI is excellent at building plans. AI is excellent at executing specific tasks. AI is terrible if allowed to execute an entire plan of action (two, in this case) without human supervision.
- There was a lot of overlap between the two plans, and the agent — in spite of understanding the connection — couldn’t implement something nuanced and structured alone, hence the constant guidance.
- Each agent is different: I found that GPT-4o could do the bulk of the work until it encountered what were to it insurmountable problems, where I’d then have to switch to Claude Sonnet 4 to get things moving again.
- A human-like absent mindedness would creep in from time to time where it would forget some technical specific, or that something that had already been done.
- As the project grew the more precise its suggestions and recommendations become.
- If allowed, the agent would keep adding and adding code!
How long would it have taken me to build Viaje without AI?
I chose some cutting edge technologies which would have incurred a steep learning curve, and with this in mind, I suspect 2-3 weeks of contiguous time, in stark contrast to the actual 4-5 days.
Would I recommend using AI?
I’ve enjoyed the most success with the AI agents when I followed these 3 simple rules:
- precision prompts;
- specificity of task(s);
- constant supervision.
In addition to software development, I’ve also used AI agents to do research, to brainstorm ideas and then attempt to validate their fitness. Here, point 2 is critical, in that it’s best to keep the tasks simple but additive, in that they’re chained: Task A contributes to Task B; Task B contributes to Task C and so on. Asking the agent to implement Tasks A through F is often when the problems begin.
AI sometimes gets things wrong, the same as we do, but the perception is that it shouldn’t. AI is not magic, and while that must seem obvious, a lot of the confusion I’ve seen has been in how people have attempted to use it, expecting magic things to happen.
AI is nascent, evolving, flawed, but also compelling and promising. Remember that we are the training data of AI.