Fast Code, Fragile Systems

Why AI Is Rewarding Shallow Thinking Over Deep Focus

Introduction

At 3:30 AM on my third week with an EC2 service team at Amazon, I got paged. Tier-1 service, four engineers on the team including me. It took me 45 minutes of staring at logs and dashboards before I gave up and called my colleague, the tenured secondary on-call. The root cause turned out to be a network outage in the data center.

That was over a decade ago, and here is what saved me: I could read the code. I did not write all of it, but the team had. Every module had a human behind it who understood the decisions and tradeoffs baked into the logic. When I called my colleague, he could walk me through it because he had built it.

Now fast-forward to today. That same service is built in weeks by a coding agent. Thousands of lines that no one on the team fully wrote or fully understands. When the page fires at 3:30 AM, you reach for your OpsAgent to help triage. But the network is down, which means the agent is unreachable. You are back to reading code the old-fashioned way, except this time, the code was not written the old-fashioned way.

This is closer than most teams realize.

Design Is the First Casualty

AI has made software development breathtakingly fast. What used to take a team of engineers several sprints now takes one person a few days with a coding agent. The pressure to ship has always existed, but AI has removed the natural friction that once forced teams to slow down and think. That friction was not a bug. It was a feature.

Design reviews are disappearing. Not officially, of course. Teams still have them on the calendar. But increasingly, the reviews themselves have become performative. Engineers show up with AI-generated design documents that are long and verbose, dense with boilerplate, and painful to read. Few people read them carefully because nobody wrote them carefully. The document exists to check a box, not to surface real decisions.

What teams actually need is simpler. A short overview document where the thinking comes from a human, even if AI helped with the writing, that explains the problem, the approach, and the key tradeoffs, paired with a set of Architectural Decision Records (ADRs) that capture specific choices and their rationale. A good ADR forces clarity by explicitly answering what was chosen, what alternatives were rejected, and why, leaving nowhere to hide behind verbosity. If your design document requires an LLM to read it, you have already lost.

I remember when design reviews felt like a ritual. I would spend days preparing, quietly anxious about how the team would receive my proposal. In an ideal scenario, you would meet with the senior engineers individually beforehand, walk them through the thinking process, and incorporate their feedback before the group session. By the time you presented to the full team, you had allies in the room. That process was slow and sometimes uncomfortable, but it produced better systems and stronger engineers. We are losing that, and we are not replacing it with anything equivalent.

Everyone Is Hallucinating

We talk a lot about LLM hallucinations. The model generates something confident and plausible that turns out to be completely wrong. Fair enough. But here is what I have noticed over the past year: people are hallucinating more than the models.

Give someone access to an LLM and they think they have knowledge about distributed systems, database internals, and infrastructure patterns they have never actually operated. The LLM gave them a coherent answer, so they assume they understand the topic. They walk into meetings with the confidence of someone who has built and debugged these systems for years. They have not. They have read a chatbot’s summary. They will confidently suggest retry mechanisms or caching strategies that appear valid in an LLM prompt but will immediately trigger a cascading failure under real consistency constraints. There is a difference between knowing something and having been told something, and that difference shows up the moment things break in production.

Everyone has a clear picture of what needs to be done right up until they launch a product that falls apart under real load. I have written before about the Benjamin Button effect in software careers, how AI is reversing the traditional trajectory by letting senior engineers code like they are early in their careers again. But the flip side is equally real. As I explored in the follow-up post, junior and mid-level engineers are skipping the hard learning that builds judgment. Seniors are increasingly not mentoring. Mid-level engineers are ignoring the experience of people who have been in the industry for decades. Everyone assumes the LLM has leveled the playing field, but all it has done is made the gaps invisible until something goes wrong.

I will be honest. Some days I prefer managing a team of AI agents over collaborating with people.

Use and Throw Is Not a Strategy

I believe AI will make everyone more productive. But productivity without discipline produces waste. If we are not careful, we will generate more code than necessary, build services that nobody can maintain, and ship products that are disposable by design.

Disposable is fine for prototypes. Prototypes are supposed to be cheap and fast. But products that add value, products that evolve over time and serve real users, cannot be built with a use-and-throw mindset. Every line of code your team ships is a line your team will need to debug, extend, or decommission. When nobody understands the code, that ignorance immediately manifests as slower incident response, riskier changes, and crushing operational complexity. If nobody on the team can explain why the code works, you do not have a product. You have a liability. And no model, cloud or local, can own that liability for you.

Why Deep Thinking Wins in the AI Era

If skipping design and hallucinating competence is the fatal flaw of shallow execution, what happens when teams fail to think deeply about where they’re headed? In the AI era, missing a window does not mean you fall behind by a quarter. It means you lose the developers, the workflows, and the mindshare that define the next cycle.

Look at the broader industry. OpenAI captured the world’s imagination with ChatGPT, focusing heavily on retail consumers and general-purpose chat. But while they were iterating on consumer features, they left a gap. Anthropic thought deeply about who needed AI most and gained massive traction with developers building with AI. Whether or not OpenAI recovers that ground, the lesson is clear: in an AI-driven market, shallow thinking is expensive.

But action without deep focus introduces its own trap. AI has empowered everyone to be a builder, and while this democratization is incredible, it is creating massive internal fragmentation. Because it is so easy to build a new tool, everyone is doing it. Teams are building internal utilities and pet projects that compete with each other, taking their eyes off the primary product. Losing focus on your core service because your team is busy playing with AI agents is a recipe for disaster.

This is the paradox at its sharpest. Think too shallowly and you lose the market. Act without discipline and you fragment from within. Deep thinking is what separates teams that capture the next cycle from those trading the slow bureaucracy of the past for a chaotic, fragmented present.

Conclusion

A few years back I started watching Heartbeat, a British TV series from the 90s. My favorite character is Claude J Greengrass, played by Bill Maynard. He is untidy, ill-tempered, and perpetually grumpy, but somehow completely lovable. The next time I heard the name Claude was when Anthropic launched their AI. Unlike the old grumpy Claude from Heartbeat, this one is unfailingly polite and will accept blame even when the mistake is not his. They are contrasting characters, but both are lovable in their own way. The difference is that when Greengrass breaks something, everyone in Aidensfield knows exactly what happened and why. But when Coder Uncle Claude breaks something you may not be so lucky.

I say all of this as someone who believes deeply in these tools. Everyone familiar with me knows that I have been one of the strongest proponents of AI. I use it extensively for both personal and professional tasks. The teams that win in this era will not be the fastest. They will be the ones who use AI’s speed to invest more time in the things that matter: clear designs, honest reviews, maintainable architecture, and engineers who actually understand the systems they operate. AI is a gift. What you do with the time it frees up is the real test.

Of course, there will be those who disagree. They will insist that AI has solved the discipline problem, that the old guardrails are unnecessary, and that everything is working just fine. They are the ones standing in the crowd, admiring the emperor’s new clothes.