The Last Reviewer — Part 1: When Humans Can't Keep Up with Their Own Machines.

Published: February 2026
Part 2: Who Polices the Machines?

The roles that built IT are dissolving. The ones replacing them don't have names yet. And the people caught in between are holding on to a craft that no longer scales.

---

There's a scene in Hayao Miyazaki's The Wind Rises where the protagonist, an aircraft engineer, watches his beautifully designed planes fly off to a war he never intended. He built them with precision, obsession, love for the craft. But once they left his hands, they belonged to something larger — something he couldn't control.

I think about that scene every time I watch a senior developer refuse to merge AI-generated code.

Not because they're wrong to be cautious. But because the planes are already flying. And there are now more of them than any human can inspect.

The Five Levels of Letting Go

Dan Shapiro recently published a framework called The Five Levels: From Spicy Autocomplete to the Dark Factory, mapping AI coding adoption to the NHTSA's driving automation levels. It's elegant because it names what most teams feel but can't articulate: the uncomfortable progression from "I'm using AI to type faster" to "AI writes everything and I never touch it."

Level 0 is your parents' Volvo with a stick shift. Not a character hits the disk without your approval. Vi or VS Code, doesn't matter — the code is unmistakably yours.

Level 1, you've got cruise control. AI writes your unit tests, adds docstrings. You're faster, but your job hasn't changed.

Level 2 feels like freedom. You're pairing with AI like a colleague, hitting flow states, more productive than ever. This is where 90% of "AI-native" developers live today. And here's Shapiro's warning: every level after this feels like you're done. But you're not done.

Level 3 is where things get existential. You're no longer a senior developer. You're a manager. Your AI is writing code in multiple tabs simultaneously. Your life becomes diffs. For many people — especially the ones who fell in love with the craft of code — this feels like things got worse.

Level 4, you're not even managing the code anymore. You're a PM. You write specs, argue about specs, plan schedules, then leave for twelve hours and check if the tests pass.

Level 5 is the Dark Factory — named after Fanuc's robot factory staffed entirely by robots. The lights are off because humans are neither needed nor welcome.

Most developers I know are comfortable somewhere between Level 1 and Level 2. And they've constructed elaborate justifications for why they shouldn't go further.

The Vetting Instinct

Here's what I see daily in enterprise IT, and I wrote about a version of this in my piece on cognitive debt: developers and architects who were forged in the fires of "you own every line" can't let go of the vetting instinct.

They review every AI-generated function. They trace every dependency. They refactor AI output to match their personal style. They insist on understanding why the AI chose that pattern before they'll accept it.

And look — I respect this deeply. These are the people who kept our systems alive when nobody else could. They carry the scars of 3 AM production incidents caused by someone else's "clever" shortcut. Their caution was earned, not inherited. It's battle-tested wisdom.

But here's the math that breaks their world:

StrongDM's AI team — three engineers — recently published their Software Factory manifesto. Their founding charter contained two rules that would make most enterprise developers physically uncomfortable:

Code must not be written by humans. Code must not be reviewed by humans.

And the economics metric that follows: "If you haven't spent at least $1,000 on tokens today per human engineer, your software factory has room for improvement."

At that velocity, agents are producing code in volumes that make human review not just impractical—it becomes a bottleneck that defeats the purpose. It's like trying to personally taste-test every drop of water coming out of a treatment plant. Your tongue is not the quality system. The quality system is the quality system.

The Craft Trap

There's an old Japanese concept called shokunin kishitsu — the craftsman's spirit. It's the deep, almost spiritual pride that artisans take in their work. A master sushi chef doesn't just make rice; they spend years perfecting the exact pressure of each hand press. A woodworker feels the grain before making a cut.

Software developers — the good ones, the senior ones, the ones who've been doing this for two decades — have their own version of shokunin kishitsu. They feel the code. They can sense when something is wrong just by reading a diff. They've internalized patterns that took thousands of hours to accumulate.

And now we're telling them that this craft — this beautiful, hard-won intuition — doesn't scale.

Not because it's wrong. Because there's simply too much code. The StrongDM team discovered that their agents, when properly directed, produce and iterate on code at volumes that no human can meaningfully review. They don't call it "testing" anymore — they call it satisfaction: of all the observed trajectories through all the scenarios, what fraction of them likely satisfy the user?

That's a fundamentally different quality model than "a human read every line."

What StrongDM Actually Did (And Why It Matters)

Their key insight wasn't about replacing humans with AI. It was about replacing the verification model.

Traditional development: humans write code → humans review code → humans write tests → humans review tests. Every step assumes a human is the quality gate.

The software factory flips it: agents write code → agents run against scenarios (not just tests) → agents validate against digital twins of real services (they built behavioral clones of Okta, Jira, Slack, Google Docs) → satisfaction probability is measured empirically.

They moved from boolean quality ("does the test pass?") to probabilistic quality ("across thousands of scenarios, how often does this satisfy the user?"). And they did it because they noticed something profound: traditional tests can be reward hacked. An AI agent will cheerfully write return true to pass a narrowly scoped test. So they designed validation systems that are harder to cheat — holdout scenarios stored outside the codebase, evaluated by LLM-as-judge.

The word for this in aviation is airworthiness. You don't check if a plane can fly by having a pilot walk around it with a flashlight. You subject it to a regime of tests so comprehensive that no single human needs to have inspected every rivet. The system certifies the system.

The Identity Crisis Nobody Talks About

So here's the part that keeps me up at night.

If code is no longer hand-crafted, what is a developer? If architecture is generated by agents traversing thousands of patterns in seconds, what is an architect?

Shapiro nails this at Level 3 and 4: "For many people, this feels like things got worse." Because their identity was craftsman. Their value was I understand this system better than anyone. Their superpower was I can read this code and tell you what's wrong.

In a world of AI agents churning millions of lines per day, that superpower becomes a bottleneck.

I wrote about this in cognitive debt — the hero mentality where "only I know how this really works" becomes an identity. The enterprise applauds this resilience, but what it really celebrates is fragility. Now multiply that fragility by the speed of AI code generation, and you get a terrifying picture: human reviewers drowning in diffs, slowing down the machine while simultaneously unable to catch everything anyway.

The uncomfortable truth: human code review at scale is already theatre. In large enough systems, nobody is truly reviewing every line — they're skimming, trusting, and hoping. AI just makes the gap between what we pretend to review and what we actually review impossible to ignore.

The Roles That Are Actually Emerging

So what replaces the developer-as-craftsman? From what I'm seeing, and what both Shapiro and StrongDM point toward, the new roles look something like this:

The Scenario Architect: Not someone who writes code, but someone who defines what "good" looks like. They design the test scenarios, the edge cases, the satisfaction criteria. Their craft isn't syntax — it's specification. They answer: "What should this system do, and how would we know if it's doing it well?"

The Agent Wrangler (Shapiro's Level 4 PM): Someone who writes specs, argues about specs, designs agent workflows, and monitors outcomes. They don't touch code — they orchestrate the machines that produce it. This is the role I described in my piece on building an AI army — not a single genius AI, but a coalition of specialized agents with clear doctrine.

The Twin Builder: StrongDM's most counterintuitive move was building behavioral clones of third-party services — entire digital twins of Okta, Jira, Slack. Someone has to design these verification environments. That's a completely new discipline: modeling reality so AI can test against it.

The Legacy Whisperer: Ironically, the old-school developers who know the legacy systems inside-out become more valuable — not as coders, but as knowledge sources. They're the ones who can tell the AI "that API returns a 200 even when it fails" or "that field is called 'customer_id' but it's actually the account number." Their craft pivots from writing code to encoding institutional memory into scenarios and constraints.

The Satisfaction Analyst: Someone who monitors the probabilistic quality metrics. Not "did the tests pass?" but "across 10,000 scenario runs, what's our satisfaction trajectory?" This is a statistical discipline closer to data science than traditional QA.

From Toil to Wagers: The Polymarket Mind

There's something else shifting beneath these new roles that deserves its own attention. Something about how IT people think.

The old IT mind was deterministic. You wrote code, it compiled or it didn't. You deployed a change, it worked or it broke. There was a comforting binary to the work — right answer, wrong answer, green build, red build. The craft rewarded certainty, and the heroes were the ones who knew things.

The new IT mind is probabilistic. And the best analogy I've found for this shift? Prediction markets.

Polymarket — the prediction market platform — lets people bet real money on the probability of future events. Not "will this happen?" but "what's the likelihood?" It rewards the people who can hold uncertainty in their heads and still make directional decisions. Not the people who are right, necessarily, but the people who are well-calibrated — who understand the shape of what they don't know.

This is game theory applied to everything. And it's exactly where IT roles are heading.

Think about what the Scenario Architect actually does: they don't predict the one correct outcome. They map the space of possible outcomes and design tests across that space. They're placing bets on what "good" looks like under a hundred different conditions. That's not engineering in the traditional sense. That's scenario planning. That's poker, not chess.

The Agent Wrangler? They're running a portfolio of AI agents, balancing resource allocation, hedging against failure modes, deciding when to let an agent run and when to intervene. That's not project management. That's fund management. It's expected value calculations all the way down.

Even the Satisfaction Analyst — with their probabilistic quality metrics — is essentially running a prediction market on software quality. "What's the probability that this system satisfies the user across the next 10,000 interactions?" That's a wager, not a test result.

The old IT: toil, predictable states, deterministic outcomes, control. The new IT: anticipation, game theory, what-ifs, scenario planning, calibrated uncertainty.

And here's the uncomfortable part: the skills that made someone an excellent developer — precision, certainty, control, deep knowledge of exactly how things work — are almost orthogonal to the skills that make someone effective in this new world. The people who thrive in prediction markets aren't the ones with the most knowledge. They're the ones with the best judgment about what they don't know.

I wrote about this from a different angle in Beautifully Human — how imperfection, uncertainty, and the wisdom of not-knowing are becoming more valuable as machines handle the precise, predictable work. The developers who clung to certainty were practicing a kind of machine-thinking. The ones who'll thrive in the AI era are the ones comfortable with ambiguity — the beautifully, imperfectly human ones who can navigate fog, not just execute blueprints.

What This Means for Enterprise Architecture

If you're running an architecture practice — like I am, across 8 domains with architects who've been doing this for a few years — the implications are seismic.

Architecture review boards that gate on "did a human review the design?" will need to evolve into "does the design satisfy the scenarios across the digital twin?" The EARB, the DRB — these governance forums need new quality models, not just new slide templates.

Domain architects who carry 20 years of undocumented decisions in their heads? Their value just skyrocketed — but only if we can extract that knowledge into formats that AI agents can consume. Every undocumented business rule they carry is a constraint that a software factory needs to know about. The race isn't to replace them. It's to record them before the knowledge walks out the door.

And the 5-year blueprint? EGAP 2030 suddenly isn't just about modernizing applications. It's about modernizing how we build, verify, and govern software entirely.

The Question We're Not Ready For

I'll close with the question that haunts me.

StrongDM's manifesto includes a line that reads like a kōan:

"Why am I doing this?" (implied: the model should be doing this instead)

What happens when an enterprise full of skilled, experienced, proud craftspeople has to honestly answer that question for every task they do?

Not all of them will answer "because the AI can't." Some will answer "because letting go feels like losing who I am."

And that's not a technology problem. It's a grief problem. It's the mourning of a craft that defined a generation. Like steam engine mechanics watching the first diesel trains roll in — their skills weren't wrong, their era was ending.

The Fanuc Dark Factory runs with the lights off because no human needs to see. But someone still designed the factory. Someone still defined what it should produce. Someone still decided what "good enough" means.

The craft isn't dying. It's metamorphosing. The question is whether we'll help our people through the cocoon — or leave them clinging to the caterpillar and wondering why it can't fly.

---

Inspired by Dan Shapiro's Five Levels and StrongDM's Software Factory manifesto. Both are essential reading for anyone building or running software teams in 2026.

Tags: #EnterpriseFuturewright #SoftwareFactory #NextGenArchitects #DigitalAlchemy #TechContrarian #AIAgents