You ran a tight process. Three rounds. A system design session. A coding challenge they crushed in 38 minutes. Your senior engineer gave a thumbs up. You extended the offer feeling confident.

Then week one hits. Their first pull request is a mess. No error handling. Hardcoded values. A docker setup that wouldn't survive five minutes in staging. You're staring at the diff thinking: did we hire the wrong person?

Maybe not. But you almost certainly measured the wrong things.

The interview tested performance, not behavior

Here's what most technical interviews actually measure: how well someone performs under artificial constraints with a known audience.

A coding challenge asks a candidate to write a function. It has a clear input, a clear output, and a time limit. There's no ambiguity. No legacy code. No undocumented dependency that breaks silently on Tuesdays.

That's not engineering. That's a recital.

Real engineering work looks like this:

• You inherit a codebase nobody documented

• You have to make a judgment call about whether to refactor or patch

• You're balancing speed against maintainability with incomplete context

• You're choosing between three "correct" approaches based on constraints nobody told you about yet

Your interview tested none of that. So why are you surprised the first commit didn't match the performance?

What you saw vs. what you needed to see

Here's a simple breakdown of interview signals versus actual job signals:

The left column is what interviews reward. The right column is what the job demands. These are fundamentally different skills.

The real gap: judgment under ambiguity

The developer who aces your interview has optimized for a specific game. They've practiced leetcode. They know how to narrate their thinking during a pair programming session. They can draw boxes and arrows on a whiteboard that look like a system.

But none of that tells you what happens when they open a terminal on day one and face a production environment with real consequences.

What you actually needed to know:

• When they hit a confusing error log, do they methodically narrow scope or shotgun random fixes?

• When they use AI to generate code, can they critically evaluate what it produced?

• When there are three valid approaches, can they articulate why they chose one over the others?

• Do they ask about constraints before building, or do they assume and ship?

These aren't personality traits. They're observable engineering behaviors. And they only show up when someone is doing real work, not performing rehearsed answers.

Why "watching someone work" matters more than hearing them talk

A pilot doesn't get certified by explaining how a 737 works. They get put in a flight simulator that replicates the actual conditions they'll face. The sim doesn't care if they can recite aerodynamics theory. It cares whether they make the right call when an engine fails at 12,000 feet.

Software engineering should work the same way. Instead of asking someone to explain why SQL reads get slow, have them connect to a database, add indexes, change the queries, and confirm latency dropped. Instead of asking about docker internals, give them a broken container on an EC2 instance and see if they fix it.

The difference between talking about work and doing work is where bad hires hide.

The uncomfortable math

Most engineering teams spend 2-3 months on a hiring cycle. They invest 30% of their team's time in interview loops. They still end up with hires who underwhelm in the first sprint.

The process isn't failing because of sourcing. It's failing because the evaluation step rewards the wrong skills. You're selecting for people who are good at interviews, then wondering why they're not good at the job.

What to take from this

If your best interview candidate wrote bad code on day one, your interview was the problem, not the candidate.

Stop testing for recital. Start observing how people actually work: how they debug, how they make tradeoffs, how they use tools, how they respond to ambiguity. The signal you need isn't in what someone says they'd do. It's in what they actually do when the terminal is open, the codebase is messy, and nobody's feeding them a clean problem statement.

That's the only evaluation that transfers to the job.