I hired someone who aced every interview. Clean code samples. Great system design explanations. Talked about scalability like they'd built Netflix. Three months in, they still hadn't shipped a single feature. That hire cost me $47,000 in salary, two missed deadlines, and a complete rethink of how I evaluate engineers.

Lesson 1: people who can explain systems aren't always people who can build them

My mistake started in the system design round. The candidate drew beautiful diagrams. Talked about load balancers, caching layers, database replication. Used all the right terms: "eventual consistency," "circuit breakers," "idempotency."

I was impressed. I shouldn't have been.

When they joined, I assigned a straightforward task: optimize a slow API endpoint. Instead of profiling the code or checking database queries, they rewrote half the service "to make it more scalable." Two weeks later, the endpoint was still slow. They'd added three new dependencies and introduced a caching bug that took another developer a day to find.

The gap I missed: knowing how something should work versus actually making it work.

Whiteboard interviews rewarded their ability to talk about architecture. But I never watched them debug real code, make tradeoffs with actual constraints, or ship something end-to-end. I tested theory. The job required execution.

Here's what I should have done:

• Give them a slow query and ask them to fix it, not explain it

• Watch them optimize actual Docker containers, not discuss containerization philosophy

• See them refactor messy code under time pressure, not design greenfield systems

System design interviews select for people who sound senior. Real work requires people who can operate senior.

Lesson 2: AI didn't break hiring—it exposed how weak our signals already were

Six months after that bad hire, I posted another role. Got 340 applications. Half had portfolios that looked identical. GitHub repos with perfect commit histories. Cover letters that hit every keyword.

I couldn't tell who was real.

One candidate's take-home project was flawless. Production-ready error handling. Perfect TypeScript types. When I brought them in for a follow-up, I asked them to explain one design decision. They couldn't. Took them three tries to describe why they used a specific data structure.

They hadn't written it. Or they'd copy-pasted it. Or ChatGPT had done the heavy lifting and they'd just stitched pieces together.

The real problem: I was still evaluating outputs, not process.

A polished GitHub repo tells me nothing about how someone thinks. I needed to see:

• How they structure a problem before writing code

• What they do when they hit an error they don't recognize

• How they use AI—as a crutch or a multiplier

• Whether they can explain tradeoffs, not just implement solutions

I started doing 20-minute live sessions. Not whiteboarding. Not leetcode. I gave candidates access to a broken staging environment and said: "This API is failing. Walk me through how you'd debug it."

The difference was immediate. Some candidates opened logs, checked database connections, traced requests. Others froze. One admitted they'd never actually deployed anything to production.

That's the signal I needed. Not what they built at home with unlimited time and tools. How they think under realistic conditions.

lesson 3: I was filtering for desperation, not talent

My original process looked like this:

1. Resume screen (keyword matching)

2. 90-minute coding test

3. Take-home project (8–12 hours of work)

4. System design round

5. Culture fit interview

Total time from candidate: ~15 hours. Total time to hire: 11 weeks.

Good candidates dropped out at stage 3. One sent me this: "I have a full-time job and two kids. I can't spend a weekend building a project for a company I might not even join."

The people who stayed were either desperate or unemployed. Not always bad, but definitely not the full talent pool.

I was selecting for free time, not skill.

What changed: I collapsed steps 2 and 3 into a single 30-minute session.

Instead of asking candidates to build something from scratch at home, I gave them a real task:

• Fix this memory leak in a running service

• Add an index to this slow query and confirm latency drops

• Deploy this containerized app and explain what's failing

It mirrored actual work. It took less time. And I could watch them do it.

Drop-off rates fell by 60%. Time-to-hire dropped to three weeks. And critically, I stopped losing great candidates who had lives outside of interview loops.

What I do differently now

I don't trust resumes. I don't run long coding tests. I don't ask people to design systems they'll never build.

Instead, I watch people work. Not on a whiteboard. Not in a sanitized coding environment. In something that looks like the job.

I give them 30 minutes and a realistic task. I watch how they approach problems. I see if they ask about constraints. I check if they can explain their choices. I evaluate how they use tools—including AI.

If they can't ship in simulation, they won't ship in production.

The developer who couldn't ship taught me that interviews measure the wrong things. They reward people who interview well, not people who execute well. Three months of missed deadlines and $47,000 later, I finally learned: stop testing what people know. Start watching what people do.