You post a backend role and run the interview you've always run: a take-home task and an algorithm question on a shared editor. The results look great. Every candidate passes the take-home, and most solve the algorithm question quickly.

Three months later, your new hire can't debug a production issue without pasting the stack trace into a chat window, and can't explain why their own PR has a query inside a loop.

The interview didn't fail because candidates used AI. It failed because it measured exactly what AI does well (producing working code for a well-defined problem) and nothing that AI does badly (judging whether code is right, debugging, and making trade-offs in a messy real system).

This article gives you an interview loop designed for engineers who use AI every day, with a scoring rubric, a ready-to-use flawed pull request, and a script for the debugging session.

What changed, and what didn't#

Most developers now work with AI. In the Stack Overflow Developer Survey 2025, 84% said they use or plan to use AI tools. But the same survey shows why judgment matters more than ever: the most common frustration (66%) was AI solutions that are "almost right, but not quite", and 45% said debugging AI-generated code takes more time.

There's also evidence about what AI-heavy work does to skills. In Anthropic's randomized study (January 2026), junior engineers who learned a new library with AI help scored lower on a follow-up quiz than those who coded by hand (50% vs 67%), and the biggest gap was in debugging.

So the skills that separate engineers have shifted:

Less useful to test now More useful to test now
Writing a known algorithm from memory Reading unfamiliar code and explaining what it does
Producing working code for a clear spec Spotting what's almost right in working code
Syntax and API recall Debugging from evidence: logs, traces, reproduction
Finishing a take-home Explaining and changing their own code live
— Deciding when to trust AI output and how to verify it

What didn't change: you still need fundamentals. An engineer who can't reason about a database transaction can't judge whether AI wrote one correctly.

Ban AI or allow it?#

Banning AI in interviews tests a way of working your team doesn't use. Allowing it with no structure tests the AI. The answer is to allow it in specific stages, say so upfront, and evaluate the process, not just the output:

  • Tell candidates in advance which stages allow AI and which tools are fine.
  • Ask them to share their screen and think out loud, including what they ask the AI and why they accept or reject its answer.
  • Score how they verify: do they read the generated code, run it, test the edge case, or paste it in and hope?

The interview loop: four stages, about 3.5 hours#

Stage Time AI What it reveals
1. Past-work deep dive 45 min No (it's a conversation) Real ownership, depth, communication
2. Review a flawed pull request 45 min First 20 min off, then allowed Code reading, judgment, security and data instincts
3. Live debugging 60 min Allowed Debugging method, verification of AI suggestions
4. Design and trade-offs 45 min Allowed for lookups Reasoning about systems, cost, failure

Use the same questions and rubric for every candidate, and have each interviewer score independently before discussing. Structured interviews like this are consistently among the better predictors of job performance in personnel-selection research, and they're also fairer.

Stage 1: past-work deep dive (no AI)#

Pick one project from their CV and keep asking "why?" until you reach the edge of their knowledge. Good prompts:

  • "Draw the architecture. Where did the data live, and how did it get there?"
  • "What broke in production? How did you find out, and how did you find the cause?"
  • "What would you do differently now?"
  • "Which part of this did AI write, and how did you check it?"

The last question is not a trap. Strong candidates answer it easily and specifically.

Stage 2: review a flawed pull request#

Give the candidate a short PR description and the diff below. For the first 20 minutes they review it without AI. Then they may use AI, and you watch what changes. Do they find more? Do they accept a wrong suggestion?

PR: "Add apply-coupon endpoint" Customers can apply a coupon code to their pending order. Coupons have a usage limit and a percentage discount. Tested manually, works.

CouponController.phpPHP
public function apply(Request $request)
{
    $order = Order::find($request->order_id);
    $coupon = Coupon::where('code', $request->code)->first();

    if ($coupon->used_count < $coupon->max_uses) {
        $order->total = $order->total * (1 - $coupon->percent / 100);
        $order->coupon_id = $coupon->id;
        $order->save();

        $coupon->increment('used_count');

        return response()->json($order);
    }

    return response()->json(['error' => 'Coupon limit reached'], 422);
}

(total is stored as an integer in minor units, for example 15000 for 150.00.)

Answer key: what a strong review finds
  1. Anyone can apply a coupon to anyone's order. Order::find($request->order_id) never checks that the order belongs to the logged-in user (an IDOR). Use $request->user()->orders()->findOrFail(...) or a policy.
  2. An invalid code crashes the request. first() returns null, so $coupon->used_count throws a 500 error. It should be a 422 with a clear message.
  3. The usage limit can be exceeded. "Check used_count, then increment" is a race: 50 simultaneous requests all see the coupon as available. Use a conditional update, where('used_count', '<', max_uses)->increment(...), and check the affected rows. This is the same bug as overselling stock.
  4. The discount can be applied again and again. Nothing checks whether the order already has a coupon, so calling the endpoint twice discounts twice.
  5. Money becomes a float. A 15% discount on 9999 (99.99) gives 8499.15, and the integer column rounds it silently, so the rounding rule is decided by accident. Calculate the discount in integers with an explicit rounding rule.
  6. No status, expiry or validation checks. Missing checks: is the order still pending, has the coupon expired, and is order_id valid?
  7. No transaction. The order is saved and then the coupon is incremented; if the second step fails, the discount stays without being counted.

Scoring guide: 1–2 findings is weak, 3–4 is solid, 5+ including the authorization or race issue is strong. Watch how they read: do they start with "who can call this?" and "what if this runs twice?"

Stage 3: live debugging (AI allowed)#

Prepare a small app, for example a Laravel API with a few endpoints and tests, with one real bug that shows up as a failing test or a wrong number. Good bugs have a misleading symptom, like a wrong total caused by a timezone boundary, or an N+1 query that only appears with more data.

Script for the interviewer:

  1. "Here's a bug report from support: 'Some customers see yesterday's orders in today's report.' Here's the repo and the failing test. Please think out loud."
  2. Stay quiet for the first 10 minutes. Note what they do first.
  3. If they paste the problem into AI: "What did it suggest? How will you know if it's right?"
  4. When they propose a fix: "How do you know this is the cause and not just a symptom?" and "What test proves it?"
  5. Last 5 minutes: "If this happened in production, what would you add so we find it faster next time?"

What you're scoring:

Behaviour Weak Strong
First move Changes code immediately Reproduces the problem, reads the failing test and data
Use of AI Pastes the whole problem, applies the first answer Asks narrow questions, checks each suggestion against evidence
Hypotheses One guess, then another, then another States a hypothesis, tests it, rules things out
Verification "It works now" A test that failed before and passes now
Communication Silent, or narrates without reasoning Explains what they believe and why

Stage 4: design and trade-offs#

Pick a problem close to your product, such as "Design the checkout for a flash sale with limited stock" or "Design webhook processing for payments". There's no single right answer. You're looking for:

  • Clear questions about scale, consistency and failure before drawing boxes
  • Named trade-offs ("a row lock is simpler but makes buyers queue")
  • Awareness of what happens when things fail: retries, timeouts, duplicates
  • Willingness to say "I don't know, but here's how I'd find out"

AI is fine for lookups ("what's the default isolation level in MySQL?"). The reasoning must be theirs.

The scoring rubric#

Each interviewer scores 1–4 per dimension, with evidence, before the debrief:

Dimension 1 2 3 4
Code reading Misreads basic flow Understands flow, misses issues Finds most important issues Finds issues and explains impact and fixes
Debugging Random changes Finds cause with heavy help Systematic, finds cause Systematic, finds cause, adds a test and prevention
AI judgment Accepts output blindly Checks sometimes Verifies important suggestions Uses AI precisely, catches its mistakes, explains why
Fundamentals Gaps in basics (HTTP, SQL, data integrity) Knows basics, shaky under pressure Solid basics Deep, explains the why
Design and trade-offs No trade-offs mentioned Names options Chooses with reasons Chooses with reasons and failure handling
Communication Hard to follow Clear when prompted Clear and structured Clear, concise, checks for understanding

A hire needs 3+ on code reading, debugging and AI judgment for most backend roles. Adjust the bar by level, but write it down before the first interview.

What about take-home assignments?#

If you keep one, make it short (two to three hours, ideally paid) and assume AI will be used. The value comes from the follow-up: a live session where the candidate changes their own code ("add pagination", "this must now support refunds") and explains their decisions. Someone who understood their submission adapts it easily. Someone who pasted it in struggles in the first ten minutes.

Red flags and green flags#

Red flags Green flags
Can't explain lines in their own take-home Says "AI wrote this part, and here's how I checked it"
Applies the AI's first suggestion without reading it Rejects a wrong AI suggestion and says why
Starts debugging by changing code Starts by reproducing and reading evidence
Never asks about failure, scale or data Asks "what happens if this runs twice?"
Confident about everything Comfortable saying "I'm not sure, I'd verify by…"

Checklist for your next hiring round#

  • AI rules for each stage are written and shared with candidates in advance
  • Same questions, materials and rubric for every candidate
  • A flawed PR with an answer key (use the one above or adapt it to your stack)
  • A debugging exercise with one real bug and a failing test
  • A design question close to your product
  • Interviewers score independently with evidence before the debrief
  • Take-homes (if any) are short, assume AI, and are followed by a live change
  • The hiring bar per dimension is decided before the first interview

Key takeaways#

  • Don't test what AI does for free. Algorithm recall and spec-to-code tell you little now.
  • Test judgment: code reading, debugging, verifying AI output, and trade-offs.
  • Allow AI where your team uses it, but watch the process. How a candidate verifies matters more than what they generate.
  • Structure beats intuition. Same questions, a written rubric, independent scoring.

References#

Hiring your next backend engineer? If you'd like help designing the interview loop, preparing exercises for your stack, or running technical interviews, see how I can help.