September 29, 2026·7 min read
Backend HiringDistributed Systems AssessmentHands On AssessmentSystem Design InterviewTechnical Interviews
Automated scoring tools promise to remove subjectivity from code review. The candidate submits their work, the model evaluates it, you get a score. No bias, no late-Friday judgment calls, no disagreement between interviewers.
It sounds clean. It is also missing several things that matter.
Style consistency. Automated tools reliably flag code that does not match a given style guide.
Test coverage. They can count tests, check that they pass, and surface obvious gaps.
Basic correctness. If the output for a given input is wrong, the model will catch it.
For high-volume screening, these checks are useful. They clear out obviously broken submissions without senior review time.
Approach quality under constraint. A candidate who wrote slightly messy but production-ready code under time pressure will often score lower than one who wrote clean code that solves the wrong version of the problem. The model has no context for which tradeoff mattered.
Debugging behavior. How did they get to the solution? Did they understand the failure mode or iterate blindly? The final code artifact cannot show you this. The session recording can.
Recovery under failure. Did the candidate hit a dead end, recognize it, and change course? Or did they keep pushing a broken approach until time ran out? This is one of the strongest signals in real engineering work. A score on the submitted code has no visibility into it.
Judgment about what to skip. A strong candidate in a 45-minute session knows which parts of a task are not worth completing. A model that scores for completeness will penalize exactly that wisdom.
When candidates use Claude during the assessment, an AI-generated score on the final code tells you even less. The submission might be high quality and entirely AI-produced. Or it might be low quality despite good AI assistance, because the candidate could not direct it.
One way to get more signal: run the same candidate through two sessions, one with AI tools and one without. The difference between the two results tells you what the candidate actually contributed. No automated score surfaces that. A reviewer watching both recordings does.
AI grading works well as a first filter. It removes the obvious bottom of the pool without consuming senior time.
After that, a human who watched the session recording will catch things no score captures. The output is not the candidate. The process that produced it is.
Use AI to filter. Use humans to decide.
Run live coding sessions and take-home challenges in real production environments. Watch sessions back, score consistently, and hire with confidence.
More posts you might like
A whiteboard conversation can describe a distributed system. It cannot show you whether a candidate can operate one when a node dies mid-write. Here is how to build a distributed systems assessment around a real cluster, the failure scenarios worth injecting, and a rubric for scoring what happens next.
Teams hire platform engineers on Kubernetes depth and Terraform fluency, then wonder why nobody notices the invoice climbing. Cost awareness is an engineering skill, it is easy to test on a running environment, and almost nobody tests it.
Read more
Under the EU AI Act, AI used to screen or evaluate candidates sits in the high risk category. Here is what that actually asks of you as an employer, the questions to put to your assessment vendor, and how to structure a process you could defend.
Read more
