September 17, 2026·5 min read
Candidate EvaluationEngineering HiringEngineering Manager Technical InterviewHiring ProcessHiring Signals
A Codility score is a measurement, not a prediction. It answers one question precisely: can this person produce correct, reasonably efficient code, alone, against a clock, in a stripped down editor, on a task with a known answer.
That is a real skill. Teams that pretend otherwise end up hiring people who cannot write a loop. It is also a small part of what an engineer does, and the gap between that part and the rest is where most bad hires live.
This post is about what the rest of it looks like, how to test it, and when a task platform is still the right tool. We build one of the alternatives, so read this with that in mind: the honest recommendation for a large share of teams is to use both.
Volume. If two thousand people apply and you have four engineers, something has to cut the pile before a human is involved. A task platform does that consistently and cheaply, and consistency is itself a fairness property: everyone gets the same task under the same conditions.
Objectivity on a narrow axis. Test cases pass or they do not. Complexity is measured, not argued about. A structured, identical exercise removes a slice of interviewer bias that unstructured phone screens reintroduce constantly.
Logistics. Scheduling, invites, reminders, results in a table your recruiters can sort. This is unglamorous and it is why these platforms sell.
Detecting the floor. Someone who cannot complete a basic task in any language in ninety minutes is telling you something true.
If your problem is "too many applicants and not enough engineer hours", a task platform is the correct tool and this article is not arguing otherwise.
Everything in this list is normal Tuesday work, and none of it fits in a task runner.
Working in a codebase somebody else wrote. Tasks are greenfield by construction. Real work is reading three files to find out why a function exists before you are allowed to change it. See reading legacy code live in a backend interview.
Everything below the code. A failing pipeline, a container that will not start, a dependency that resolves differently on the build machine, a config that drifted. No task platform has an opinion about any of this because there is no system, only a function.
Debugging with incomplete information. In a task the failure is a failed assertion and the input is printed. In production the failure is a user complaint and the input is gone. How to assess debugging skill is about the second one.
Judgment and tradeoffs. Tasks have a known optimal answer, so there is nothing to trade off. Real decisions are made where two defensible options exist and the reasoning is the deliverable.
Collaboration. Whether someone asks a clarifying question at minute three rather than building the wrong thing for an hour is one of the strongest predictors we see, and a solo timed task is designed to exclude it.
AI supervision. This is the newest and biggest hole. When the model can produce a correct solution to a classic task in seconds, the score stops measuring the candidate. Vendors have responded with similarity checks and, increasingly, their own AI usage detection. Check what any specific product claims and how that claim is evaluated, because similarity checking on its own compares a submission against other candidates and known solutions, which is a different problem from a model generating a correct solution fresh. The industry answer of harder proctoring is a treadmill. We wrote about the alternative in how to design a technical interview a candidate cannot cheat.
The alternative is not "a harder task". It is a different substrate: put the candidate in a real environment and give them a real problem in it.
In EasyEnv that means a live machine, or several, with an actual service, actual data, an actual test suite, and a fault you chose. The candidate has a terminal, an editor, logs and the internet. The session is recorded, so the review is about what they did and in what order, not only about what they ended with.
That changes what you can score. Instead of a percentage you get evidence: they checked the logs before editing the config, they reproduced the bug before fixing it, they asked what the acceptable latency was before optimising, they noticed the test the model deleted. Those are the behaviours that survive contact with your codebase.
It also changes the candidate experience. Engineers who dislike timed puzzle tests usually do not dislike being handed a broken system, because it is recognisably the job. The tradeoff is honest: this takes more of your engineers' time per candidate than a score does.
Ask what your failure mode actually is.
"We cannot process the volume." Task platform at the top of the funnel. Keep the task short, tell candidates the time cost up front, and do not use the score as a ranking for the people who pass it, only as a floor.
"People pass our process and cannot do the job." Your screen is probably fine and your final stage is measuring the wrong thing. Replace one round with a real environment exercise before touching anything else.
"Good candidates drop out." Look at total unpaid hours you are asking for. A two hour timed task plus a take home plus four interviews is a funnel designed to lose senior people who have other offers. See why your best candidates drop out of the funnel.
"We cannot tell if they used AI." Stop trying to tell. Design the assessment so that using AI well is the thing being measured.
"We hire for infrastructure, platform or SRE roles." A task score is close to useless here, because the work is systems, not functions. Start with what a real DevOps interview should look like.
For most teams above a hundred applicants per role: a short task platform screen as a floor test, then one real environment session that decides the hire, then a debrief with a written rubric. The task removes the people who cannot code. The environment session tells you who to hire. Do not let the funnel end at the score, because the score cannot see the part of the job that gets people fired.
Which of your last five engineering hires would a timed task score have ranked correctly?
Run live coding sessions and take-home challenges in real production environments. Watch sessions back, score consistently, and hire with confidence.
More posts you might like
Engineering managers should not take coding tests, and most EM loops overcorrect into pure behavioural questions that everyone has rehearsed. Here is a middle path that tests the judgment calls the job actually consists of.
Most QA interviews test vocabulary, and vocabulary is not the skill. Here is how to assess testers and test automation engineers on a running application, starting with the single best question there is: a genuinely flaky test.
Read more
Almost every Python candidate can write a comprehension and a decorator. Far fewer can say why the worker grew to 6GB overnight. Here are the tasks that separate them, run on a real box with a real process.
Read more
