September 23, 2026·5 min read
The model has shell access now. It reads the repository, edits files, runs the tests and reports back. Which means the interesting question in an engineering interview is no longer whether the candidate can write the code. It is whether they can supervise something that writes it faster than they can read it.
That is a real, separable, testable skill. Most hiring loops do not test it, and a growing number of them are actively trying to prevent it from appearing.
Most AI-in-hiring conversations are still about autocomplete: does the candidate use a model to write a function, and is that cheating. That framing is already a year behind the work. Agentic tools change the shape of the task, because the unit of delegation is no longer a line, it is a change across several files that the engineer did not watch being made.
The skills that matter shift accordingly:
Scoping. Giving an agent a task small enough to verify. "Fix the failing test" is verifiable. "Refactor the service layer" is a blank cheque.
Verification. Reading a diff you did not write, in code you did not write, and deciding whether it is correct. This is a rarer skill than writing the code was.
Interruption. Noticing that it has gone the wrong way in minute two rather than minute fifteen.
Knowing what not to delegate. There are tasks where the model is excellent and tasks where checking its output costs more than doing it. Engineers who cannot tell them apart lose time on net while feeling fast.
We cover the underlying framework in AI literacy versus AI fluency and four AI skills and how to test each. This post is about the agentic layer specifically.
Give the candidate a real repository, a real failing state, a working agent, and forty minutes. Tell them explicitly that using the agent is expected and that you will be reviewing what it did, not only what they ended with.
Three tasks that work:
1. A bug in unfamiliar code. A failing test in a service they have never seen, where the actual cause is two files away from the symptom. What you watch: whether they read enough to write a useful instruction, or paste the failure and hope. Engineers who first spend three minutes orienting themselves get dramatically better output, and they can tell when it is wrong.
2. A change with a trap. A task where the fastest path makes the suite green while breaking something the tests do not cover, or where an agent will plausibly modify a test rather than the code. This is our favourite, because it is the failure we see most often in real sessions: the suite goes green, the candidate moves on, and nobody has read what changed. A candidate who reviews the diff and says "it deleted an assertion, that is not a fix" has demonstrated the whole skill in one sentence.
3. A task not worth delegating. Something small, specific, and fiddly, where handing it over and checking the result takes longer than doing it. What you watch: whether they notice. There is no penalty for using the agent here and the strong candidates often stop halfway and finish it themselves, which is exactly the judgment you want.
Do they scope before handing over? A one line instruction with no context produces a plausible, wrong change. A candidate who says which file, which constraint, and what "done" means is managing, not gambling.
Do they read the diff? Not "did they glance at it". Watch whether they open the files the agent touched that they did not expect it to touch. That is the moment the good ones catch things.
Do they run the tests themselves? Accepting the agent's claim that the tests pass is a small and very revealing shortcut.
Do they interrupt? An agent going in the wrong direction is usually obvious three steps in. Candidates who wait for it to finish and then start again have not yet developed the reflex.
Do they keep the task decomposed? Strong candidates do a sequence of small verifiable steps. Weaker ones ask for the whole feature and then debug the result, which is slower than writing it by hand and feels faster.
Can they explain the result? At the end, ask them to walk you through the change as if they were about to review it for a colleague. A candidate who cannot explain why a line is there has shipped code they do not own. This is the same test as how to review AI generated code, applied to the code they just produced.
If you run enough of these sessions, one pattern dominates. The code compiles. The tests pass. The candidate moves on. Nobody, including the candidate, has read what actually changed.
It is not laziness. It is that reviewing a fifty line diff in unfamiliar code is genuinely harder than writing twenty lines yourself, and the incentive in a timed session is to accept the green checkmark. That is exactly the incentive their first month on your team will have, which is why watching them under mild time pressure is informative rather than unfair.
Score it explicitly. A candidate who finishes two tasks with verified changes should beat one who finishes three with unread ones, and your rubric has to say so, or interviewers will quietly reward speed. See how to write a technical interview rubric.
An agent needs a repository, a test suite, dependencies that install, and a shell. A shared document cannot host any of it, and a screen share of the candidate's own machine gives you their configuration, their model, and their shell history rather than a comparable exercise.
In EasyEnv the candidate gets a real machine with the repository and tooling, and the session is recorded, including the terminal. The review is over the sequence: what they asked for, what came back, what they checked, what they missed. Two candidates can produce the same final diff by completely different routes, and the route is the hire decision.
Do not ban the tools and call it integrity. A loop that measures unassisted typing speed is measuring a task that has largely stopped existing. Proctoring harder is a treadmill, discussed in how to design a technical interview a candidate cannot cheat.
Do not reward volume. More generated code is not more signal. The engineer who deleted most of what the agent proposed is usually the better hire.
Do not test prompt phrasing. Clever wording is a fashion and it changes with every model release. Supervision does not.
How many of the diffs merged into your codebase last week were fully read by a human?
Run live coding sessions and take-home challenges in real production environments. Watch sessions back, score consistently, and hire with confidence.
More posts you might like
Observability interviews usually become tool interviews, which sort candidates by which vendor their last employer bought. Here is how to test the actual skill: narrowing down a failure that only happens to four percent of requests.
Ask a candidate to describe a good CI pipeline and everyone gives the same clean answer. Hand them a build that passed yesterday and fails today, and you find out who has actually lived with one.
Read more
The best Linux interview question is not a question, it is an SSH prompt on a machine where something is wrong. Here are six scenarios, what each one separates, and how to read the first ninety seconds.
Read more
