The junior loop rested on one signal: give them a small problem, see if working code comes back. That signal is gone. Allow the agent and everyone passes; ban it and you are deliberately measuring the one condition that will never occur again in their career. Most teams picked the ban and are now screening hard for a skill they do not want. The replacement is not a harder puzzle. It is handing them code the agent wrote and asking whether it should merge.
github.com/tacoda/fulcorum-hiring-juniorsThe coding exercise measured production. Production is now free, so the exercise measures either nothing or the wrong thing, depending on which rule you picked.
AGENT ALLOWED
everybody produces working code
→ measures: prompting speed, and only at the low end
→ discriminates: barely, and mostly on typing
AGENT BANNED
some produce working code from memory
→ measures: unassisted recall under observation
→ a condition that will never recur in the job you are hiring for
The banned branch feels rigorous, which is why it won. It is worth stating what it actually does: it selects for the candidates who best simulate a working environment that no longer exists, and it selects against people who have already adapted to the one that does. If your team writes code with an agent every day, an interview that forbids one is not a higher bar. It is a different bar, pointed somewhere else.
The way out is not a cleverer puzzle. It is to notice that the thing you were using production as a proxy for was never production. You wanted to know whether this person will eventually be able to tell right from plausible. That is directly testable, in forty-five minutes, and it always was. Nobody bothered because the proxy used to be cheaper.
Old question: can you produce a correct change? New question: can you tell whether a change is correct? The second one was always the job. It just used to be inseparable from the first.
Not because candidates cheat. Because authorship is no longer attributable, and no honour system fixes an unfalsifiable claim.
A take-home now returns a well-structured, tested, documented submission from almost everyone, and you cannot tell from the artifact who understood it. Asking candidates not to use an agent puts the honest ones at a disadvantage against the rest, which is a bad design in any process and an especially bad one in a process about integrity.
There is a partial rescue that some teams use: let them use anything, then spend thirty minutes interrogating the submission. That works, and it is strictly worse than skipping the take-home and spending the same thirty minutes on material you prepared. You lose nothing and you stop asking for six hours of unpaid work whose output you have decided to ignore.
Filtering by willingness to spend a weekend, which correlated with enthusiasm and also with not having caring responsibilities, a second job, or a current employer. That was always a cost. It is now a cost with no matching signal, which makes the decision easy.
An interview is a low-bandwidth, high-noise channel. It always was. The useful move is matching each stage to something it can actually resolve.
| Stage | Can resolve | Cannot |
|---|---|---|
| Résumé | Exposure. Whether they finished something hard once. | Anything about judgment. Stop reading it for that. |
| Unsupervised take-home | Nothing attributable. | Everything. See §1. |
| Comprehension exercise | Reading, classification, hedging behaviour, whether they check. | Whether they improve with feedback. |
| Pushback probe | Whether a position survives disagreement. | Whether it survives disagreement from their manager, six months in. |
| Ambiguity probe | Whether they ask before assuming. | Whether they keep asking once it is socially expensive. |
| Every stage combined | A rough floor on comprehension today. | Rate of improvement, which is what you are actually buying. |
That last row is the honest summary of the whole page. You are hiring a junior for their trajectory, and trajectory is a derivative that no single point can show you. Everything below is about extracting the best available point estimate, and §9 is about the fact that the point estimate was never going to be enough.
One agent-written change with one planted defect. Agent fully available. The question is not "fix it". The question is "should this merge?"
If every candidate's diff is defective, word gets out within a month and you are then testing whether people have been briefed. More importantly, a candidate who finds a bug in a correct change has told you something serious: they will do that to their colleagues' work every week for years.
Same four points as reverse-review, which gives the interview a property most interviews lack: it predicts performance on the programme they will actually be put through.
| Point | For a junior hire, read as |
|---|---|
| Found | Baseline. Nice to have, and the least predictive of the four at this level. |
| Proved | Did they check, or assert? A candidate who says "let me run it" unprompted is showing you a habit, not a result. |
| Classified | Rare in a junior and a genuine signal when present. Do not require it. |
| Durable fix | Almost never present, and that is expected. If someone asks "what would stop this happening again", that is a strong hire signal on its own. |
Weight proved above found for a junior. Finding the defect is partly luck at this level; going and checking before committing to an answer is a disposition, and disposition is what survives training. A candidate who misses the defect, says "I am not sure about this line, here is how I would find out", and is then right about the method is a better outcome than one who spots it and cannot say why.
Two interviewers, independent written scores before discussion, same as the calibration drill in the mentor's playbook. Interview debriefs anchor even harder than review sessions do, because the second person usually has less material.
Ninety minutes of candidate time, total. If your loop is six hours, the extra five are measuring stamina.
That plus the exercise is the loop. Everything else people put in a junior process, the system design round, the second technical screen, the culture panel, is measuring something you cannot act on or something you already know. Cutting them costs you nothing and buys you a candidate pool that has not been ground down before day one.
Some were always weak. All five are now close to zero, and two of them are actively misleading.
All five are observable in the ninety minutes above, and none of them require the candidate to already be good at your stack.
"What would have to be true for this to be wrong?" Ask it in the exercise and again about their own past project. It is the same question the mentor asks in reverse-review, which means a strong answer here genuinely forecasts a strong first quarter rather than merely correlating with one.
A comprehension exercise has its own biases. They are manageable, and only if you name them first.
Interviewers who cannot themselves name the defect class will grade on whether the candidate reached their conclusion. Anyone running this exercise should have run the exercise, from the other chair, at least once.
You cannot see rate of improvement in ninety minutes. You can see it clearly in thirty days. Spend accordingly.
Every organisation that is unhappy with its junior hiring responds by lengthening the interview, and it does not work, because the variable that actually decides the outcome is not present in the room. How fast does this person improve when told something? No format resolves that from a single sitting, and a sixth round does not get you closer to it.
What does resolve it is a first month with a gate in it. If a new engineer is doing spoken claims from day one and a weekly rep from week three, you will know by day thirty whether feedback lands, and you will know it from evidence rather than impression. That is a far better instrument than any interview, and it is available to anyone willing to run the first month on purpose.
TYPICAL
6-stage loop, 6 hours ──▶ hire ──▶ unstructured first 90 days
high cost, low resolution problems surface at month 7
BETTER
2-stage loop, 90 minutes ──▶ hire ──▶ a first month with a gate
low cost, honest resolution problems surface at day 30, cheaply
This is not an argument for a careless interview. It is an argument that the marginal hour is worth far more after the offer than before it, and that a team with a real first ninety days can afford a shorter loop precisely because it is no longer asking the interview to do something interviews cannot do.
Replace the take-home with the forty-five minute exercise and change nothing else. It costs less of everyone's time than what it replaces, it produces a scored artifact two people can disagree about productively, and it is the only stage in most loops that is measuring the thing the job now consists of.