Harness Engineering / Hiring Juniors / the interview kit

Stop asking whether they can build it.

The junior loop rested on one signal: give them a small problem, see if working code comes back. That signal is gone. Allow the agent and everyone passes, but ban it and you are deliberately measuring the one condition that will never occur again in their career. So most teams picked the ban, and they now screen hard for a skill they do not want. The replacement is not a harder puzzle: it is handing them the agent's code and asking whether it should merge.

who it's for · the lead designing a junior loop, and the hiring manager who has to defend it
interview kit · github.com/tacoda/fulcorum-hiring-juniors
assumes · you have already decided to hire juniors. That argument is a separate talk
reuses · the defect taxonomy and rubric from reverse-review, as the interview instrument
costs · about 90 minutes of candidate time, and rather more of yours
Visible in an hour
Reading a change, naming a failure class, asking before assuming, holding a position. Test these.
Visible over weeks
Follow-through, coachability, whether feedback lands twice. No interview format reaches these.
Never visible
Rate of improvement, the single most predictive variable for a junior. §9 is what to do about that.
0 The break

Both branches are bad, and everyone took the worse one

The coding exercise measured production. Production is now free, so the exercise measures either nothing or the wrong thing, depending on which rule you picked.

what the classic junior screen now measurespick one
AGENT ALLOWED
  everybody produces working code
  → measures: prompting speed, and only at the low end
  → discriminates: barely, and mostly on typing

AGENT BANNED
  some produce working code from memory
  → measures: unassisted recall under observation
  → a condition that will never recur in the job you are hiring for

The banned branch feels rigorous, which is why it won. Worth stating what it does: it selects for the candidates who best simulate a working environment that no longer exists. It also selects against people who have already adapted to the one that does. If your team writes code with an agent every day, an interview that forbids one is not a higher bar, just a different bar, pointed somewhere else.

The way out is not a cleverer puzzle, but noticing that the thing you were using production as a proxy for was never production. You wanted to know whether this person will eventually be able to tell right from plausible, which is testable in forty-five minutes, and always was. Nobody bothered because the proxy used to be cheaper.

The reframe

Old question: can you produce a correct change? New question: can you tell whether a change is correct? The second one was always the job. It just used to be inseparable from the first.

1 Retire it

The unsupervised take-home cannot be saved

Not because candidates cheat, but because authorship is no longer attributable, and no honour system fixes an unfalsifiable claim.

A take-home now returns a well-structured, tested, documented submission from almost everyone, and the artifact cannot say who understood it. Asking candidates not to use an agent penalises the honest ones relative to the rest, which is bad design in any process and worse in a process about integrity.

There is a partial rescue that some teams use: let them use anything, then spend thirty minutes interrogating the submission. That works, and is strictly worse than skipping the take-home and spending the same thirty minutes on material you prepared. You lose nothing and you stop asking for six hours of unpaid work whose output you have decided to ignore.

What the take-home was measuring

Filtering by willingness to spend a weekend, which correlated with enthusiasm and also with not having caring responsibilities, a second job, or a current employer. That was always a cost. It is now a cost with no matching signal, which makes the decision easy.

2 Bandwidth

Be honest about what each stage can see

An interview is a low-bandwidth, high-noise channel. It always was. The useful move is matching each stage to something it can resolve.

resolution by stage
StageCan resolveCannot
Résumé Exposure. Whether they finished something hard once. Anything about judgment. Stop reading it for that.
Unsupervised take-home Nothing attributable. Everything. See §1.
Comprehension exercise Reading, classification, hedging behaviour, whether they check. Whether they improve with feedback.
Pushback probe Whether a position survives disagreement. Whether it survives disagreement from their manager, six months in.
Ambiguity probe Whether they ask before assuming. Whether they keep asking once it is socially expensive.
Every stage combined A rough floor on comprehension today. Rate of improvement, which is what you are buying.

That last row is the honest summary of the whole page. You are hiring a junior for their trajectory, and trajectory is a derivative that no single point can show you. Everything below is about extracting the best available point estimate, and §9 is about the fact that the point estimate was never going to be enough.

3 45 minutes

Forty-five minutes on code they did not write

One agent-written change with one planted defect. Agent fully available. The question is not "fix it". The question is "should this merge?"

agent allowed, encouraged ask should this merge defect exactly one, sometimes zero
  1. Sent ahead Tell them the format in advance, in writing. Surprise measures composure, not judgment, and you are not hiring for composure under surprise.
  2. 0:00 → 0:05 Context: what the program is for, what the ticket asked for, what the business rule is. State the business rule up front. Withholding it tests telepathy.
  3. 0:05 → 0:25 They read. Anything goes: run it, test it, ask the agent, ask you. Say out loud that you expect them to use the agent, because half will assume it is a trap.
  4. 0:25 → 0:40 The verdict, and the defense. Merge or not, and why. Then you push, once, on whatever they were most confident about.
  5. 0:40 → 0:45 Tell them the answer, and what you were watching for. Every candidate leaves knowing something. This costs five minutes and is the reason people speak well of your process.

Choosing the material

  • Self-contained, ~60 lines, common language. Offer a choice of two languages. A defect in your stack tests familiarity with your stack, which is not what you are hiring a junior for.
  • One defect, from the taxonomy. A policy inversion or a test that cannot fail are the two best for this purpose. Both are invisible to the suite, and neither requires local knowledge once you have stated the business rule.
  • About one candidate in four gets a clean diff. And you say nothing about it. Correctly approving a clean change, with reasons, is a full pass and the most informative outcome you can get.
The clean diff is not a trick

If every candidate's diff is defective, word gets out within a month and you are testing whether people were briefed. And a candidate who finds a bug in a correct change has told you something serious: they will do that to their colleagues' work every week for years.

4 Four points

Score it on the rubric you already train with

Same four points as reverse-review, which gives the interview a property most interviews lack: it predicts performance on the programme you will put them through.

what to write down, per candidate
PointFor a junior hire, read as
FoundBaseline. Nice to have, and the least predictive of the four at this level.
ProvedDid they check, or assert? A candidate who says "let me run it" unprompted is showing you a habit, not a result.
ClassifiedRare in a junior and a genuine signal when present. Do not require it.
Durable fixAlmost never present, and that is expected. If someone asks "what would stop this happening again", that is a strong hire signal on its own.

Weight proved above found for a junior. Finding the defect is partly luck at this level; going and checking before committing to an answer is a disposition, and disposition is what survives training. Take the candidate who misses the defect and says "I am not sure about this line, here is how I would find out". If the method they name is the right one, that beats spotting the defect and not saying why.

Two people, written first

Two interviewers, independent written scores before discussion, same as the calibration drill in the mentor's playbook. Interview debriefs anchor even harder than review sessions do, because the second person usually has less material.

5 90 minutes

The rest of the loop, in two more stages

Ninety minutes of candidate time, total. If your loop is six hours, the extra five are measuring stamina.

  1. 25 min · one thing One thing they built or worked on. Not "tell me about it": what would you do differently, and what did you not understand at the time? The second half of that question separates people, and someone whose project was mostly agent-written can answer it without lying.
  2. 20 min · ambiguity Hand over a two-sentence spec with a hole in it. Not a puzzle, a genuine omission: what happens to a refund on a cancelled order. Watch for whether they ask or assume. Assuming is not disqualifying; assuming silently is the signal.

That plus the exercise is the loop. Everything else people put in a junior process is measuring something you cannot act on or something you already know. That covers the system design round, the second technical screen, and the culture panel. Cutting them costs you nothing and buys you a candidate pool that has not been ground down before day one.

6 Cut these

Five things that have stopped carrying information

Some were always weak, but all five are now close to zero, and two of them mislead.

  1. Syntax recall Whiteboard-from-memory API questions. Weak in 2015, worthless now, and negatively correlated with people who have spent their time on harder things.
  2. Algorithm puzzles Measures preparation, and preparation is now nearly free and wholly automated, so you are ranking candidates on how recently they ground through a practice set.
  3. Build a small app Measures prompting. Fine as a warm-up if you enjoy it. Not evidence.
  4. Side-project portfolio Once a decent proxy for curiosity, now a proxy for having had unstructured time, since the artifact costs a weekend to produce regardless of understanding. Ask about one project in depth instead of counting them.
  5. "Explain this code" as a quiz Asking someone to narrate what a function does tests what the agent does best. Ask what would make it wrong, which is the question the model cannot answer for you.
7 Watch for

Five things worth more than they used to be

All five are observable in the ninety minutes above, and none require the candidate to already know your stack.

  1. Calibrated doubt "I don't know" followed immediately by how they would find out. The pair is the signal; either half alone is not.
  2. Observed vs inferred Do they distinguish "I ran it and saw" from "I think it does"? People who keep that boundary clean under mild pressure are rare and trainable into everything else.
  3. Asks about the rule Reaching for the business rule rather than the algorithm: the single most transferable instinct on this list, and the one the agent structurally lacks.
  4. Holds, or folds well Under one round of pushback: do they hold with a reason or concede with a reason? Both are fine. Silent capitulation is the thing to note.
  5. Notices the hole Spots the missing case in the spec without anyone pointing at it. Cheap to test, hard to fake, and it is most of what a first year of usefulness consists of.
One question that does a lot of work

"What would have to be true for this to be wrong?" Ask it in the exercise and again about their own past project. It is the same question the mentor asks in reverse-review, so a strong answer here forecasts a strong first quarter instead of just correlating with one.

8 Noise

What you will measure by accident instead

A comprehension exercise has its own biases, and they are manageable only if you name them first.

  • Familiarity with your stack. Offer two languages, keep the program self-contained, and state the business rule out loud. Anything the candidate has to already know about your domain is measuring prior access, not aptitude.
  • Thinking aloud. Fluent narration is a separate skill from close reading, and it is unevenly distributed for reasons unrelated to engineering. Let people work in silence and ask at the end. Read the written verdict, not the monologue.
  • Surprise. Send the format in advance. A candidate told what to expect shows you their judgment; one who was not shows you their startle response.
  • Your own defect. If you wrote the planted defect and find it obvious, you have lost the ability to judge its difficulty. Have someone else on the team attempt it cold first, and time them.
The one that will bite you

Interviewers who cannot themselves name the defect class will grade on whether the candidate reached their conclusion. Anyone running this exercise should have run the exercise, from the other chair, at least once.

9 The payoff

The interview is not where the accuracy comes from

You cannot see rate of improvement in ninety minutes, but you can see it in thirty days. Spend where the signal is.

Every organisation unhappy with its junior hiring responds by lengthening the interview, and it does not work: the variable that decides the outcome is not in the room. How fast does this person improve when told something? No format resolves that from a single sitting, and a sixth round does not get you closer to it.

What does resolve it is a first month with a gate in it. If a new engineer is doing spoken claims from day one and a weekly rep from week three, day thirty tells you whether feedback lands. And it tells you from evidence rather than impression. That is a far better instrument than any interview, and it is available to anyone willing to run the first month on purpose.

where to put the effortthe conclusion
flowchart LR
%% declared bottom-up: the layout engine stacks the later subgraph first,
%% so BETTER is written first and renders under TYPICAL
  subgraph better["BETTER"]
    direction LR
    b1["2-stage loop, 90 minutes · low cost, honest resolution"] -- hire --> b2["a first month with a gate"]
    b2 ~~~ bwhen["problems surface at day 30, cheaply"]
  end
  subgraph typical["TYPICAL"]
    direction LR
    t1["6-stage loop, 6 hours · high cost, low resolution"] -- hire --> t2["unstructured first 90 days"]
    t2 ~~~ twhen["problems surface at month 7"]
  end
  class t2 warn
  class twhen block
  class b2,bwhen pass

None of this argues for a careless interview. It argues that the marginal hour is worth far more after the offer than before it. A team with a real first ninety days can afford a shorter loop, because it is no longer asking the interview to do something interviews cannot do.

Where to start

Replace the take-home with the forty-five minute exercise and change nothing else. It costs less of everyone's time than what it replaces, and it produces a scored artifact two people can argue over and get somewhere. In most loops it is also the only stage measuring the thing the job now consists of.