Harness Engineering / Hiring Juniors / the interview kit

Stop asking whether they can build it.

The junior loop rested on one signal: give them a small problem, see if working code comes back. That signal is gone. Allow the agent and everyone passes; ban it and you are deliberately measuring the one condition that will never occur again in their career. Most teams picked the ban and are now screening hard for a skill they do not want. The replacement is not a harder puzzle. It is handing them code the agent wrote and asking whether it should merge.

who it's for · the lead designing a junior loop, and the hiring manager who has to defend it
interview kit · github.com/tacoda/fulcorum-hiring-juniors
assumes · you have already decided to hire juniors. That argument is a separate talk
reuses · the defect taxonomy and rubric from reverse-review, as the interview instrument
costs · about 90 minutes of candidate time, and rather more of yours
Visible in an hour
Reading a change, naming a failure class, asking before assuming, holding a position. Test these.
Visible over weeks
Follow-through, coachability, whether feedback lands twice. No interview format reaches these.
Never visible
Rate of improvement, the single most predictive variable for a junior. §9 is what to do about that.
0 The break

Both branches are bad, and everyone took the worse one

The coding exercise measured production. Production is now free, so the exercise measures either nothing or the wrong thing, depending on which rule you picked.

what the classic junior screen now measurespick one
AGENT ALLOWED
  everybody produces working code
  → measures: prompting speed, and only at the low end
  → discriminates: barely, and mostly on typing

AGENT BANNED
  some produce working code from memory
  → measures: unassisted recall under observation
  → a condition that will never recur in the job you are hiring for

The banned branch feels rigorous, which is why it won. It is worth stating what it actually does: it selects for the candidates who best simulate a working environment that no longer exists, and it selects against people who have already adapted to the one that does. If your team writes code with an agent every day, an interview that forbids one is not a higher bar. It is a different bar, pointed somewhere else.

The way out is not a cleverer puzzle. It is to notice that the thing you were using production as a proxy for was never production. You wanted to know whether this person will eventually be able to tell right from plausible. That is directly testable, in forty-five minutes, and it always was. Nobody bothered because the proxy used to be cheaper.

The reframe

Old question: can you produce a correct change? New question: can you tell whether a change is correct? The second one was always the job. It just used to be inseparable from the first.

1 Retire it

The unsupervised take-home cannot be saved

Not because candidates cheat. Because authorship is no longer attributable, and no honour system fixes an unfalsifiable claim.

A take-home now returns a well-structured, tested, documented submission from almost everyone, and you cannot tell from the artifact who understood it. Asking candidates not to use an agent puts the honest ones at a disadvantage against the rest, which is a bad design in any process and an especially bad one in a process about integrity.

There is a partial rescue that some teams use: let them use anything, then spend thirty minutes interrogating the submission. That works, and it is strictly worse than skipping the take-home and spending the same thirty minutes on material you prepared. You lose nothing and you stop asking for six hours of unpaid work whose output you have decided to ignore.

What the take-home was really doing

Filtering by willingness to spend a weekend, which correlated with enthusiasm and also with not having caring responsibilities, a second job, or a current employer. That was always a cost. It is now a cost with no matching signal, which makes the decision easy.

2 Bandwidth

Be honest about what each stage can see

An interview is a low-bandwidth, high-noise channel. It always was. The useful move is matching each stage to something it can actually resolve.

resolution by stage
StageCan resolveCannot
Résumé Exposure. Whether they finished something hard once. Anything about judgment. Stop reading it for that.
Unsupervised take-home Nothing attributable. Everything. See §1.
Comprehension exercise Reading, classification, hedging behaviour, whether they check. Whether they improve with feedback.
Pushback probe Whether a position survives disagreement. Whether it survives disagreement from their manager, six months in.
Ambiguity probe Whether they ask before assuming. Whether they keep asking once it is socially expensive.
Every stage combined A rough floor on comprehension today. Rate of improvement, which is what you are actually buying.

That last row is the honest summary of the whole page. You are hiring a junior for their trajectory, and trajectory is a derivative that no single point can show you. Everything below is about extracting the best available point estimate, and §9 is about the fact that the point estimate was never going to be enough.

3 45 minutes

Forty-five minutes on code they did not write

One agent-written change with one planted defect. Agent fully available. The question is not "fix it". The question is "should this merge?"

agent allowed, encouraged ask should this merge defect exactly one, sometimes zero
  1. Sent ahead Tell them the format in advance, in writing. Surprise measures composure, not judgment, and you are not hiring for composure under surprise.
  2. 0:00 → 0:05 Context: what the program is for, what the ticket asked for, what the business rule is. Give the business rule explicitly. Withholding it tests telepathy.
  3. 0:05 → 0:25 They read. Anything goes: run it, test it, ask the agent, ask you. Say out loud that using the agent is expected, because half of them will assume it is a trap.
  4. 0:25 → 0:40 The verdict, and the defense. Merge or not, and why. Then you push, once, on whatever they were most confident about.
  5. 0:40 → 0:45 Tell them the answer, and what you were watching for. Every candidate leaves knowing something. This costs five minutes and is the reason people speak well of your process.

Choosing the material

  • Self-contained, ~60 lines, common language. Offer a choice of two languages. A defect in your stack tests familiarity with your stack, which is not what a junior is being hired for.
  • One defect, from the taxonomy. A policy inversion or a test that cannot fail are the two best for this purpose: both are invisible to the suite and neither requires local knowledge once you have stated the business rule.
  • Roughly one candidate in four gets a clean diff. And you say nothing about it. Correctly approving a clean change, with reasons, is a full pass and is the single most informative outcome you can get.
The clean diff is not a trick

If every candidate's diff is defective, word gets out within a month and you are then testing whether people have been briefed. More importantly, a candidate who finds a bug in a correct change has told you something serious: they will do that to their colleagues' work every week for years.

4 Four points

Score it on the rubric you already train with

Same four points as reverse-review, which gives the interview a property most interviews lack: it predicts performance on the programme they will actually be put through.

what to write down, per candidate
PointFor a junior hire, read as
FoundBaseline. Nice to have, and the least predictive of the four at this level.
ProvedDid they check, or assert? A candidate who says "let me run it" unprompted is showing you a habit, not a result.
ClassifiedRare in a junior and a genuine signal when present. Do not require it.
Durable fixAlmost never present, and that is expected. If someone asks "what would stop this happening again", that is a strong hire signal on its own.

Weight proved above found for a junior. Finding the defect is partly luck at this level; going and checking before committing to an answer is a disposition, and disposition is what survives training. A candidate who misses the defect, says "I am not sure about this line, here is how I would find out", and is then right about the method is a better outcome than one who spots it and cannot say why.

Two people, written first

Two interviewers, independent written scores before discussion, same as the calibration drill in the mentor's playbook. Interview debriefs anchor even harder than review sessions do, because the second person usually has less material.

5 90 minutes

The rest of the loop, in two more stages

Ninety minutes of candidate time, total. If your loop is six hours, the extra five are measuring stamina.

  1. 25 min · one thing One thing they built or worked on. Not "tell me about it": what would you do differently, and what did you not understand at the time? The second half of that question is the one that separates people, and it is answerable honestly by someone whose project was mostly agent-written.
  2. 20 min · ambiguity Hand over a two-sentence spec with a hole in it. Not a puzzle, a genuine omission: what happens to a refund on a cancelled order. Watch for whether they ask or assume. Assuming is not disqualifying; assuming silently is the signal.

That plus the exercise is the loop. Everything else people put in a junior process, the system design round, the second technical screen, the culture panel, is measuring something you cannot act on or something you already know. Cutting them costs you nothing and buys you a candidate pool that has not been ground down before day one.

6 Cut these

Five things that have stopped carrying information

Some were always weak. All five are now close to zero, and two of them are actively misleading.

  1. Syntax recall Whiteboard-from-memory API questions. Weak in 2015, worthless now, and negatively correlated with people who have spent their time on harder things.
  2. Algorithm puzzles Measures preparation, and preparation is now nearly free and thoroughly automated. You are ranking candidates on how recently they ground through a practice set.
  3. Build a small app Measures prompting. Fine as a warm-up if you enjoy it. Not evidence.
  4. Side-project portfolio Once a decent proxy for curiosity. Now a proxy for having had unstructured time, since the artifact costs a weekend to produce regardless of understanding. Ask about one project in depth instead of counting them.
  5. "Explain this code" as a quiz Asking someone to narrate what a function does tests the same thing the agent does best. Ask what would make it wrong, which is the question the model cannot answer for you.
7 Watch for

Five things worth more than they used to be

All five are observable in the ninety minutes above, and none of them require the candidate to already be good at your stack.

  1. Calibrated doubt "I don't know" followed immediately by how they would find out. The pair is the signal; either half alone is not.
  2. Observed vs inferred Do they distinguish "I ran it and saw" from "I think it does"? People who keep that boundary clean under mild pressure are rare and trainable into everything else.
  3. Asks about the rule Reaching for the business rule rather than the algorithm. The single most transferable instinct on this list, and the one the agent structurally lacks.
  4. Holds, or folds well Under one round of pushback: do they either hold with a reason or concede with a reason? Both are fine. Silent capitulation is the thing to note.
  5. Notices the hole Spots the missing case in the spec without being led there. Cheap to test, hard to fake, and it is most of what a first year of usefulness consists of.
One question that does a lot of work

"What would have to be true for this to be wrong?" Ask it in the exercise and again about their own past project. It is the same question the mentor asks in reverse-review, which means a strong answer here genuinely forecasts a strong first quarter rather than merely correlating with one.

8 Noise

What you will accidentally measure instead

A comprehension exercise has its own biases. They are manageable, and only if you name them first.

  • Familiarity with your stack. Offer two languages, keep the program self-contained, and state the business rule out loud. Anything the candidate has to already know about your domain is measuring prior access, not aptitude.
  • Thinking aloud. Fluent narration is a separate skill from reading carefully, and it is unevenly distributed for reasons that have nothing to do with engineering. Let people work in silence and ask at the end. Read the written verdict, not the monologue.
  • Surprise. Send the format in advance. A candidate who has been told what to expect shows you their judgment; one who has not shows you their startle response.
  • Your own defect. If you wrote the planted defect and find it obvious, you have lost the ability to judge its difficulty. Have someone else on the team attempt it cold first, and time them.
The one that will bite you

Interviewers who cannot themselves name the defect class will grade on whether the candidate reached their conclusion. Anyone running this exercise should have run the exercise, from the other chair, at least once.

9 The payoff

The interview is not where the accuracy comes from

You cannot see rate of improvement in ninety minutes. You can see it clearly in thirty days. Spend accordingly.

Every organisation that is unhappy with its junior hiring responds by lengthening the interview, and it does not work, because the variable that actually decides the outcome is not present in the room. How fast does this person improve when told something? No format resolves that from a single sitting, and a sixth round does not get you closer to it.

What does resolve it is a first month with a gate in it. If a new engineer is doing spoken claims from day one and a weekly rep from week three, you will know by day thirty whether feedback lands, and you will know it from evidence rather than impression. That is a far better instrument than any interview, and it is available to anyone willing to run the first month on purpose.

where to put the effortthe conclusion
TYPICAL
  6-stage loop, 6 hours       ──▶  hire  ──▶  unstructured first 90 days
  high cost, low resolution          problems surface at month 7

BETTER
  2-stage loop, 90 minutes    ──▶  hire  ──▶  a first month with a gate
  low cost, honest resolution       problems surface at day 30, cheaply

This is not an argument for a careless interview. It is an argument that the marginal hour is worth far more after the offer than before it, and that a team with a real first ninety days can afford a shorter loop precisely because it is no longer asking the interview to do something interviews cannot do.

Where to start

Replace the take-home with the forty-five minute exercise and change nothing else. It costs less of everyone's time than what it replaces, it produces a scored artifact two people can disagree about productively, and it is the only stage in most loops that is measuring the thing the job now consists of.