We've hired fifty-odd engineers since 2020, and we've changed the interview loop three times. This is the shape it's taken this year, and — more usefully — the reasons we changed each part.

The most controversial change: we dropped the take-home test at the end of last year. This piece is mostly about why.

What the loop looks like now

Four conversations, spread over about two weeks. No unpaid work, no whiteboard puzzles about balanced binary trees, no "how many piano tuners are there in Lucknow."

  1. A 30-minute call with Kavya from ops. Not a screener in the tech-lead sense. This is where we describe how we actually work, what the current projects look like, and what the money and the pension look like. Candidates get to ask everything they'd normally save for later. If either side isn't interested after this, we save both parties a week.
  2. A 60-minute paired review. The candidate brings a piece of code they've written that they're willing to talk about. It can be from work, from a side project, from open source — anything they can share. We spend an hour reading it together, asking about the tradeoffs, and — this is the important part — we contribute to the conversation, not just extract from it. If they wrote something we don't understand, we ask. If we would have done it differently, we say so.
  3. A 90-minute pair-programming session on a real thing. This one used to be the take-home. Now it's live, with one of us on the call, working on a genuine problem from a codebase we've prepared. We're not testing whether they know the API. We're seeing how they read code they've never seen, how they ask for help, and how they respond when something surprises them.
  4. A 45-minute conversation with Rohan or one of the practice heads. Bigger-picture: how they think about product, how they've handled disagreements on teams, what kind of work they want more of and less of. This one is genuinely two-way — the practice head is also being interviewed, and we tell candidates that outright.

An offer, or a "no, and here's specifically why" note, within three business days of the last conversation.

Why the take-home went away

We used to send a three-to-four-hour take-home after the first interview. It was well-scoped: a specific small feature to add to a small starter repo, with tests and a README. We thought it was fair — everyone got the same problem, we could evaluate the code on its merits, and it was less biased than a whiteboard.

Three things eventually convinced us to stop.

First: the honest ones were penalised. A candidate who told us they spent four and a half hours on it — because they carefully read the docs, wrote real tests, and didn't submit until they were proud of it — was implicitly disadvantaged against a candidate who submitted a scrappier version in "two hours" and glossed over how much time they'd actually spent. We couldn't tell the difference from the code alone.

Second: LLMs made the signal noisier. This one is obvious now but it snuck up on us over 2024–2025. Candidates who used an assistant well weren't being dishonest — they were doing what they'd do on the job. But we couldn't tell from the artifact alone whether the person understood their own submission. That defeats the purpose.

The question stopped being "can you write this code" and started being "do you understand this code." A submitted artifact can't answer the second question. A conversation can.

Third: it was rude. Asking someone to spend an evening of their unpaid time on a bespoke problem, when we hadn't yet spent an hour of ours talking to them, felt increasingly one-sided. Especially for senior candidates with families and commitments outside of work. The take-home was, structurally, filtering for candidates with the most free time — not the ones we most wanted to hire.

The 90-minute paired session that replaced it is not a perfect signal either. But it's a fair one — we're in the room, we can help when someone gets stuck, and we can see how they think in real time, which is what we actually needed to know all along.

What we still get wrong

Interview loops are hard to evaluate. We can measure whether people we hired worked out. We can't easily measure whether the people we didn't hire would have. Every hiring team is grading their own homework.

A few things we know we haven't figured out:

  • We over-index slightly on candidates whose written communication is very good. This is probably correlated with what we do, but not perfectly.
  • We're not great at hiring for roles we don't currently have someone senior in. When we hired our first designer, we didn't have a designer to interview them. We eventually flew a friend in for the loop. There's probably a better answer.
  • Our loop is fine for mid-to-senior engineers. It's less clearly fair for people early in their careers, who may not have a piece of code they're comfortable walking through. We're piloting a different second step for those candidates this quarter.

What we tell candidates

The loop above, verbatim, on the first call. Including the parts we're still working on. If someone is going to spend a couple of weeks talking to us, they deserve to know how we make the decision and where we know we're weakest. Nobody has ever pulled out because of the honesty. A few have because of it in a good way — they'd been through processes elsewhere that pretended to be more rigorous than they were, and they were tired of it.

We hire slowly. We say no more than we say yes. But when we say yes, we mean it — and the person on the other side of the yes is walking into a team that has actually met them, not a team that has read their commit log.

Filed under: hiring, studio, practices · ← Back to journal