How to Set Up an Ashby Interview Plan in 20 Minutes

October 27, 2026

An Ashby interview plan is the structure attached to a job that defines each interview stage, who conducts it, and which scorecard attributes it covers. You build it under Admin → Interview Plans, then attach it to a job. Everything downstream — scheduling, scorecards, debrief packets — inherits from it.

The reason to care: without a plan, two interviewers on the same panel score the same candidate on “communication” and disagree, and you cannot tell whether the candidate was inconsistent or the rubric was. Ashby’s plan model pushes you to assign distinct competencies per stage, which is the actual fix.

What does an Ashby interview plan contain?

Four layers, nested:

  • Stages — the ordered steps, e.g. Recruiter Screen → Technical Screen → Onsite (3 interviews) → Offer.
  • Interviews — individual sessions inside a stage, each with an interviewer or interviewer pool.
  • Scorecard attributes — the competencies scored, from your shared attribute library.
  • Guidelines — the questions or focus areas shown to the interviewer before the session.

Ashby recommends 4–6 total interviews for most roles. Beyond that, candidate experience drops faster than signal improves, and the marginal interviewer adds noise rather than information.

How do you create the plan?

Start from Admin → Interview Plans → New Interview Plan. Name it after the role family (Engineering — Backend — Senior), not the individual job, so you can reuse it.

  1. Add your stages in order and drag to reorder.
  2. For each stage, add interviews and pick a Default Interviewer or a pool (e.g. Backend Interviewers) for auto-assignment.
  3. Click into each interview and attach 2–3 scorecard attributes. Not five. Three attributes per interview is roughly the ceiling for an interviewer to assess well in 45 minutes.
  4. Write 2–4 guideline questions per interview. These show in the packet, and they keep interviewers from improvising the same question four times.
  5. Save, then attach the plan to a job under the job’s Interview Plan tab.

Why interviewer pools beat named interviewers

Pools let Ashby auto-assign based on availability and interviewer load. Named interviewers are fine for a five-person startup where everyone is on every panel, but they break the moment someone takes leave. Pools also give you a report on who is interviewing too much, which is how you catch burnout before it becomes attrition.

How do scorecard attributes work?

Attributes live in a shared library, so Problem Solving is the same object across every job. That is what makes cross-job reporting possible — the same attribute can be compared across roles and time.

Ashby supports two scales: a numeric 1–4 and a custom scale you define. The 1–4 scale maps to strong no / no / yes / strong yes, and it is the default for a reason — forced choice beats a 1–5 scale where everyone picks 3.

One rule worth enforcing: every attribute in a plan should be scored by at least two interviewers across the loop. Single-score attributes are the ones that get argued about in debrief.

What should go in the guidelines?

Guidelines are the lever most teams ignore. Write them as behaviour probes, not topics:

  • Bad: “Ask about system design.”
  • Good: “Have them design a rate limiter. Probe how they handle a change in requirements mid-design.”

Specific probes produce comparable answers. Vague topics produce four different conversations and a debrief where nobody remembers the details.

What breaks in practice?

  • Attaching the wrong plan to a job. Ashby lets you swap plans after candidates are in the pipeline; scores already submitted stay, which leaves a mixed-rubric pipeline. Swap before sourcing starts.
  • Too many stages. Five stages for a mid-level role triples time-to-hire for marginal signal.
  • No debrief owner. The plan defines structure but not the meeting. Assign a debrief facilitator per panel or the packet gets read after the decision, not before.
  • Scorecards submitted late. Ashby nudges, but interviewers still submit 12 hours later and reconstruct from memory. Enforce same-day submission as a team norm.

Who is Ashby not right for?

Companies hiring fewer than about five people a year will find the interview plan setup, attribute library, and structured debrief machinery heavier than the problem. A shared doc and a calendar work at that volume. Ashby’s own pricing reflects this — it is built for teams that hire continuously, and the per-seat cost only makes sense once recruiting is a standing function rather than an occasional scramble.

Key Takeaways

  • An Ashby interview plan defines stages, interviews, interviewer pools, scorecard attributes, and guidelines for a role family.
  • Keep it to 4–6 interviews and 2–3 attributes per interview.
  • Attach each attribute to at least two interviewers so debrief has more than one data point.
  • Write guideline questions as concrete behavioural probes, not topics.
  • Swap plans before sourcing begins — mid-pipeline changes leave mixed rubrics.
Tags: