How to interview for a platform nobody on your team has run

Hiring for a platform nobody on the team has run puts the panel in an impossible position. You cannot judge an answer you could not give yourself, so the interview drifts toward what you can check: certifications, years of experience, whether the CV says the right words. Those are exactly the signals a candidate who has read about the platform, but never run it, can satisfy.

Test sequencing, not knowledge

Knowledge is searchable. What production experience actually changes is order: what someone checks first, what they refuse to do until they know more, and how they separate a symptom from its cause. Those are visible to a non-expert, because they show up in the questions a candidate asks rather than the answers they recite.

Give the candidate a short, realistic incident with an obvious-looking wrong move built in, and ask them to walk through the first ten minutes. You do not need to know the right answer to notice whether they ask about the environment before touching it.

Have someone who has run it write the scenario

The one thing a non-expert panel genuinely cannot do is write a good scenario. A useful one has a trap that only experience reveals — a failure that is actually expected behaviour, or a metric that is lying. If nobody internal can write it, borrow the expertise for an hour: a consultant, a vendor's engineer, a peer at another firm. One well-built scenario can be reused for every candidate. Our own screens are published on each role page as worked examples of the shape.

Listen for specifics, then verify one

Ask about a real incident the candidate owned. Experienced engineers volunteer detail you did not ask for — the version, the setting that mattered, what they would do differently. Then pick the single most important claim, such as the migration they led or the outage they ran, and confirm it with a former colleague. One specific confirmation is worth more than three generic references.

What not to rely on

• Quizzes and trivia. They measure recall, which is the one thing that does not distinguish someone who has run the platform from someone who has studied it.
• Take-home projects. They test whether a candidate will do unpaid work, and they cannot reproduce the conditions that separate seniority levels: partial information, time pressure, a live system.
• Certification as a proxy. A certification shows someone passed an exam on the platform. It says little about whether they have been on call for it.
• Confidence mistaken for competence. When a panel cannot judge the technical answer, it tends to judge the delivery instead, which rewards fluency rather than experience.