The short answer
Ask who performs the technical screen and whether you can see it. A firm that screens with recruiters cannot assess senior platform work, and the failure only surfaces after the engineer is embedded. Every other criterion — rate, bench size, response time — is secondary to that one.
The standard vendor evaluation measures the wrong things well. Response time, bench size, a logo wall and a rate card are all easy to compare, and none of them predicts whether the engineer who arrives can do the work. They measure a vendor's sales function, which is not the function you are buying.
There is one question that does most of the work, and eight that clean up after it.
The question that matters
Who performs your technical screen, what do they ask, and can I see it?
A generalist recruiter screens on keywords because they cannot screen on substance. That is not a criticism of recruiters — it is a description of the job. Nobody can assess whether a candidate genuinely understands quorum queue failover unless they have run quorum queues themselves.
So the answer sorts vendors into two groups immediately. Either a practitioner who runs that platform performs the screen, or somebody is pattern-matching a CV against a job description. The second group can still fill a role. It cannot tell you whether the role was filled well — and neither can you, until the engineer is three weeks into your estate.
Eight more, in priority order
1. Where is your bench genuinely deep, and where is it thin?
A vendor who claims depth everywhere has depth nowhere, or is willing to send you approximations. The answer you want names specific gaps. A firm that volunteers what it cannot staff is telling you its other claims are load-bearing.
2. What happens in month two if it is not working?
Ask for the replacement window, and ask who raises the problem. If the answer is that you raise it, the vendor has no ongoing relationship with the placement and no structural reason to notice.
3. Is the rate all-in?
Recruitment, replacement and account management either sit inside the rate or arrive as line items later. Settle it in writing before the SOW.
4. Can the engineer who screened my candidate join a call?
This is the cheapest verification available and it is remarkably clarifying. A vendor whose screener is real will produce them.
5. What is your definition of senior for this platform?
"Senior" is uncontrolled. A vendor who cannot say what a senior engineer must be able to do on your platform cannot price the role, and neither can you.
6. Do residency and clearance constraints filter the pool up front?
Data residency, US-person requirements and clearance are staffing filters, not preferences. Discovering one at contract stage costs you the whole search.
7. What are your published placement statistics?
Expect a number, then ask how it is measured — retention at what horizon, counted how, across how many placements. Unmeasured statistics are the most common unverifiable claim in this industry. We do not publish ours yet, for exactly that reason, and say so on the vetting page.
8. Who owns the relationship after the placement starts?
If the account manager disappears at contract signature, the vendor has optimised for closing rather than for delivery.
A scoring sheet you can actually use
| Signal | Weak answer | Strong answer |
|---|---|---|
| Who screens | A recruiter, or "our technical team" | A named practitioner who runs the platform |
| Can you see the screen | It is proprietary | Published before you engage |
| Bench gaps | We cover everything | Specific named gaps |
| Replacement | Raise it with your account manager | Defined window, vendor raises it |
| Rate | Day rate, details later | All-in, stated in writing |
| Statistics | A round number with no method | A measured number, or an honest none yet |
If a vendor scores strongly on the first two rows, the rest usually follows. If they score weakly there, the rest is decoration.
Related questions
What should I ask an IT staff augmentation company first?
Who performs the technical screen, what they ask, and whether you can see it. A firm that screens with recruiters cannot assess senior platform work, and that failure only surfaces once the engineer is embedded in your estate.
How do I compare staff augmentation vendors fairly?
Score them on who screens and whether the screen is visible before you engage, then on named bench gaps, replacement terms and whether the rate is all-in. Response time and bench size measure a vendor's sales function, not its engineering judgement.