How to evaluate AI recruitment software (a buyer’s guide written by a vendor)

We sell AI recruitment software. So treat the next 1,800 words accordingly — except for one thing: we will tell you exactly which questions to ask us, and which questions we'd lose to other tools in this category. If a buyer's guide written by a vendor concedes nothing, it is an ad. This one concedes.

"AI recruitment software" now covers everything from resume-keyword filters that re-branded in 2023 to voice agents that run live interviews to enterprise talent-intelligence suites. They are not the same product, do not solve the same problem, and should not be evaluated against each other on a feature checklist. Before the seven questions, the category split.

The category, honestly

There are roughly four things people mean when they search "AI recruitment software" today:

• Sourcing & matching — tools that surface or rank candidates before you talk to them. Eightfold AI's "Talent Intelligence Platform" is the canonical example; Phenom is a sibling on the CRM/CX side. Both are enterprise-tier suites, sold to large TA orgs. (Eightfold product page, accessed 2026-06-14; Phenom product page, accessed 2026-06-14.)
• Conversational / chat assistants — bot-led scheduling, FAQs, and structured chat screens. Paradox's Olivia is the category leader, McDonald's-and-Marriott scale. (Paradox product page, accessed 2026-06-14.)
• Asynchronous video / one-way interviews — candidates record answers to fixed prompts; recruiters review later. HireVue is the long-standing name here. (HireVue product overview, accessed 2026-06-14.)
• Live AI interviewing — software that runs a real-time conversation with the candidate, scores it against the recruiter's rubric, and returns a transcript and report. This is where Vettika sits, and where Metaview's interview-intelligence layer increasingly overlaps. (Metaview product page, accessed 2026-06-14.)

If you do not yet know which of these four you are buying, the seven questions below are wasted effort. The right move is to write down which step of your funnel is on fire — sourcing, scheduling, screening, or post-interview synthesis — and then shortlist within that subcategory. We have a comparison hub that maps the live-interview category in detail; for the other three, the right buyer's guides are not written by us.

What follows applies to live AI interviewing and asynchronous screening tools — the tier we know best and the one most exposed to candidate-experience and compliance risk.

The 7 questions to ask every vendor (including us)

1. Can I see a full sample artifact before I sign up?

The artifact is the product. Demos are theater; the report a recruiter actually receives at 9am is the thing.

Ask each vendor to send the complete output of one interview: the score, the criteria, the per-criterion evidence, the full transcript. Not a screenshot. Not a redacted slide. The actual file.

Our answer: our sample report is public, no signup, no demo call. It is demonstration data (labeled as such on the page), but the structure, scoring breakdown, evidence quotes, and transcript format are byte-for-byte what every live interview produces.

If a vendor cannot or will not show you a sample, you are buying a black box and a promise.

2. Are scores tied to my rubric, or to a generic one?

Scoring "communication skills" out of 10 against an opaque template is a vendor's opinion of your candidates wearing the disguise of math. The question is whether you define:

• The criteria for the role.
• The weights between them.
• The thresholds that map a score to a recommendation (advance / review / decline).

Our answer: yes, all three are configurable per campaign and applied consistently across every candidate. We described what that looks like in last week's walkthrough: What a 12-minute AI first-round actually produces.

If the answer is "we have a proprietary scoring model," you are renting someone else's hiring bar.

3. Does every score come with evidence quoted from the transcript?

A score without a receipt is unauditable. You cannot overrule what you cannot inspect, and a tool that cannot be overruled cannot be your tool — it is its own.

Our answer: every rubric criterion in our report includes an "Evidence (from transcript):" line with the actual quote, including the moments the candidate corrected themselves on a follow-up. The receipts are how the recruiter stays the decision-maker.

This question is the single fastest filter in the category. Run it on every shortlisted vendor.

4. What does the candidate get?

A first round is the moment most candidates form their opinion of your employer brand — and most tooling treats them as raw material. The candidate-experience picture in this category is not flattering:

• A 2024 industry pulse from Yello, surveying job seekers, found that 38% of candidates would withdraw from a hiring process that used an AI interview. (Yello, "Job Seeker Perspectives on AI in the Recruitment Process," 2024, accessed 2026-06-14.)
• HireVue carries a 2.3/5 ("Poor") Trustpilot rating as of 2026-06-11, on a small review base (6 reviews) — small sample, but consistent in direction with the broader candidate-experience picture above and aligned with our /vs/hirevue page. (Trustpilot, hirevue.com page.)
• Paradox's parent McHire experienced a publicly reported credential exposure affecting ~64 million applicant records in mid-2025. (Wired, "A Default Password and Default Account Exposed 64 Million McDonald's Job Applications," 2025-07-09, accessed 2026-06-14.)

So ask each vendor, in writing:

• Does the candidate receive their own transcript and score? (Ours: yes — design commitment, visible at the foot of every report.)
• Is the interaction disclosed as AI before consent is collected? (Ours: yes — PRD §"Consent & Compliance Layer," verified on production candidate flow.)
• What recourse does a candidate have to a human? (Ours: the recommendation is "advance / review / decline" for the first round only — the hiring decision is the recruiter's, and that boundary is in the rubric language itself.)

We have not yet published our own measured candidate-experience numbers. We will when our production instrumentation is verified and demo data is excluded; until then we cite third-party research and design commitments and label them as such. A vendor citing self-reported candidate-NPS without disclosing methodology, sample size, and date range is not citing a number — they are quoting a marketing department.

5. What does it explicitly not do?

The honest version of "what does it do" is much shorter than the slide deck.

Our answer: no hire/no-hire verdict (advance/review/decline only, your thresholds); no personality inference; no facial or voice-tone analysis anywhere in the scoring path; no auto-ranking of candidates across roles. We score what the candidate said, quoted back to you.

If a vendor's answer to "what does it not do" is "great question, let me think" — you have your answer.

6. How do you handle the parts of recruiting that are regulated?

This is not a yes/no compliance question. The honest version is: what assumptions about regulation does this product make, and where does the liability live when something goes wrong?

Some hard, primary-sourced reference points to bring into the conversation:

• NYC Local Law 144 (in effect since 2023) requires bias audits and candidate notice for "automated employment decision tools." (NYC DCWP final rule, accessed 2026-06-14.)
• Illinois HB 3773 / AIVIA amendments add AI-specific employment protections effective 2026-01-01. (Illinois Public Act 103-0804, accessed 2026-06-14.)
• EU AI Act classifies most recruitment AI as "high-risk," with the bulk of obligations applying from 2026-08-02. (European Commission AI Act overview, accessed 2026-06-14.)
• Mobley v. Workday is an ongoing class action testing whether a SaaS hiring tool can be a "joint employer" for purposes of discrimination claims. (Mobley v. Workday, N.D. Cal. 23-cv-00770; Reuters coverage 2024-07-12, accessed 2026-06-14.)
• HireVue / BIPA — facial-analysis litigation history is part of the public record and a primary reason HireVue retired its facial-analysis component in 2021. (HireVue press release, accessed 2026-06-14.)

Ask each vendor:

• Where in the product is the AI disclosure to the candidate, before consent?
• What data does the candidate consent to, and what is the retention default?
• Can you export, per interview, a full audit log (questions, transcript, scores, who reviewed)?
• Does any part of the scoring path use facial, voice-tone, or other biometric signals?

Our answer: AI is disclosed on the candidate-facing interview-invite page before consent (PRD §"Consent & Compliance Layer"; verified on the live candidate flow 2026-06-14). Scoring is content-only — no facial or voice-tone analysis is used in the scoring path (design commitment; verified against current production scoring code; carried as the standing claim on /vs/hirevue and /vs/paradox). We do not claim to be "LL144 certified" or "EU AI Act compliant" — those phrases are not certifications you can earn — but we have published which obligations apply where so you can make your own call.

7. What's the smallest commitment I can make to try it on a real role?

If a vendor's smallest experiment is a 12-month enterprise contract, the experiment is not the product — it is the procurement. Real evaluation requires running it on a real candidate, not a sandbox.

Our answer: the first 3 interviews are free, no credit card, no subscription. Run them on a live role, compare the report against your manual phone-screen notes from the last hire you made, keep whichever is better. (Pricing page, verified live 2026-06-14.)

If you are evaluating an enterprise platform that does not offer a real-data trial, the alternative is to ask for three full sample reports from three of their live customers, with the customers' permission. This rarely happens. The reason it rarely happens is the answer to the question.

When you should buy someone else

We maintain a /vs hub where every page includes a section called "When [competitor] is the right call." Some of those answers, condensed:

• Buy HireVue if you're a Fortune-500 TA team hiring at massive scale and need game-based assessments, coding tests, FedRAMP authorization for public-sector work, or a Workday-certified integration with a 140-customer track record. (/vs/hirevue, verified live 2026-06-14.)
• Buy Paradox if you're a direct employer hiring hourly workers at hundreds of locations and your pain is application drop-off and scheduling chaos rather than interview quality — especially inside a Workday shop, where Olivia's integration gravity is real. (/vs/paradox, verified live 2026-06-14.)
• Buy Eightfold if you're a large enterprise consolidating talent acquisition and management on one AI platform — especially with Workday, SAP, or Oracle integrations in play (Eightfold's AI Interviewer is embedded in Oracle Fusion Cloud Recruiting as of May 2026), or with FedRAMP requirements. (/vs/eightfold, verified live 2026-06-14.)
• Buy Metaview if you want sourcing, application review, and a note-taking bot on human calls — i.e., AI sits in on interviews humans conduct, rather than running them itself. (/vs/metaview, verified live 2026-06-14.)
• Buy Phenom if you're an enterprise TA organization that needs the whole experience layer — career site, CRM, scheduling, assessments — under one vendor with deep ATS integrations, and its high-volume hiring tooling is proven at scale. (/vs/phenom, verified live 2026-06-14.)

If your situation is none of the above — small-to-mid recruiting team, real screening rubrics that change per role, you want a transcript and a scored report you can audit, and you want to start on one role this week — we built it for you.

What "trust over reach" means in this category

A buyer's guide written by a vendor is suspect by default. The way we are trying to earn the read is:

• Every external claim above is primary-sourced and date-stamped.
• Every competitor recommendation is the actual case we recognize as theirs, not a strawman.
• Every claim about our product is verifiable on the live site or in a public artifact today.
• No metric we cannot attest to.
• When something on this page becomes wrong — a competitor changes their product, a regulation moves — we update it and timestamp the change.

If you read one of the linked comparison pages and find we soft-pedaled a competitor's actual strength, tell us. We ship corrections weekly. That is the only durable answer in a category this noisy.

Try the artifact, not the pitch

1. Read the sample scored report — two minutes, no signup.
2. Read last week's walkthrough of what a 12-minute AI first-round actually produces.
3. If it's the right shape, the first 3 interviews are free. Run one on a real role.