Tools · Beauty parade

Questions to ask a applicant tracking and recruiting vendor

48 questions, written to be read out loud in the meeting. Each one asks for the thing rather than the claim, because every vendor says yes to "do you support that" and almost none of them can show you.

Free to use, copy or put your own logo on. No sign-up, and nothing on this page is sponsored.

Missing a question you always ask? Send it in on the Ask page and it goes on the list with your name on it if you want it there.


Before you start

What the work in this category actually looks like

Reachable today Needs software built first Cannot be sped up

Human Resources Specialists, 26 rated tasks in the exposure report. 12 of them need software built first, and that is what a tool in this category has to earn its price on. The 9 in the first band are already reachable with a chat window you have.

Most of what an ATS demo shows you is scheduling and templated messages, and a chat window already writes those. Ask instead whether it does the screening: reviewing applications against job requirements, checking selection criteria, running the background checks. That is the work sitting in the middle band, and it is the reason to pay.

How this is organised

Four places a question can be answered

A salesperson cannot show you real candidates, cannot post to your job boards, and cannot open another customer's account. Asking anyway wastes the meeting and tells them you have not bought one of these before. So every question below says where it belongs.

Demo

In the demo

Their sandbox and their invented people. Everything here is something a salesperson can do live in forty minutes without touching a real record.

Break the rehearsed route

A demo is a path through invented data that the salesperson has walked a hundred times. All of this runs on that same sandbox, and none of it is on the path.

  1. Take us through posting a job, then tell us which of our job boards are built in, which need their own contract, and which nobody has ever connected.

    One-click syndication is the most oversold thing in the category. They cannot post to your boards in a demo, because that needs your credentials and your money. They can name yours one at a time and say how each works. Bring the list, including the niche board nobody has heard of.

    Worry ifBoards described as supported where support turns out to mean you paste a link.

  2. Show me the application form on a phone, and let me count the screens.

    Most applications start on a phone and every extra screen loses people you paid to attract. You can count them yourself in the room. Ask their typical completion rate too, but treat the count as the fact and the rate as a claim.

    Worry ifA form that takes a CV and then asks the candidate to type it in again.

  3. Book an interview between three of your demo users across two calendars, with one of them busy.

    Scheduling is where recruiter hours go, and their sandbox has invented users with invented calendars, so this is entirely doable. Watch what it does with the clash and how many clicks a reschedule takes, because reschedules are half the work.

  4. Show me the hiring manager view, and tell me how long their training takes.

    A recruiter will learn anything. A hiring manager gets about half an hour of attention, uses the system twice a month, and is the person most able to sink the whole thing by going back to email. If their view is the recruiter view with permissions stripped out, they will not use it.

  5. A hiring manager replies to the notification email instead of logging in. What happens?

    This will happen every week for the life of the contract. Systems that capture the reply as feedback keep working. Systems that lose it teach your managers the tool is optional.

If it scores or ranks candidates

Plenty of good systems do not rank at all, and that is a fine answer. What matters is knowing which one you are buying and how far you could defend it.

  1. Does this rank or score candidates, and what exactly is it scoring against?

    Get a straight answer on whether it matches against the advert text, a structured requirement list, or a model trained on who you hired before. Those are three different products, and only one of them plays your past hiring patterns back at you.

  2. Show me a score in your demo data with the reason next to it. If the reason is another percentage, we will read the CV anyway.

    The common failure is a 73 per cent match with nothing behind it. Recruiters do not trust it, so they read the CV, and the screen has saved nobody anything. A reason in words is what makes a score usable rather than decorative.

    Worry ifA star rating or a percentage with no way to see what drove it.

  3. A rejected candidate asks how the decision was made. What do you hand us?

    Ask it as a practical question rather than a legal one. Several large vendors in this category are defending class actions over alleged algorithmic discrimination, and whatever comes of those, being unable to answer this is a commercial risk as much as a compliance one.

The two questions that get honest answers

Ask these at the end, when the demo is over and the room has relaxed. They are worth more than the previous hour.

  1. Which of the things you showed me today are live for every customer, and which are on the roadmap?

    The most common complaint from buyers in this market is feeling misled afterwards, with promised features never arriving. Ask in the room, write the answer down, and put anything you are depending on into the contract with a date against it.

    Worry ifAnything demonstrated in an environment no customer is on yet.

  2. What are the three things your customers complain about most?

    A good salesperson answers honestly, because they know you will find out in month two. "Nothing really" means either they do not talk to their customers or they will not tell you.

    Worry ifA strength dressed up as a weakness.

Trial

In a trial, with your own test data

Ask for a sandbox and a fortnight. Use made-up employees and CVs with the names taken off, so nothing real is involved. Whether they hand one over without a sales process is worth knowing on its own.

Drive it yourself

  1. Parse these three CVs. A clean one, a two-column designed one and a badly formatted one, all with the names taken off.

    Parsing quality decides how much retyping your team does every day and it is the fastest thing to test. Anonymised CVs keep it a test rather than a data protection problem, and doing it yourself stops it being their three favourite files.

    Worry ifOnly wanting to use their sample CVs.

  2. Build one of our real hiring stages. Ours has a task, a panel and a two-week gap.

    Every organisation has a stage that does not fit the default. Find out whether you can build it or whether it is a services ticket, because that answer applies to every change you make for three years.

  3. Send a rejection to a test candidate and read what arrives.

    Silence after applying is the most consistent complaint candidates make about hiring, and it usually traces back to a system nobody configured on the candidate side. Read the email and see how easy it is to change.

Getting a sandbox at all

  1. Can we have a sandbox for two weeks with our own test data, without a sales process attached?

    Everything you learn in a demo is about their product being used the way they use it. A fortnight with your own made-up data tells you whether your team can use it. A vendor confident in the product hands one over.

    Worry ifA trial only available after a commercial conversation, or a guided tour rather than an account.

  2. Set up one thing we would need on day one, ourselves, while you watch.

    Pick something ordinary: a workflow step, an approval, a template. If it needs the vendor, then every change for the next three years needs the vendor, and nobody put that in the quote.

    Worry ifConfiguration that turns out to be a professional services ticket.

Reference

From a customer, not the salesperson

Ask for two customers your size, and ask each for a screen share rather than a phone call. Real volume, real reporting and real mess only exist here, and almost nobody asks.

Ask a customer

  1. Show me your time to hire, source of hire and stage conversion, in the system.

    The vendor genuinely cannot show you this, because real reporting needs a year of real hiring. A customer can, and this is where thin reporting shows up. Watch whether they change the date range without help, or export the lot to a spreadsheet.

    Worry ifA customer who rebuilds every report in Excel.

  2. How many of your job boards ended up working the way you expected?

    The gap between the posting demo and the posting reality is the most common disappointment in this category, and only somebody a year in knows its size.

  3. Do your hiring managers use it, honestly?

    The answer is often no, and the reason is the most useful thing you will hear all week.

Worth asking any customer, in any category

Ask the vendor for two customers your size, then ask each for a screen share. These four are for that call, with the vendor not on the line.

  1. How long did implementation actually take, against what you were told?

    The most useful number in the whole process, and the vendor will quote you their fastest. A customer will tell you what it cost in their own team's time, which never appears in any quote.

  2. What did you assume it did that it turned out not to do?

    Everybody has one. It is usually a reporting gap or an integration that needed building, and it is the thing you would otherwise find in month three.

  3. What happened the last time you raised something serious with support?

    Support quality tends to drop once a contract is signed, and a reference customer is a year past that point. Ask for a specific incident rather than an opinion.

In writing

In writing, before you sign

Not a conversation and not a slide. Ask for the document, and get anything you are relying on into the contract rather than into the product.

What is in the box, and what costs extra

This is the real negotiation, and far more useful than the feature grid. Texting, video interviewing, assessments, onboarding and a candidate CRM are commonly sold on top of a price that looked complete.

  1. Send us every module and what each one costs. Where does the core product stop?

    The core licence is often a fraction of what a working setup costs once texting, video interviews, assessments and onboarding are on. The full menu changes what you negotiate on.

    Worry ifA single price with "everything you need included" and no module list.

  2. Which features are gated to a higher tier rather than sold as add-ons? Start with the API and the integrations.

    API access and integrations on the top plan only is a common structure, and it turns a cheap plan into an expensive one the moment you connect your HRIS. Tier gating is easy to miss because it never appears as a line item.

  3. If we sign this quarter, which add-ons will you include, for how long, and what is the renewal price for them?

    Add-ons cost the vendor almost nothing to switch on, which makes them the most negotiable part of the deal. Get the free period and the renewal price in the same email, or you have bought a bill you cannot forecast.

    Worry ifA discount now with the renewal price left open.

  4. Are we charged per user, per employee or per open job? Model the bill at thirty per cent more hiring, and at a freeze.

    The three models behave completely differently. Per open job punishes exactly the busy quarter when you most need the system, and rewards nothing when hiring stops.

  5. Send the integration list for our HRIS, payroll and calendar, marking which are native, which go through a connector, and which are an API we would build against.

    Those three cost very different amounts. Manual re-entry between recruiting and onboarding is the most common reason the promised time saving never appears, so find where a person retypes something.

    Worry ifA logo wall where the answer to "how" is "we have an open API".

  6. For a team our size, how long is implementation, who does the migration, and can we run both systems side by side for two weeks?

    A rollout with historical candidate data, single sign-on and approval chains is a multi-week project whatever the deck says. Running old and new together and reconciling before switching off is the normal safe way, and a vendor who has done a few migrations expects the question.

  7. What comes across from our current system, and what gets left behind?

    Usually the candidate records come and the history does not: notes, interview feedback, email threads, rejection reasons. Ask about those four by name, because they are what you miss when a past applicant reapplies.

Candidate data

Shorter than you might expect, because most of what applies here applies to you rather than to them. These three are the ones where the vendor has to do something.

  1. How long do you keep the data of a candidate we did not hire, and can we set that ourselves?

    You will want to keep unsuccessful applicants for a period and then delete them, and the period is your decision. The ICO audit of this category found tools collecting more than they needed and keeping it indefinitely.

    ICO, AI tools in recruitment audit outcomes report, November 2024

  2. Have you completed a data protection impact assessment for this product, and will you share it?

    The ICO expects one before processing starts, ideally at procurement. A vendor who has one and will send it saves you a fortnight. One who has never done it means you are starting from nothing.

    ICO, six questions to ask when procuring AI recruitment tools, November 2024

  3. Do you have a candidate-facing privacy notice we can put on the careers site, and can we edit it?

    Somebody has to write it and it has to describe what the tool actually does. If they have a template, take it and check it against what you saw in the demo.

    ICO, six questions to ask when procuring AI recruitment tools, November 2024

What it costs, all in

Implementation, migration, integrations and support tiers are where the quoted price stops being the price. Ask for the total before the demo ends.

  1. What does year one cost in total, in writing, including implementation, migration and integration fees?

    The licence figure is rarely the bill. Implementation, data migration, API charges, premium support and mid-term true-ups are all common and all appear after signature. A vendor who will put the year-one total in an email can be held to it.

    Worry ifA licence price now and implementation "scoped later".

  2. Is implementation a fixed fee or time and materials?

    Fixed fee puts the risk of a slow implementation on the people running it. Time and materials puts it on you, and HR data is always messier than anyone expects.

    Worry ifDay rates with no cap.

  3. What is the renewal price, and is the increase capped in the contract?

    Year-one discounts often disappear at renewal, which turns a good deal into an expensive one two years in. A cap written into the contract is a normal thing to ask for.

    Worry ifUplift "in line with the market".

  4. What notice do we give to leave, and does it renew automatically?

    Thirty to sixty days is reasonable. Ninety or more with auto-renewal means one missed diary entry costs another full year. Ask for the clause and read it in the room.

    Worry ifNinety days or more, auto-renewing, multi-year.

  5. On the day we leave, what do we get, in what format, and at what cost? Is that in the contract?

    You want every record you put in, plus anything the system generated about your people, in CSV or JSON, at no charge. A feature can change in a release note. A clause cannot, and this one costs nothing to ask for at the point of buying.

    Worry ifA PDF export, a fee, or access cut off before the export arrives.

Data protection and security

  1. For each thing this system does, are you controller or processor, and is the data processing agreement attached to the contract?

    The roles have to be settled activity by activity rather than once for the whole product, and the ICO expects a contract that says so. If they are a processor you have to give explicit written instructions, so ask what those will say.

    Worry ifHesitation, or "we are the processor" as a blanket answer.

    ICO, six questions to ask when procuring AI recruitment tools, November 2024

  2. Where is the data hosted, which sub-processors touch it, and how do we hear when that list changes?

    Employee data includes salaries, health information and grievances. You need to know who holds it and to have a route to object when a new sub-processor appears.

  3. Do you train any model on our data? Is it off by default, and is that in the contract?

    Ask plainly and get it in writing. "We do not sell your data" is a different sentence and is often the one that comes back.

    Worry ifAn answer about anonymisation that never says no.

  4. Send us your most recent penetration test summary and your ISO 27001 certificate or SOC 2 report.

    A certificate number is easy to claim and easy to check. The pen test summary tells you whether anything serious was found and how fast it closed.

    Worry if"Available under NDA once we have progressed further."

  5. When an employee makes a subject access request touching your system, what do we get from you and how quickly?

    You have a statutory deadline and the vendor does not. Ask for their internal service level in days and check it leaves you time to review before you answer.

Only if you hire outside the UK

Where the rules actually differ

Skip this if you only employ in the UK. All of it is for the written stage rather than the demo, and the dates have already moved once this year, so check them before you rely on them. None of it is legal advice.

If you hire in the United States

One city and three states now regulate AI in hiring directly, and they do not agree with each other. Ask which ones the vendor supports rather than whether they are compliant, because most of the duty is yours rather than theirs.

  1. For New York City, do you provide an annual independent bias audit we can publish, with selection rates and impact ratios by sex, race and ethnicity?

    Local Law 144 requires the audit, requires you to post a summary publicly, and requires candidates to be told before the tool is used and offered an alternative. Penalties run per violation and each day counts separately. The State Comptroller called enforcement ineffective in December 2025, and the department has since committed to proactive rather than complaint-driven enforcement, so the quiet period is ending.

    Worry ifAn audit run by the vendor rather than an independent auditor, or one you are not allowed to publish.

    DLA Piper, New York State Comptroller audit of New York City AI hiring law, 2026

  2. Can you produce the candidate notice ten business days ahead, and handle somebody who opts out?

    The notice has to say which qualifications and characteristics the tool assesses, and the opt-out route has to exist in practice. A tool that cannot move an opting-out candidate into a human process leaves your team doing it by hand.

    DLA Piper, New York State Comptroller audit of New York City AI hiring law, 2026

  3. For Illinois, can you produce the notice that AI is being used, and confirm you never infer a protected characteristic from a postcode?

    Illinois amended its Human Rights Act with effect from 1 January 2026 to require applicants be told. Implementing rules were proposed in May 2026 and withdrawn in June for more work, so the duty is live while the detail is not settled.

    Seyfarth Shaw, Colorado postpones its AI law, California finalises regulations, Illinois disclosure law takes effect, 2026

  4. For California and Colorado, what are you building, and to what date?

    California finalised employment discrimination regulations covering automated decision systems from 1 October 2025, and separate automated decision-making rules under its privacy regime start on 1 January 2027. Colorado repealed and replaced its AI Act in May 2026, cutting the obligations back and moving the date from 30 June 2026 to 1 January 2027. A vendor still working to June 2026 is reading old material.

    Covington, round-up of automated decision-making technology laws for employers, August 2026

  5. Can you give us selection rates by group as data, so we can run our own four-fifths check?

    Federal discrimination law applies to the outcome whatever any state statute says, so you want the numbers regardless. Ask for something you can analyse rather than a certificate saying it was fine.

If you hire in the EU

Recruitment and selection are high risk under the AI Act. The date moved. The classification did not.

  1. What is your plan for the high-risk obligations, and what date are you working to?

    The deadline for these systems moved from 2 August 2026 to 2 December 2027 under the Digital Omnibus, approved in June 2026. The delay is not a repeal, and the transparency duties and the AI literacy requirement were not deferred with it.

    Worry if"That got cancelled", or an August 2026 date nobody has revisited.

    Morgan Lewis, changes to EU AI Act deadlines for employers and HR technology providers, 2026

  2. If a decision about a candidate is made without a person involved, what do you give us for the right to an explanation and to human review?

    A solely automated decision with legal or similarly significant effects sits under Article 22 of the GDPR, which is separate from the AI Act and has been in force since 2018. You need a mechanism, not a paragraph.


Sources

Anything above that states a legal duty or a date carries its source. Everything else is our judgement about what is worth asking, and you should treat it as that. None of it is legal advice, and the law here is moving quickly enough that you should check a date before you rely on it.

Band figures generated from the exposure report. Ratings from Eloundou, Manning, Mishkin and Rock, GPTs are GPTs, Science 384, 1306–1308, 2024, applied to O*NET 30.3 task statements. O*NET® is a trademark of USDOL/ETA, used under CC BY 4.0 and modified here.