How to evaluate an AI voice agent vendor

To evaluate an AI voice agent vendor, get their agent on a real phone call in your buyers' language, then work through the questions below and insist that each answer is a demonstration or a document, not a promise. The questions cover how fast it replies, which languages it really speaks, what it does when the answer is not in your brochure, how it handles an opt-out, who does what for compliance, where your data and recordings live, how calls are billed, and what testing happens before it rings a real lead.

This checklist is meant to be fair. It is the list we would want a buyer to use on us, and it works on anyone. If you are new to AI voice agents, start with what an AI voice agent is and how it works.

First, get a real call

Demos are recorded, rehearsed or run in a browser on good Wi-Fi. Your buyers are on mobile networks, in traffic, with the TV on. Ask for a live call to your own phone, and do not follow a script. Every question below is easier to judge after one unscripted call than after an hour of slides.

1. How long does it take to reply?

The pause between you finishing and the agent starting, called voice latency, decides whether it sounds like a person or a machine.

• A good answer describes how they measure it: per reply, on real phone calls, from the caller's last word to the agent's first sound, with attention on the slow replies rather than the average.
• A red flag is one figure from a lab, or "it's instant".
• Test it by asking a question that needs the brochure, asking three things in one breath, and saying "haan" (yes) while it talks. It should answer all three, and it should not stop for the "haan".

We explain where the delay comes from in why latency decides whether a voice agent sounds human.

2. Which languages, exactly, and on audio?

"Supports Hindi" can mean anything from native conversation to a translated script read in an American accent.

• A good answer is a demonstration in your callers' languages, on real audio and with your own questions, plus a straight answer about how well it speaks each one. BlackWolf sets up each business's agent in its callers' languages and tries it on the business's own questions before a caller hears it.
• A red flag is a long list of languages with no demonstration in yours.
• Test it by starting in one language and switching to another mid-sentence, then asking in English for it to speak Hindi. Read the transcript afterwards.

Why this is harder than it looks is in what it takes for voice AI to sound local in Hindi and Gujarati.

3. What happens when the answer is not in the brochure?

• A good answer: facts come only from your documents, looked up when the caller asks, and when the answer is not there the agent says so and offers a callback.
• A red flag: the agent always has an answer. Somewhere, it is guessing.
• Test it by asking about a detail you know your documents do not cover.

4. What must it never say, and how is that enforced?

Every business has things an agent must not say: a pre-launch price, a registration status not yet confirmed, a promise of returns, a comment about a competitor.

• A good answer: the vendor asks you for these before building anything, writes them into the agent, and tests each one before go-live.
• A red flag: nobody asks.
• Test it by asking for the forbidden thing directly, then sideways ("roughly how much, just a ballpark?").

5. How is an opt-out handled?

A caller who says "dobara call mat karna" (don't call again) must not be called again. Ask:

• Is the opt-out caught from the caller's own words every time, or does it depend on how the conversation goes?
• When is it recorded: the moment it is said, or after the call, if someone remembers?
• Does the system tell "not interested" (a lead for next quarter) apart from "stop calling me" (a number never to ring again)?
• Does it work in every language the agent speaks, and which phrasings does it catch in each?
• What happens when a caller says "I am on DND"? That is an instruction, not small talk.

BlackWolf detects an opt-out on the caller's own turn, in set phrasings in English, Hindi and Gujarati, and puts the number on the suppression list the moment it is said, automatically. The regulator takes opt-outs seriously too: TRAI's February 2025 amendment (PDF), opens in a new tab says a sender may not ask a customer who has opted out for consent again within 90 days.

6. Who does what for compliance?

Commercial calls in India fall under TRAI's TCCCPR framework, and the stakes sit with the business making the calls. As of July 2026, TRAI states (PDF), opens in a new tab that promotional calls by entities of any sector must come from the 140 number series, and that the 1600 series is mandated for service and transaction calls from financial-sector entities regulated by RBI, SEBI, IRDAI and PFRDA to their existing customers, and from government. Under the February 2025 amendment (PDF), opens in a new tab, a sender that keeps violating the rules has all its telecom resources, including SIP trunks, disconnected across all operators for a year, and is blacklisted. The same amendment also brings in disclosure of the use of auto-diallers and robocalls.

TRAI's Third Amendment of September 2026 (PDF), opens in a new tab speaks to AI calling directly. It defines an application-to-person (A2P) call as a voice call initiated by an application, software system or automated platform without direct human dialling, including through autodialling, robocalls and pre-recorded or artificial voice technologies. Every sender must declare its use of A2P calls to its originating operator in advance, with the CLI ranges it will call from, and the operator records that declaration on DLT. A call made without that declaration is treated as unsolicited commercial communication. The amendment comes into force in stages after its publication in the Official Gazette, and the A2P declaration is among the provisions that take effect after sixty days.

So ask the vendor to write down the split. The parts that usually stay with you:

• Registration as a principal entity on a DLT platform, with your telecom operator.
• A registered telemarketer placing the calls, linked to you on DLT.
• The right number series for the kind of call.
• Declaring your A2P calling, and the numbers it will use, to your operator in advance.
• Scrubbing your list against the national NCPR register, through your DLT access.

The parts software can enforce on every call: checking its own suppression list before dialling, dialling only inside a calling window, honouring opt-outs and keeping every call on record. How the agent introduces itself is different: at BlackWolf the opening that names your company is scripted with you, not enforced by the platform, so read it and hear it on a test call.

• Ask which numbers the agent will call from, whether they stay fixed so your A2P declaration stays accurate, and how the agent introduces itself.
• A red flag is "fully TRAI-compliant" or "we check DND" with no explanation of how, or through whose DLT access.

Our compliance page sets out BlackWolf's split, and our guide to TRAI's rules for AI calling goes further. This is a summary, not legal advice.

7. Where does my data live, and who else can see it?

• Is each client's data kept apart, and is that tested or merely assumed?
• Which third parties process it, and where is it hosted? Ask for the list.
• Does the agent learn from your calls in a way that benefits other clients?
• How does the vendor's handling fit your own obligations under the DPDP Act? Our piece on the DPDP Act for sales teams covers what to ask.

BlackWolf keeps every lead, call, recording and document in one workspace, scopes every query to it, and tests isolation between workspaces on every change. More on our security page.

8. Who can hear the recordings, and for how long?

• Are recordings copied into private storage, or left on a URL that anyone with the link can open?
• Do playback links expire?
• How long are recordings kept, and can you have them deleted?

BlackWolf copies recordings off the carrier into private storage, filed by workspace and played through links that expire. For the wider question, see call recording and retention in India.

9. How is a call billed?

Ask how minutes are counted (per second, or rounded up per call), whether unanswered attempts cost anything, what setup covers, whether telephony is included and where volume tiers break. The same rate can produce very different bills. We work through the arithmetic in what AI calling costs in India.

10. What lands in the CRM, and does it ever re-ask?

• Ask to see the lead record from your own test call: which fields are filled, whether there is a summary, the transcript and the recording.
• Ask whether it can push into the CRM you already use, and whether that works today or is on a roadmap. At BlackWolf, call outcomes are written back today for leads that came from DaeBuild; other CRMs are not connected yet.
• Mention your budget in passing early in a test call, and see whether the agent asks for it again later.

How a well-built agent avoids re-asking is in how a voice agent remembers and follows a language switch.

11. What testing happens before go-live?

A vendor should place real calls on your number with your documents before any lead hears the agent. A useful list:

• It opens in the right language and the right voice.
• It answers real questions from your documents correctly.
• It refuses what is not in them.
• Every prohibition holds.
• Interrupting it works; "haan" and "ji" (yes, and a polite yes) do not stop it.
• It asks the qualifying questions as a conversation, not a form.
• The lead appears with its fields filled, and the transcript is readable.
• It says goodbye out loud and then ends the call.
• The recording arrives.
• An opt-out lands on the suppression list.

12. What happens after go-live?

Agents drift and buyers ask new things. Ask who listens to calls after launch, how you report a problem, how a fix is tested before it ships, and how the vendor would notice if the agent got worse. A vendor that measures on real calls will have specific answers. One that relies on impressions will not.

What to do next

Put this list in front of two or three vendors and score them on real, unscripted calls rather than demos. If you sell property, see how BlackWolf approaches it on the real estate page.