AI vendor risk assessment: how to vet AI tools before you buy

Somebody on your team is buying AI tools right now. A meeting recorder here, a drafting assistant there, most adopted after one demo, none checked for where the data goes. An AI vendor risk assessment catches the problem before it costs you: a short, structured vetting of every AI tool before the subscription starts. You do not need a procurement department. You need the right questions, asked in the right order, with written answers on file.

Quick Answer: An AI vendor risk assessment is a structured check you run on an AI tool before buying it: what data the tool collects, whether that data trains the vendor's models, how it is secured, whether performance claims hold up, and how you exit. A small business can run one in a few hours per tool with a fixed question list.

What is an AI vendor risk assessment?

An AI vendor risk assessment is a review of an AI tool and the company behind it before you sign, focused on four things: where your data goes, how the tool is secured, whether the vendor's claims are true, and what happens when you leave. It is the software version of checking references before a hire.

Large companies run this inside formal third-party risk programs, with questionnaires that run to hundreds of items. Many of those questionnaires borrow from the NIST AI Risk Management Framework, a voluntary framework released in January 2023. A 10 to 70 person firm needs the same substance at a fraction of the weight: a fixed question list, one owner, one page of answers per tool.

One boundary worth drawing: this is how you vet a tool. Vetting the people you might hire to build automation is a different exercise, covered in our 12 questions to ask an AI consultant.

Why do AI tools need more vetting than regular software?

Because an AI tool usually processes your live business data on someone else's servers, and some vendors use that data to train their models. Regular software mostly stores what you type. AI tools read, summarize, and generate from client files, emails, and recordings, which raises the cost of a bad pick.

The numbers back the caution. IBM's 2025 Cost of a Data Breach Report found that 20 percent of breached organizations had breaches linked to shadow AI, meaning unsanctioned tools employees adopted without oversight. Those incidents added as much as $670,000 to the average breach cost, and 63 percent of breached organizations had no AI governance policy at all.

Shadow AI is what happens anywhere there is no sanctioned way to say yes to a tool. When approval is slow, staff swipe a card, and client data starts flowing through a product nobody vetted. The assessment exists to make the sanctioned path faster than the shortcut.

What questions should you ask an AI vendor before buying?

Ask ten questions across four areas: data (where it is stored, whether it trains models, how to delete it), security (certifications, breach history, access controls), claims (proof, references, error rates), and exit (contract terms, export formats). A vendor that answers all ten in writing clears most of the risk.

Data

1. Where is our data stored, and in which region?
2. Is our content used to train your models, and is the default off for our plan?
3. Who inside your company can access our data?
4. Which subprocessors handle our data, and where is that list published?

Security

5. Which security certifications do you hold, and can we see the reports?
6. Have you had a breach, and what notification commitment will you put in writing?

Claims

7. What evidence backs your accuracy or savings claims: references, error rates, test results?
8. Can we pilot the tool on our own real work before any annual commitment?

Exit

9. When we cancel, what gets deleted, on what timeline, and in what format can we export?
10. When the tool gets something wrong, how do we reach a human?

Send the list by email and keep the replies. Question 7 matters more than buyers expect: in September 2024 the FTC launched Operation AI Comply, five enforcement actions against companies that leaned on AI hype or sold AI tools that enabled deception. One of them, DoNotPay, had marketed "the world's first robot lawyer" without testing whether its output matched a human lawyer's work, and settled for $193,000. A vendor that cannot show evidence for a claim is asking you to run the test for them, on your clients.

How do you find out what happens to your data?

Read three documents before the demo sways you: the data processing agreement, the privacy policy section on model training, and the subprocessor list. Together they answer whether your files train the vendor's models, which third parties touch your data, where it is stored, and what gets deleted when you leave.

The data processing agreement (DPA) is the contract that binds the vendor to specific handling rules; no DPA on offer is a walk-away signal for any tool touching client or employee data. The training question is the one AI adds: consumer and free tiers often use customer content to improve models by default, while business plans usually turn it off. Confirm it in writing either way. The subprocessor list names the companies your vendor itself depends on, such as cloud hosts and model providers. Your data is only as contained as that list.

For law firms this is an explicit professional duty. ABA Formal Opinion 512, issued in July 2024, makes lawyers responsible for knowing how a generative AI tool uses client data and for safeguards against unauthorized disclosure; boilerplate consent in an engagement letter is not enough. HR teams carry the same weight with employee records, real estate firms with transaction and financial data.

Which certifications and standards actually matter?

Three names cover most of what you need: SOC 2 (independent audit of security controls), ISO 27001 (certified information security management), and ISO 42001 (the first management standard specific to AI). None guarantees a good product. They tell you the vendor treats security and AI governance as a process, not a promise.

The NIST framework rounds out the picture: free, voluntary, organized around four functions (govern, map, measure, manage), and easy to borrow questions from. Treat all of these as signals to weigh, not gates to pass. Certifications are expensive, so a young vendor without them can still be a sane choice for low-sensitivity work if its written data answers are clean.

Where does AI vendor vetting go wrong?

The two failure modes we see most are vetting nothing and vetting everything. Firms either let tools in on a demo and a credit card, or they freeze every purchase behind a checklist built for enterprise procurement. Both end the same way: the team quietly uses whatever works, unvetted.

Three patterns repeat:

Buying the tool before mapping the work. The demo defines the problem, the firm signs, and then someone hunts for a workflow to justify the license. Vetting cannot save a purchase that was backwards from the start. Mapping which workflow needs help, and what it costs you manually, is the first half of a full AI automation audit, and it is the half most firms skip. In the law-firm audits we run, each one has found ten or more AI and automation opportunities, and the list rarely matches the tools the firm had already bought.

Nobody owns the check. Five partners buy five tools against five private standards, or no standard at all. The fix costs nothing: one named owner, one shared question list, answers kept in one folder.

Flat vetting. Every tool gets the same scrutiny, so either everything crawls or the risky ones coast through with the trivial ones. Scale the check to the data. A transcription tool that hears client calls gets the full ten questions and a document review. A copy tool that never sees a client name needs fifteen minutes; more than that teaches your team to route around you.

We have also seen where the no-vetting road ends. We built a single source-of-truth data layer for a 500-employee New York real estate company: the fix for data that had ended up spread across too many disconnected systems. Every unvetted tool you add today is another future silo someone will pay to untangle.

How do you run an AI vendor risk assessment in a small business?

Give one person the job, use one shared question list, and size the effort to the data involved. A tool that touches client or employee records gets the full ten questions and a document review. A tool that never sees sensitive data gets a fifteen-minute check. Written answers, one page, kept on file.

1. List every AI tool in use today. Include the free ones staff adopted on their own. Make it an amnesty: the goal is a complete list, not blame.
2. Sort the list by data sensitivity. Client and employee data at the top, financial data next, public-content tools at the bottom.
3. Run the ten questions on the top tier. By email, answers in writing, one page per tool.
4. Fix the paperwork. Training opt-out confirmed, DPA signed, export format named in the contract, notification terms in writing.
5. Recheck once a year and on trigger events. An acquisition, a terms-of-service change, a major new AI feature, or a breach at the vendor restarts the clock.

If the list raises a bigger question, whether to buy tools at all or get help choosing and connecting them, start with do you need an AI consultant or just software, and see what an AI automation audit includes before you spend on either.

---

Want to know which workflows are worth an AI tool in the first place? That is what our free AI assessment shows you. Six questions, about two minutes, and you get a personalized preview of where automation would pay off in your business, before you sign any vendor contract. Start your free AI assessment.

Frequently asked questions

What is an AI vendor risk assessment?

An AI vendor risk assessment is a structured review of an AI tool and the company behind it before you buy. It covers four areas: what happens to your data (storage, model training, deletion), how the vendor secures its systems (certifications, breach history), whether performance claims are backed by evidence, and how you exit (export, deletion, contract terms). In a small business it is a fixed question list owned by one person, and the output is one page of written vendor answers kept on file.

Do small businesses need an AI vendor risk assessment?

Yes, scaled to the stakes. A 20-person firm does not need enterprise procurement, but it routes client files, employee records, and financials through AI tools the same way a big company does. IBM's 2025 Cost of a Data Breach Report found that 20 percent of breached organizations had breaches linked to AI tools staff adopted without oversight. A fixed question list and one accountable owner give a small business most of the protection at almost none of the overhead.

What questions should I ask an AI vendor about my data?

Ask five: Where is our data stored, and in which region? Is our content used to train your models, and is the default off for our plan? Who at your company can access our data? Which subprocessors handle it? And what gets deleted when we cancel, on what timeline? Ask by email so the answers are in writing. A vendor that answers clearly within a few days tells you as much through the speed as through the content.

What certifications should an AI vendor have?

For tools handling sensitive data, look for SOC 2 or ISO 27001, which cover security controls. Ask also whether the vendor is pursuing ISO 42001, the first international management standard specific to AI, published in December 2023. Treat certifications as strong signals rather than hard requirements: they are expensive, so early-stage vendors often lack them. A vendor with no certification but clear written answers on storage, training, and deletion can still be a reasonable pick for lower-stakes work.

Are free AI tools safe for business use?

Free tiers are where most trouble starts. Consumer versions of AI tools often use your content for model training by default, and they come with weaker admin controls, no contract, and no data processing agreement. For anything involving client or employee data, use a business plan with training turned off and a signed DPA, or keep that work out of the tool entirely. A free tool applied to public, non-sensitive tasks carries a different, much lower risk.

How often should we reassess AI vendors?

Once a year, plus after trigger events: the vendor is acquired, changes its pricing or terms of service, ships a major new AI feature, or reports a breach. AI products change faster than regular software, and a tool that passed your check last year may now route data through new subprocessors or new models. The annual pass is short: confirm the training setting, reread the subprocessor list, and check that your export path still works.

What are the biggest red flags when buying an AI tool?

Five walk-away signals: the vendor cannot say in writing whether your data trains its models; there is no data processing agreement on offer; performance claims come with no evidence, references, or trial; there is no way to export your data in a usable format; and pressure to sign an annual contract before a pilot. None of these requires technical knowledge to spot. They show up in how the vendor answers, or refuses to answer, ten plain questions.