Year-end is when the AI vendor pitches pile up, everyone wants to close before their fiscal year ends, and the pressure to sign something before December makes this exactly the wrong time to skip diligence. The questions to ask AI vendors haven't changed much since earlier this year, but the market has, there are more genuinely capable products now and also more thin wrappers around the same base model dressed up as something proprietary. Here are the ten questions that tell you which one you're looking at, each with the answer that should reassure you and the one that should worry you.

1. Which model or models power this, and what happens if that model changes?

Good answer: they name it specifically (GPT-4 class, Claude 3 class, an open-weight model they've fine-tuned) and explain how they'd handle a provider deprecating or repricing it. Alarming answer: "that's proprietary" or vague deflection. If they won't tell you what's under the hood, they usually don't want you to notice there's very little under the hood.

2. What happens to our data after we send it?

Good answer: a clear statement on retention, whether your data trains their model or the underlying provider's model, and a written data processing agreement. Alarming answer: "don't worry about it" or a privacy policy that's clearly boilerplate copied from elsewhere.

3. Can you show me accuracy numbers on a task like ours, not a generic benchmark?

Good answer: a real number, on real customer data if possible, with the failure cases disclosed alongside the successes. Alarming answer: a demo video with a rehearsed, perfect example and nothing about the cases where it's wrong.

4. What happens when it gets something wrong?

Good answer: a described fallback, escalation to a human, a confidence threshold, a way for staff to flag and correct errors. Alarming answer: "it doesn't really get things wrong." Every system does. The ones that pretend otherwise are the ones you'll be debugging alone at 11pm.

5. How is this priced, and what happens if our usage spikes?

Good answer: transparent per-seat, per-usage, or flat pricing with a clear description of overage costs. Alarming answer: pricing that only becomes clear after you're already a few months in, or "we'll figure it out together" as a substitute for a number.

6. What do we own if we leave?

Good answer: you keep your data, your prompts or configurations, and can export in a usable format. Alarming answer: everything you've built inside their platform, prompts, workflows, integrations, stays locked in if you cancel.

7. Who else uses this for something similar to our use case?

Good answer: a specific, checkable reference, ideally one you can actually call. Alarming answer: "we can't disclose our clients" for a product that isn't handling anything remotely sensitive. That's not confidentiality, that's an absence of references.

8. What's your actual uptime and incident history?

Good answer: a status page, or at minimum a straight answer with a number. Alarming answer: "we've never had downtime." Nobody hasn't.

9. How much of this is you, and how much is the underlying model provider?

Good answer: an honest breakdown, most vendors are a layer on top of a foundation model and that's fine, as long as they're clear about where their value actually is (workflow, integration, domain tuning) versus the raw model capability. Alarming answer: claiming proprietary AI when it's a thin prompt wrapper around a general-purpose API, priced as if it were something built from scratch.

10. What's the exit clause?

Good answer: month-to-month or a reasonable notice period, no punitive lock-in. Alarming answer: a multi-year contract with an early termination fee, pitched to you as a "founding customer discount" to make the lock-in feel like a favor.

Reading the pattern, not just the answers

No single vague answer should kill a deal, vendors are often smaller teams who haven't formalized every policy yet, and that's not automatically disqualifying. What matters is the pattern across all ten. A vendor who's specific about the model, honest about failure modes, and clear about data ownership is building a real relationship with you. A vendor who's vague on four or five of these is selling momentum, not a product, and December's close-the-deal urgency is exactly the pressure that makes that easy to miss.

If you're weighing an AI purchase against building something narrower and owned in-house, this is the same fork covered in build versus buy, and it's worth running before you sign, not after.

The takeaway

Print these ten questions and bring them to the next vendor call, literally, as a checklist. The specific answers matter less than whether the vendor answers plainly at all. Vagueness on model dependency, data use, or exit terms is the tell, not a technicality to wave through because the sales rep is friendly and the demo looked good.