CFO’s Guide to Hiring an AI Vendor

Although finance has been slower to adopt AI than other functions, that’s starting to change. More companies are now seriously exploring AI—not just for experiments, but for real work in reporting, forecasting, and compliance.

AI vendors promise to save time, improve accuracy, and make your team more efficient. But when you look closer, things don’t always hold up. Some tools barely use AI at all. Others break down when your data isn’t clean. And a few come with hidden costs no one talks about upfront.

In this issue, I’ll share what to watch for, what questions to ask, and how to choose a vendor that can actually deliver what your team needs.

📢 Upcoming Events for Finance Leaders – Save the Dates!

As many of my readers might know, I’m not only a newsletter author but also an expert in the AI Finance Club, a community of finance executives and leaders committed to mastering AI. 

Nicolas Boucher, who leads the community, is hosting a live webinar on Friday, June 27th, exclusively for CFOs, Fractional CFOs, and Finance Directors.

In this session, Nicolas will break down how AI is reshaping the CFO role, the tools that top finance leaders are already using, and practical strategies that can elevate your finance processes. Highly recommend!

📅 When: Friday, June 27th | 5:00 PM CET / 11:00 AM ET

Seats are limited—grab yours now!

The Balanced View: How to Hire an AI Vendor that Actually Delivers

Evaluating an AI solution is not like buying an FP&A tool or ERP module. The models behind these tools are different. The pricing model is different. The security risks are different. And most importantly, the long-term ownership responsibilities are unclear unless you ask the right questions.

1. What Type of Vendor Are You Talking To? (And What Are They Actually Building?)

Here are the four vendor types CFOs are most likely to meet—and what they really deliver:

Tip for CFOs: Ask for a workflow diagram before you look at a demo. Make the vendor explain what AI is doing, where human inputs happen, and how outputs are validated.

2. Cost Breakdown – AI Has a Hidden Meter Running

Unlike traditional SaaS, AI tools charge you while they work. You need to understand both the build and run costs.

Build Costs:

• Prompt design or app setup

• Workflow integration

• Testing and initial training

Ask:

• What’s included in the scope of work?

• Who owns the IP after implementation?

• If I switch vendors, can I keep using this?

Run Costs (where surprises happen):

• Most vendors use APIs (OpenAI, Claude, etc.) priced by tokens (i.e., how much text is processed).

• You pay every time a user runs a query or a bot triggers automatically.

Ask:

• What’s your monthly cost forecast based on our usage?

• Can I cap usage or set thresholds?

• Who monitors the run cost drift?

3. Security & Compliance – What You Can’t Afford to Miss

AI vendors aren’t always upfront about how data moves or who sees it. But as CFO, you own the risk if data leaves your systems insecurely.

Key questions to ask:

Data Handling

• Where is the data stored? (If it’s in the vendor’s environment, is it encrypted at rest and in transit?)

• Do they use OpenAI’s public API or an enterprise-secure endpoint?

• Can you opt out of data being used for model training?

Compliance

• Are they SOC 2, ISO 27001, or GDPR compliant?

• Do they offer audit trails for generated content or decisions?

• Who can access the logs, and what’s retained?

Human Review

• Is there a step where a human must approve AI output before it’s saved or sent?

• Can you override or flag faulty AI responses?

You don’t need to speak AI fluently, but you do need to know when something doesn’t add up. Behind every slick demo lies a real cost, real complexity, and real risk if you don’t ask the right questions. As AI transitions from just a concept to a budget line, your role isn’t just to approve the spending. You need to ensure that what you’re buying actually works, scales, and protects the business.

Real Things I’ve Heard from AI Vendors—and What Set Off Red Flags

When you’re a CFO evaluating AI vendors, you will encounter many confident pitches. Some sound impressive—until you dig deeper. Others may seem vague, but that’s not necessarily a bad thing. Here’s a list of actual statements I’ve heard during vendor meetings, and what they really signaled.

🚩 “This tool uses AI throughout the workflow.”

Red flag: When asked where exactly the AI lives, they couldn’t answer.
What I found: It was just a basic chatbot layered on top of a reporting interface—no real analysis, learning, or decision support. The “AI” was just a UI gimmick.

Ask: “Which step uses machine learning or language models? What task is the AI actually performing?”

🚩 “We’ll build a custom AI model for your finance team.”

Big red flag: That’s a massive, expensive undertaking—and 99% of finance use cases don’t require it.
What I recommended instead: Use existing open-source or commercial models (Mistral, GPT-4, Claude) and build a Retrieval-Augmented Generation (RAG) layer or light fine-tuning if needed.

Ask: “Why can’t this be done with a pre-trained model and good prompt engineering?”

🚩 No one asked about your data quality.

Red flag: If a vendor doesn’t review the quality, structure, or availability of your data—but still promises great results—the solution is almost guaranteed to fail or stall.

In practice: We had to pause a project mid-way because assumptions about ERP consistency didn’t hold.

Ask: “What assumptions are you making about our data structure? What happens if it’s incomplete or inconsistent?”

🚩 “It just works—you won’t need to touch it.”

Red flag: There’s always some configuration, adjustment, or learning curve. If a vendor downplays that, they’re setting you up for disappointment.

What happened: The team had no idea how to update prompts or troubleshoot broken logic. We had to reverse-engineer the solution just to make a basic change.

 Ask: “What skills or tools will we need internally to maintain and evolve this solution?”

✅ “We don’t know if this will work yet, but we can start with a low-cost proof of concept.”

Green Flag: AI isn’t plug-and-play, especially for custom workflows. If a vendor admits there’s some uncertainty and suggests a POC before full rollout, that shows maturity, not weakness.

✅ “We’ll use your real workflows, not just a sandbox demo.”

Green flag: A good vendor wants to replicate your actual process, not just show you canned examples. They’ll ask for a real Excel file, a recent report, or sample ERP exports to test with.

✅ “Here’s how you’ll maintain this after we hand it off.”

Green flag: The best vendors bake in handoff planning early. They’ll provide documentation, training, and clearly define who owns what, especially if prompt or model tweaks are needed over time.

Bottom line? Flashy demos are easy. But responsible AI vendors will:

• Scope based on your actual data

• Use proven models instead of reinventing the wheel

• Build solutions you can maintain

• Offer realistic, phased implementation—not AI theater
  

If a vendor gets nervous when you ask how things work or what happens when they don’t, they’re not the right partner.

Closing Thoughts

AI vendors are getting better at selling, but not necessarily better at delivering. As CFOs, we’re stepping into uncharted territory where buzzwords can quickly turn into budget leaks. That’s why asking sharp, grounded questions isn’t a sign of skepticism; it’s leadership.

The best AI projects aren’t the ones with the most features or the flashiest demos—they’re the ones your team can use, maintain, and scale. Get that part right, and AI won’t just be a tech decision—it’ll be a lasting strategic advantage.

See you next week.

We Want Your Feedback!

This newsletter is for you, and we want to make it as valuable as possible. Please reply to this email with your questions, comments, or topics you'd like to see covered in future issues. Your input shapes our content!

Want to dive deeper into balanced AI adoption for your finance team? Or do you want to hire an AI-powered CFO? Book a consultation! 

Did you find this newsletter helpful? Forward it to a colleague who might benefit!

Until next Tuesday, keep balancing!

Anna Tiomina 
AI-Powered CFO