Three Questions Every CIO Should Ask of AI Vendors
Most enterprise AI purchases fail the same way: a vendor demo looks impressive, procurement approves the spend, and six months later the system sits unused because it can't actually do the job.
A recent Forbes article, “The Smart CIO’s Guide to Choosing the Right AI Tech Stack” by Keith Ferrazzi, cuts through the noise with three questions that reveal whether AI will actually work for your business:
1. Who trained the model?
If your vendor is reselling someone else's foundation model, you're limited to what that model was designed to do. Real differentiation comes from working with vendors who 1) can design, optimize, and continuously improve the model architecture itself; and 2) are focused on the problems in your specific domain.
2. What data was it trained on?
Generic internet data can teach a system to chat or summarize text, but it can’t forecast your demand or optimize your supply chain. Models that are trained on domain-specific data (reflecting your operational realities) deliver insights that are both explainable and actionable.
3. What was it trained to do?
A model built to generate sentences will underperform when forecasting sales. The underlying training objective of any AI system defines what it can actually do. Taking a language based model and asking it to perform quantitative prediction is like fitting the proverbially round peg into a square hole. Purpose-built systems start with the end goal and work backwards, which gives them an advantage in final performance.
Why “Purpose-Built” Matters
Ferrazzi highlights companies like Omnifold that are examples of this new class of AI: systems engineered for complex computational problems that general-purpose LLMs can’t solve. In supply chain forecasting and optimization, purpose-built AI learns from the network, seasonality, and behavioral dynamics unique to supply chain, not from generic internet data.
The Takeaway for CIOs
The CIO’s job is to architect a relationship with AI that aligns with their enterprise’s data, strategy, and ambition. LLMs and generative AI agents are excellent tools for processing documents, writing code or automating known workflows, but complex use cases, such as forecasting and optimization, require purpose built AI.