Questions to Ask AI Vendors in an ERP RFP
questions to ask AI vendors in an ERP RFP
Also searched as
- AI vendor RFP questions
- questions to ask ERP vendors about AI
- AI vendor due diligence questions for ERP
- what to ask a vendor about their ERP copilot
Short answer
Ask AI vendors how, not whether: how the assistant reaches ERP data, whose permissions it uses, where each model runs, how answers are tested, what it costs at volume and what you can take with you on exit. Below are the questions grouped by topic, with what a good and a worrying answer sounds like.
Applies to: ERP vendor copilots, third-party ERP AI products, and custom AI builds; relevant to any ERP
How to run the vendor question round
- 1Send the same written question set to every bidder, grouped under the headings below, and require answers in writing before demos.
- 2Ask for architecture diagrams showing where data, models and logs sit, and compare them to your security policy.
- 3Hold a technical session with your security lead and an ERP power user, not only sales engineers.
- 4Request reference calls with customers who run the product on an ERP similar to yours and ask what they would change.
- 5Run the proof of value on your sanitized data using your test questions and compare against known ERP results.
- 6Record answers in a table so you can score them consistently after the demos.
- 7Follow up unclear answers in writing; a verbal assurance has no contract value.
Data access and grounding
Ask: How does the assistant read our ERP data, through documented APIs, database views, or a replicated copy? How fresh is the data? Which tables or business objects are in scope for our modules? Can it show the query or source for each answer? A good answer names the interface and the refresh model. A worrying answer is 'it learns from your data' with no mechanism.
Ask: Is any of our data used to train or fine-tune a model, ours or yours? Where do embeddings and vector indexes live? Can we delete them? Ask for the answer per component, since a vendor can truthfully say 'no training' while still retaining prompts in a logging system.
Ask: What happens when the answer requires joining ERP data with documents such as quality procedures, contracts or drawings? Many products do one well and fake the other.
Security, roles and write-back
Ask: Whose credentials does the assistant use when answering? Does it enforce the asking user's ERP roles and data restrictions? A product using one service account should explain in detail how it filters results, and you should test it with a low-privilege user during the proof of value.
Ask: What actions can the assistant take, and what approval is required? Can we run it fully read-only? Is every action logged with user, input and outcome? Ask for the audit record format. Ask about prompt injection: what happens if a supplier document or free-text field contains instructions aimed at the model?
The /tools/ai-agent-security-review-checklist and /tools/ai-vendor-evaluation-checklist tools provide longer lists; use them to extend this section if your risk profile is high.
Models, deployment and quality
Ask: Which models run in the product, who hosts them, and can we choose or replace them? Can it run on our GPUs or in our private cloud? What is the minimum hardware? What happens to quality if we use a smaller open-weight model? Vendors that depend on a single external model API should state the data terms of that provider.
Ask: How do you measure accuracy? Do you have a regression test set, and what happens when you change the model or prompts? Ask for failure examples, not only successes. A vendor who cannot name a failure mode has either not tested widely or is not being candid.
Ask: What is your support model for wrong answers? Is there a feedback loop that lets our admins review flagged responses?
Commercial, roadmap and exit
Ask: What is the pricing unit, and what would our bill be at our expected volume and at three times that? Which AI features are included in the base ERP subscription, which are add-ons, and how have prices changed over the last two renewals?
Ask: What can we export if we leave: configuration, prompts, evaluation sets, embeddings, tuned weights? Which parts rely on proprietary components that cannot move? See cio-erp-vendor-copilot-lock-in for how to weigh the answers.
Ask: Which of the capabilities you described are generally available today versus roadmap, and will you commit roadmap items contractually with dates and remedies?
Common pitfalls
- !Asking yes/no questions that every vendor answers yes.
- !Letting the demo environment stand in for answers about production architecture.
- !Not testing role enforcement with a low-privilege account.
- !Skipping the data-use and logging questions because 'no training' was stated once.
- !Treating roadmap statements as commitments.
- !Sending questions only to sales rather than involving the vendor's security and engineering staff.
How an ERP-grounded AI assistant handles this
Use a grounded assistant on your own side of the table: with read-only access to your ERP it can answer 'which tables hold this data and how many records are involved', which helps you check vendor claims about scope and volume. ERPray-style tooling that shows the generated query also gives you a reference for how a transparent answer should look when you grade vendor demos.
Frequently asked questions
What is the single most important question to ask?
Whose permissions does the assistant use, and can you prove it with a low-privilege test user. Role enforcement failures are the commonest serious defect in ERP AI and are invisible in a demo run by an administrator.
How do I check a vendor's accuracy claims?
Do not rely on a quoted percentage. Give them 10 to 20 of your own questions with known answers from the ERP, include a few hidden ones, and grade results yourself. Ask how they would measure it in production after go live.
Should I ask about the underlying model provider?
Yes. If the product sends your data to a third-party model API, you need that provider's data retention, location and training terms, and you need to know whether the vendor can swap models without notice. Ask for it in the contract.
Are these questions only for third-party AI vendors?
No. Ask your ERP vendor the same questions about their built-in copilot. Bundled does not mean secure or private by default, and its deployment, pricing and exit terms may be less flexible than a separate layer.
Related
ERP AI Vendor Due Diligence: What CIOs Should Actually Check
The single highest-signal question in ERP AI vendor due diligence is 'show me it answering a real question against our actual ERP data, live, not a demo dataset.' Beyond that, focus diligence on data grounding method, security and access model, exit/portability terms, and references from customers on your specific ERP platform - not on feature checklists, which most vendors can match on paper.
CIO briefingHow to Write an RFP for ERP AI
Write the ERP AI RFP as two linked scopes: the ERP platform and the AI layer that reads or acts on its data. Define the business questions the AI must answer, where data may and may not travel, which ERP roles apply, and how you will test answers before you sign. Vague asks like 'must have AI' produce slideware, not comparable bids.
CIO briefingDoes Your ERP Vendor's AI Copilot Increase Lock-In?
Native vendor copilots such as SAP Joule, Oracle Fusion AI agents and Microsoft Copilot in Dynamics 365 are usually bundled or low-cost to start, but they only see that vendor's native data model and typically bill on consumption, which raises both switching cost and long-run cloud spend. Use them for what they do well inside the vendor's own UI, and keep cross-system or custom-field questions on a vendor-neutral layer.
How-toHow to Score ERP AI Vendors in an RFP
Score in two passes: pass/fail gates first (deployment, data residency, security, mandatory functionality), then weighted criteria on a defined 0 to 4 scale with evidence required for every score. Fix the weights before bids arrive, score independently, then reconcile, and keep price as a separate, calculated factor.
CIO briefingERP Upgrade or AI First? A CIO Decision Framework
In most cases, add a grounded AI layer on top of your current ERP first, because it proves value in weeks and shows exactly what an upgrade would need to fix. Reserve a full ERP replacement for cases where the platform itself is end of support, unsupported, or structurally blocking the business, not simply because it feels dated.
CIO briefingHow Much Does ERP AI Actually Cost?
A scoped pilot against one ERP module typically runs 15,000 to 50,000 USD; a single-department production deployment runs 50,000 to 150,000; a multi-module enterprise rollout runs 150,000 to 500,000 or more, plus ongoing hosting and usage costs. Native vendor copilots are usually priced per active user per month on top of existing licensing, not included free.
AI for ERPBuild vs Buy: Should You Use Your ERP Vendor's AI Copilot, or Build Your Own?
SAP Joule, Copilot for D365, Infor GenAI, Oracle AI Agent Studio, or a private LLM on your own data. A CIO framework for the build vs buy decision, with real trade-offs.
AI for ERPA Private LLM Grounded on Your ERP Data
How a private LLM answers questions on your ERP data: RAG plus text-to-SQL, role-based permissions inherited from the ERP, and where each fits.
Stuck on ERP selection and AI?
Talk to engineers who work inside ERP selection and AI every week, and who build private AI that answers these questions from your own ERP data.