Any ERPBuyer Guide

ERP AI Buyer Guide

How to Choose an ERP AI Implementation Partner

Short answer

Choosing an ERP AI implementation partner comes down to five checks: do they know your ERP's actual data model, will the architecture keep sensitive data inside your boundary, can they demonstrate a working query against a real schema rather than a slide, what does the pricing model actually charge for, and who owns the system when the engagement ends. This guide gives CIOs a structured scorecard for evaluating any vendor, Netray included, against those five checks.

ERP: SAP S/4HANA, Infor CloudSuite Industrial, Oracle E-Business Suite, Microsoft Dynamics 365, Epicor Kinetic
Industries: Manufacturing, Aerospace, Defense, Electronics
Written for: CIO

Discuss your SAP S/4HANA environment See use cases

Most CIOs evaluating ERP AI in 2026 are fielding three kinds of pitches at once: a generic AI consultancy that talks fluently about RAG and agents but has never opened an IDO or a BAPI; the incumbent ERP VAR that knows the schema cold but treats 'AI' as a chatbot bolted onto the support desk; and a hyperscaler systems integrator whose default architecture routes ERP data to a managed cloud API. None of the three is automatically wrong, but each has a failure mode that only shows up after the contract is signed.

The risk is structural, not vendor-specific. An AI layer over an ERP has to touch the database, the middleware, or the application APIs to be useful, which means the partner you choose inherits access to production financial, engineering, and personnel data the moment the pilot starts. A partner without ERP fluency will either build something brittle that breaks on the next patch, or default to sending queries through a public model API because that is the fastest way to a demo. A partner without AI engineering depth will ship a keyword search with a chat wrapper and call it done.

This matters more in regulated manufacturing than in most software categories, because the data in a SyteLine, S/4HANA, or Costpoint instance often includes export-controlled technical data, CUI, or supplier pricing that cannot legally or contractually leave a defined boundary. A partner selection mistake here is not just wasted budget; it can be a compliance event. The questions in this guide are designed to surface that risk before the statement of work is signed, not after the first data export.

What follows is a scorecard: the architecture questions to ask, the pricing models to expect, the phase structure a competent engagement follows, and the honest cases where hiring an external partner is the wrong call and building in-house or waiting is the better one.

What usually gets in the way

The problems we hear most from cio teams running SAP S/4HANA.

AI-fluent vendors without ERP fluency

Many AI consultancies can build a RAG pipeline in a week but have never queried a SyteLine IDO, mapped a SAP CDS view, or dealt with an Oracle interface table. They discover the ERP's quirks on your dime, mid-pilot.

ERP-fluent vendors without AI engineering depth

VARs and resellers know the modules and the customizations, but 'AI' often means a support bot wired to a knowledge base, not grounded retrieval with citations, permission-aware answers, or an agent approval workflow.

Default-to-cloud architecture

The fastest way for any integrator to hit a demo date is to route queries through a public model API. For ITAR technical data, CUI, or unreleased engineering BOMs, that is a deemed-export or contractual breach, not a shortcut.

Pilots that never reach production

A slide-deck proof of concept with no defined success criteria, no owner for the model after go-live, and no plan for who patches prompts when the ERP schema changes on the next upgrade.

Time-and-materials scope creep

Open-ended hourly billing with no fixed milestones turns 'we ran into an integration issue' into a line item, and gives the vendor no incentive to finish rather than extend.

Where AI earns its place in SAP S/4HANA

Each use case names the ERP objects it reads or writes, so your ERP team can judge the integration effort before anyone commits budget.

Live query against your actual schema

Before signing anything, ask the partner to run a natural-language question against a read-only copy or sandbox of your own ERP, not a generic demo tenant, and show the underlying query.

Touches: IDO CustList / ItemWhse, BAPI_MATERIAL_AVAILABILITY, SuiteQL saved search, OData CDS view

Outcome: Separates vendors who can adapt to your customizations from vendors who only demo on clean sample data.

Read-only rollout before any write-back

A competent partner proposes a phase where the AI layer only reads and answers questions, with write-back to the ERP (creating a PO, updating a status) gated behind explicit approval later.

Touches: ERP roles/permissions table, service account with SELECT-only grants, audit log of queries

Outcome: Limits blast radius during the riskiest phase of the engagement, when the retrieval logic is least tested.

Role and permission mirroring

The AI layer should answer a shop floor supervisor and a controller differently for the same question, because it inherits the same row- and field-level restrictions the ERP already enforces.

Touches: ERP security groups, IDO/BAPI authorization objects, row-level filters on cost and margin fields

Outcome: Prevents an AI shortcut from becoming a permission bypass that IT security has to explain later.

Structured data and document retrieval side by side

Most ERP questions blend a transaction lookup (open orders, on-hand quantity) with unstructured context (a spec sheet, a quality deviation, an email thread). Ask how the partner handles both in one answer.

Touches: ERP tables/views plus PDM/PLM documents, quality records, engineering change notices

Outcome: A single grounded answer instead of two separate tools the user has to reconcile manually.

Agent approval workflow for any write action

When the engagement moves past read-only, every write an agent proposes (release a PO, update a due date) should surface as a draft a named human approves inside the ERP's own workflow, not an autonomous action.

Touches: ERP approval/workflow engine, task queue, change log

Outcome: Keeps the ERP's existing segregation-of-duties controls intact instead of routing around them.

Reference architecture for air-gapped or private-cloud

Ask for an actual network diagram, not a slogan: where does the model run, where do embeddings live, what crosses the boundary, and what happens if the internet connection is cut entirely.

Touches: GPU server or private cloud tenancy, vector store, ERP connector, network segmentation

Outcome: Confirms the partner has actually deployed this pattern, not just proposed it.

Continuity across an ERP version upgrade

Ask what happens to the AI layer's connectors, prompts, and retrieval mappings when the ERP moves versions (SyteLine 9 to CloudSuite, ECC to S/4HANA). A good partner designs for this from day one.

Touches: Connector configuration, schema mapping layer, IDO/BAPI/CDS version references

Outcome: Avoids a rebuild-from-scratch bill the first time the ERP team patches or upgrades.

Reference architecture

Whatever partner you choose, the proposal should describe five layers explicitly, not gesture at them. If a proposal collapses these into one box labeled 'AI platform,' that is itself a signal to ask harder questions.

1
ERP connectors
Named, version-specific connectors (ION API, IDO, BAPI/OData, SuiteQL, AIS/Orchestrator) with a defined refresh cadence and a documented service-account permission set, not a generic 'database connection.'
2
Data and semantic layer
A mapping from raw ERP tables and codes to business-readable terms (item master fields, status codes, cost elements) so the model answers in your vocabulary and can explain what it queried.
3
Model serving
Where the LLM actually runs: on customer-owned GPUs via vLLM or Ollama, in a private/sovereign cloud tenancy, or (the one to scrutinize) a public model API that the vendor's default config points to.
4
Retrieval and agents
RAG over ERP data plus documents, with citations back to source records, and any agent actions expressed as proposed changes routed through approval, not direct writes.
5
Governance and audit
Every query and every proposed write logged with who asked, what was retrieved, what the model answered, and who approved any resulting change, retained per your existing records policy.

Integration notes for your ERP team

Ask for the specific API or protocol per ERP: ION API for Infor, OData/BAPI/RFC for SAP, AIS/Orchestrator for JD Edwards, SuiteQL/RESTlets for NetSuite, data entities/OData for Dynamics 365.
Confirm whether the connector reads a replicated/reporting copy of the database or the live transactional system, and what load it adds during MRP runs, period close, or other batch windows.
Get the service-account permission model in writing: which roles, which tables, read-only vs. write, and how those permissions are reviewed on a schedule.
Ask how prompts, connector configs, and schema mappings are version-controlled, so a change is reviewable and revertible like any other code change.
Clarify what happens to cached or embedded ERP data if the contract ends: is it deleted, exported to you, or does it remain on vendor infrastructure.
Request a network diagram showing every system the AI layer talks to, including any third-party API, monitoring service, or logging destination.
Ask how the partner handles a schema change from an ERP patch or upgrade: automated detection, manual review, or silent failure.

Deployment options

Air-gapped on-prem

Defense contractors, ITAR technical data, CMMC Level 2 enclaves, classified-adjacent programs

Model, vector store, and connectors run entirely inside your network with no outbound path; the right choice when the data itself cannot legally reach a shared cloud, regardless of that cloud's certifications.

Private or sovereign cloud

Multi-site manufacturers who need central management but still require data residency guarantees

A dedicated tenancy under your control, run by you or a partner under contract, giving central visibility across plants without the deployment overhead of true air-gapping.

Hybrid

Organizations with a mix of sensitive and non-sensitive workloads across sites

Sensitive ERP data and export-controlled content stay on-prem or in a private tenancy; lower-sensitivity workloads (general HR policy Q&A, public product documentation) can use faster cloud paths, with the split defined by data classification, not convenience.

Compliance and data control

How the architecture supports your obligations. Certification and accountability stay with your organisation; the design keeps the evidence straightforward.

ITAR / export control

Verify the partner's default architecture keeps technical data inside your boundary and never transits a third-party model API; ask them to point to the specific control in their design, not just assert compliance.

CMMC 2.0 / NIST SP 800-171

If CUI is involved, confirm the AI layer sits inside your existing enclave boundary and inherits, rather than duplicates, your access controls, logging, and incident response process.

GDPR / EU AI Act (for multinational operations)

Ask how the partner classifies the AI system's risk tier under the EU AI Act and what documentation (data lineage, human oversight, logging) they produce as a byproduct of the engagement, not as a separate deliverable.

SOC 2 / internal audit

Require the partner's own access to your production systems during the engagement to be scoped, time-boxed, and logged the same way any other contractor's access would be under your existing SOC 2 controls.

Data residency

For any region with sovereignty requirements (EU, Middle East, parts of APAC), confirm in writing where model weights, embeddings, and logs physically reside, and what changes if the vendor relationship ends.

Where Netray fits

ERPray

For grounded question-answering, dashboards, and agents across SAP, Infor, Oracle, or other supported ERPs, evaluate any partner's proposal against what a purpose-built ERP question-answering layer already handles: read-only defaults, visible queries, and role inheritance.

Custom build

When the requirement is a bespoke workflow (a bespoke quoting agent, a fine-tuned classifier for a niche document type) rather than general Q&A, the evaluation criteria in this guide still apply to the custom-build proposal itself.

How an engagement runs

Phase 1 . 2-3 weeks

Discovery

-Data and access inventory across the ERP modules in scope
-Data classification pass (what is export-controlled, CUI, or otherwise restricted)
-Architecture proposal naming the specific connector, model, and deployment target
-Written scope and fixed-fee or milestone pricing for the pilot

Phase 2 . 6-8 weeks

Pilot

-Read-only deployment against a defined use case with real users
-Query accuracy review against a held-out set of known-answer questions
-Permission and audit log validation
-Go/no-go decision with named success criteria set before the pilot started

Phase 3 . 8-12 weeks

Production

-Hardened deployment with monitoring and alerting
-Documented runbook for prompt and connector changes
-Named owner on your side for ongoing operation
-First write-back use case, if any, gated behind approval workflow

Phase 4 . Ongoing

Scale

-Rollout to additional plants, business units, or ERP instances
-Quarterly review of query patterns and accuracy drift
-Update plan tied to ERP patch and upgrade cadence
-Clear internal capability so the partner is not a single point of failure

Questions to ask any vendor, including us

A short list that separates real SAP S/4HANA AI work from a chatbot demo.

Can you run a live query against a sandbox copy of our actual ERP, not a demo tenant, in the first meeting?
Where does the model physically run, and what is the exact path data takes to reach it?
What happens to our data, prompts, and any fine-tuned weights if we end the contract?
Who on your team has touched a system like ours before, by name, not by logo on a slide?
What is the pricing model: fixed milestones, time and materials, or a hybrid, and what triggers a change order?
What is your rollback plan if the pilot fails to hit the agreed accuracy threshold?
How do you handle a schema change from an ERP patch mid-engagement?
What do we own outright at the end of the engagement versus what remains licensed from you?

Frequently asked questions

Should we hire an ERP AI implementation partner or build in-house?

Build in-house if you already have staff who know your ERP's APIs and can learn LLM/RAG patterns, and your timeline allows a few months of ramp-up. Hire a partner when you need production results faster than that ramp allows, or when the compliance stakes (ITAR, CMMC) make a first attempt too costly to get wrong. Many organizations do a hybrid: a partner for architecture and the first deployment, in-house ownership after.

How long should a first pilot take?

A focused pilot on one use case, one ERP module, and a small user group typically runs 6-8 weeks after a 2-3 week discovery phase. Anything quoted at 'a few days' is likely reusing a generic demo rather than adapting to your schema; anything open-ended past 12 weeks without a defined checkpoint usually signals scope was never fixed.

Is it a red flag if a vendor wants to use a public model API by default?

Not automatically, but it should trigger a direct conversation about what data crosses that boundary and under what data processing terms. For non-sensitive general questions it can be a reasonable cost trade-off; for ERP data that includes export-controlled technical data, CUI, or unreleased engineering content, it is usually disqualifying, and a competent partner will say so rather than default to it silently.

What should a fixed-fee milestone actually cover?

A well-structured milestone ties payment to a demonstrable outcome: a working read-only query against your real schema, a documented accuracy result against a test set, or a signed-off go-live, not hours logged. If a proposal cannot describe what 'done' looks like for a phase, it is time and materials in disguise.

Do we need a different partner for each ERP if we run more than one?

Not necessarily. The connector layer differs by ERP, but the data/semantic, model serving, retrieval, and governance layers can be shared architecture across SAP, Infor, and Oracle instances in the same organization. Ask any candidate partner directly whether they have built that shared layer before or would be starting from zero on your second ERP.

How do we evaluate accuracy before committing to production?

Build a held-out set of 30-50 real questions with known correct answers pulled from your own ERP, covering easy lookups and harder multi-step questions, and score the pilot against it before go-live. Ask the partner to propose this test themselves; if they resist an objective accuracy benchmark, treat that as a signal.

What ongoing cost should we expect after go-live?

Expect three ongoing lines: GPU or private-cloud infrastructure, a support/maintenance retainer for connector and prompt updates (often tied to ERP patch cycles), and internal staff time to own the system day to day. Ranges vary widely by scale; the erp-ai-cost-pricing-guide page breaks down the components in more detail.

Related guides

Infor AI Buyer Guide

What an Infor AI Consulting Partner Should Actually Deliver

A CIO's guide to Infor AI consulting: what a partner should deliver on ION API, IDOs, and Data Lake, how it relates to Coleman AI, and questions to ask.

SAP AI Buyer Guide

SAP AI Consulting for On-Prem and Private Deployments

What to demand from an SAP AI consulting partner when data must stay on-prem or private: OData/BAPI mechanics, Joule vs. private LLM, and buyer questions.

Oracle AI Buyer Guide

Oracle ERP AI Consulting: What a Partner Should Deliver

What to demand from an Oracle ERP AI consulting partner across EBS, JD Edwards, NetSuite, and Fusion Cloud: interface tables, APIs, and buyer questions.

ERP AI Cost Guide

What ERP AI Actually Costs: A CFO's Guide

A CFO's guide to what ERP AI actually costs: GPU hardware, model licensing, integration and connector work, and realistic ongoing run-rate ranges.

ERP AI readiness

Is Your ERP Ready for AI? A Readiness Assessment Checklist

Is your ERP actually ready for AI? A practical checklist covering master data quality, access and permissions, GPU sizing, and governance before you fund a pilot.

CIO playbook: strategy, budget, org

A CIO's Guide to Sequencing AI on Your Manufacturing ERP

A practical CIO guide to sequencing AI on top of your manufacturing ERP: where to start, how to budget, who should own it, and how to avoid a pile of stalled pilots.

Plan it with numbers

Free Tool

Talk it through with an engineer who knows SAP S/4HANA

Bring one real question your team cannot answer from the ERP today. We will map the data path, the model, and where it runs, and tell you honestly if AI is the wrong tool for it.

Start the conversation See ERPray

All AI for ERP guides