Any ERPBuyer Guide

Vendor copilots vs private AI

Build vs Buy: Should You Use Your ERP Vendor's AI Copilot, or Build Your Own?

Short answer

Vendor copilots (SAP Joule, Microsoft Copilot for Dynamics 365, Infor GenAI/Coleman, Oracle AI Agent Studio) are fastest to turn on but tie you to the vendor's cloud tier, data residency terms, and roadmap. A private LLM grounded on your ERP data via RAG and read-only query tools gives you control over where data goes and which customizations it understands, at the cost of an integration project. The right answer usually depends on whether your data can legally and commercially sit in the vendor's cloud, and whether your value is in standard transactions or in your own custom objects and workflows.

ERP: SAP Joule, Microsoft Copilot for Dynamics 365, Infor GenAI / Coleman, Oracle AI Agent Studio
Industries: Manufacturing, Aerospace, Defense, Electronics
Written for: CIO

Discuss your SAP Joule environment See use cases

Every ERP vendor now has an AI story. SAP has Joule and the Business AI portfolio running on BTP AI Core. Microsoft has Copilot for Dynamics 365, wired into Dataverse and Azure OpenAI Service. Infor markets GenAI and Coleman AI capabilities across CloudSuite, reachable through ION. Oracle has AI Agent Studio and embedded Fusion AI features backed by OCI Generative AI. Each pitch sounds similar: ask a question in plain language, get an answer or a drafted action, inside the ERP you already run.

As a CIO evaluating this, the real question is rarely whether AI is useful, it clearly is for structured questions against clean master data. The real question is whether your organization's constraints, data residency, export control, customization depth, multi-ERP landscape, budget model, point toward the vendor's managed AI tier or toward a private stack you control. Those constraints differ enormously between a single-instance SAP shop on RISE and a defense supplier running SAP for finance, Infor LN for engineer-to-order, and Costpoint for government contracts.

Vendor copilots are optimized for the vendor's own transactions and their own cloud. They are usually strongest on documented, standard-configuration scenarios and weakest on the Z-tables, custom IDOs, bolt-on MES, and heavily configured workflows that most manufacturers actually run on after a decade of use. A private LLM grounded through retrieval on your specific schema, including your customizations, can answer questions the vendor copilot was never trained to handle, but someone has to build and maintain that grounding.

This page lays out the decision honestly: what each vendor copilot actually requires from a data and licensing standpoint, where a private LLM changes the calculus, and the questions to ask before committing budget either way. Netray builds the private-LLM side of this equation, ERPray for natural-language query and agents, custom RAG and fine-tuning where needed, so the comparison below is written to be useful even if you end up choosing the vendor path.

What usually gets in the way

The problems we hear most from cio teams running SAP Joule.

Vendor copilots often require a specific cloud or licensing tier

SAP Joule generally assumes S/4HANA Cloud or RISE with a BTP subscription; Microsoft Copilot for Dynamics 365 assumes Dataverse and an Azure OpenAI allocation; Infor's GenAI features lean on Infor OS and ION in the multi-tenant cloud. Shops that are on-prem, air-gapped, or on an older release often cannot turn the feature on at all without a migration project they did not budget for.

Cost scales with seats and consumption, not with value delivered

Copilot licensing is typically per named user or per Copilot Studio message/capacity unit, which makes the multi-year cost hard to forecast once adoption grows past the pilot group and easy for finance to push back on when the per-user math is compared to a flat infrastructure cost.

Generic training data means generic answers on your specific setup

Vendor copilots are strongest on out-of-the-box transactions. They struggle with your custom fields, bolt-on MES, non-standard approval chains, and the tribal-knowledge exceptions that make up a large share of what your ERP team actually gets asked about daily.

Security and export-control teams cannot get a straight answer on data flow

For ITAR, CUI, or customer-confidential BOM and pricing data, 'the vendor's AI cloud' is rarely a sufficient answer for a facility security officer or a CMMC assessor. Proving exactly where prompts, embeddings, and logs are processed and retained takes real documentation, and vendor copilots do not always make that documentation easy to get.

IT ends up comparing apples to oranges

A vendor copilot quote bundles licensing, support, and roadmap promises. A private-LLM build quote is a project cost plus a run-rate. Without a common framework, CIOs end up choosing based on which number was presented last rather than which one is actually lower over three years.

Where AI earns its place in SAP Joule

Each use case names the ERP objects it reads or writes, so your ERP team can judge the integration effort before anyone commits budget.

Natural-language ERP query

A planner or finance user asks a question in plain English instead of building a report or writing a query.

Touches: SAP Joule chat over CDS views and Fiori apps, or a private text-to-SQL layer over SuiteQL, ION API, or OData with a documented semantic model

Outcome: Vendor copilots answer well on standard views; a private layer grounded on your actual reporting views and custom fields answers the questions your team already emails to the ERP admin.

PO exception follow-up

Drafting supplier follow-up emails and flagging at-risk POs based on promise dates and receipt status.

Touches: PO/PR tables, supplier confirmation records, ASN data, ME2M/ME2N-style exception lists or the equivalent purchasing worklist in your ERP

Outcome: Either approach can draft the email; the difference is whether the exception logic understands your specific expedite rules and supplier tiers, which usually favors a grounded, configurable build.

Quality NCR and CAPA drafting

Turning a scanned inspection note or a shop-floor description into a structured NCR/CAPA record.

Touches: QM notifications, inspection lots, or an AS9100 quality module's NCR/CAPA tables, plus prior similar-defect records for pattern matching

Outcome: A vendor copilot rarely has quality-module depth out of the box; teams that need this usually build it as a custom agent regardless of which base ERP they run.

Custom object and Z-table awareness

Answering questions that touch heavily customized objects, custom IDOs in SyteLine, Z-tables in SAP, or bolt-on fields added over years of configuration.

Touches: Customer-specific tables, custom IDOs, BAdIs, user exits, and the mapping documentation that (hopefully) describes them

Outcome: This is the clearest build-favoring use case: vendor copilots were not trained on your customizations, and grounding a private model on your own schema documentation closes that gap directly.

Cross-system question spanning PLM, ERP, and MES

A single question that needs data from a PLM system, the ERP, and a shop-floor MES to answer accurately.

Touches: Teamcenter or Windchill change objects, ERP BOM/routing tables, and MES work-order and scrap records

Outcome: Vendor copilots are scoped to their own ERP; a private layer with connectors to each source can answer the cross-system question in one pass, which is usually the harder and more valuable case.

Air-gapped or classified-network deployment

Running an AI assistant on a network with no path to any public cloud, common for ITAR/CUI environments.

Touches: Read-only database replicas, ION or IDoc extracts staged inside the enclave, local vector store

Outcome: Vendor copilots that depend on a public cloud tenant are not an option here; this is a build-only use case by definition.

Multi-language shop-floor assistant

Operators asking work-instruction and traveler questions in their own language on the floor.

Touches: Work order/traveler records, routing steps, document management attachments (drawings, work instructions)

Outcome: Either approach can add multilingual UI; the deciding factor is usually whether shop-floor terminals can reach the vendor's cloud endpoint at all, which many plants restrict.

Reference architecture

Whichever path you take, the underlying reference architecture is similar: a connector layer into the ERP, a semantic or retrieval layer that grounds answers in your actual data and documentation, a model-serving layer, an agent/action layer with approval gates, and a governance layer that logs everything. The difference is who owns and can modify each layer, and where the model itself runs.

1
ERP connectors
Vendor copilots use the vendor's own APIs (ION, Dataverse, OData/CDS, OCI Fusion connectors). A private build uses the same APIs, plus direct database read replicas and file/document stores where the vendor API does not expose what you need.
2
Data and semantic layer
A documented mapping from raw tables and custom fields to business terms; this is the layer most vendor copilots leave thin for customizations, and the layer a private build invests in first.
3
Model serving
Vendor copilots call the vendor's own hosted model (Azure OpenAI, OCI Generative AI, SAP's BTP AI Core model library). A private build serves an open-weight model (Llama, Qwen, Mistral, Gemma, gpt-oss class) on infrastructure you control, on-prem or in a private cloud tenant.
4
Retrieval and agents
Both approaches can do retrieval-augmented answers and multi-step agent actions; the private build controls exactly which sources are indexed, including customer-confidential documents the vendor cloud may not be an acceptable home for.
5
Governance and audit
Every write-back action needs a human approval step and a full audit trail tied to the ERP's own user and role model, regardless of which side of build vs buy you choose; this layer is where most compliance failures actually happen.

Integration notes for your ERP team

SAP Joule runs through BTP AI Core and typically expects S/4HANA Cloud, Public or Private Edition, or RISE; pure on-prem ECC or classic S/4HANA on-prem has limited or no Joule availability today, confirm current entitlement before planning around it.
Microsoft Copilot for Dynamics 365 is built on Dataverse and Azure OpenAI Service; on-prem Business Central or older AX installations generally cannot use it without a Dataverse migration.
Infor's GenAI and Coleman AI capabilities are delivered through Infor OS and ION in the multi-tenant CloudSuite; SyteLine and LN customers on classic on-prem deployments should verify which features are actually licensed and reachable versus roadmap items.
Oracle AI Agent Studio and embedded Fusion AI features are built for Fusion Cloud ERP; EBS and JD Edwards customers evaluating Oracle's AI story should distinguish between Fusion-only features and anything genuinely available on their release.
A private build typically starts with the same official APIs (ION API, OData/CDS views, SuiteQL, AIS/Orchestrator for JDE) for anything low-risk, then adds read-replica database access for reporting-heavy or customization-heavy questions the API does not expose.
Whichever path you choose, put write-back actions behind an explicit approval workflow tied to your existing ERP role model; do not let either a vendor copilot or a private agent post transactions without a human in the loop during the first 6-12 months.
Budget separately for the semantic/documentation layer. This is the work of mapping custom fields and business terms to the underlying schema, and it is the single biggest driver of answer quality regardless of which model or vendor sits behind it.

Deployment options

Vendor copilot, vendor cloud

Standard-configuration shops with data that is contractually and legally fine to process in the vendor's cloud tier

Fastest to enable if you are already on the right subscription level (RISE, Dataverse, Infor OS multi-tenant, Fusion). Weakest on customizations, export-controlled data, and cross-system questions.

Private LLM, air-gapped on-prem

ITAR/CUI environments, classified or isolated networks, sites with no reliable outbound connectivity

The model, retrieval index, and ERP connectors all run inside your boundary on your own GPU hardware. This is not available from any vendor copilot today; it is a build-only path.

Private LLM, hybrid

Multi-site organizations where some ERPs and sites are cloud-friendly and others are not

Non-sensitive sites can use a lighter deployment while regulated sites keep everything on-prem, sharing the same semantic layer and connector patterns so the organization is not maintaining two unrelated AI stacks.

Compliance and data control

How the architecture supports your obligations. Certification and accountability stay with your organisation; the design keeps the evidence straightforward.

ITAR / EAR (deemed export)

Sending ITAR-controlled technical data to a public LLM API, including most vendor copilot cloud tiers, can constitute a deemed export to the operator's personnel. A private, on-prem or single-tenant deployment keeps the data inside your defined boundary and access-controlled by U.S. person status where required.

CMMC 2.0 / NIST SP 800-171

CUI processed by an AI system falls inside your assessment boundary. A private deployment lets you scope the AI system into your existing enclave rather than adding a new SaaS vendor to your System Security Plan and POA&M.

GDPR / data residency

For EU operations, confirm exactly which region the vendor's AI cloud processes and retains data in; several vendor copilot tiers still route through U.S. regions by default. A private deployment keeps the answer simple: your data stays in your chosen country.

Customer flow-down and NDA terms

Many aerospace and defense prime contracts prohibit sending program data to third-party cloud AI services without explicit approval. Review flow-down clauses before assuming a vendor copilot is contractually available for that program's data.

Where Netray fits

ERPray

If the deciding factors point toward a private path, ERPray is built specifically to answer natural-language questions and run agents grounded on your live ERP data, read-only by default, with the underlying query shown and ERP roles respected.

Custom build

For deep customization awareness, cross-system questions spanning PLM/MES, or air-gapped deployment, a bespoke RAG and agent build tailored to your specific schema and network constraints is usually the right scope.

How an engagement runs

Phase 1 . 2-3 weeks

Discovery

-Inventory of vendor copilot entitlements you already have or could license
-Data residency and export-control constraint map by site and program
-Short list of 3-5 use cases scored against build vs buy fit
-Rough three-year TCO comparison for the top option in each path

Phase 2 . 6-8 weeks

Pilot

-Working prototype on the chosen path against one ERP module or one vendor copilot trial
-Answer accuracy review against a held-out question set
-Security review covering data flow and audit logging
-Go/no-go recommendation with a documented rationale

Phase 3 . 8-12 weeks

Production

-Hardened deployment with approval workflows for any write-back action
-Role-based access mapped to existing ERP security
-Monitoring and cost tracking dashboard
-Runbook for the ERP and IT teams

Phase 4 . Ongoing

Scale

-Additional modules or ERPs onboarded onto the same semantic layer
-Quarterly cost and accuracy review
-Model refresh plan as open-weight or vendor models improve
-Documented decision log for future build vs buy calls as new use cases arise

Questions to ask any vendor, including us

A short list that separates real SAP Joule AI work from a chatbot demo.

Exactly which cloud region and legal entity processes and retains our prompts, and for how long?
Does this feature work on our current release and deployment model, or does it require a cloud migration we have not budgeted?
What happens to answer quality when the question touches a custom field, Z-table, or bolt-on system?
What is the true three-year cost per active user, including any capacity unit or message-based consumption charges?
Can we get a written answer on ITAR/CUI applicability from the vendor's compliance team, not just a marketing page?
Who owns the semantic layer and documentation if we switch models or vendors later?
What write-back actions does the copilot or agent take without human approval, and can that be turned off?
If we build privately, what is the realistic timeline and internal effort required versus turning on a vendor feature?

Frequently asked questions

Is a vendor AI copilot always cheaper than building a private one?

Not necessarily. Vendor copilots avoid an upfront integration project but scale with seats and consumption, which can exceed a flat infrastructure cost within two to three years for larger user bases. Run both as a three-year TCO, not a first-year license quote, before deciding.

Can we use SAP Joule or Copilot for D365 if we are ITAR-regulated?

It depends entirely on which cloud region and tenant configuration is in play, and on your specific technical data classification. Get a written statement from the vendor's compliance team for your exact deployment before assuming either public cloud AI feature is available for controlled data.

Do vendor copilots understand our custom fields and Z-tables?

Generally not well out of the box. Vendor copilots are trained and tuned on standard configurations and documented objects. Custom fields, bolt-on modules, and non-standard workflows usually need additional grounding work, which is effectively a smaller build project layered on top of the vendor feature.

Can we run a vendor copilot on-prem instead of in the vendor's cloud?

Mostly no. SAP Joule, Copilot for Dynamics 365, and Infor's GenAI features are built around the vendor's own cloud AI services and generally require the corresponding cloud or hybrid ERP tier. If a fully on-prem or air-gapped deployment is a hard requirement, a private build is currently the only path.

What is the middle ground between build and buy?

A common middle path is grounding a private retrieval layer on your ERP data while still calling a vendor-hosted or your own served open-weight model, giving you control over what data is indexed and where it lives without building every layer from scratch.

How long does a private build typically take before it is useful?

A scoped pilot answering questions against one ERP module can usually be working in six to eight weeks. Production hardening with approval workflows and role-based access typically adds another two to three months before broad rollout.

Does choosing a private build lock us out of future vendor AI features?

No. A well-designed semantic and connector layer is largely independent of which model or agent framework sits on top, so you can add a vendor copilot for standard transactions later while keeping the private layer for customization-heavy or regulated use cases.

Related guides

ERP AI Cost Guide

What ERP AI Actually Costs: A CFO's Guide

A CFO's guide to what ERP AI actually costs: GPU hardware, model licensing, integration and connector work, and realistic ongoing run-rate ranges.

ERP AI Buyer Guide

How to Choose an ERP AI Implementation Partner

A CIO checklist for picking an ERP AI implementation partner: the architecture questions to ask, red flags, pricing models, and what to demand in the SOW.

SAP Joule + on-prem alternative

SAP Joule or a Private LLM Beside SAP: An Honest Comparison

An honest comparison of SAP Joule and Business AI against a private, self-hosted LLM beside SAP: what each covers, where they overlap, and where CIOs run both.

RAG + SQL + permissions

A Private LLM Grounded on Your ERP Data

How a private LLM answers questions on your ERP data: RAG plus text-to-SQL, role-based permissions inherited from the ERP, and where each fits.

Agents + approval gates

AI Agents for ERP, Running On-Prem

A practical guide to on-prem AI agents for ERP: what they can safely automate, where human approval belongs, and how to design the guardrails.

Ask your ERP anything

Natural Language Query for ERP Data: Ask SAP, Infor, or Oracle a Question in Plain English

See how natural language query over SAP, Infor, Oracle, and NetSuite data works: grounded text-to-SQL, role-based permissions, and a visible audit trail.

Plan it with numbers

Free Tool

Talk it through with an engineer who knows SAP Joule

Bring one real question your team cannot answer from the ERP today. We will map the data path, the model, and where it runs, and tell you honestly if AI is the wrong tool for it.

Start the conversation See ERPray

All AI for ERP guides