SAPBuyer Guide

SAP AI Buyer Guide

SAP AI Consulting for On-Prem and Private Deployments

Short answer

SAP AI consulting for on-prem or private-cloud customers needs to answer one question honestly: does the proposed architecture keep S/4HANA or ECC data inside your boundary, or does it depend on SAP Business AI, Joule, or a third-party model API that your compliance posture rules out. This guide covers the OData, BAPI/RFC, and CDS mechanics a serious proposal should reference, and the questions to ask any SAP-focused AI partner before committing.

ERP: SAP S/4HANA, SAP ECC 6.0, SAP Business One
Industries: Manufacturing, Aerospace, Defense, Electronics
Written for: CIO

Discuss your SAP S/4HANA environment See use cases

SAP's own generative AI push, Joule and the broader SAP Business AI portfolio, is built around SAP's cloud and RISE with SAP. That is a reasonable path for organizations already committed to that model. It is the wrong default for a defense supplier, an aerospace tier-2, or any manufacturer whose data cannot leave an on-prem or private-cloud boundary, whether the instance is S/4HANA on-prem, a private-cloud RISE deployment, or ECC 6.0 running out mainstream maintenance toward the 2027 deadline.

For those organizations, an AI consulting partner needs a different default: private model serving, grounded retrieval over SAP data via OData, BAPI/RFC, or CDS views, and an honest answer about which parts of SAP Business AI they can still use versus which parts they cannot. A partner who pitches Joule as a drop-in answer without checking your data residency requirements has not done the diligence the engagement needs.

SAP's own vocabulary is a useful filter here. A partner who talks about 'connecting to SAP' in the abstract, without naming a transaction code, an IDoc message type, a BAPI, or a CDS view, likely has not built against a real S/4HANA or ECC instance before. The mechanics matter because they determine what is actually feasible on your timeline and your compliance posture.

This guide sets out the architecture a serious SAP AI proposal should describe, the honest trade-off between Joule/SAP Business AI and a private layer, and the specific questions to put to any SAP-focused consulting partner, whether pitched by an AI-native shop or a traditional SAP implementation partner adding AI to its offering.

What usually gets in the way

The problems we hear most from cio teams running SAP S/4HANA.

Joule pitched as a universal answer

SAP's Joule assumes participation in SAP's cloud AI hub. For on-prem ECC, on-prem S/4HANA, or any instance where data cannot reach SAP's cloud, it is not available in the same way, and a partner should say so plainly.

Generic connectors that ignore CDS views and authorization objects

SAP's authorization model (roles, authorization objects, org-level restrictions) is granular by design. An AI layer that queries around it rather than through it recreates a permissions gap SAP spent decades closing.

No plan for the ECC-to-S/4HANA migration timeline

With ECC 6.0 mainstream maintenance winding down toward 2027, an AI project built without accounting for a coming migration risks a rebuild the moment the underlying system changes.

BTP proposed as the only path

SAP Business Technology Platform is a legitimate option for some customers, but it is not the only way to add AI to SAP, and a partner should present self-hosted alternatives with the same seriousness.

Vague statements about 'SAP-certified' AI

SAP certification programs exist for specific products and partners; a general claim of being 'SAP-certified' for AI work should be verified against SAP's actual partner directory, not taken at face value.

Where AI earns its place in SAP S/4HANA

Each use case names the ERP objects it reads or writes, so your ERP team can judge the integration effort before anyone commits budget.

Natural-language MD04 exception triage

A planner asks which MRP exceptions matter most today and gets a grounded, prioritized answer instead of scrolling MD04 manually.

Touches: MD04 stock/requirements list, planned orders, PP/MRP tables via OData or BAPI

Outcome: Cuts the time spent triaging MRP exceptions from a manual MD04 review to a prioritized shortlist.

QM notification and 8D drafting

A quality engineer gets a first-draft 8D or CAPA response grounded in the actual QM notification and inspection lot data, for review and edit, not blind acceptance.

Touches: QM notifications, inspection lots, QM01/QM02 transactions, CAPA records

Outcome: Reduces drafting time on routine notifications so quality engineers spend more time on investigation than documentation.

VA01/ME21N order status Q&A

Sales and procurement teams ask plain-language questions about sales order or purchase order status and get an answer sourced from the live SAP data with the query shown.

Touches: VA01/VA02 sales orders, ME21N/ME22N purchase orders, IDoc status

Outcome: Reduces routine status-check tickets to the ERP or planning team.

PM/EAM work order and failure-code assistance

Maintenance technicians get plain-language help finding the right work order, failure code, or equipment history without navigating PM transaction codes.

Touches: PM notifications, work orders, equipment master, failure code catalogs

Outcome: Speeds up work order documentation, particularly for less SAP-fluent shop floor maintenance staff.

Private retrieval layer alongside BTP

For customers already using SAP BTP for other integration, a private retrieval and model-serving layer runs alongside it for the specific data classes BTP's own AI hub cannot process.

Touches: BTP integration suite, CAP model, SAP AI Core (for non-sensitive workloads), private model serving for sensitive workloads

Outcome: Lets you use BTP where it fits and keep sensitive workloads private, rather than an all-or-nothing platform decision.

ECC-to-S/4HANA migration knowledge capture

During a migration project, an AI layer indexed against custom Z-transactions and configuration helps the project team answer 'what does this customization do' faster.

Touches: Z-transactions, custom tables, configuration documentation, IDoc mappings

Outcome: Shortens the discovery phase of a migration by giving consultants a faster way to understand existing customizations.

Business One AI for SMB manufacturers

For smaller manufacturers on SAP Business One, a lighter-weight private layer answers order, inventory, and financial questions without the BTP/S4 overhead a larger deployment assumes.

Touches: SAP B1 Service Layer, HANA/SQL database views

Outcome: Brings grounded question-answering to a segment SAP's larger AI investments generally do not prioritize.

Reference architecture

A serious SAP AI proposal for on-prem or private deployment should name OData, BAPI/RFC, IDoc, or CDS views specifically, and state plainly where it overlaps with or diverges from SAP Business AI and BTP.

1
SAP connectors
OData services, BAPI/RFC calls, or CDS views with a defined communication user and authorization profile, scoped to the specific transactions and tables in play.
2
Data and semantic layer
Mapping from SAP field and transaction codes (material master fields, movement types, order status codes) to business terms, respecting company code and plant-level scoping.
3
Model serving
Open-weight models served on customer GPUs or in a private/sovereign cloud, explicitly distinguished from SAP AI Core, Joule, or any third-party API in the proposal.
4
Retrieval and agents
RAG over SAP transactional data plus documents (quality records, spec sheets), with any write-back proposed as a draft routed through SAP's own workflow (release strategies, approval hierarchies).
5
Governance and audit
Logging aligned to SAP's authorization objects and org-level restrictions, so the AI layer's access mirrors, rather than bypasses, existing SAP security.

Integration notes for your ERP team

Ask whether the partner uses OData services, BAPI/RFC, or direct CDS view access, and why, since the choice affects performance and maintainability.
Confirm the communication user's authorization profile is scoped to only the transactions and company codes in play, reviewed on a schedule.
For IDoc-based integration, ask how the partner handles IDoc status monitoring and error handling for the AI layer's own traffic.
Clarify how custom Z-tables and Z-transactions are handled, since a generic connector will not know about them out of the box.
Ask directly which parts of SAP Business AI, Joule, or BTP the proposal assumes versus explicitly avoids, and why.
For ECC customers, ask how the connector layer is designed to survive an eventual S/4HANA migration.
Request the actual network path for any component that could touch SAP's cloud services, even indirectly through BTP.

Deployment options

Air-gapped on-prem

Defense suppliers and A&D manufacturers on S/4HANA on-prem or ECC with data that cannot reach any cloud

Private LLM and retrieval stack on customer infrastructure, connected via OData/BAPI over the internal network, with no path to SAP's cloud AI services.

Private or sovereign cloud

Manufacturers on RISE with SAP private cloud or a self-managed private cloud wanting central AI management

A dedicated tenancy under your control, connected to S/4HANA via the same integration patterns, keeping data out of SAP's shared multi-tenant AI hub.

Hybrid alongside SAP BTP / Business AI

Customers on RISE public cloud or S/4HANA Cloud who want Joule for general use cases but need privacy for sensitive data

Joule and SAP Business AI handle general, lower-sensitivity workloads; a private layer handles the data classes (export-controlled, CUI, unreleased engineering) that cannot go through SAP's cloud AI hub.

Compliance and data control

How the architecture supports your obligations. Certification and accountability stay with your organisation; the design keeps the evidence straightforward.

ITAR / export control

Confirm the connector and model path keep technical data inside your boundary; if SAP Business AI or Joule is in scope for any workload, verify explicitly which data classes are excluded from it.

CMMC 2.0 / NIST SP 800-171

Verify the AI layer's access to S/4HANA or ECC inherits your existing authorization objects and org-level restrictions rather than a broader service account.

GDPR / EU AI Act

For EU operations, confirm where embeddings, logs, and any cached SAP data physically reside, and how the system's risk classification under the EU AI Act is documented.

SOX / segregation of duties

Ensure any AI-assisted financial posting or approval respects SAP's existing release strategies and segregation-of-duties rules rather than introducing a bypass.

Where Netray fits

ERPray

For grounded natural-language question-answering, dashboards, and agents over S/4HANA, ECC, or Business One, evaluate proposals against a purpose-built, architecture-agnostic ERP AI layer with read-only defaults and visible queries.

Custom build

For bespoke agents tied to specific SAP workflows (an 8D drafting assistant, a Z-transaction-aware tool), the evaluation criteria in this guide still apply to a custom SAP-specific proposal.

How an engagement runs

Phase 1 . 2-3 weeks

Discovery

-Inventory of relevant transactions, Z-objects, and authorization profiles
-Explicit statement of Joule/SAP Business AI overlap and exclusions
-Architecture proposal naming OData/BAPI/CDS specifically
-Fixed-fee pilot scope

Phase 2 . 6-8 weeks

Pilot

-Read-only deployment against a defined transaction set
-Accuracy test against SAP-specific known-answer questions
-Authorization object and org-level scoping validation
-Go/no-go review

Phase 3 . 8-12 weeks

Production

-Hardened OData/BAPI connector with monitoring
-Runbook for CDS/Z-object mapping updates
-Named internal SAP Basis or functional owner
-First approved write-back use case, if in scope

Phase 4 . Ongoing

Scale

-Rollout to additional plants or company codes
-Update plan tied to SAP support package and S/4HANA release cadence
-Quarterly accuracy review
-Internal capability handoff

Questions to ask any vendor, including us

A short list that separates real SAP S/4HANA AI work from a chatbot demo.

Does your default architecture depend on SAP Joule, SAP Business AI, or BTP, and what happens if our data cannot go through them?
Which specific OData services, BAPIs, or CDS views have you actually built against before?
How is the communication user's authorization profile scoped, and who reviews it?
What is your plan if we migrate from ECC to S/4HANA during or after this engagement?
How do you handle our custom Z-transactions and Z-tables?
Where does the model physically run, and can you show the network diagram?
What SAP-specific projects, by name if permitted, have you delivered before?

Frequently asked questions

Is Joule enough, or do we need a private AI layer?

If you are already on RISE with SAP public cloud or S/4HANA Cloud and your data can legally and contractually go to SAP's cloud AI hub, Joule may cover a meaningful share of your use cases. On-prem makes sense when data residency, export control, or contractual restrictions rule out SAP's cloud path, or when you need capability outside what Joule currently covers.

Does SAP AI consulting require a SAP-certified partner?

For work that touches SAP's own cloud AI products (Joule configuration, BTP AI Core), SAP partner status can matter for support and licensing. For a private, self-hosted AI layer built against OData/BAPI/CDS, formal SAP certification is less critical than demonstrated hands-on experience with your specific SAP version; verify any certification claim directly with SAP's partner directory.

How does the 2027 ECC maintenance deadline affect an AI project now?

Building a private AI layer on ECC now is not wasted work if the connector and mapping layer are designed to be reusable; the OData/BAPI patterns used against ECC largely carry forward to S/4HANA. Ask any partner directly how they design for that continuity rather than assuming a rebuild.

Can a private AI layer coexist with BTP and Joule?

Yes. A common pattern is Joule and SAP Business AI for general, lower-sensitivity workloads, with a private, self-hosted layer for the data classes that cannot go through SAP's cloud AI hub. The split should be defined by data classification, documented explicitly in the architecture proposal.

What SAP data is typically too sensitive for a cloud AI hub?

Export-controlled technical data (ITAR), CUI under a DoD contract, unreleased engineering BOMs, and in some cases supplier pricing under contractual confidentiality are common categories manufacturers keep out of any shared cloud AI service, SAP's included.

How long does a first SAP AI pilot take?

A focused pilot against one transaction set (order status, MD04 exceptions, or QM notifications) with a defined user group typically runs 6-8 weeks after a 2-3 week discovery phase, assuming the partner has real OData/BAPI experience. Longer timelines without a defined checkpoint usually mean the partner is still learning your SAP configuration.

What should we own at the end of the engagement?

At minimum, you should own the connector configuration, prompt library, and any fine-tuned artifacts specific to your data, with the ability to run the system without the vendor if needed. Confirm this in writing before signing, since some proposals bundle these as licensed rather than owned.

Related guides

ERP AI Buyer Guide

How to Choose an ERP AI Implementation Partner

A CIO checklist for picking an ERP AI implementation partner: the architecture questions to ask, red flags, pricing models, and what to demand in the SOW.

SAP S/4HANA + private AI

AI for SAP S/4HANA, Running On-Prem or in Your Private Cloud

Run AI on SAP S/4HANA without sending ERP data to a public API. On-prem and private-cloud architecture, CDS views, OData, and honest deployment trade-offs.

SAP Joule + on-prem alternative

SAP Joule or a Private LLM Beside SAP: An Honest Comparison

An honest comparison of SAP Joule and Business AI against a private, self-hosted LLM beside SAP: what each covers, where they overlap, and where CIOs run both.

SAP BTP + self-hosted AI

SAP BTP and a Private LLM: Where the Generative AI Hub Fits and Where It Doesn't

Where SAP BTP's Generative AI Hub fits and where a self-hosted LLM belongs instead. CAP, Integration Suite, and an architecture decision framework.

ERP AI Cost Guide

What ERP AI Actually Costs: A CFO's Guide

A CFO's guide to what ERP AI actually costs: GPU hardware, model licensing, integration and connector work, and realistic ongoing run-rate ranges.

Vendor copilots vs private AI

Build vs Buy: Should You Use Your ERP Vendor's AI Copilot, or Build Your Own?

SAP Joule, Copilot for D365, Infor GenAI, Oracle AI Agent Studio, or a private LLM on your own data. A CIO framework for the build vs buy decision, with real trade-offs.

Plan it with numbers

Free Tool

Talk it through with an engineer who knows SAP S/4HANA

Bring one real question your team cannot answer from the ERP today. We will map the data path, the model, and where it runs, and tell you honestly if AI is the wrong tool for it.

Start the conversation See ERPray

All AI for ERP guides