SAP + private AI in Germany
AI for SAP in the German Mittelstand: On-Prem and DSGVO-Ready
Short answer
German Mittelstand manufacturers add AI to SAP S/4HANA or Business One by running an open-weight model on-prem or in a German or EU data center, grounding it in SAP data through OData services and CDS views, and logging every query for DSGVO and the Betriebsrat. That keeps production, quality, and HR data inside the company's own boundary while giving planners, quality engineers, and controllers a natural-language way to work with SAP instead of another dashboard.
- ERP
- SAP S/4HANA, SAP ECC 6.0, SAP Business One
- Industries
- Manufacturing, Automotive Supply, Machinery and Equipment
- Written for
- CIO / IT-Leiter
Most Mittelstand SAP shops did not choose S/4HANA or Business One as a blank slate. They carry twenty years of custom Z-transactions, plant-specific variant configuration, and a Basis team that is often two people or an outsourced partner. Any AI initiative has to work with that reality, not against it: reading SAP through the interfaces that already exist, without demanding a re-implementation project first.
Two constraints show up in almost every German conversation before the technical ones do. The first is the Betriebsrat: under Section 87 of the Betriebsverfassungsgesetz, a works council has co-determination rights over any tool that is technically capable of monitoring employee performance or behavior, and an AI copilot that reads order entry or shop floor data can look like exactly that unless it is scoped and documented carefully from the start. The second is DSGVO, which German data protection authorities (Landesdatenschutzbehorden) interpret strictly, particularly around sending personal data - HR infotypes, customer contacts, supplier contacts - to a processor outside the EU.
This is why the practical AI conversation in the Mittelstand is rarely 'which copilot' and almost always 'where does it run and who can see the prompts.' A cloud LLM API answers the first question with a shrug and the second with a data processing agreement that a DPO has to review line by line. An open-weight model served on GPUs the company owns, or in a German or EU-based private cloud, answers both questions before anyone asks.
The upside is real but has to be sized to Mittelstand economics: a 200 to 2,000 employee manufacturer does not have a data science team, and the AI business case has to survive a capital committee that is used to scrutinizing SAP add-on spend closely. The mechanism that works is narrow, well-defined use cases - MRP exception triage, quality notification drafting, purchase order follow-up - each with a clear before/after on a task the SAP team already does today, not a platform promise.
What usually gets in the way
The problems we hear most from cio / it-leiter teams running SAP S/4HANA.
Betriebsrat co-determination risk
Any tool that could be used to monitor individual employee performance triggers Section 87 BetrVG co-determination rights. Rolling out an SAP copilot without a Betriebsvereinbarung (works council agreement) in place risks a grievance that stalls the whole project, not just delays it.
DSGVO exposure from cloud LLM APIs
SAP order, HR, and supplier data routed to a US-hosted model API creates a third-country transfer question that most Mittelstand legal teams are not equipped to resolve quickly, and German DPAs have shown they will ask hard questions about it.
Thin SAP Basis and ABAP bench
Many Mittelstand IT departments run SAP with two to five people, often supplemented by an outsourced Basis partner. There is no capacity to build a custom AI integration layer from scratch, so anything added has to plug into existing OData services and authorization objects, not require new ABAP development as a precondition.
Reporting fragmented across ECC/S4, BW, and Excel
Plant managers and controllers still pull numbers into Excel because ad hoc questions ('how many open orders for this customer this week') do not map cleanly to an existing BW query, and the two-person BI team cannot build a new report for every question.
Budget discipline tighter than at DAX-listed peers
Mittelstand capital committees, often family-owned or Gesellschafter-controlled, expect a specific mechanism for payback, not a platform vision. AI spend competes directly against tooling, headcount, and plant capex in the same review.
Where AI earns its place in SAP S/4HANA
Each use case names the ERP objects it reads or writes, so your ERP team can judge the integration effort before anyone commits budget.
MRP exception triage
Planners ask, in plain language, which materials in MD04/MD07 need attention today instead of scrolling the full stock/requirements list plant by plant.
Touches: MD04, MD07 exception lists, planned orders, purchase requisitions via SAP Gateway OData
Outcome: cuts the time to work the daily exception list from a couple of hours to under one, with the reasoning shown alongside the recommendation
Quality notification and 8D drafting
A quality engineer describes a defect and the assistant drafts the QM notification and a first-pass 8D structure, pulling the relevant material, batch, and supplier history from SAP QM.
Touches: QM01/QM02 notifications, batch and inspection lot data, vendor master
Outcome: cuts the time to a usable first 8D draft from most of a day to under an hour for routine defects
Sales order entry copilot
Reviews a draft VA01 order against plant, material master, and customer-specific pricing conditions before it is saved, flagging the kind of missing data that normally surfaces as a delivery block days later.
Touches: VA01/VA02, ATP checks, condition records, customer master
Outcome: fewer orders block in delivery processing for missing plant or pricing master data
Purchase order follow-up
Drafts and tracks routine supplier follow-up on open POs and confirmations, escalating only genuine exceptions like a missed confirmation date to the buyer.
Touches: ME21N, ME2N open PO reports, vendor confirmations
Outcome: buyers spend follow-up time on the exceptions that matter instead of routine status chasing
Plant maintenance notification quality
Helps technicians write consistent, well-coded breakdown notifications on a phone or tablet at the machine, which feeds cleaner downtime and MTTR reporting.
Touches: PM notifications (IW21/IW28), damage and cause codes, equipment master
Outcome: more consistent notification coding, which improves the accuracy of OEE and MTTR reporting without extra training overhead
Month-end variance commentary
Gives controllers a first draft of cost center variance commentary based on FI/CO actuals versus plan, which they edit rather than write from a blank sheet.
Touches: FI/CO actual and plan line items, cost center reports
Outcome: shortens the drafting portion of month-end variance commentary, leaving more of the close window for review
Natural-language questions over CDS views and BW
Lets plant managers ask ad hoc questions directly instead of routing every one-off request to the BI team, with the underlying query shown for verification.
Touches: CDS views, BW queries, replicated reporting tables
Outcome: fewer ad hoc report requests reach the two-person BI team, and the ones that do are the genuinely new ones
Reference architecture
The pattern that works in a Mittelstand SAP environment keeps the SAP system itself untouched and adds a layer alongside it: read access through standard interfaces, a model that runs where the company controls it, and a governance layer that gives IT and the Betriebsrat visibility into exactly what the AI can see and do.
- 1
SAP connectors
OData services via SAP Gateway, BAPI/RFC calls, and IDoc consumption for asynchronous events, all read-only unless a write-back is explicitly approved per use case.
- 2
Data and semantic layer
CDS views and a small set of replicated or cached tables give the model a stable, documented shape of the SAP data, plus a vector index over quality manuals, work instructions, and SOPs.
- 3
Model serving
An open-weight model (Llama, Qwen, Mistral, or Gemma class) served with vLLM or Ollama on GPUs the company owns, or in a German or EU-based private cloud instance sized to the workload.
- 4
Retrieval and agents
Retrieval-augmented generation grounds answers in the current SAP data and documents; any agent that proposes a write (a PO follow-up email, a QM notification draft) stops for human approval before it touches SAP.
- 5
Governance and audit
Every prompt, retrieved record, and response is logged against the user's SAP authorization profile, feeding both IT audit needs and the DSGVO Verzeichnis von Verarbeitungstatigkeiten (records of processing activities).
Integration notes for your ERP team
- Read access to SAP S/4HANA or ECC through SAP Gateway OData services, avoiding custom ABAP development as a precondition for the first use case.
- BAPI/RFC calls for transactional data not exposed via existing OData services, using the same RFC user and authorization profile pattern IT already manages.
- IDoc consumption for near-real-time events such as order changes or PO confirmations where polling OData is not fast enough.
- Authorization mapping so the assistant only surfaces what a given SAP user could already see through their existing SAP authorization objects, no privilege escalation via the AI layer.
- SSO via SAP Identity Authentication Service or the company's existing Active Directory/LDAP, so users authenticate the same way they do for SAP GUI or Fiori.
- SAP Business One integration via the Service Layer (OData-based) or DI API for smaller Mittelstand entities on B1 rather than S/4HANA.
- Any write-back to SAP goes through the normal change process: a human reviews and confirms in the assistant, and the transaction is posted exactly as if a user had entered it, preserving the standard SAP change log.
Deployment options
Air-gapped on-prem
Automotive and machinery suppliers with customer security addenda or an OT-adjacent plant network
The model, retrieval index, and SAP connectors run entirely inside the company's own network with no outbound internet path, satisfying customers who require it contractually even without a specific regulation demanding it.
Private or sovereign EU cloud
Mittelstand firms without in-house GPU capacity or a team to operate one
The same stack runs in a German or EU-based private cloud instance dedicated to the customer, keeping data inside German or EU jurisdiction and simplifying the DSGVO transfer question without requiring the company to buy and run its own hardware.
Hybrid
A head office with several plants, some with tighter security requirements than others
Sensitive plants run air-gapped locally while head office and less sensitive sites use a shared private cloud instance, with the governance layer giving one consistent view across both.
Compliance and data control
How the architecture supports your obligations. Certification and accountability stay with your organisation; the design keeps the evidence straightforward.
DSGVO / GDPR
On-prem or EU-hosted deployment keeps personal data (HR, customer, supplier records) inside the company's boundary or a documented EU processor, avoiding the third-country transfer analysis that a US-hosted API forces.
Section 87 BetrVG (Betriebsrat co-determination)
Use cases are scoped and documented before rollout so the works council can see exactly what data the assistant touches and that it is not used for individual performance monitoring, supporting a Betriebsvereinbarung rather than a dispute.
BSI C5
Where a private cloud option is used, the company can request the provider's BSI C5 attestation directly and evaluate it as part of vendor selection, the same as any other cloud service.
NIS2
For larger Mittelstand firms that fall into scope as important or essential entities, the same audit logging and access control layer that supports DSGVO doubles as evidence for NIS2 supply chain and incident reporting obligations.
IATF 16949 traceability
For automotive suppliers, the audit trail on AI-assisted quality notifications and 8D drafts is designed to be reviewable by the same auditors who check SAP QM traceability today, not a separate, opaque log.
Where Netray fits
ERPray
Natural-language question answering and dashboards over S/4HANA, ECC, or Business One data fit the recurring 'why is this number what it is' questions plant managers and controllers ask most often.
Custom build
Use cases like the order entry copilot or PM notification assistant need workflow-specific logic and a tighter SAP write-back path than a general question-answering layer provides.
How an engagement runs
Phase 1 . 2-3 weeks
Discovery
- -Review of current SAP authorization structure and the OData/BAPI surface already exposed
- -Betriebsrat briefing pack outlining scope, data touched, and what the assistant will not do
- -Shortlist of two to three use cases ranked by effort and expected time saved
- -GPU/private-cloud sizing estimate based on user count and use case
Phase 2 . 6-8 weeks
Pilot
- -One use case live in read-only mode with a defined user group
- -Model evaluation against real SAP queries, tuned for accuracy on the company's own data
- -Draft DSGVO Verzeichnis von Verarbeitungstatigkeiten entry for the pilot
- -User feedback loop and adoption metrics reviewed with IT and the pilot group
Phase 3 . 6-10 weeks
Production
- -Hardened deployment with role-based access tied to existing SAP authorizations
- -Full audit logging wired into existing SIEM or log management
- -Signed Betriebsvereinbarung covering the production scope
- -Runbook for IT covering model updates, monitoring, and incident response
Phase 4 . Ongoing
Scale
- -Additional use cases added from the original shortlist based on pilot results
- -Rollout to additional plants or legal entities under the hybrid model where relevant
- -Refresher training for new SAP users joining the assistant's scope
- -Quarterly review of model performance and any newer open-weight model worth evaluating
Questions to ask any vendor, including us
A short list that separates real SAP S/4HANA AI work from a chatbot demo.
- Where does the model physically run, and can we see the server or the private cloud contract ourselves?
- Has the Betriebsrat been briefed and is a Betriebsvereinbarung part of the plan before go-live, not after?
- Does the assistant respect our existing SAP authorization objects, or does it need a separate, broader service user?
- Can we export the full query and response log for DSGVO records of processing and for our own audit?
- If we stop working with the vendor, do we keep the models, the connectors, and the configuration, or does the whole thing stop working?
- How is data from SAP HCM infotypes handled, and can it be excluded entirely from the assistant's scope?
- What is the fallback if the on-prem GPU or private cloud instance is unavailable for a day?
- Can the entire stack run with no outbound internet connection, and has that actually been tested, not just claimed?
Frequently asked questions
Do we need to involve the Betriebsrat before deploying AI on SAP?
In most cases, yes. Under Section 87 BetrVG, a works council has co-determination rights over tools technically capable of monitoring employee performance or behavior, and an SAP copilot that touches order entry, quality, or maintenance data can fall into that category. Scoping the use case, documenting what data it touches, and bringing the Betriebsrat in early, with a Betriebsvereinbarung as the outcome, avoids a dispute that can stall the whole project.
Can we run AI on SAP without sending data to the United States?
Yes. Open-weight models such as Llama, Qwen, Mistral, or Gemma class models can be served with vLLM or Ollama on GPUs the company owns, or hosted in a German or EU-based private cloud instance. Neither path requires sending SAP data to a US-hosted model API, which removes the third-country transfer question that DSGVO would otherwise raise.
Do we need SAP BTP to add AI to S/4HANA?
No. BTP is one option for building integrations, but the same result is achievable directly against the on-prem or private-cloud S/4HANA or ECC system through SAP Gateway OData services, BAPI/RFC, and IDoc, without a BTP subscription as a prerequisite. BTP becomes relevant only if the company already uses it for other integration work and wants to reuse that investment.
How much SAP customization does this require?
Little to none for the first use cases. The interfaces most SAP systems already expose - OData services, BAPI/RFC, IDoc - are usually sufficient to read the data needed for MRP triage, quality drafting, or PO follow-up. Custom ABAP development becomes relevant only for write-back use cases with heavy custom logic, and even then it follows the same transport process as any other SAP change.
Is this realistic for a 300-employee Mittelstand manufacturer, not just a DAX-listed group?
Yes, and the deployment model is often simpler for a smaller company: fewer legal entities, a smaller set of authorization roles, and a private-cloud option that avoids buying GPU hardware outright. The scope should still start with one or two use cases with a clear before/after, sized to what a capital committee used to reviewing SAP add-on spend will actually approve.
What happens to data from SAP HR (HCM)?
It should be excluded from the assistant's scope by default unless there is a specific, documented use case for it, since HR infotypes carry the highest DSGVO sensitivity and the strictest co-determination scrutiny. Most of the practical value in a Mittelstand SAP environment - planning, quality, procurement, maintenance, finance close - does not require touching HCM data at all.
How is this different from SAP's own Joule or Business AI roadmap?
Joule runs as part of SAP's cloud offering and is a reasonable fit if the company is already moving to RISE with SAP and comfortable with data processed in SAP's cloud. An on-prem or private-cloud private LLM is the alternative when the company wants to stay on-prem, keep model choice open, or extend AI to non-SAP systems like a document store or MES using the same governance layer.
Related guides
AI for SAP S/4HANA, Running On-Prem or in Your Private Cloud
Run AI on SAP S/4HANA without sending ERP data to a public API. On-prem and private-cloud architecture, CDS views, OData, and honest deployment trade-offs.
SAP Business One + AIAI for SAP Business One, Without Sending Your Books to a Public API
Add AI to SAP Business One without sending customer data or financials to a public API. Service Layer and DI API integration, sized for an SMB budget.
GDPR + AI on ERP dataBuilding GDPR-Compliant AI on Top of Your ERP
Design AI on ERP data that satisfies GDPR: lawful basis, DPIA, data minimisation, and an on-prem architecture that avoids the Schrems II transfer problem.
EU AI Act + ERP AI systemsEU AI Act Compliance for AI Built on Your ERP
How the EU AI Act's risk classes, documentation, and timeline apply to AI copilots and agents on ERP systems, and what compliance officers need in place.
SAP Joule + on-prem alternativeSAP Joule or a Private LLM Beside SAP: An Honest Comparison
An honest comparison of SAP Joule and Business AI against a private, self-hosted LLM beside SAP: what each covers, where they overlap, and where CIOs run both.
Swiss manufacturing AIAI for Swiss Manufacturers Without Sending Data Abroad
Ground AI on your SAP, Infor, or Oracle ERP without data leaving Switzerland, matching revFADP and the confidentiality standard Swiss customers expect.
Plan it with numbers
Sovereign AI Readiness Assessment
Score your organization across eleven dimensions of sovereign AI readiness, from data residency and model provenance to cleared personnel and air-gapped operations.
Free ToolGDPR Compliance Cost Calculator
Estimate your fully loaded annual GDPR cost across DSAR fulfillment, DPIA workload, DPO staffing, and privacy tooling, then see the cost per 1,000 data subjects.
Free ToolAI Governance Maturity Assessment
Score your AI governance across policy, inventory, risk classification, data handling, monitoring, and executive oversight, and get a banded improvement roadmap.
GuideERP GDPR Data Protection Compliance Guide
Achieve GDPR compliance in ERP systems with data mapping, consent management, right-to-erasure implementation, and data protection impact assessments.
GuideEU AI Act Implications for On-Prem AI Deployments
EU AI Act implications for on-prem deployments: risk tiers, high-risk obligations, and how self-hosted models simplify enterprise compliance.
GuideERP Cloud Compliance and Regulatory Guide
Ensure ERP cloud compliance with SOX, GDPR, HIPAA, and industry regulations. Covers data residency, audit trails, encryption, and compliance automation strategies.
Talk it through with an engineer who knows SAP S/4HANA
Bring one real question your team cannot answer from the ERP today. We will map the data path, the model, and where it runs, and tell you honestly if AI is the wrong tool for it.