# How to add AI to SAP ECC 6.0 on-prem without moving data to a cloud

> Source: https://netray.co/erp-help/private-ai-layer-sap-ecc-on-prem-architecture
> Publisher: Netray (netray.co), AI and integration engineering for ERP and manufacturing systems.
> Section: ERP Help

## Short answer

You add AI to SAP ECC 6.0 by leaving ECC where it is and building a read-only layer beside it: RFC-enabled function modules and BAPIs for live lookups, an SLT or database replica for bulk analytics, a retrieval index over documents, and an open-weight model served on your own GPUs. SAP's authorization objects are enforced in the tool layer so users only see what they could see in the GUI. Writes go back only through BAPIs after human approval.

## Kind

advanced

## System

sap

## Erp

SAP ECC 6.0

## Module

Integration / Private AI

## Query

how to add AI to SAP ECC 6.0 on-prem

## Alternate Queries

- SAP ECC AI integration without cloud
- run a local LLM on SAP ECC data
- SAP ECC private AI assistant architecture
- SAP ECC 6.0 chatbot on-premise RFC BAPI

## Applies To

SAP ECC 6.0 (any enhancement package) on Oracle, SQL Server, DB2, MaxDB, ASE or HANA (Suite on HANA). Gateway-based steps need SAP NetWeaver Gateway (embedded from NetWeaver 7.40, add-on before that).

## Steps

### Build the private AI layer on SAP ECC 6.0

**Items:**

Inventory the interfaces you already have: NetWeaver release and enhancement package, whether SAP Gateway is active (transaction /IWFND/MAINT_SERVICE), whether SLT is licensed and installed, and which RFC destinations and technical users exist (SM59, SU01).

Choose the read path per question type. Live single-document lookups go through BAPIs or a small custom RFC-enabled function module. Bulk and analytical questions go to a replica fed by SLT or by your existing database replication, limited to the tables you approve.

Create a dedicated technical user for the AI layer with only S_RFC for the function groups it needs plus display-level authorizations. Prefer per-user identity (SNC with SSO, or trusted RFC) for any live call so the SAP check runs as the real person.

Load the approved tables into a reporting store (PostgreSQL or a column store on your own servers) with a column allow-list. Leave out HR infotypes, bank details and anything your data classification marks restricted.

Build the retrieval index over attachments and documents: Generic Object Services attachments, ArchiveLink or content-server documents, DMS document info records (table DRAW), plus SOPs and drawings from file shares. Store the owning object key (plant, company code, purchasing org) with every chunk.

Run a tool server (an MCP server or a plain internal API) that exposes named, parameterised read actions such as get_po_status or open_po_lines. The model never receives a database connection or free-form SQL against production.

Serve an open-weight model with vLLM or Ollama on local GPUs, log every question, tool call and returned row to an append-only audit store, and start with a pilot group before opening it to all users.

## Sections

### The supported ways to read ECC data

**Body:**

ECC sits on an ABAP application server with the data in a relational database, but the supported path to that data is not a direct database login. SAP's licence terms and its own notes restrict direct reads of application tables from third-party tools, so confirm your contract before you point a reporting tool at production tables. The sanctioned options are RFC (including BAPIs and custom RFC-enabled function modules), IDocs and ALE for event-style outbound data, OData services through SAP Gateway, and replication through SLT.

For lookups, BAPIs return business objects with their authorization checks applied inside the ABAP code. BAPI_PO_GETDETAIL1 returns a purchase order header and items, and similar read BAPIs exist for sales orders, materials and vendors. Check the exact parameters in SE37 on your release before coding against them. For bulk data, the core MM tables are EKKO and EKPO (purchasing header and item), EKET (schedule lines), EKBE (history), MARA, MARC and MARD for materials, and VBAK and VBAP for sales. Note that the client field MANDT must be in every query.

RFC_READ_TABLE is tempting but a poor foundation: it is not a released interface, it truncates wide rows, and it can expose any table the calling user is allowed to read through S_TABU_DIS. Write a small purpose-built function module per question family instead, and have its authority checks live in ABAP next to the data.

**Code:**

-- Replica query: open PO schedule lines (illustrative, run on the SLT target, not on production)
SELECT h.EBELN, h.LIFNR, p.EBELP, p.MATNR, p.WERKS,
       s.EINDT AS delivery_date, (s.MENGE - s.WEMNG) AS open_qty
FROM   EKKO h
JOIN   EKPO p ON p.MANDT = h.MANDT AND p.EBELN = h.EBELN
JOIN   EKET s ON s.MANDT = p.MANDT AND s.EBELN = p.EBELN AND s.EBELP = p.EBELP
WHERE  h.MANDT = '100' AND p.LOEKZ = ' ' AND s.MENGE > s.WEMNG
ORDER  BY s.EINDT;

# Live read through a BAPI with the pyrfc library (verify parameter names in SE37)
from pyrfc import Connection
conn = Connection(ashost='ecc-app01', sysnr='00', client='100', user='ZAI_READ', passwd=get_secret('zai'))
res = conn.call('BAPI_PO_GETDETAIL1', PURCHASEORDER='4500001234')
items = [(i['PO_ITEM'], i['MATERIAL'], i['QUANTITY']) for i in res['POITEM']]

### Reference architecture: everything stays on your network

**Body:**

The layer has six parts. A reporting copy of the approved tables, fed by SLT or log-based change data capture, so analytical questions never touch the production database. A retrieval index (vectors plus keyword search) over documents, with the owning plant or company code stored as filterable metadata. A tool server exposing approved read actions. An open-weight model served by vLLM or Ollama on your GPUs. An identity layer that maps the person asking to their SAP user. And an append-only audit log.

A typical request goes: user asks 'which open POs for plant 1000 are late?'; the orchestrator picks the open_po_lines tool; the tool queries the replica with the plant filter derived from the user's authorizations; the rows come back to the model, which writes the answer and cites PO and item numbers. Nothing leaves the data centre. For an air-gapped site, load model weights, container images and Python wheels through a one-way transfer, and run the replica and index inside the same enclave.

Keep the model out of the SQL business. It chooses among named tools with typed parameters; the tool code builds the query. That makes every answer reproducible and lets security review a finite list of actions rather than an open-ended text-to-SQL surface.

**Code:**

ECC (ABAP app + DB)
  |-- SLT / CDC --> reporting store (approved tables only)
  |-- RFC/BAPI (per-user, SNC) --> tool server (MCP / internal API)
Documents (GOS, ArchiveLink, DMS, shares) --> parser --> retrieval index (+ plant/company metadata)
User --> chat UI --> orchestrator --> vLLM (local GPUs) --> tool server --> answer + cited keys
Every step --> audit log (who, what question, which tool, which rows)

### Permissions: the assistant must not out-see the user

**Body:**

ECC authorizations are role based (PFCG) and checked in ABAP with AUTHORITY-CHECK against authorization objects. For purchasing, M_BEST_WRK limits by plant, M_BEST_EKG by purchasing group and M_BEST_BSA by document type. Finance uses F_BKPF_BUK for company code, and sales uses V_VBAK_VKO for sales organization. S_TABU_DIS and S_TABU_NAM govern generic table display, and S_RFC governs which function groups an RFC caller may run.

There are two workable patterns. Per-user: the tool server calls BAPIs as the real user through SNC or trusted RFC, so SAP itself does the check. This is the most faithful but slower and needs identity plumbing. Mirrored: you export the effective authorization values per user (SUIM reports, or a function module that returns them) into the AI layer and apply them as mandatory filters on replica queries and on retrieval metadata. This is faster but must be refreshed whenever roles change, so schedule it and treat staleness as a defect.

Write-back is the narrow case. The assistant may draft a purchase requisition and place it in an approval queue; on approval a service user calls a BAPI such as BAPI_PR_CREATE followed by BAPI_TRANSACTION_COMMIT. See the BAPI commit page below for why the commit matters. Never give the model a direct update path to tables.

### Sizing, rollout and honest limits

**Body:**

As ranges: a pilot for 50 to 150 named users with a 7B to 32B parameter open-weight model typically fits one to two data-centre GPUs; a plant-wide rollout for several hundred users with a 70B-class model is usually two to four GPUs with a spare. The retrieval index for 100,000 documents is commonly tens of gigabytes. Use the sizing calculator linked below with your own question volumes rather than these ranges.

Roll out in three phases. Phase one is read-only questions over the replica and documents with a pilot group. Phase two adds drafted outputs (emails to vendors, requisition drafts, exception summaries) that a human sends. Phase three adds approved actions through BAPIs with an approval step and an audit trail. Do not skip to phase three.

Limits to state plainly. ECC mainstream maintenance is scheduled to end in 2027 (SAP offers extended maintenance at a premium into 2030), so this layer is often built to survive an S/4HANA move: keep tool contracts stable and swap the data access underneath. Custom Z tables and heavy user exits need mapping by someone who knows your system. Cluster and pool tables are awkward to replicate and read directly. Finance line items in BSEG are a cluster table in ECC, so use the secondary index tables or standard reports for them. And if your question is simply 'show me a standard report faster', a well-built ALV variant or SAP's own search may be enough.

## Pitfalls

- Pointing a text-to-SQL tool at production with a broad database user, which bypasses ABAP authority checks and may breach your SAP licence terms.
- Using RFC_READ_TABLE as the integration, then discovering wide tables truncate and the technical user can read tables it should not.
- Forgetting MANDT in joins, which silently mixes clients on shared databases.
- Indexing documents without the owning plant or company code, so a retrieval hit can leak a document the user could not open in SAP.
- Letting authorization mirrors go stale after a role change; the assistant then answers from yesterday's permissions.
- Building the layer so tightly around ECC table names that the S/4HANA move forces a rewrite.

## Ai Angle

ERPray connects to ECC through the read paths above, shows the query or BAPI call behind every answer, and respects the SAP user's roles. For a team that wants a private pilot, the quickest scope is a handful of read tools (open POs, stock by plant, delivery status) over a replica plus the SOP and drawing library, served from a local model. Write-back stays behind approval and BAPIs.

## Faq

### Can an LLM read SAP ECC tables directly?

**Answer:** Technically yes, but it is the wrong design. Direct table reads skip ABAP authorization checks and can conflict with SAP licence terms on third-party database access. Use BAPIs, custom RFC function modules, Gateway services, or a replica fed by SLT, and expose them as named tools the model can call.

### Do I need SAP BTP or Joule to do this?

**Answer:** No. The pattern here uses RFC, BAPIs, SLT and open-weight models on your hardware. Joule and BTP are SAP's cloud-oriented route and their availability for ECC differs by release, so check SAP's current statements. Many on-prem plants prefer the self-hosted route for data residency reasons.

### How do I stop the assistant showing data a user cannot see in SAP?

**Answer:** Either call BAPIs as the real user through SNC or trusted RFC so SAP checks authorization, or mirror each user's effective authorization values (plant, company code, sales organization) and apply them as mandatory filters on queries and retrieval. Refresh the mirror on a schedule and log every request.

### Is this still worth building if we will move to S/4HANA in two years?

**Answer:** Often yes, if you keep the tool contracts and prompts stable and treat the data access layer as replaceable. The retrieval index, model serving, identity and audit parts carry over unchanged. Expect to rewrite the table-level queries and BAPIs against S/4HANA APIs and CDS views.

## Search terms

add AI to SAP ECC, SAP ECC on-prem AI, SAP ECC LLM integration, SAP RFC BAPI AI assistant, SAP SLT replica reporting, private LLM SAP ECC, SAP ECC air-gapped AI, SAP authorization AI assistant, SAP ECC chatbot on-premise

---
Talk to an engineer: https://netray.co/contact?intent=erp_ai | Machine catalog: https://netray.co/agent/catalog.json