AdvancedIBM i (AS/400) custom RPG applicationsDb2 for i and RPG integration

How to Add AI to an IBM i (AS/400) RPG Application Without Moving It

Question
how to add AI to an IBM i AS/400 RPG application on-prem

Also searched as

  • AS/400 AI integration without cloud
  • IBM i Db2 SQL access for LLM
  • modernize RPG application with private AI

Short answer

A custom RPG system does not need to be rewritten to get AI. Db2 for i already exposes your physical files as SQL tables, journals give you change capture, and stored procedures or Integrated Web Services can call your RPG logic. Read through a replica or read-only profile, describe the cryptic file and field names in a semantic layer, and serve an open-weight model on local GPUs.

Applies to: IBM i 7.3 and later (row and column access control from 7.2, SQL services in recent Technology Refreshes), RPG III, RPG IV and free-form RPG, DDS or SQL defined files.

Add a private AI layer to a custom IBM i system

  1. 1List the physical and logical files behind the screens and reports users ask about. Use the catalog (QSYS2.SYSTABLES, QSYS2.SYSCOLUMNS) and DSPFD or DSPFFD output to document them.
  2. 2Create a dictionary of meaning: for each field add the business name, units, code values and date format, using LABEL ON, COMMENT ON, or a separate table. Many fields are CHAR or packed numeric dates such as YYYYMMDD.
  3. 3Choose the access route: a read-only user profile over JDBC (JTOpen) or ODBC, or a replica fed by remote journaling or a replication product.
  4. 4Expose reviewed SQL views and parameterized procedures; do not give the model open SQL on the production library.
  5. 5Where business logic lives only in RPG (pricing, availability), call it through a stored procedure or an Integrated Web Services endpoint instead of re-implementing it.
  6. 6Mirror security: use separate profiles per user group, and apply row and column access control for sensitive columns.
  7. 7Run the model on a separate x86 GPU server, with the tool server in the same network zone as the IBM i partition, and log all calls.

Data access paths

Db2 for i exposes every physical file as an SQL table. The system name can be 10 characters (CUSTMAST, ORDHDR), while SQL names, if present, are longer. Field names are often 6 to 10 characters (CMCUST, OHDATE) and the meaning sits in documentation, field text or someone's head. The catalog in QSYS2 lets you list tables, columns and column text, and IBM i SQL services give you views on objects, jobs and journals.

Access from outside is by JDBC (the open-source JTOpen driver), ODBC through IBM i Access, or Mapepire, an open-source SQL server for IBM i. For logic that lives in RPG, you can create SQL stored procedures with EXTERNAL NAME pointing at a program, or expose programs as REST services using Integrated Web Services. Native record-level access does not appear in SQL: if a program applies business rules while reading, SQL on the file will not reproduce them.

-- Document the schema from the catalog
SELECT table_schema, table_name, column_name, ordinal_position, data_type, length, column_text
FROM   qsys2.syscolumns
WHERE  table_schema = 'PRODLIB' AND table_name IN ('ORDHDR','ORDDTL','CUSTMAST')
ORDER  BY table_name, ordinal_position;

-- Add meaning that both people and the model can use
LABEL ON COLUMN prodlib.ordhdr.ohdate IS 'Order date YYYYMMDD';

-- Read-only view that converts numeric dates
CREATE VIEW ai_lib.open_orders AS
SELECT ohord AS order_no, ohcust AS customer_no,
       DATE(TIMESTAMP_FORMAT(DIGITS(ohdate),'YYYYMMDD')) AS order_date, ohstat AS status
FROM prodlib.ordhdr WHERE ohstat <> 'C';

Reference architecture

Reference architecture used on every page of this series: (1) a read-only replica or reporting copy of the data, (2) change capture so the index stays current, (3) a retrieval index with the source ACLs stored beside every chunk, (4) a tool server (MCP or plain HTTP) that exposes only approved read actions, (5) an open-weight model served with vLLM or Ollama on local GPUs, (6) an append-only audit log. On IBM i, journaling is the natural change source: Db2 for i journals physical file changes, and remote journaling or a replication tool can feed a replica or an index without touching the application.

Because the model server is x86 with GPUs, it sits next to the Power system on the same network, not on it. The tool server connects with JDBC to a read-only profile, serves only named views and procedures, and converts EBCDIC, packed decimal and numeric date fields into clean JSON. A retrieval index covers spool files, PDFs, manuals and the field dictionary you created. The air-gapped variant is the same stack inside the closed network.

Permissions and write-back

IBM i security is object-based: library and object authority, user and group profiles, adopted authority in programs, and often application-level menus that restrict users. Menus do not protect SQL. Give the assistant its own low-privilege profiles, one per role group, with authority only to AI-specific views. Row and column access control (RCAC) lets you add permissions and masks on the tables so that a user sees only their own branches or a masked salary column.

Write-back goes through the existing application logic, not direct file updates. Call an RPG program through a stored procedure or a web service so validation, locks and audit fields behave as they do from a green screen, require a human approval in your workflow, and log the call. Direct SQL INSERT into an order file bypasses the program's checks and is the quickest way to corrupt data.

Sizing, rollout and honest limits

A custom IBM i system for a mid-sized plant often serves 100 to 500 users, with databases from tens of GB to a few TB. Analytical questions run on a replica so they never compete with batch windows. Model load is small: one GPU serving a 7B to 14B model is a common starting point, scaled by concurrent users.

Phase one: read-only questions through reviewed views. Phase two: explain and document, such as describing what an RPG program does or what a batch job produces. Phase three: approved actions via stored procedures. Limits: undocumented field names need human effort first, logic hidden in RPG cannot be inferred from tables, and if your only goal is code understanding, IBM's own tooling such as watsonx Code Assistant for i may be enough.

Try it here, free

IBM i (AS/400) Custom RPG Applications Private AI Layer Blueprint Generator

Open full tool

Fill in your IBM i RPG environment and export an architecture plan for a private, on-prem AI assistant: components, security controls, rollout and open questions.

Common pitfalls

  • !Letting the model guess field meanings from names like CMCUST. Build the dictionary first.
  • !Running analytical SQL on the production partition during the batch window.
  • !Assuming SQL equals the application. RPG programs apply rules that raw file reads do not.
  • !Using one powerful profile for the assistant. Use per-role profiles plus RCAC.
  • !Ignoring CCSID and packed or zoned numeric formats, which produce garbled text and wrong numbers.
  • !Writing to files directly with SQL instead of calling the RPG program.

How an ERP-grounded AI assistant handles this

DataRay and ERPray both rely on a semantic layer over the data, which is the heavy lifting on IBM i: turning short field names and numeric dates into things a user and a model can talk about. Netray builds that layer with your RPG developers, then serves the assistant on your own GPUs. IBM's own tools are worth testing first if your need is limited to code explanation.

See ERPray

Frequently asked questions

Do we need to rewrite the RPG application first?

No. Db2 for i exposes the files as SQL tables and journals give you change capture. The work is documenting fields, building reviewed views and calling RPG for logic, not rewriting the application.

Can the model write SQL against our files?

Only against reviewed views or a replica with a low-privilege profile. Cryptic names and numeric dates make free SQL error-prone, so give the model named views and procedures with descriptions.

How do we call business logic that only exists in RPG?

Wrap the program in an SQL stored procedure with EXTERNAL NAME or expose it with Integrated Web Services, then call that from the tool server. This keeps validation and audit behavior identical to the existing screens.

Is IBM i row and column access control worth using?

Yes for sensitive data. RCAC lets Db2 filter rows and mask columns by user or group at the database, so the assistant cannot bypass it through SQL even if a prompt tries.

Can the model run on the Power system?

Practically, run it on a separate x86 GPU server on the same network. The IBM i partition stays focused on the application, and the tool server connects with JDBC.

Related

CIO briefing

Adding AI to a Legacy ERP Without Replacing It

Yes, for most legacy ERPs still running in production. If the system exposes its data in any queryable form - ODBC, an API, a scheduled export, or even a read replica of the database - a grounded AI layer can sit alongside it, answering questions and automating workflows without touching core code, buying years of runway before a forced migration.

CIO briefing

Is Your ERP Data Ready for AI? A Readiness Checklist

Most ERP data is ready enough to start a scoped AI pilot immediately; full data-quality remediation is not a prerequisite. A handful of specific gaps - duplicate item or customer masters, inconsistent units of measure, missing descriptions, and orphaned records - will visibly degrade AI answers and are worth checking before the pilot, not after.

CIO briefing

On-Prem vs Cloud ERP AI for Regulated Manufacturers

For ITAR, CMMC, and export-controlled manufacturers, the deciding question is not on-prem versus cloud in general but whether controlled unclassified information (CUI) or export-controlled technical data ever leaves your boundary to reach an AI model. An on-prem or private-VPC-deployed AI layer keeps that data inside your control boundary; a vendor's shared cloud AI service usually does not, regardless of how the core ERP itself is hosted.

Advanced

How to add a private AI layer to Infor LX / BPCS on-prem

BPCS and Infor LX run on IBM i with their data in Db2 for i, in files that carry short coded names by module. A private AI layer reads those files through read-only SQL views over a replica, indexes documents and attachments separately, exposes approved queries through a tool server, and serves an open-weight model on local GPUs. Writes, if any, go through the supported transaction programs with human approval.

Advanced

How to add a private AI layer to Infor System21 on-prem

System21 runs on IBM i with its business data in Db2 for i. The practical private AI pattern is a read-only replica or extract, SQL views that give files readable names, a retrieval index over documents, a tool server of approved queries, and a locally served open-weight model. User visibility is carried over from System21 security and company set-up, and writes only go through its supported entry points after approval.

Advanced

How to add a private AI layer to Infor EAM on-prem

On-prem Infor EAM stores its data in an Oracle or SQL Server schema whose tables use the R5 prefix, and it exposes web services for supported access. A private AI layer reads equipment, work order and parts data through read-only views on a reporting copy, indexes failure notes and manuals for retrieval, mirrors EAM organisation security, and serves an open-weight model locally. Work orders created or changed by the assistant go through EAM's web services after a person approves.

AI for ERP

A Private LLM Grounded on Your ERP Data

How a private LLM answers questions on your ERP data: RAG plus text-to-SQL, role-based permissions inherited from the ERP, and where each fits.

AI for ERP

AI Agents for ERP, Running On-Prem

A practical guide to on-prem AI agents for ERP: what they can safely automate, where human approval belongs, and how to design the guardrails.

Stuck on IBM i (AS/400) custom RPG applications?

Talk to engineers who work inside IBM i (AS/400) custom RPG applications every week, and who build private AI that answers these questions from your own ERP data.

Talk to an ERP engineerBrowse ERP Help
All ERP Help articles

Keep exploring

Free tools

ERP fixes and how-tos

AI on your ERP

Integrations

Software