AdvancedMicrosoft SharePoint Server (on-prem)Content, Search and Permissions

How to Add AI to SharePoint Server On-Prem Without Moving Your Data

Question
how to add AI to SharePoint Server on-prem

Also searched as

  • SharePoint Server 2019 AI integration without cloud
  • private LLM over SharePoint on-premises documents
  • SharePoint on-prem RAG with security trimming

Short answer

You can put a private assistant over SharePoint Server 2016, 2019 or Subscription Edition without any cloud service. Read content through the SharePoint REST API and the Search service, index it locally with the source permissions attached, and serve an open-weight model with vLLM or Ollama on your own GPUs. Microsoft 365 Copilot does not cover SharePoint Server on-prem, so this is the route for farms that stay in the data center.

Applies to: SharePoint Server 2016, 2019 and Subscription Edition, farms on SQL Server, claims-based authentication with AD DS. Check Microsoft's lifecycle page for support dates of your version.

Build the private AI layer in six steps

  1. 1Inventory what is worth answering from: document libraries, lists, wiki and site pages, and which web applications and site collections hold them. Skip personal sites and archives at first.
  2. 2Create a dedicated service account with read-only rights and register it as the crawler identity. Do not read the SQL content databases directly; use REST, CSOM or the Search service.
  3. 3Pull documents and metadata through the REST API or an incremental crawl, and use the change log (GetChanges) so only modified items are re-processed.
  4. 4Extract text (Office, PDF, OCR for scans), chunk it, embed it locally, and store every chunk with its ACL principals and the source URL.
  5. 5Stand up an MCP or HTTP tool server with approved read tools such as search_documents and get_document_summary, and put an open-weight model behind vLLM or Ollama on local GPUs.
  6. 6At question time resolve the asking user's AD groups, filter the index by those principals, and ask the model to answer only from the returned passages with citations.
  7. 7Log the question, the passages retrieved and the answer to an append-only store, then review it weekly with the farm owner.

Data access paths: use the supported surface, not the content database

A SharePoint farm keeps its content in SQL Server content databases, with document metadata and binary streams spread across tables such as AllDocs, AllUserData and AllDocStreams. Microsoft does not support reading or changing these tables directly, the schema moves between versions, and remote BLOB storage can place the bytes outside SQL entirely. Treat the database as an implementation detail.

The supported surfaces are the SharePoint REST API under /_api, the client-side object model (CSOM), and the Search service application with its query endpoint. REST gives you lists, files and permissions. Search gives you a security-trimmed ranked result set and a managed crawl that already handles many file formats. For bulk ingestion most teams use REST plus the change log, and keep Search as an additional retrieval signal.

# Search as the end user (Kerberos constrained delegation or an app that passes the user claim)
GET https://sp.corp.local/sites/quality/_api/search/query?querytext='AS9100 corrective action'&selectproperties='Title,Path,LastModifiedTime,Author'&rowlimit=20
Accept: application/json;odata=nometadata

# Incremental ingestion: items changed on a list since a stored change token
POST https://sp.corp.local/sites/quality/_api/web/lists/getbytitle('Procedures')/GetListItemChangesSinceToken

# Fetch the file bytes for text extraction
GET https://sp.corp.local/sites/quality/_api/web/GetFileByServerRelativeUrl('/sites/quality/Procedures/SOP-114.docx')/$value

Reference architecture

Reference architecture used on every page of this series: (1) a read-only replica or reporting copy of the data, (2) change capture so the index stays current, (3) a retrieval index with the source ACLs stored beside every chunk, (4) a tool server (MCP or plain HTTP) that exposes only approved read actions, (5) an open-weight model served with vLLM or Ollama on local GPUs, (6) an append-only audit log. For SharePoint the replica is not a database copy. It is a normalized store of extracted text and metadata that your ingestion job builds from REST responses.

The vector index (pgvector, Qdrant or OpenSearch run on your own servers) holds chunks, embeddings, the source URL, the modified date and the allowed principals. A change-detection job reads the change log every few minutes and updates or deletes chunks. In an air-gapped farm the same stack runs with model weights, container images and Python wheels brought in through a transfer process and checked by hash.

The tool server is where you enforce policy. Expose search and fetch only, return the citation with every passage, and cap result size. The model never gets a connection string or a service account token.

Permissions: mirror SharePoint, do not approximate it

SharePoint security is inherited and then broken: site, library, folder and item level permissions, SharePoint groups, AD security groups and claims. If your index is built by a service account that sees everything, a naive assistant becomes a way to read documents the user could not open. There are two safe patterns.

Pattern one: query SharePoint Search as the user, so the platform's own security trimming applies, and let the model see only what comes back. Pattern two: store the ACL principals resolved at crawl time on every chunk, resolve the user's AD groups at question time, and filter before retrieval. Pattern two is faster and works for a vector index, but you must refresh ACLs on a schedule and handle permission changes and deleted users. Test with a user who has limited rights and prove that restricted files never appear in the answer or the citation list.

Sizing, rollout and write-back

Text extraction and embedding are the heavy one-time work, about 1 to 3 chunks per page; a 500,000 document farm averaging 8 pages lands in the 4 to 12 million chunk range and tens of GB of index. Serving a 7B to 32B class open-weight model for a few thousand named users is usually one to four data-center GPUs, depending on concurrency and context length. Use the sizing calculator for your numbers.

Roll out in three phases. Phase one is read-only questions with citations over a few libraries. Phase two drafts content, for example a summary or a change notice, that a person pastes into SharePoint. Phase three allows narrow approved actions, such as creating a list item or starting a workflow, through the REST API with a human approving each call and the action logged. Never let the model write with a broad service account.

Try it here, free

SharePoint Server On-Prem Private AI Layer Blueprint Generator

Open full tool

Fill in your SharePoint Server environment and export an architecture plan for a private, on-prem AI assistant: components, security controls, rollout and open questions.

Common pitfalls

  • !Reading the content databases directly. It is unsupported, breaks on cumulative updates, and bypasses security trimming.
  • !Indexing as a farm admin and filtering later in the prompt. The model cannot be trusted as an access control layer; filter in the retrieval step.
  • !Ignoring item-level permission breaks. Libraries with thousands of uniquely permissioned items slow ACL resolution and need a plan.
  • !Treating scanned PDFs as text. Without OCR the index is empty for much of an engineering library.
  • !Forgetting deletes and moves. A moved or deleted file must disappear from the index quickly, or the assistant cites documents that no longer exist.
  • !Running a version near or past end of support and planning AI as a reason to stay. Decide the upgrade path first.

How an ERP-grounded AI assistant handles this

ERPray and DataRay can run over a SharePoint library the same way they run over an ERP: they connect read-only, show the passages and the query behind each answer, and respect the asking user's permissions. Netray designs and runs this private layer with your IT team. If you only need search inside a hybrid Microsoft 365 tenant, Microsoft's own tools may be enough and cheaper.

See ERPray

Frequently asked questions

Does Microsoft 365 Copilot work with SharePoint Server on-prem?

Copilot grounds on Microsoft 365 content in the cloud. A farm that stays on SharePoint Server on-prem is outside that boundary, apart from whatever hybrid search you have configured. A private layer over REST and Search covers the on-prem content.

Can the model read the SQL content database for speed?

It should not. Direct reads are unsupported by Microsoft, depend on version specific schema, and skip security trimming. REST, CSOM and Search are slower to set up but are the surface that survives updates and keeps permissions intact.

How do we keep answers limited to what a user can open?

Either call Search as the user so SharePoint trims the results, or store resolved ACL principals on each chunk and filter by the user's AD groups before retrieval. Test with restricted accounts and keep the audit log.

Can it work air-gapped?

Yes. Embedding model, LLM, vector store and tool server all run inside the network. Weights and containers enter through a controlled transfer and are verified by hash. Nothing calls an external API at question time.

What GPU do we need?

For a few thousand named users with modest daily use, one to four data-center GPUs serving a 7B to 32B open-weight model is a typical range. Concurrency, context length and model size move that; measure with a pilot.

Related

CIO briefing

Adding AI to a Legacy ERP Without Replacing It

Yes, for most legacy ERPs still running in production. If the system exposes its data in any queryable form - ODBC, an API, a scheduled export, or even a read replica of the database - a grounded AI layer can sit alongside it, answering questions and automating workflows without touching core code, buying years of runway before a forced migration.

CIO briefing

On-Prem vs Cloud ERP AI for Regulated Manufacturers

For ITAR, CMMC, and export-controlled manufacturers, the deciding question is not on-prem versus cloud in general but whether controlled unclassified information (CUI) or export-controlled technical data ever leaves your boundary to reach an AI model. An on-prem or private-VPC-deployed AI layer keeps that data inside your control boundary; a vendor's shared cloud AI service usually does not, regardless of how the core ERP itself is hosted.

CIO briefing

Is Your ERP Data Ready for AI? A Readiness Checklist

Most ERP data is ready enough to start a scoped AI pilot immediately; full data-quality remediation is not a prerequisite. A handful of specific gaps - duplicate item or customer masters, inconsistent units of measure, missing descriptions, and orphaned records - will visibly degrade AI answers and are worth checking before the pilot, not after.

Advanced

Private AI for OpenText Content Server and Documentum On-Prem

Both products expose supported APIs that a private AI layer can use without touching the cloud: the Content Server REST API and search for OpenText, DQL, DFC, DFS and Documentum REST for Documentum. Extract text and metadata through those interfaces, keep each object's permissions beside its chunks, and serve an open-weight model on local GPUs.

Advanced

How to add a private AI layer to Infor LX / BPCS on-prem

BPCS and Infor LX run on IBM i with their data in Db2 for i, in files that carry short coded names by module. A private AI layer reads those files through read-only SQL views over a replica, indexes documents and attachments separately, exposes approved queries through a tool server, and serves an open-weight model on local GPUs. Writes, if any, go through the supported transaction programs with human approval.

Advanced

How to add a private AI layer to Infor System21 on-prem

System21 runs on IBM i with its business data in Db2 for i. The practical private AI pattern is a read-only replica or extract, SQL views that give files readable names, a retrieval index over documents, a tool server of approved queries, and a locally served open-weight model. User visibility is carried over from System21 security and company set-up, and writes only go through its supported entry points after approval.

AI for ERP

A Private LLM Grounded on Your ERP Data

How a private LLM answers questions on your ERP data: RAG plus text-to-SQL, role-based permissions inherited from the ERP, and where each fits.

AI for ERP

AI Agents for ERP, Running On-Prem

A practical guide to on-prem AI agents for ERP: what they can safely automate, where human approval belongs, and how to design the guardrails.

Stuck on Microsoft SharePoint Server (on-prem)?

Talk to engineers who work inside Microsoft SharePoint Server (on-prem) every week, and who build private AI that answers these questions from your own ERP data.

Talk to an ERP engineerBrowse ERP Help
All ERP Help articles

Keep exploring

Free tools

ERP fixes and how-tos

AI on your ERP

Software