# On-Prem AI for CUI in Small Manufacturers > Source: https://netray.co/resources/on-prem-ai-for-cui-small-manufacturers > Publisher: Netray (netray.co), AI and integration engineering for ERP and manufacturing systems. > Section: Knowledge Base ## Short answer On-prem AI for CUI: how small defense manufacturers run an LLM on their own GPUs inside the NIST 800-171 boundary, with hardware sizing and real trade-offs. ## Category ai-automation ## Content **Intro:** On-prem AI for CUI means running an open-weight language model on hardware inside your own CUI boundary, so drawings, specs and ERP data never travel to an outside AI service. A small defense manufacturer can do this with one or two GPU servers, a retrieval layer over its documents and ERP, and the same access controls it applies to other CUI systems. It trades some model strength and convenience for control, which is the right trade when the data cannot leave. **Sections:** #### Why Public AI Tools Are a Problem for CUI **Body:** Pasting a controlled drawing or specification into a public chatbot puts CUI on a system outside your boundary, which can violate DFARS 252.204-7012 and, for ITAR data, can be an unauthorized export. Even enterprise AI tiers vary: some provide data-handling commitments, but only a provider authorized for the right impact level, with the right contractual terms and US-person administration, may hold CUI. For most small shops, the simplest policy is clear: no controlled data in any AI tool unless it runs inside your boundary or the provider is explicitly authorized. Write that policy now, because staff are already using these tools on their own. #### What an Inside-the-Boundary Setup Looks Like **Body:** A practical stack has four parts. An inference server with one or more GPUs running an open-weight model through vLLM or Ollama. A retrieval layer that indexes your documents and ERP data into a vector store on the same network. An application layer with login tied to your identity provider, so the AI only retrieves what the user may already see. And logging, so queries, retrieved sources and outputs are recorded for audit. All of it sits in the same enclave segment as your other CUI systems, administered by vetted staff, with encryption at rest and in transit. No call leaves the boundary. **Bullets:** - Open-weight models such as Llama, Qwen, Mistral, Gemma or gpt-oss class served locally - Vector index and embeddings stored inside the enclave, since embeddings can leak source content - Role-aware retrieval that mirrors ERP and document permissions - Prompt and response logs protected like any other CUI record #### Hardware Sizing for a Small Shop **Body:** You do not need a data center. A model in the 7B to 32B parameter range handles RFQ summarization, drawing note extraction, clause screening and ERP question answering for a few concurrent users on a single server with one or two professional GPUs. Larger models help with harder reasoning but need more memory and power. Quantization reduces memory use at some accuracy cost. Budget for the server, GPUs, storage, backup, a UPS, and the staff time to patch and monitor it, because an unpatched AI server is a weak point in an otherwise compliant boundary. Total first-year cost for a small deployment is commonly in the tens of thousands of dollars, not millions. #### What On-Prem AI Does Well and Badly **Body:** It works well on bounded, document-heavy tasks: reading an RFQ package, extracting material and special process callouts, finding similar past jobs, comparing a customer PO to your accepted terms, answering questions about ERP data with the query shown. It is weaker on open-ended reasoning against a frontier hosted model, and it needs maintenance. Accuracy on scanned drawings depends on OCR quality and on tolerances written in tight notation, so keep a human reviewing anything that becomes a price, a commitment or an inspection record. Anything that writes back to the ERP should require explicit approval. #### When It Is Not Worth Building **Body:** If you handle little CUI, quote a few jobs a month, or your volume is mostly repeat parts with stable prices, a rules-based quoting tool or better ERP reports may deliver more value for less. If the real bottleneck is messy data, such as inconsistent routings or missing costs, AI will surface that mess, not fix it. Start with one narrow use case, measure estimator hours or schedule-meeting time saved, and expand only if the numbers justify it. A private deployment is an investment in capability, not a shortcut. **Cta:** Netray designs and runs private, on-prem AI for manufacturers and defense contractors, with read-only ERP access by default and human approval on every write-back. **Faq:** #### Can I use ChatGPT or Copilot with CUI? **Answer:** Only if the specific service is authorized for CUI at your required level, with contractual and technical safeguards, and your prime accepts it. Standard consumer and many business tiers are not. For ITAR-controlled technical data, additional US-person and data-residency requirements apply. When uncertain, treat it as not allowed. #### How much does it cost to run an LLM on-prem for a small shop? **Answer:** A small deployment typically needs one GPU server, storage, networking and support time, with first-year costs commonly in the tens of thousands of dollars depending on model size, redundancy and integration work. Ongoing costs are power, patching, monitoring and model updates. Compare that with estimator hours saved and compliance risk avoided. #### Do embeddings and vector databases contain CUI? **Answer:** They can. Embeddings derived from controlled documents may allow partial reconstruction of the source, so assessors and prudent shops treat the vector index as holding the same classification as the documents. Keep it inside the boundary, encrypt it and apply the same access controls. ## Related Slugs - on-prem-ai-defense-contractors - itar-cmmc-ai-workloads-guide - cmmc-compliant-ai-deployment - on-prem-llm-deployment-architecture-guide - on-prem-ai-cost-benchmarks-2026 ## Search terms on-prem AI for CUI, private LLM for defense contractors, AI for ITAR data, local LLM NIST 800-171, can I use ChatGPT with CUI, air-gapped AI manufacturer, on-prem AI small manufacturer --- Talk to an engineer: https://netray.co/contact?intent=erp_ai | Machine catalog: https://netray.co/agent/catalog.json