# AI Audit Trail Readiness Checklist: Can You Prove What Your AI System Did? > Source: https://netray.co/tools/ai-audit-trail-readiness-checklist > Publisher: Netray (netray.co), AI and integration engineering for ERP and manufacturing systems. > Section: Free Tools ## Short answer A practical control checklist for building AI audit trails that satisfy compliance assessors, covering request-level logging, log integrity, identity evidence, and model lineage. ## Category on-prem-ai ## Tool **Kind:** checklist **Groups:** #### Request-level logging **Items:** **Text:** Every inference request is logged with requesting identity, timestamp, model version, and a unique correlation ID **Critical:** true Logs capture which documents or data sources were retrieved and passed into context for each RAG-based response Tool calls and agent actions are logged individually, not only the final summarized output Failed requests, rate-limit rejections, and errors are logged with the same rigor as successful requests Logging captures the full prompt template and system instructions in effect at request time, not just the user-visible message #### Log integrity and retention **Items:** **Text:** Logs are shipped to storage outside the AI platform itself, so a compromise of the platform cannot also erase its own evidence **Critical:** true **Text:** Log storage is tamper-evident, using write-once storage, cryptographic hashing, or an equivalent integrity control **Critical:** true Retention periods are explicitly defined per your compliance framework, not left to storage system defaults Log access itself is logged, so you can show who reviewed or exported audit data and when A tested backup and restore process exists for the log storage system, separate from the AI platform's own backups #### Identity and authorization evidence **Items:** **Text:** Every logged request ties to an authenticated identity, never a shared service account with no per-user attribution **Critical:** true Retrieval-layer authorization decisions are logged, showing which documents a user was and was not entitled to see for a given query Administrative actions on the AI platform (model deployment, configuration changes, permission grants) are logged separately from inference activity Role and permission changes affecting AI system access are logged with who made the change and when #### Model and data lineage **Items:** **Text:** Each logged request records the exact model version that served it, including quantization and fine-tuning variant if applicable **Critical:** true A model registry ties every deployed version to its training data sources, evaluation results, and approval record Data source updates (new documents added to a RAG corpus, corpus removals) are logged with timestamp and the identity that triggered them Fine-tuning runs are logged with the exact training dataset version, hyperparameters, and resulting model checksum #### Audit reporting and assessor readiness **Items:** **Text:** A defined process can produce a complete audit export for a specified time window within one business day **Critical:** true Log data can be filtered and reported by identity, model version, or data source without a custom engineering effort each time A documented mapping exists between your logging controls and the specific compliance framework requirements they satisfy (CMMC, SOC 2, ISO 27001, or equivalent) A mock audit or internal review of the logging system has been performed at least once, not only tested in theory **Scoring Note:** AI audit trail readiness is defensible when at least 90% of all items are complete and every critical item is closed. Any open critical item should concern you regardless of your overall score, since request-level logging, tamper-evident storage, and per-identity attribution are the three controls every assessor checks first, and a gap in any one of them undermines confidence in everything else the log claims to show. ## Content **Intro:** This free AI audit trail readiness checklist covers the logging and evidence controls needed to prove what an AI system did, when, and for whom, and it is written for compliance leads, security architects, and platform engineers preparing AI systems for a CMMC assessment, SOC 2 audit, customer security review, or internal governance requirement. It spans five domains: request-level logging, log integrity and retention, identity and authorization evidence, model and data lineage, and audit reporting readiness. Most AI systems ship with basic application logs that were never designed to answer an auditor's actual questions: who asked what, what data was retrieved, which model version responded, and can you prove none of that has been altered since. **Sections:** #### Why AI systems need audit logging that goes beyond application logs **Body:** A standard application log typically captures errors and high-level events, which is not the same as an audit trail. An assessor asking about an AI system wants to know exactly what data a specific user's query retrieved and whether that retrieval respected their entitlements, exactly which model version generated a specific output and whether it was an approved version, and whether the logs proving all of this could have been altered after the fact. None of these questions are answerable from generic error logs, and building the capability to answer them after an incident or an audit request arrives is far more expensive than building it into the system from the start. **Bullets:** - Retrieval-layer logging is the piece most systems miss, since it is the layer that determines whether a user saw data they were entitled to see. - Model version logging matters because serving stacks are updated frequently and behavior can shift between versions. - Tamper-evident storage separate from the AI platform itself protects the log's credibility if the platform is ever compromised. - Per-identity attribution, not shared service accounts, is what makes a log actually useful for an investigation. #### The controls that matter most **Body:** The critical items reflect the specific questions every assessor asks first: is every request tied to an identity, is the log storage tamper-evident and external to the platform, and does the log record which exact model version served each request. These three controls, together, let you answer the questions an investigation or audit actually needs answered. Everything else in the checklist, retention policy, reporting tooling, framework mapping, matters for efficiency and completeness but does not substitute for these three fundamentals being solid. **Bullets:** - Per-identity attribution on every logged request, never a shared or anonymous service account. - Tamper-evident log storage that lives outside the AI platform itself. - Model version recorded on every request, including fine-tuning and quantization variant. - A tested process for producing a complete audit export within one business day. #### How to work through the checklist **Body:** Build logging into the system architecture from the first deployment rather than retrofitting it after a growth in scale or a customer's security questionnaire forces the issue. Assign a single owner for the logging pipeline who is accountable for its completeness, separate from whoever owns application functionality, since logging requirements tend to lose priority against feature work unless someone is specifically responsible for defending it. Run a mock audit internally at least once before a real one arrives, because the gap between logs existing and logs being usable in the specific format and timeframe an assessor requires is often larger than teams expect. #### How Netray builds audit-ready AI systems **Body:** Netray builds AI logging and audit infrastructure for manufacturers who need to satisfy CMMC assessors, customer supplier security reviews, and internal governance requirements simultaneously. We architect request-level logging that captures retrieval decisions and model versions, ship logs to tamper-evident storage outside the AI platform, and map the resulting evidence directly to the compliance frameworks our customers are assessed against. Because we also integrate with SyteLine, LN, and M3 permission models, our retrieval-layer logging shows not just what was retrieved but whether the requesting user was actually entitled to see it. **Cta:** Get an audit readiness review of your AI logging mapped to the compliance framework your assessors actually use. **Faq:** #### What is the minimum logging needed to satisfy most AI compliance audits? **Answer:** At minimum, every inference request needs an authenticated identity, timestamp, model version, and correlation ID, stored in tamper-evident storage outside the AI platform with a defined retention period. For RAG systems, you also need retrieval-layer logging showing what was fetched and whether the user was entitled to see it. This baseline satisfies the most common questions across CMMC, SOC 2, and customer security reviews, though specific frameworks may require additional detail. #### Why does log storage need to be separate from the AI platform itself? **Answer:** If the AI platform is ever compromised, an attacker with access to the platform could also access and alter its own logs, destroying the evidence needed to investigate the compromise. Shipping logs to external, tamper-evident storage means the log's integrity does not depend on the security of the system it is monitoring. This separation is one of the first things a mature security assessment checks for, and its absence is a common and serious finding. #### How long should we retain AI inference logs? **Answer:** It depends entirely on your compliance framework and contractual obligations, not on storage system defaults. CMMC and similar frameworks typically specify retention periods that apply to your AI logs the same as any other system log. Do not leave this to whatever your logging tool's default happens to be; define the retention period explicitly and document the compliance requirement it satisfies. #### Can we retrofit audit logging into an AI system that is already in production? **Answer:** Yes, but it is more work than building it in from the start, and you will have a gap in historical evidence for the period before retrofitting. Prioritize the three fundamentals first: per-identity attribution, tamper-evident external storage, and model version tracking. Retrieval-layer authorization logging is usually the hardest to retrofit because it requires understanding entitlement decisions that may not currently be recorded anywhere in the pipeline. ## Search terms AI audit trail checklist, LLM audit logging readiness, AI compliance audit trail, inference logging requirements, AI system audit evidence, model lineage audit trail --- Talk to an engineer: https://netray.co/contact?intent=erp_ai | Machine catalog: https://netray.co/agent/catalog.json