Accuracy Questions: How Does It Know My Schema?
Ask how the assistant knows your tables and columns, specifically your customizations and user-defined fields, and what happens when the model generates SQL referencing something that does not exist. Weak answers involve prompting harder. Strong answers involve a compiled schema index refreshed from the live database and a validation layer that structurally rejects phantom references before execution.
Then ask the definitional question: when I ask for open orders, whose definition is used, and can my team read and change it? Look for semantic views or an equivalent single source of definitions.
- Demand a live demo against a realistic SyteLine schema, not a toy database.
- Ask what happens when generated SQL references a nonexistent column.
- Ask where the definition of open orders lives and who can inspect it.
- Ask whether the dashboard number and the chat number can ever disagree.
Trust Questions: Evidence and Writes
Two questions expose a system's trust model. First: for any answer, can I see the exact SQL that ran and the tables it read? Evidence-first systems show this by default; black boxes ask to be believed.
Second: what happens when a user asks the assistant to change data? The only acceptable answer involves a governed flow, previewed changes, explicit confirmation, a full audit trail, and undo, with writes disabled by default until you decide otherwise. An assistant that writes to your ERP as a conversational side effect is a liability, not a feature.
Deployment Questions: Where Does My Data Go?
Finally, the architecture question: does ERP data or anything derived from it leave your network? For manufacturers under ITAR, CMMC, or customer flow-downs, cloud-only assistants often fail security review outright. Ask whether the system deploys fully on-premise, whether local models are a first-class option, and whether air-gapped operation is supported.
SyteRay's answers, for the record: fully on-prem, local models supported, air-gap supported, evidence shown with every answer, writes governed and off by default. We built it to pass our own checklist.
Try it here, free
SyteLine Database Tables Reference
Searchable lookup of the main SyteLine SQL tables, what each holds and its usual business key, for writing queries and reports.
Frequently asked questions
What should a SyteLine AI assistant demo prove?
Three things: correct answers over a realistic schema including customizations, visible SQL evidence for every answer, and graceful failure, wrong or ambiguous questions should produce clarifying dialogue or validation errors, never a confidently invented number.
Should an ERP AI assistant be able to change data?
Only through governance: previewed change sets, explicit per-change confirmation, a permanent audit trail, and undo. Write capability should ship disabled by default and be enabled deliberately per deployment. Anything looser is unacceptable for a production ERP.
Do on-prem AI assistants sacrifice quality versus cloud ones?
For structured ERP work, far less than assumed. The hard part of ERP question-answering is schema knowledge and validation, not raw model brilliance, and those come from the grounding layer, which works the same on-premise. Local open-weight models handle grounded SQL generation well.
Put SyteRay through this checklist against your own data. Book an evaluation.