# ERP Vendor Demo Scorecard: Rate What You Actually Saw, Not What You Were Told > Source: https://netray.co/tools/erp-vendor-demo-scorecard > Publisher: Netray (netray.co), AI and integration engineering for ERP and manufacturing systems. > Section: Free Tools ## Short answer Score each ERP vendor demonstration on the ten factors that predict implementation reality, from scripted scenario coverage to whether integrations and error handling were shown live. ## Category erp-migration ## Tool **Kind:** assessment **Questions:** #### Did the vendor demonstrate your scripted business scenarios end to end? **Help:** A scripted scenario means your process, your terminology, and a defined start and finish - not a feature tour the vendor chose. **Options:** ##### No script was used - the vendor drove a standard product tour **Score:** 0 ##### The script was acknowledged but the demo drifted into vendor-selected features **Score:** 1 ##### Most scripted scenarios were shown, with a few deferred to follow-up **Score:** 2 ##### Every scripted scenario was demonstrated end to end without skipping steps **Score:** 3 #### Was your own data used in the demonstration? **Options:** ##### Entirely vendor demo data with unfamiliar parts and customers **Score:** 0 ##### A handful of your part numbers pasted into an otherwise generic dataset **Score:** 1 ##### A representative sample of your BOMs, routings, and customers was loaded **Score:** 2 ##### A realistic slice including your most complex BOMs, routings, and customer requirements **Score:** 3 #### Was it clear which functionality was standard, configured, or custom code? **Help:** This single question predicts more implementation cost surprises than any other. Ask for it in writing after the demo. **Options:** ##### No distinction was made - everything was presented as the product **Score:** 0 ##### The vendor said most of it was standard when asked, with no specifics **Score:** 1 ##### Standard versus configured was mostly clear, custom work was vague **Score:** 2 ##### Each capability was explicitly labeled standard, configured, or custom, with effort estimates **Score:** 3 #### Who actually ran the demonstration? **Options:** ##### A sales representative reading from a prepared script **Score:** 0 ##### A pre-sales engineer with no connection to the delivery team **Score:** 1 ##### A solution consultant who would be involved in the project part-time **Score:** 2 ##### The named implementation consultant who would lead your project **Score:** 3 #### Were required integrations demonstrated live? **Options:** ##### Integrations were discussed only, with no working demonstration **Score:** 0 ##### A slide or architecture diagram was shown instead of working software **Score:** 1 ##### One integration was shown live, the rest described as achievable **Score:** 2 ##### Your critical integrations - EDI, CAD, MES, shop floor devices - ran live in the demo **Score:** 3 #### Was exception and error handling shown, or only the happy path? **Help:** Real operations are mostly exceptions. A demo that never shows a shortage, a hold, or a failed transaction has told you nothing about daily use. **Options:** ##### Only clean, successful transactions were shown **Score:** 0 ##### One exception was mentioned verbally but not demonstrated **Score:** 1 ##### Some exceptions were shown when the evaluation team asked **Score:** 2 ##### The vendor proactively demonstrated shortages, holds, rejections, and correction workflows **Score:** 3 #### Were reports or dashboards built live in response to your questions? **Options:** ##### Only pre-built reports were shown, no live creation **Score:** 0 ##### The vendor described the reporting tool but did not use it **Score:** 1 ##### A simple report was built live, complex requests were deferred **Score:** 2 ##### The vendor built a non-trivial report or dashboard live from an unscripted question **Score:** 3 #### Was shop floor, warehouse, or mobile execution shown on real devices? **Options:** ##### Not shown at all **Score:** 0 ##### Shown as screenshots or a simulated browser view **Score:** 1 ##### Shown on a tablet or phone in the vendor environment **Score:** 2 ##### Demonstrated on real scanners and terminals matching what your floor would use **Score:** 3 #### Was roadmap content clearly separated from generally available functionality? **Options:** ##### Roadmap and shipping features were blended with no distinction **Score:** 0 ##### Roadmap items were flagged verbally but without dates or commitments **Score:** 1 ##### Roadmap was separated with target releases but no contractual commitment **Score:** 2 ##### Everything demonstrated was generally available today, roadmap discussed separately **Score:** 3 #### Were implementation effort, pricing, and reference customers disclosed openly? **Options:** ##### No pricing, no hours estimate, and references were declined **Score:** 0 ##### A rough license price was given with no implementation estimate **Score:** 1 ##### License and services estimates provided, references limited to vendor-selected accounts **Score:** 2 ##### Detailed pricing, a phased hours estimate, and references you could select and contact directly **Score:** 3 **Bands:** #### Inconclusive demo - you have not seen enough to decide **Min:** 0 **Max:** 39 **Verdict:** What you watched was a marketing event, not an evaluation. A demo at this level tells you the product exists and looks modern, but nothing about whether it fits your processes or what implementation will actually cost. Do not shortlist on this basis; go back with a tighter script before spending more selection time. **Recommendations:** - Send a written scenario script and require the vendor to demonstrate every step without substitutions. - Provide a real data sample including your most complex BOM and routing, and require it to be loaded. - Demand written classification of every demonstrated capability as standard, configured, or custom. - Insist the named implementation lead runs the second demo, not a pre-sales engineer. - Reject any capability presented as roadmap unless it is contractually committed with a date. #### Partial evidence - core function seen, delivery reality unproven **Min:** 40 **Max:** 59 **Verdict:** You saw enough to believe the product can handle your core transactions, but the areas that drive implementation cost and post-go-live pain went unexamined. Integrations, exception handling, and the standard-versus-custom line are where budgets get destroyed. A focused second session can close these gaps quickly. **Recommendations:** - Schedule a deep-dive session dedicated entirely to integrations running live against test endpoints. - Require an exception-handling walkthrough covering shortages, quality holds, and transaction reversals. - Get the standard, configured, and custom classification in writing with hour estimates attached. - Ask for three references you select from a full customer list, not three the vendor picked. - Have the vendor build one unscripted report live to test whether your team could maintain reporting. #### Solid evaluation - verify the commercial and delivery details **Min:** 60 **Max:** 79 **Verdict:** This was a real evaluation and you have credible evidence about product fit. What remains is commercial and delivery diligence: confirming the estimate reflects the scope you demonstrated, that the named consultants are contractually assigned, and that references confirm the vendor delivers what the demo promised. Do not let a good demo shortcut reference checking. **Recommendations:** - Reconcile the services estimate line by line against the scenarios and integrations you actually saw. - Get named implementation resources written into the statement of work with replacement conditions. - Call references who implemented the same modules and integrations, and ask specifically about scope changes. - Confirm every demonstrated capability is generally available in the release you would receive. - Score all shortlisted vendors with this same instrument so the comparison is defensible. #### Excellent demo - move to contracting with confidence **Min:** 80 **Max:** 100 **Verdict:** You saw your scenarios, on your data, with your integrations, run by the people who would deliver the project, with the standard-versus-custom line drawn clearly. That is a rigorous evaluation and it substantially reduces implementation risk. Carry the same rigor into contracting so the demo commitments become contractual obligations. **Recommendations:** - Attach the demo script and recording to the contract as an acceptance baseline for the demonstrated scope. - Lock the named implementation team into the statement of work with notice requirements for changes. - Convert the standard-versus-custom classification into a fixed-scope schedule with change control. - Negotiate price protection and support terms before signing, while your leverage is highest. - Run the same scorecard on the runner-up so the negotiation has a credible alternative behind it. ## Content **Intro:** This free scorecard rates an ERP vendor demonstration on the factors that actually predict implementation outcomes, and it is built for selection team leads, IT directors, and executive sponsors evaluating manufacturing ERP. Ten questions assess whether your scripted scenarios were demonstrated, whether your own data was used, how clearly standard was separated from custom, who ran the session, whether integrations and exception handling ran live, and whether pricing and references were disclosed. Score each vendor immediately after their session while detail is fresh, then compare results side by side. The instrument is deliberately harsh, because comfortable demos produce uncomfortable projects. **Sections:** #### How to use this scorecard during selection **Body:** Score every vendor with the same ten questions within a day of their demonstration, ideally as a group exercise with each evaluator scoring independently before discussion. Independent scoring surfaces disagreement that a group conversation would smooth over, and the disagreements are usually the most informative part. Each question offers four graded answers worth zero to three points, giving a thirty-point maximum converted to a percentage. Keep the raw answers, not just the totals, because a vendor scoring seventy percent with a zero on integrations is a different risk than one scoring seventy percent evenly. Compare vendors on their weakest dimensions, not their averages. #### Why these ten factors predict implementation cost **Body:** Demo quality correlates with implementation reality because the same organizational habits show up in both. Vendors who prepare your data before a demo prepare your data before go-live. Vendors who blur the line between standard and custom in a demo will blur it in a change order, and vendors who cannot spare the implementation consultant for a day of selling will not have that consultant available for your project either. The demo is the only sample of the vendor's working behavior you get before signing, which makes it far more informative as a delivery signal than as a product signal. The four factors below are the strongest predictors of budget overrun in manufacturing ERP projects. **Bullets:** - Unclear standard-versus-custom classification, which converts directly into unbudgeted development change orders. - Integrations described rather than demonstrated, historically the largest single source of schedule slip. - Happy-path-only demos, which hide the exception handling that consumes most user training and support effort. - Roadmap functionality presented as available, leaving you dependent on a release date you cannot enforce. #### Interpreting scores across a shortlist **Body:** Do not simply pick the highest score. A vendor can score well because they invested heavily in the sales process rather than because the product fits. Read the scores alongside your functional fit findings and reference calls. The most useful pattern is the spread: a vendor with consistent twos and threes across all ten questions is usually a safer partner than one with several threes and two zeros, because the zeros mark exactly where your project will be surprised. If two vendors are close, weight the questions covering integrations, exception handling, and the named implementation team most heavily, since those three drive the majority of post-signature variance. #### How Netray supports your ERP selection **Body:** Netray runs vendor-neutral ERP selection support for discrete manufacturers, including writing demonstration scripts based on your real processes, preparing representative data sets that stress the product where it matters, and sitting in demos to ask the questions vendors hope nobody asks. Because we implement Infor SyteLine, CloudSuite Industrial, Infor LN, and Baan every day, we know which capabilities are genuinely standard and which require development, and we can price the gap honestly before you sign. We also help translate demonstration commitments into contractual scope so what you saw becomes what you get. Selection support engagements typically run four to eight weeks alongside your internal team. **Cta:** Get expert help scripting your ERP demos and scoring vendors objectively from Netray's manufacturing ERP selection specialists. **Faq:** #### How long should an ERP vendor demo be? **Answer:** Plan a full day per shortlisted vendor for a serious evaluation, split into functional scenario blocks with your subject matter experts present for the sessions that matter to them. Anything under three hours forces the vendor to skim, which produces a polished tour rather than evidence. If a vendor resists a full-day scripted format or wants to control the agenda entirely, treat that as a data point about how they will run your project. #### Should we let vendors use their own demo data? **Answer:** For an initial overview it is acceptable, but never for a shortlist evaluation. Vendor demo data is curated to avoid the product's weak spots, and manufacturing edge cases like deep multi-level BOMs, phantom assemblies, subcontract routings, and customer-specific requirements are exactly where products differ. Supply a real data slice including your most complex parts and require it to be loaded before the session. #### What if a vendor refuses to classify features as standard or custom? **Answer:** Treat refusal as an answer. Every experienced implementation team knows which capabilities ship in the box and which require development, so refusal signals either inexperience or an intent to bill for the difference later. Ask for the classification in writing as a condition of shortlisting, and attach the response to the contract. This one document prevents more change-order disputes than any other artifact in the selection process. ## Search terms ERP demo scorecard, ERP vendor evaluation criteria, how to evaluate ERP demos, ERP demo script template, ERP selection scoring matrix, ERP vendor comparison tool, ERP software demo checklist --- Talk to an engineer: https://netray.co/contact?intent=erp_ai | Machine catalog: https://netray.co/agent/catalog.json