AI Agent Stack Availability and Downtime Calculator
An AI assistant depends on a chain: gateway, model server, vector store, ERP API and network. Each link has its own uptime, and they multiply. This calculator gives the end-to-end figure and shows what a second inference node adds.
Your numbers
A single GPU server or a hosted API endpoint as you measure it.
Your results
Assumes independent failures and perfect failover. Real redundancy shares power, network and software, so treat results as an upper bound on availability.
Annual downtime vs LLM node availability
Sweeps LLM inference node availability from 95 to 99.9, holding your other inputs fixed.
Review your AI infrastructure design
Optional: we can look at your inference and failover design.
Optional. The tool and every export work without this. No spam.
How to use it
Enter measured availability for each component, from your monitoring over the last quarter rather than vendor claims. For hosted APIs use observed success rate, not the contractual SLA. Pick how many independent LLM nodes serve traffic. The result is the expected uptime if every component must work for a request to succeed. The chart sweeps the model tier's availability to show how much it moves the total.
The math
For components in series, availability is the product of the individual values. For n identical independent nodes in parallel, tier availability = 1 - (1 - a) to the power n. Annual downtime = (1 - availability) x 8,760 hours. Five nines is about five minutes a year; three nines is almost nine hours.
- The weakest serial link dominates.
- Redundancy helps only the tier it duplicates.
- Maintenance windows count unless you can drain traffic.
Practical notes
Decide what happens during an outage: graceful degradation to the ERP's native screens is better than a broken assistant. A single GPU server is often the weakest link, but a second node doubles hardware cost, so compare saved downtime hours against the business value of the assistant. Also test failover, including model reload times, since a node that takes minutes to load a large model is not instantly available.
Frequently Asked Questions
Why do components multiply?
If every part must work for a request to succeed, the chance all work together is the product of each chance, assuming failures are independent.
Is a second GPU node worth it?
It depends on how critical the assistant is. If it supports shift operations, yes. For occasional analytics, a documented manual fallback may be enough.
Do hosted LLM APIs have better uptime?
Not necessarily. Providers publish SLAs but incidents and rate limits still occur. Use your own measured success rate, and consider a second provider or local fallback model.
Are failures really independent?
Rarely. Shared power, switches, storage and software versions create correlated failures, so real availability is usually lower than the calculation.
Netray deploys on-prem inference with health checks and fallback behaviour. Talk to us if you want a high availability design reviewed.
Related Tools
ERP AI Agent API Load Impact Calculator
Convert user count and calls per question into peak ERP calls per second, concurrent sessions and rate limit headroom before you connect an agent.
On-Prem AILLM Gateway Queueing Delay Calculator
Use queueing theory to see how request wait times explode as an inference service approaches full utilization.
On-Prem AIGPU Server Buy vs Rent Calculator
Turn GPU price, power cost, and cloud hourly rates into an annual owned-versus-rented comparison, plus the utilization rate at which buying starts to win.
AI Agents & AutomationAI Agent Human Review Queue Staffing Calculator
Estimate how many reviewers an approval-gated ERP AI agent needs, and what the labor saving really is once review and rework are counted.
More On-Prem AI tools
See all- On-Prem LLM Total Cost of Ownership Calculator
- GPU Sizing Calculator for LLM Inference
- On-Prem vs Cloud AI Cost Comparison Calculator
- Self-Hosted LLM Hardware Estimator
- Air-Gapped AI Readiness Assessment
- LLM API vs Self-Hosted Cost Calculator
- RAG Infrastructure Sizing Calculator
- Open-Weight Model Selector
- On-Prem AI ROI Calculator