Extraction Precision, Recall and F1 Calculator
Accuracy alone hides the trade-off in document extraction between returning wrong values and failing to return values. Precision, recall and F1 separate the two so that you can tune a model or threshold with intent.
Your numbers
Fields the model extracted with the right value.
Fields the model extracted with a wrong value, or extracted where none exists.
Fields present in the document that the model did not return.
Rework, payment error or scrap cost when a wrong value is accepted.
Manual keying cost when a field is absent.
Your results
Count results from a labeled sample of your own documents. Define a match rule (exact, normalized) before scoring.
Precision, recall and F1 vs wrong extractions
Sweeps Wrong extractions (false positives) from 0 to 300, holding your other inputs fixed.
Extraction evaluation help
Optional: discuss building an evaluation harness for your documents.
Optional. The tool and every export work without this. No spam.
How to use the calculator
Take a labeled sample, such as 100 invoices or certificates with the correct values entered by hand. Count fields the model got right, fields it returned wrong and fields it missed. Enter the counts, and add the cost of an undetected wrong value and of a missed field to see financial exposure for the sample. The chart sweeps the wrong-extraction count so that you can see the effect on precision and F1.
- Fix the match rule first: exact match or normalized (dates, currency, whitespace).
- Score per field type, because invoice totals and descriptions behave differently.
- Use a sample that includes your worst suppliers and scan quality.
The definitions
Precision is true positives divided by true positives plus false positives. Recall is true positives divided by true positives plus false negatives. F1 is their harmonic mean, which simplifies to twice true positives over twice true positives plus false positives plus false negatives. A miss is cheap in a system where people fill gaps; a wrong value is costly where nobody checks it. That is why cost weights matter more than a single score.
How to read the result
High recall with low precision means the model returns values for almost everything but some are wrong, which is dangerous without review. High precision with low recall means it is cautious and leaves work to people. A confidence threshold moves a model along this trade-off: raising it improves precision and lowers recall. Choose the point from the cost of each error type, not from the F1 alone.
Common mistakes
Scoring on a sample that is too clean gives optimistic results. Counting a field returned with the wrong value only as a false positive is a convention, and some teams also count it as a miss, so state your rule. Finally, a single overall F1 hides the fields that matter: report the critical fields separately.
Frequently Asked Questions
What is the difference between precision and recall?
Precision asks how many returned values are right. Recall asks how many of the existing values were found. A model can be strong on one and weak on the other, which is why both are reported.
When should I use F1?
Use F1 as a single comparison number when false positives and false negatives matter roughly equally. When their costs differ greatly, use the cost exposure or weight the metrics accordingly.
How big should the evaluation sample be?
Large enough to cover the main suppliers, layouts and scan qualities, and to give a stable count for each field you care about. A few hundred documents is common, but rare fields may need targeted sampling.
How does the confidence threshold relate to these metrics?
Raising the threshold removes low-confidence values, which usually increases precision and reduces recall. Plot both against several thresholds on your sample and pick the point that fits your review capacity.
If you need an evaluation harness that scores an extraction model against your labeled documents on-prem, Netray can build one with you.
Related Tools
OCR Field Accuracy to Document Accuracy Calculator
See why 98 percent field accuracy does not mean 98 percent clean documents, and what field accuracy a target requires.
AI Agents & AutomationOCR Human Review Queue Staffing Calculator
Size the number of reviewers needed for a document capture exception queue, including peak days and the impact of the confidence threshold.
Quality & InspectionOCR Extraction Error Audit Sample Worksheet
Record a sample of reviewed documents by type and get field accuracy and clean-document rate per type and overall.
Costing & FinanceDuplicate Invoice Payment Leakage Calculator
Estimate annual cash lost to duplicate invoices that slip past controls, and what a higher catch rate would save.
More AI Agents & Automation tools
See all- OCR Straight-Through Processing Rate Savings Calculator
- Business Process Automation ROI Calculator
- AI Agent Use Case Prioritizer
- Document Processing Automation Savings Calculator
- AP Invoice Automation ROI Calculator
- Order Entry Automation Savings Calculator
- Quote-to-Cash Cycle Time Calculator
- Manufacturing AI Readiness Assessment
- RPA vs AI Agent Selector