Fuzzy Match Duplicate Review Workload Calculator
Every duplicate detection tool forces the same trade-off: merge automatically and risk wrong merges, or send pairs to people and pay for review time. This calculator shows the cost of each threshold choice for vendors, customers or items, so you can set bands deliberately.
Your numbers
Share of auto-merged pairs that really are the same entity.
Your results
Pairs below the review band are assumed ignored and are not counted as found. Precision and band values must come from a labelled sample of your own data.
Review hours vs share of pairs sent to review
Sweeps Pairs sent to manual review from 0 to 80, holding your other inputs fixed.
Matching threshold tuning
Optionally ask for help labelling a sample and tuning match bands on your own records.
Optional. The tool and every export work without this. No spam.
How to use it
Run your matching on the full table and count the candidate pairs. Split them into three bands: above the upper threshold, between the thresholds, and below the lower one. Label a random sample of each band by hand to measure precision. Enter those shares and precisions here, along with review minutes and the cost of unwinding a wrongly merged record, which for vendors may include payment reversals and reissued 1099 or tax documents. The outputs show hours, wrong merges and cost per confirmed duplicate, so you can compare threshold scenarios side by side.
The method
Auto-merged pairs equal total pairs times the auto share. Wrong merges equal auto-merged pairs times one minus precision. Review hours equal reviewed pairs times minutes divided by sixty. Confirmed duplicates combine correct auto merges with the true duplicates found in the review band. Total cost is review labor plus unwind cost, and the efficiency metric is cost per confirmed duplicate. The chart sweeps the review share to show the cost of widening the manual band: hours rise linearly, but the extra pairs have a lower hit rate, so the marginal cost per extra duplicate found climbs.
- Wrong merges = auto pairs x (1 - precision)
- Review hours = reviewed pairs x minutes / 60
- Cost per duplicate = (review cost + unwind cost) / confirmed
Practical guidance
Merging is asymmetric: a missed duplicate costs a little every month, but a wrong merge can corrupt history, so keep the auto-merge precision high, and be conservative with vendors that have bank details or tax identifiers. Strong keys such as tax ID, DUNS number or normalized bank account give much better precision than names alone; use them to define the upper band. Revisit thresholds quarterly, because new data sources and acquisitions change the score distribution. Always keep an audit trail of merged record keys so a merge can be reversed.
Frequently Asked Questions
What precision is acceptable for auto-merge?
Most governance teams want 99 percent or better, and many never auto-merge vendors at all. The right level depends on the unwind cost: the higher it is, the more you should push pairs to human review.
How do I measure precision in a band?
Randomly sample at least 100 pairs from the band, have two people independently label them same or different, resolve disagreements, and divide confirmed matches by the sample. More samples narrow the uncertainty.
Why are pairs below the lower threshold ignored?
They are assumed to be not duplicates for planning. Some true duplicates hide there, so periodically sample that band to estimate recall and decide whether the lower threshold is too high.
Does this work for items as well as vendors?
Yes. For items, matching on normalized description, manufacturer part number and unit of measure works well, and unwind cost is usually higher because of where-used and stock history.
Embedding-based matching with a human approving ambiguous pairs can narrow the review band without raising wrong merges. Netray can prototype this on a copy of your vendor or item data, kept inside your network.
Related Tools
Duplicate Item Merge Savings Calculator
Size the business case for merging duplicate item master records: yearly admin saving, split inventory carrying cost and payback on the merge project.
Quality & InspectionMaster Data Audit Sample Size Calculator
Work out how many item, vendor or customer records you must inspect to measure error rate with a defensible margin, and read the interval from your findings.
ERP Migration & SelectionData Cleansing Batch Effort Calculator
Plan a data cleansing or migration clean-up: how many records need manual attention, how many hours, how many weeks and what it costs.
ERP OperationsMaster Data Steward Staffing Calculator
Convert monthly master data requests into the number of stewards you need, with rework, review time and utilization built in.