# Attribute Agreement Analysis Worksheet

> Source: https://netray.co/tools/attribute-agreement-analysis-worksheet
> Publisher: Netray (netray.co), AI and integration engineering for ERP and manufacturing systems.
> Section: Free Tools

## Short answer

Enter two appraisers' repeated pass/fail calls against a known standard and see agreement percentages.

## Category

quality

## Audience

- Quality engineer
- Inspection supervisor
- Six Sigma practitioner

## Tool

**Kind:** worksheet

**Mode:** editable

**Columns:**

#### Sample no.

**Key:** sample

**Type:** text

#### Reference (1 = pass, 0 = fail)

**Key:** ref

**Type:** number

#### Appraiser A, trial 1

**Key:** a1

**Type:** number

#### Appraiser A, trial 2

**Key:** a2

**Type:** number

#### Appraiser B, trial 1

**Key:** b1

**Type:** number

#### Appraiser B, trial 2

**Key:** b2

**Type:** number

#### A repeats (1 = yes)

**Key:** aw

**Type:** number

**Expr:** eq(a1, a2)

**Format:** integer

#### B repeats (1 = yes)

**Key:** bw

**Type:** number

**Expr:** eq(b1, b2)

**Format:** integer

#### A matches reference

**Key:** as

**Type:** number

**Expr:** eq(a1, ref) * eq(a2, ref)

**Format:** integer

#### B matches reference

**Key:** bs

**Type:** number

**Expr:** eq(b1, ref) * eq(b2, ref)

**Format:** integer

#### All four match reference

**Key:** all

**Type:** number

**Expr:** eq(a1, ref) * eq(a2, ref) * eq(b1, ref) * eq(b2, ref)

**Format:** integer

**Rows:**

**Sample:** 1

**Ref:** 1

**A1:** 1

**A2:** 1

**B1:** 1

**B2:** 1

**Sample:** 2

**Ref:** 0

**A1:** 0

**A2:** 0

**B1:** 0

**B2:** 0

**Sample:** 3

**Ref:** 1

**A1:** 1

**A2:** 1

**B1:** 1

**B2:** 0

**Sample:** 4

**Ref:** 0

**A1:** 0

**A2:** 1

**B1:** 0

**B2:** 0

**Sample:** 5

**Ref:** 1

**A1:** 1

**A2:** 1

**B1:** 1

**B2:** 1

**Sample:** 6

**Ref:** 0

**A1:** 1

**A2:** 1

**B1:** 0

**B2:** 0

**Sample:** 7

**Ref:** 1

**A1:** 1

**A2:** 1

**B1:** 1

**B2:** 1

**Sample:** 8

**Ref:** 0

**A1:** 0

**A2:** 0

**B1:** 0

**B2:** 0

**Totals:**

#### Samples

**Key:** sample

**Agg:** count

**Format:** integer

#### A repeats

**Key:** aw

**Agg:** sum

**Format:** integer

#### B repeats

**Key:** bw

**Agg:** sum

**Format:** integer

#### A matches reference

**Key:** as

**Agg:** sum

**Format:** integer

#### B matches reference

**Key:** bs

**Agg:** sum

**Format:** integer

#### All match

**Key:** all

**Agg:** sum

**Format:** integer

**Summary:**

#### A within-appraiser agreement

**Key:** a_within

**Expr:** 100 * sum_aw / count

**Format:** percent

#### B within-appraiser agreement

**Key:** b_within

**Expr:** 100 * sum_bw / count

**Format:** percent

#### A versus reference

**Key:** a_ref

**Expr:** 100 * sum_as / count

**Format:** percent

#### B versus reference

**Key:** b_ref

**Expr:** 100 * sum_bs / count

**Format:** percent

#### All appraisers versus reference

**Key:** overall

**Expr:** 100 * sum_all / count

**Format:** percent

**Notes:** Enter 1 for pass and 0 for fail on every call. Use at least 30 samples in a real study, mixing clearly good, clearly bad and borderline units, and randomize the order. Kappa statistics need a statistics package; this sheet gives the percent agreement figures.

## Content

**Intro:** Not every measurement is a number. Visual inspection, go/no-go gaging and cosmetic checks produce pass or fail decisions, and those decisions need their own study. Attribute agreement analysis measures how consistent inspectors are with themselves and with the truth.

**Sections:**

#### How to use it

**Body:** Select samples that include clear passes, clear fails and borderline cases, and have an expert establish the reference decision for each. Give them to each appraiser in random order, at least twice, without letting anyone see previous calls. Enter 1 for pass and 0 for fail in the sample rows. The sheet marks whether each appraiser repeated their own call, whether they matched the reference, and totals the percent agreement at the bottom. Add rows until you have at least 30 samples.

#### Reading the results

**Body:** Within-appraiser agreement shows repeatability: does the same inspector give the same answer twice. Appraiser versus reference shows accuracy. The all-match figure is the toughest: every call on a sample agrees with the standard. AIAG guidance commonly treats kappa of 0.9 or higher as good and below 0.7 as needing improvement. Percent agreement alone does not remove chance agreement, so use kappa from a statistics package when the result drives a decision. Separately look at miss rate (failing parts passed) and false alarm rate.

**Bullets:**

- Within-appraiser: repeatability
- Versus reference: accuracy
- Miss rate matters more than false alarms for safety
- Kappa corrects for chance

#### Common mistakes

**Body:** Using only obvious good and bad samples makes everyone look perfect. Using too few samples gives wide error bars, so a 100% result on 10 parts proves little. Letting appraisers discuss calls ruins independence. Neglecting the reference decision is the biggest gap: without a trusted standard you can measure consistency, but not correctness. If results are poor, improve the work instruction, add boundary samples or limit samples to retrain, then repeat the study.

**Cta:** If inspection pass or fail results are already captured in your ERP or MES, Netray can help analyse that data for consistency issues.

**Faq:**

#### How many samples do I need?

**Answer:** A common recommendation is at least 30 samples, with roughly equal numbers of good, bad and borderline units, and 2 or 3 trials per appraiser. Larger studies narrow the confidence interval on the agreement figures.

#### What is a good agreement percentage?

**Answer:** Targets differ by industry. As a rule, kappa of 0.9 or above is considered good and below 0.7 unacceptable under AIAG guidance. Percent agreement should be as close to 100% as practical, with extra attention to missed defects.

#### What is the difference between this and gage R&R?

**Answer:** Gage R&R in the variable sense measures continuous readings and splits variation into repeatability and reproducibility. Attribute agreement analysis handles discrete decisions such as pass or fail and uses agreement rates and kappa.

#### Do I need a reference standard?

**Answer:** For accuracy, yes. Without a known reference you can still test whether appraisers agree with themselves and each other, but not whether they are right. Have an experienced expert or a measuring system establish the reference.

## Search terms

attribute agreement analysis excel, attribute gage R&R template, attribute MSA worksheet, inspector agreement study, visual inspection agreement test, attribute agreement analysis template, pass fail measurement system analysis, appraiser agreement percentage

---
Talk to an engineer: https://netray.co/contact?intent=erp_ai | Machine catalog: https://netray.co/agent/catalog.json