Claims Integrity Audit

Instructions

Audit a document for claims integrity. Extract every factual, quantitative, comparative, or capability claim; classify it; grade it against available evidence; and produce a structured report with verdicts and remediations.

Step 1 — Claim Extraction

Read the document and extract every statement a reasonable technical buyer, journalist, or regulator could interpret as factual. Include:

• Express claims: Explicit statements of fact (“We have 192 agents”)
• Implied claims: Meaning conveyed by context, juxtaposition, or omission (listing 20 products in a flat table without status implies all are production-ready)
• Quantitative claims: Numbers, percentages, multipliers, counts
• Comparative claims: “faster than,” “better than,” “leading” — note whether a comparator is named
• Capability claims: Assertions about what the product or organization can do
• Temporal claims: “Since 2024,” “in production,” “currently” — check currency

Step 2 — Claim Classification

Classify each claim into one of four categories (determines the required evidence bar):

Category	Definition	Evidence Bar
Puffery	Subjective, non-falsifiable superlatives (“world-class,” “best-in-class”)	Low — but flag if context makes it sound factual
Objective-verifiable	Can be proven true or false (“42 skills,” “PHP 8.0+”)	Must be accurate and current
Comparative	Claims superiority or difference vs. a baseline or competitor	Must name comparator; must have evidence for the comparison
Regulated	Health, safety, financial, legal, or AI-capability claims subject to regulatory scrutiny	Highest bar — must meet FTC/industry substantiation standards

Step 3 — Evidence Grading

For each objective or comparative claim, assess the evidence:

Verdict	Criteria	Action Required
Verified	Evidence on file, reproducible, independently confirmable	None — include in approved claims registry
Supported	Reasonable basis exists; claim is appropriately caveated	None — but document the evidence source
Needs Context	Factually grounded but misleading without qualification	Add qualifier, footnote, or caveat
Unsupported	No evidence trail; may be true but unverifiable	Must add evidence or soften to puffery
Misleading	Omits material information or creates false impression	Must rewrite or add material context
False	Contradicted by available evidence or internally inconsistent	Must correct or remove immediately

Step 4 — Red Flag Heuristics (Fast Triage)

Flag any claim matching these patterns for immediate review:

Quantitative red flags:

• Precision without measurement methodology (“30-50% savings” with no test protocol)
• Counts that conflate different entity types (indexed agents + embedded agents + role definitions = one number)
• Non-additive metrics presented in a stacking table (savings that don’t compound but table implies they do)
• Percentages or multipliers with no named baseline (“3-4x faster” — than what?)

Comparative red flags:

• Comparative with no named comparator (“faster,” “more secure,” “leading”)
• Absolute guarantees in stochastic systems (“zero hallucinations,” “100% accurate”)
• Category labels with no standard definition (“enterprise-grade,” “clinical-grade,” “production-grade”)
• “Up to X%” framing that implies typical performance

Structural red flags:

• Flat lists that imply uniform status (products without maturity indicators)
• Demo capabilities presented as production features
• Aspirational specifications presented alongside operational ones (org-chart roles listed as “agents”)
• Cherry-picked time windows or single-customer results generalized

Rhetorical red flags:

• Weasel words: “helps,” “enables,” “supports,” “leverages” (what does it actually do?)
• Passive voice hiding agency: “errors were reduced” (by whom? by how much? measured how?)
• Testimonial without attribution or permission
• “AI-powered” applied to rule-based or scripted functionality

Step 5 — Remediation Recommendations

For each claim rated Needs Context or worse, provide a specific remediation:

• Needs Context: Provide the qualifying language to add
• Unsupported: Specify what evidence would substantiate it, or provide softened alternative wording
• Misleading: Provide rewritten version that includes the omitted context
• False: Provide corrected statement or recommend removal

Output Format

## Claims Integrity Audit — [Document Name]
**Date**: YYYY-MM-DD
**Auditor**: Claims Ombudsman Agent / [Human reviewer]
**Document**: [path or URL]

### Summary
- Total claims extracted: [N]
- Verified: [N] | Supported: [N] | Needs Context: [N] | Unsupported: [N] | Misleading: [N] | False: [N]
- **Overall Integrity Rating**: Clean / Minor Issues / Significant Issues / Fails Review

### Claims Register

| # | Claim | Location | Category | Verdict | Evidence | Remediation |
|---|-------|----------|----------|---------|----------|-------------|
| 1 | [claim text] | Line/section | Obj/Comp/Puff/Reg | Verified/etc. | [source] | [action] |

### Critical Findings (fix before publication)
[List any Misleading or False claims]

### High-Priority Findings (fix this sprint)
[List any Unsupported claims]

### Context Additions (recommended)
[List any Needs Context claims with suggested qualifiers]

Integrity Rating Criteria

• Clean: Zero claims rated Unsupported or worse; fewer than 3 Needs Context
• Minor Issues: 1-3 Unsupported claims, zero Misleading/False
• Significant Issues: 4+ Unsupported or any Misleading claims
• Fails Review: Any False claims, or 3+ Misleading claims

Outputs: Claims register with per-claim verdicts, overall integrity rating, prioritized remediation list, approved claims for evidence registry.