News Credibility Scoring

Instructions

Evaluate and score news sources on a multi-dimensional credibility scale. Credibility scoring is foundational for news curation, fact-checking, and AI-assisted journalism — it determines how much weight a source’s claims carry.

Credibility Assessment Framework

1. Ownership and Funding Analysis

Assess who controls the source and how it’s funded:

Factor	Scoring Impact
Independent ownership	Positive — editorial independence more likely
Corporate conglomerate	Neutral — assess editorial firewall
State-owned/funded	Negative for domestic coverage; may be reliable for international
Nonprofit/foundation	Generally positive — less commercial pressure
Individual/activist	Variable — check for disclosed conflicts of interest
Undisclosed ownership	Strongly negative — transparency is a baseline requirement

Document:

• Parent company and ownership chain
• Funding sources (advertising, subscriptions, grants, government)
• Known conflicts of interest
• Ownership changes in last 5 years

2. Editorial Standards Assessment

Evaluate the source’s editorial practices:

Standard	Present?	Evidence
Published editorial policy	Y/N	Link
Bylined articles with author credentials	Y/N	Examples
Clear separation of news and opinion	Y/N	Section structure
Named editors with contact information	Y/N	Masthead
Fact-checking process (stated or evident)	Y/N	Policy page
Corrections and retractions policy	Y/N	Corrections page
Source attribution in reporting	Y/N	Article review
Multiple sources per story	Y/N	Article review
Transparent methodology for investigations	Y/N	Examples

3. Correction History Analysis

A source’s handling of errors reveals its commitment to accuracy:

• Frequency of corrections: Some corrections indicate accountability, not weakness
• Timeliness: How quickly are errors corrected after identification?
• Visibility: Are corrections prominent or buried?
• Scope: Do corrections address the substance or just minor details?
• Pattern: Repeated errors on the same topic suggest systemic bias

4. Domain Authority and Reputation

Indicator	How to Assess
Journalism awards	Pulitzer, Peabody, George Polk, IRE awards
Press freedom index recognition	RSF, CPJ references
Academic citations	Frequency as source in scholarly work
Peer recognition	Referenced by other credible outlets
Longevity	Years of continuous publication
Legal track record	Libel suits and outcomes

5. Bias Assessment

Evaluate bias separately from credibility — a source can be biased but factually accurate:

Bias Dimension	Indicators
Political lean	Editorial endorsements, story selection patterns, framing
Topic selection bias	What stories are covered vs. ignored
Framing bias	How headlines and leads frame neutral events
Source selection bias	Which experts and voices are quoted
Omission bias	What relevant context is consistently excluded

Use established media bias assessments (Ad Fontes Media, AllSides, MBFC) as starting references, but verify independently.

Credibility Scoring Model

Composite Score (0-100)

Component	Weight	Score Range
Ownership transparency	15%	0-100
Editorial standards	25%	0-100
Correction practices	15%	0-100
Source attribution	15%	0-100
Domain authority	15%	0-100
Track record accuracy	15%	0-100

Tier Classification

Tier	Score Range	Usage Guidance
Tier 1 — Authoritative	85-100	Suitable as primary source; minimal verification needed
Tier 2 — Reliable	70-84	Suitable as corroborating source; verify key claims
Tier 3 — Mixed	50-69	Use with caution; always corroborate with Tier 1-2
Tier 4 — Questionable	25-49	Do not use as source without extensive independent verification
Tier 5 — Unreliable	0-24	Do not cite; note for tracking purposes only

Staleness Detection

Credibility scores degrade over time and with organizational changes:

Trigger	Action
Ownership change	Immediate re-evaluation
Editor-in-chief change	Re-evaluate within 90 days
Major correction/retraction	Review and adjust score
12 months since last assessment	Scheduled re-evaluation
Credibility controversy reported	Immediate re-evaluation
Significant staff layoffs in newsroom	Re-evaluate within 30 days

Flag stale scores in any credibility database with last_evaluated date and staleness_risk indicator.

Credibility Database Schema

{
  "source_id": "",
  "name": "",
  "domain": "",
  "tier": 1,
  "composite_score": 87,
  "scores": {
    "ownership_transparency": 90,
    "editorial_standards": 85,
    "correction_practices": 80,
    "source_attribution": 90,
    "domain_authority": 95,
    "track_record": 85
  },
  "bias": {
    "political_lean": "center-left",
    "confidence": "high"
  },
  "last_evaluated": "2026-04-14",
  "staleness_risk": "low",
  "notes": ""
}

Inputs Required

• News source name and domain URL
• Available information about ownership and funding
• Sample articles for editorial standards review (5-10 recommended)
• Existing bias assessments from third parties (if available)
• Context for evaluation (general credibility vs. topic-specific)

Output Format

## Credibility Assessment: [Source Name]

### Summary
- **Tier**: [1-5] — [Authoritative / Reliable / Mixed / Questionable / Unreliable]
- **Composite Score**: [X]/100
- **Bias**: [Political lean] (Confidence: [High/Med/Low])
- **Last Evaluated**: [Date]

### Component Scores
| Component | Score | Key Finding |
|-----------|-------|------------|
| Ownership transparency | X/100 | [finding] |
| Editorial standards | X/100 | [finding] |
| Correction practices | X/100 | [finding] |
| Source attribution | X/100 | [finding] |
| Domain authority | X/100 | [finding] |
| Track record | X/100 | [finding] |

### Ownership
[Ownership chain and funding analysis]

### Editorial Assessment
[Key observations on editorial practices]

### Usage Recommendation
[How to use this source in journalism/research contexts]

### Staleness Triggers to Monitor
[Known upcoming events that could affect credibility]

Anti-Patterns

• Conflating bias with credibility — A source can lean left or right and still be factually reliable; score them separately
• Binary credible/not-credible — Credibility is a spectrum; use the tiered scoring system
• Static credibility lists — Sources change; implement staleness detection and regular re-evaluation
• Ignoring correction history — Sources that correct errors are more credible, not less
• Single-dimension scoring — A source may be excellent on one topic and poor on another; note topic-specific variations
• Relying solely on third-party ratings — MBFC, AllSides, and Ad Fontes are starting points, not final answers
• Penalizing paywalls — Subscription models often indicate higher editorial investment, not lower credibility
• Ignoring wire service sourcing — Many outlets republish AP/Reuters; assess the original source, not the republisher