How Icarus checks a claim

What the four conclusions mean

Icarus is an evidence-review system, not a truth oracle. It identifies factual claims in the specific X post a requester tagged, gathers current evidence, separates genuinely independent origins from repeated coverage, applies a published scoring policy, and explains what that evidence supports as of a stated UTC time.

• ✅ Supported — at least two independent origins provide sufficiently strong support.
• ⚠️ Misleading or missing context — the statement misrepresents cited material, omits a load-bearing qualification, or the evidence is materially mixed.
• ❌ Not supported by evidence — at least two independent origins provide sufficiently strong refutation, or the evidence contradicts the claim under the rules below.
• ❔ Unverifiable — reliable evidence is absent, insufficiently independent, inaccessible, too early, or outside what this check can establish.

Every conclusion has a low, medium, or high confidence band. These four are the only labels we publish: we deliberately do not reduce a check to a two-way verdict word, because available evidence can change and many factual statements combine accurate details with missing context. These labels summarize the claim-level explanation; they do not describe a person's honesty or intent.

What gets checked

The checked subject is the tagging reply's direct parent. The conversation root and one quoted post may provide context but do not replace that subject. V1 covers English factual assertions anchored in captured post text, note_tweet text, quoted-post text, or legible text read from attached still images and screenshots. Linked HTML and PDF material and non-text image descriptions are evidence and context, not claim wording. Video and audio, named private individuals, personal advice, and non-literal opinion, prediction, satire, or jokes are not given factual verdicts.

The subject is first split into atomic claims, each keeping its original character span and the context that makes it checkable. Models may propose spans and normalized search text, but the displayed claim is sliced verbatim from the captured source artifact. Implied claims are checked only when the implication is necessary and can be stated without guessing the author's intent. Claims not selected because of relevance, scope, or budget are listed as not assessed.

The overall result is the most adverse decided claim: ❌ before ⚠️ before ✅. A ❔ claim does not overwrite a decided claim; when every claim is unresolved, the overall result is ❔. This is a product policy, not a model vote.

Evidence collection and self-evidence

Icarus first asks whether material cited by the subject actually says what the subject claims. It snapshots linked pages, PDFs, and images before analysis. If a quote is incomplete, a population is changed, a qualifier is dropped, or a figure is reversed, that self-evidence mismatch is shown using a verbatim substring of the snapshot.

It then searches in multiple ways: close wording, normalized claim, date-bounded event terms, refutation and negation terms, primary-source queries, and correction or retraction queries. X search is not used as corroboration. A professional ClaimReview is one high-reputation origin; Icarus does not inherit its rating.

Every evidence item records a canonical URL, publisher and domain, publication and event dates when available, retrieval time, an immutable content hash and snapshot, the exact excerpt span, a source-reputation record, directness, validity flags, stance toward each claim, and its provenance cluster. Models refer to server-assigned evidence identifiers, never to model-authored URLs.

Source reputation

Reputation is a prior about a source's editorial and correction practices, not evidence that a particular claim is correct. Political viewpoint never enters the arithmetic. A bias label, when available, is display-only.

1. the Lin et al. research dataset of 11,520 domains as the free continuous baseline — not yet live;
2. Iffy Index (not yet live) and the Wikipedia perennial-sources list as high-precision risk flags;
3. IFCN/Duke identity for professional fact-checking organizations;
4. manually reviewed classifications for primary official sources, wires, aggregators, satire, platforms, and release services;
5. licensed commercial feeds only if coverage measurements justify them;
6. a clearly labeled heuristic estimate for unknown domains.

Each record is versioned and says how it was produced. Unknown is a valid result: deterministic features may estimate a score, but that evidence receives a ×0.60 provenance discount and the report says “reputation estimated, not on file.” Any estimate that could change a verdict is queued for human review — the queue is not yet live, so today an estimate carries the discount and the caveat and nothing more. Verification badges and follower counts do not establish credibility; a state-affiliation label is context, never a refutation.

How independent sources are counted

Ten articles can be one source. Icarus computes an independent corroboration count (ICC) with this algorithm:

1. Canonicalize. Normalize URLs and domains, AMP and print variants, timestamps, bylines, titles, and known wire or release markers.
2. Similarity graph. Remove boilerplate, identify verbatim shingles, embed available text, combine similarity with metadata overlap, and build a graph. Community detection groups probable copies.
3. Citation graph. Parse links and explicit attribution such as “according to,” wire datelines, press-release overlap, and source naming. A calibrated judge may also label semantic attribution that the parser cannot see, and such an answer does create an attribution edge; every edge records whether it came from a parsed link or from a judge, and a judge-only edge can only make an item derivative — never independent.
4. Reachability. If an item links or attributes through any path to another item — or back to an artifact whose claim is under review — it is derivative of that origin, however reputable the republisher is.
5. Cycles. Strongly connected components collapse circular-reporting loops. Cycles are disclosed and can only reduce confidence.
6. Origin election. For each similarity community plus citation closure, choose the earliest first-seen time; ties go to higher reputation and then primary official or wire precedence. When an item carries no publication date its retrieval time stands in, so chronology is rarely absent outright; the election is instead marked uncertain when the cluster sits in a citation cycle, when no member has any timestamp at all, or when the elected origin carries a date/event mismatch, a future publication date, or anonymous sourcing. An uncertain election costs 0.15 of confidence.
7. Weight once. A cluster contributes the maximum weight of any member, never the sum of its articles.

✅ and ❌ require ICC ≥ 2 on their respective side. V1 deliberately has no single-primary-source exception, even when an official source would normally be definitive. A single origin produces ❔ with “single source, developing.”

Evidence weight

Each evidence item receives a deterministic weight:

evidenceWeight(i) = R(i) × D(i) × V(i) × C(i) × P(i)

Validity adjustments are multiplicative: retracted ×0; expression of concern ×0.50; corrected ×0.80; preprint ×0.80; anonymous sourcing ×0.85; disclosed conflict of interest ×0.80; each statistical flag ×0.93; invalid C2PA manifest ×0.30. Satire (unless the claim concerns the satire itself), pure aggregators, content farms, self-citation, and anything reachable from evidence under consideration have zero corroboration weight.

A cluster takes the stance of its highest-weight member, and MIXED when members above the counting threshold disagree; a MIXED cluster counts toward neither side.

clusterWeight(c) = max(evidenceWeight of members in c)
S  = sum of supporting cluster weights
Rf = sum of refuting cluster weights
polarity = (S - Rf) / (S + Rf), or 0 when S + Rf = 0
ICC_s = supporting clusters with weight >= 0.45
ICC_r = refuting clusters with weight >= 0.45
diversity = number of distinct source tiers among counted clusters

Claim decision thresholds

The policy applies these rules in order:

1. A verified self-evidence misrepresentation — overstatement, reversal, dropped qualifier, changed population, or cherry-picked figure — produces ⚠️.
2. Genuine conflict produces ⚠️ when min(S, Rf) >= 0.30 × max(S, Rf).
3. Mixed evidence produces ⚠️ when abs(polarity) < 0.50 and S + Rf >= 1.20.
4. Otherwise, ✅ requires polarity >= +0.50 and ICC_s >= 2.
5. Otherwise, ❌ requires polarity <= -0.50 and ICC_r >= 2.
6. Otherwise, ❔ applies, including S + Rf < 0.60, fewer than two qualifying origins, an evidence chain deeper than three unsupported inference hops, or an ineligible disposition.

Rule 1 has highest precedence because a post can cite a real source yet misstate it. Rule 6 means finding nothing is not refutation: no usable evidence always produces ❔, never ❌.

Confidence

Confidence is computed from the evidence graph, not requested from a model:

confidenceScore =
    0.30 × min(1, (S + Rf) / 2.5)
  + 0.25 × min(1, max(ICC_s, ICC_r) / 3)
  + 0.15 × min(1, diversity / 3)
  + 0.15 × abs(polarity)
  + 0.10 × meanStanceConfidence
  + 0.05 × temporalFreshness
  - penalties

Penalties are 0.15 if a counted origin is uncertain; 0.10 per counted heuristic-reputation cluster, capped at 0.20; 0.20 for any circular-reporting component; and 0.10 when a correction or an expression of concern is open on counted evidence, whatever the claim's temporal scope. Scores are clamped to 0–1. High is ≥0.70, medium is ≥0.45, and low is below 0.45. Published confidence is the band, not a spurious percentage.

Hard gates G0–G7

Gates run after scoring and can only weaken or prevent a conclusion:

Deterministic validity rules H1–H30

These rules do not depend on free-form model reasoning. “Exclude” means exclude from corroboration, not erase from the evidence ledger.

C2PA can establish a chain, not the truth of a caption; no manifest is neutral; platform re-encoding often removes credentials. Icarus never cites an AI-text-detector score as evidence about authorship.

The role of models and judges

Models assist with claim-span proposals, query plans, OCR and vision, narrow stance and attribution questions, and cited explanatory drafts. Typed judges answer typed questions; they do not decide verdicts. Every answer records its provider and version and must pass calibration and schema checks. Versioned code applies the published rubric, and code validation — not a model — is the final authority on published output.

Developing stories and rechecks

When a story is still developing, Icarus publishes ❔ immediately with an “as of” UTC time, then checks again at +15 minutes, +1 hour, +6 hours, and +24 hours. Every completed version appears in the report timeline. A follow-up reply on X is considered only when the four-label conclusion changes; evidence and confidence changes remain visible on the report.

Today a check goes on that schedule when any claim is about an ongoing situation, is ❔ because only one origin survived (G1), or is held open by an active correction window (G5). Those triggers share the same four offsets and idempotent schedule repair.

Published but not yet live

This page is the rubric every report is scored against, and it is published in full rather than trimmed to whatever happens to be finished. Some individual rules above are written but not yet active in the running system. They are marked “not yet live” where they appear and collected here. A rule on this list affects no published conclusion today.

• Reputation datasets. Only manually reviewed records, the Wikipedia perennial-sources list, and IFCN identity are loaded. The Lin et al. 11,520-domain baseline and the Iffy Index are not ingested, so a domain missing from those sources falls through to the labeled heuristic estimate rather than to a dataset score.
• H4 registry-review queue. An estimated reputation carries its ×0.60 discount and its caveat, but nothing is queued for a human to review.
• H9 first-report exclusion. A publication/event date mismatch is flagged and makes an elected origin uncertain; it does not exclude the item from the election.
• H10 domain registration. No registration data is held, so a publication date earlier than a domain's registration cannot be detected. Future dates are detected.
• H12 load-bearing statistics. The percentage/percentage-point flag applies its ×0.93 but does not force ⚠️.
• H13 arithmetic recomputation and H14 order-of-magnitude conflict are not implemented.
• H15 numeric-context caveat. Relative risk without a base rate is flagged and discounted, but the numeric-context-missing caveat is never attached.
• H16 retraction lookup and H22 C2PA verification. Both rules are implemented and tested, but no production lookup is wired in, so neither observes a real retraction record or a real manifest today. A retraction that reaches us another way still triggers G4.
• H17 journal metadata. Preprints are detected from the host and from preprint wording; missing journal metadata alone is not a trigger.
• H19 release overlap. A release host or release-wire flag demotes directness. The ≥0.70 overlap measure is used for clustering only.
• H21 coordinated amplification. Near-simultaneous near-identical items are collapsed into one cluster, which is the part that protects the count; the caveat naming the pattern is not emitted.
• H24 soft 404 and content drift. An archived snapshot is used when a page cannot be snapshotted; soft-404 pages and pages whose content hash has changed since retrieval are not detected.

Limitations

• Search coverage is never complete. Paywalls, removals, indexing delay, regional access, and provider outages can hide relevant evidence.
• Source reputation is a fallible prior at the domain level. It cannot prove an article correct and may be estimated for new domains.
• Publication dates and first-seen timestamps are often noisy. Origin election is disclosed as uncertain when chronology is weak.
• Independence analysis detects links, wording, timing, and attribution but may miss a shared unnamed source.
• OCR can misread or invent image text; chart interpretation, sarcasm detection, identity resolution, and image-context analysis can fail. Image-derived claims are labeled “text read from image,” and material OCR uncertainty causes abstention.
• Two independent origins can still share the same error. ICC is a necessary corroboration rule, not a guarantee.
• “Not supported by evidence” describes the documented search and evidence set as of the report time; it does not prove metaphysical impossibility.
• Our thresholds encode a conservative preference for abstention. Their effect and calibration are continuously measured.

Corrections, deletion, and version history

Every report records its evidence snapshots, methods, policy, prompts and models, and “as of” time. Rechecks and corrections create a new version; they never silently overwrite the old result. The latest report explains what changed and why. If evidence weakens — for example, a supporting paper is retracted — confidence may fall and the verdict is recomputed.

We correct the report first, then issue a concise follow-up on X when needed. If the source X content is deleted or protected, its redistributed text or image is hidden within 24 hours; the verdict, the evidence and the update log remain. Reports are indexed by post and check, never by person.

Methodology changes receive a new version and date. Threshold changes are evaluated against a held-out, multi-annotator set, and adversarial fixtures for circular reporting, wire pickup duplication, press-release laundering, retractions, screenshots, statistics, and prompt injection run on every release.