How the Watchdog Works

Every score runs from 0 to 10. It is directional, not a generic reading of bias against neutrality. Each topic measures one specific direction of framing, and only that direction:

• Gazaanti-Palestinian framing

  Measures anti-Palestinian framing in Gaza coverage. 0 means none detected, 10 means severe.

• Lebanonanti-Lebanese framing

  Measures anti-Lebanese framing in Lebanon coverage. 0 means none detected, 10 means severe.

• Irananti-Iran framing

  Measures anti-Iran framing in Iran coverage. 0 means none detected, 10 means severe.

• UK Politicsuneven treatment of the UK parties

  Measures uneven treatment of the UK parties and their leaders. 0 means even-handed coverage, 10 means severe asymmetry.

This is the most important thing to understand about the number. An outlet whose framing runs the other way scores near 0, because the scale only ever counts framing in the single direction each topic is built to detect. It never goes negative. A low score records the absence of the measured framing, not a reward for leaning the opposite way, and a 0 is not a certificate of neutrality. The score is not a general trust rating.

The watchdog scores a fixed, curated panel of 52 outlets. The same panel is scored the same way on every topic, so a score on one outlet is directly comparable with any other. The panel spans five source types:

The panel does not change from topic to topic. That is deliberate: a fixed panel is what makes the leaderboard comparable across topics and over time.

Each article we collect on a topic is read by a large language model, Anthropic's Claude. For every metric in the topic's rubric the model returns a 0 to 10 score and, crucially, must ground that judgement in a verbatim quote from the article itself. We call these the receipts.

The quotes are stored alongside the score. They are what you see when you click any score on the site: the exact words that led to the judgement, so you can check the working rather than take it on trust.

The individual metric scores are then combined into the article's overall 0 to 10 score using fixed weights. The weights do not change between articles. They are listed in full below.

Each topic has its own rubric. Every metric carries a fixed weight, shown here as a percentage of the article score. These are the live weights the scorer uses. Open a topic to see its full breakdown.

An outlet's score on a topic is the median of its article scores over a rolling 90-day window. We use the median, not the mean, so a single unusual article does not swing the figure.

Alongside the median we show a consistency range: the middle 50% of an outlet's article scores, from the 25th to the 75th percentile (the interquartile range). A narrow range means the outlet frames the topic much the same way across its coverage; a wide range means its framing comes and goes.

An outlet is only ranked once it has published enough articles on the topic for the median to mean something. Below that threshold it shows as "not enough data" rather than being handed a score it has not earned. The minimums are:

• Gaza8 articles
• Lebanon5 articles
• Iran5 articles
• UK Politics10 articles

The full panel has only been scored continuously since 22 June 2026, so the 90-day windows are still filling. Expect ranges to tighten and more outlets to cross the ranking threshold as coverage accumulates.

The method has real limits. We would rather state them than have you assume they are not there.

• It measures one direction of framing per topic, by design. It is not a general trust score and not a verdict on whether an outlet is accurate, honest, or good.

• It is generated by an automated model, which can misjudge an individual article, especially opinion pieces, explainers, and very short items. There is no published accuracy figure and no human re-scores every article. The safeguard is transparency: every score links to the quotes it was based on, so you can check any judgement yourself.

• Coverage depends on what each outlet publishes and on what we are able to collect and access. An outlet that publishes little on a topic will have a thinner, less settled score.

• The score is a measure of the text as published. It is not a claim about any individual journalist's intent, and it does not attribute motive to any writer or editor.