Y12W25RC Evaluating information, seriously

This week’s reading examines how experts actually evaluate the reliability of online information — and the answer is not what most people think.


Stage 1 of 4

Prior knowledge activation

  • The last time you read a striking online claim — something surprising or that challenged what you believed — what made you decide whether to believe it? Did you check anything, or did you go with your gut?
  • Have you ever believed something false online and then discovered it was wrong? What made you realise? How did that feel?
  • When you encounter a claim online that you’re unsure about, what steps do you actually take, if any, to figure out whether to believe it?

Stage 2 of 4

Purpose-setting statement

This article examines how experts actually evaluate the reliability of online information — and the answer is not what most people think. The article presents research on how academics, students, and professional fact-checkers evaluate websites, and identifies a specific skill that separates people who evaluate well from those who don’t. Importantly, this skill is learnable and quite different from traditional ‘information literacy’ instruction.


Stage 3 of 4

Prediction: What do you think?

The article mentions a specific method called ‘lateral reading’ that experts use. Before reading the details, predict: would you evaluate a website by reading it more carefully, or by leaving it and looking up information about it elsewhere? Which strategy seems more likely to reveal whether a source is trustworthy?


Stage 4 of 4

A question to carry into the reading

As you read, notice how the author presents research findings. Are the researchers claiming that most people are stupid for not evaluating well? Or is something else going on? What does their analysis suggest about the gap between how people are taught and what actually works?


Now read

Evaluating information, seriously

~12 min read · ~1,800 words

Here’s a test. You’re reading an article online that makes a striking claim — something surprising, something that contradicts what you previously believed, something that would, if true, matter. The article cites a study. The study sounds credible. The writer sounds confident. The site is one you’ve seen before, though you’d struggle to describe its general reliability.

What do you do next?

If you’re like most people, you do one of two things. You either accept the claim — because the presentation seems reasonable and you don’t have the time or expertise to check — or you reject it — because something about the framing seems off, or because it contradicts your prior beliefs. Neither of these is really evaluation. Both are reactions, not investigations.

There’s a specific skill that distinguishes the people who evaluate online information well from the people who don’t. The skill isn’t intelligence, education, or domain expertise — at least, not primarily. It’s a specific habit, described best by researchers who have spent years studying how experts actually work, and it’s radically different from what most education currently teaches.

What the researchers found

The most important research on how people evaluate online information comes from Sam Wineburg, who directs the Stanford History Education Group. Wineburg’s team has spent years comparing how different groups — academics, undergraduate students, and professional fact-checkers — evaluate the reliability of websites and online claims.

The results were striking enough that they reshaped how information literacy is now taught in many schools.

Academics, it turned out, were not particularly good at evaluating websites. They read carefully. They stayed on the site. They looked for signs of credibility within the site itself — the design, the tone of writing, the presence of citations, the apparent credentials of the authors. But they often failed to notice that the entire site was being produced by a partisan organisation, an industry lobby, or a propaganda operation. They were being evaluative in a way that looked rigorous but was systematically vulnerable to deception.

Students were worse — unsurprisingly — but often in the same way. They also stayed on the site, also evaluated on surface cues, also failed to detect sophisticated deceptions. Training in traditional “information literacy” — checklists of what to look for, tips about evaluating URLs, warnings about grammatical errors as red flags — had surprisingly little effect. The students who had received this training weren’t meaningfully better than those who hadn’t.

Professional fact-checkers did something different. Almost as soon as they arrived at a site they were evaluating, they left it. They opened new tabs. They searched for the organisation, the author, the study being cited, the claims being made. They cross-referenced with multiple other sources. They looked for coverage of the topic from established news outlets they already had reason to trust. Within a few minutes, they typically had a reasonably accurate sense of what the original site was, what its biases were likely to be, and how much weight to give its claims.

Wineburg’s team called this lateral reading — evaluating a site not by examining it more deeply, but by leaving it and checking what the broader web said about it. The insight is important because it runs directly against what traditional reading education emphasises. Schools teach careful, attentive, deep reading of a text. Lateral reading is shallow reading of many texts rapidly. The first is what students are trained to do. The second is what actually works online.

The SIFT method

A practical version of lateral reading, designed for use by students and ordinary readers, comes from the American digital-literacy educator Mike Caulfield. His SIFT framework has become the most widely-taught practical approach in digital literacy programmes.

S — Stop. Before engaging with the content, pause. Don’t react yet. Don’t share yet. Don’t form an opinion yet. Most online misinformation travels because people react before they evaluate. Inserting a moment of deliberation between encounter and response is itself a significant improvement.

I — Investigate the source. Open a new tab. Search for the organisation or author. Look for a Wikipedia entry if one exists — Caulfield, against some traditional academic advice, recommends Wikipedia as a first-pass reality check, because its structural biases are at least well-documented and its entries for major organisations are usually reliable summaries of what the organisation actually is. Find out who’s publishing the content before evaluating the content.

F — Find better coverage. Is the claim being made elsewhere? If so, by whom? Established news organisations with reputations to protect have different error rates and different incentives from obscure sites with no such reputation. This isn’t a claim that mainstream coverage is always right — it often isn’t — but it’s a claim that coverage from organisations whose credibility depends on being roughly accurate is usually a better first-pass signal than coverage from organisations whose credibility depends on nothing.

T — Trace claims. When a specific study, quote or statistic is cited, try to find the original. A startlingly large fraction of online claims, when traced, turn out to misrepresent the underlying source — sometimes accidentally, often deliberately. The original paper often says something more modest, more caveated, or more context-dependent than the article citing it suggests. The three minutes required to find the original claim often tells you more than an hour of debate about whether the claim is right.

The SIFT method has the advantage of being simple enough to actually practise. Most of the traditional information-literacy frameworks are too elaborate for ordinary use. SIFT has four steps, each of which takes seconds to perform, and together they produce a substantial improvement in the accuracy of evaluation.

The context-sensitivity problem

A specific concern raised in the more recent research is that traditional information-literacy training often fails to transfer from neutral tests to emotionally charged content.

Students trained on neutral evaluation skills can demonstrate those skills in class exercises. When the content is politically, emotionally or personally engaging — when it confirms or contradicts beliefs the student holds strongly — the same students often revert to reactive patterns. They fail to apply the techniques they’ve demonstrated they know. The motivation to evaluate carefully depends on emotional distance from the content, and that distance is exactly what’s absent in the content that matters most.

This finding, from researchers including Wineburg himself and also Kathleen Hall Jamieson at the University of Pennsylvania, suggests that information-literacy training has to be designed around the emotional conditions in which evaluation actually happens. Skills that work in neutral conditions don’t automatically work when the content activates the reader’s existing commitments.

The practical response, incompletely developed, is to build specific habits that trigger in the moments of highest activation — the moments when you most want to believe something, or most want to dismiss it. These are the moments when your evaluation is least reliable and most important. Building the habit of stop before sharing, specifically when you feel strongly, is probably more useful than any additional technique.

The domain-expertise caveat

Before making any of this sound simpler than it is, an important complication. Genuine evaluation of technical claims usually requires genuine domain expertise, and there are limits to what lateral reading can do when the subject matter is genuinely specialised.

A non-physicist cannot reliably evaluate claims about quantum physics. A non-lawyer cannot reliably evaluate claims about legal interpretation. A non-epidemiologist cannot reliably evaluate claims about the design of epidemiological studies. Lateral reading helps — you can check whether the claim is being made by someone whose credentials suggest they know the field, you can see whether experts in the relevant field generally endorse it — but this is second-order evaluation rather than genuine understanding.

The honest position is that on most technical questions, most people’s evaluation consists of choosing whom to trust rather than evaluating the underlying evidence directly. This isn’t intellectual laziness; it’s a necessary feature of specialisation. Nobody can be an expert in everything. What matters is whether your trust-choices are themselves well-considered.

This points to a specific skill that’s worth cultivating: learning to recognise the markers of genuine expertise versus confident-sounding amateurism. Experts in a field usually express more uncertainty than amateurs do — they know what they don’t know. They typically situate their claims in ongoing debates rather than presenting them as settled facts. They often disagree with each other in specific, technical ways. They rarely produce confident generalisations about their field. A confident, sweeping claim about a specialised topic is, paradoxically, often a signal that the speaker isn’t a genuine expert in the field they’re speaking about.

What to actually practise

For most readers, the research suggests a few working habits that reliably improve information evaluation.

Pause before reacting. The single most important habit. The content that moves you quickly is the content most likely to be misleading you.

Leave the site to investigate the site. Don’t evaluate on surface credibility. Open new tabs. Check what the broader web says about this source.

Trace specific claims. When a study, a statistic, or a quote is cited, try to find the original. Your evaluation improves dramatically with this single move.

Notice when you’re emotionally activated. This is when your evaluation is least reliable. In these moments, the practice of slowing down, checking sources, looking at coverage from other angles is most valuable and hardest to do. Build it as a habit when it’s easy so it’s available when it’s hard.

Recognise the limits of your own evaluation. On genuinely technical questions, you’re mostly choosing whom to trust. Do that well by being willing to defer to actual expertise — which usually looks calmer, more uncertain, and more specific than the confident voices around it.

The question that remains

The deep thing the research on information evaluation teaches is that the skill isn’t about being clever or being well-read. It’s about a specific set of habits that most people don’t currently have and that most education doesn’t currently teach. The habits can be built. They take some deliberate practice. Once built, they substantially change what you believe to be true about the world.

In an information environment where vast amounts of content are being generated by actors whose incentives don’t align with accuracy — commercial, political, ideological, and now increasingly automated — the skill of evaluating well isn’t a luxury. It’s a condition of being able to think clearly about the world you live in. People without it drift, over years, toward increasingly distorted pictures of reality without noticing the drift. People with it have some chance of staying oriented.

The question worth carrying, the next time you encounter a striking claim online:

Before you decide what to do with this, could you spend two minutes investigating where it actually came from — and if not, why are you about to respond as if you know?

Key research referenced: Sam Wineburg and Sarah McGrew’s Stanford History Education Group research on lateral reading (Teachers College Record, 2019); Mike Caulfield’s SIFT framework (Web Literacy for Student Fact-Checkers, 2017); Kathleen Hall Jamieson’s research on political information and evaluation.