STATISTICS IN ONE HOUR: Smelling the rat between the numbers, by Hans van Maanen
- I. General introduction: Smelling the rat
Here, we’ll first try to emphasise that numbers are always produced and presented by people, usually after being polished, edited and thoroughly cleaned. There is no such thing as raw data, so we always have to wonder :
- why we get these figures
- why we get only these figures
- how they were put together and assembled for our consumption.
- How did they measure it?
- Is this the complete data set?
- What else do we know?
- II. Basic tools: Finding the rat
In the second part, we’ll show how to put these simple questions to work and find out how to get the answers from one scientific article — if we cannot find the answer, maybe the article is at fault, and not the journalist. If we do find the answer, how do we translate the answers to a format our readers will understand?
- Is a ‘statistically significant’ finding always relevant?
- What is the net benefit for a patient or a consumer?
- III. For fun and profit: Dissecting the rat
Next, the audience gets a quarter of an hour to read and analyse another scientific paper, and shoot some holes in it — we’ll take an example that has quite enough holes to keep everybody busy and amused… There were 14 more heart deaths on the day of an exciting soccer game — could that really be the cause, as the authors implied? How about other exciting matches, how about other days with a lot of heart deaths? >>> http://www.bmj.com/content/321/7276/1552
- IV. Picking up the remains: Post-mortem
Finally, the objections raised to this paper will be discussed and put into context. Just by discussing them, more and more questions arise, until we wonder why the paper has ever made it through peer review published. Bad examples can be very educational, and by the end of the session, the participants will hopefully have developed a taste for critical thinking about science and numbers.
Blood Infusion for Staggering Science Journalism
Capacity Building Workshop at the World Conference of Science Journalism WCSJ 2013 Helsinki
**) For some more and in-depth statistics, see http://www.wfsj.org/course/, lesson 9.
A few impressions from the presenters and some participants
(7 short videos, you can skip from one to another
– Apologies for the bad quality of the audio)
Hans van Maanen is a veteran Dutch science journalist. In 1981, after finishing his sociology studies he started as a reporter for Haarlems Dagblad. In 1998 he became science editor for the Amsterdam newspaper Het Parool. Since 2003 he has been a freelance journalist, writing both for newspapers and for (popular) science magazines. Today he is best known for a weekly critical column in De Volkskrant in which he exposes sloppy science and bad thinking.
He wrote some twenty books, mostly on games, statistics and popular science. In 2007, he was awarded both the ‘Van Walree Prize’ for medical writing by the Royal Dutch Academy of Arts and the ‘Eureka Prize’ for science writing by the Netherlands Organisation for Scientific Research. He teaches science writing, journalism, and statistics at several Dutch universities.
Van Maanen authored the chapter ‘Understanding statistics’ for the WFSJ Online Course in Science Journalism, and was co-organiser of a session ‘How to read medical studies (and avoid pitfalls)’ at the World Conference of Science Journalists 2011.