What a shambles. A study came out that supposedly proved Dutch media have a pro-Israel bias. Nobody in their right mind believed it, but everyone was curious about the evidence. After all, it was in Trouw! In De Groene Amsterdammer! It was on NPO Radio 1! Then the methodological account turned out to be a joke, and the eventual ‘full study’ a panicked retreat. It takes a long-form piece to map out this shambles.

A study appeared that supposedly showed Dutch news coverage to be skewed in Israel’s favour. The researchers were given space to write pieces for Trouw, once a Dutch resistance newspaper, and for the weekly De Groene Amsterdammer. NPO Radio 1 — on its programme Mediaforum — reported approvingly on the study. But how much of it held up?

The study’s methodological account is awash with figures and technical terms. Lay readers soon lose their way and will assume, in awe, that the scientific underpinning must be rock-solid. But one golden rule of scientific models is this: the output of the calculations is never better than the data the researcher put in.

Anyone who looks through the smokescreen of academic jargon sees a research design that worked in every respect towards the conclusion already packaged in its central research question.

Tweet text: To everyone’s surprise, our media turn out to be PRO-Israel.🤔 The PDF of the study: https://groene.nl/uploads/upload/file/145/Gaza_verantwoording__1_.pdf. The starting point of the ‘neutral’ researchers: there is/was a genocide in Gaza. Was that the yardstick against which the media were measured?😳 This calls for a fact-check.😋 Quote tweet: You know what I find so remarkable? That a ‘study’ claiming to show (where’s their data?🧐) that the anti-Israel media are in fact PRO-Israel🤣 gets picked up by ALL the anti-Israel media. Our study suited them less well.😏 https://archive.ph/wC5Zc

Genocide as the starting point

The researchers set out to examine ‘…how Dutch news media have reported, since the Hamas attack of 7 October 2023, on the Hamas terrorist attack and the subsequent genocide in Gaza.’ So they were not examining the journalistic quality of the coverage; they were examining the extent to which these media reflected their own activist opinion.

This wording was the first cardinal sin of science, and since it was the central research question, the entire study could really have gone straight into the bin. The researchers then sent this skewed central question to more than a hundred journalists at the media outlets under study, asking to interview them about it.

It is no wonder that more than half of the journalists and editors they approached declined the honour. In the end, the researchers interviewed the remaining group alongside a group of experts they had selected themselves, without making clear in the methodological account which answers came from which group.

Panic?

And then the final version of the study appeared. In it, the central research question had been rewritten as: ‘To what extent was there bias in Dutch reporting on the conflict between Israel and Hamas from 7 October 2023 onwards?’ Hey presto, and genocide as the starting point was gone. And this after the researchers had approached all those journalists with the original question.

It gets even crazier. While both Trouw and De Groene Amsterdammer reported at length on conversations with these journalists — which also featured in the methodological account — none of these interviews could be found in the full version of the study. It does not even contain a single reference to them.

Interviews

In the methodological account, under the heading Interview Themes, the conversations are summarised. This section is full of activist language such as white/Western European gaze, Holocaust guilt, fear of antisemitism, self-censorship and old white men. Such language is characteristic of identity politics and anti-Israel activism, which is hardly surprising, given that these interviewees saw nothing wrong with the activist invitation.

Under the heading Capacity for Self-Learning appears the telling term genocide turn. Is it a sign of a capacity for self-learning when media start saying that Israel is committing genocide? This says more about the researchers’ bias than about the media. These interviews also mentioned CIDI intimidation. Intimidation means things like disrupting meetings, making threats, vandalising the property of people who think differently and harassing people at their homes. CIDI (the Centre for Information and Documentation on Israel) writes, fact-checks and occasionally takes legal action. That is not intimidation; it is substantive debate.

Tweet text: ‘Let’s hope Roelen was a better soldier than he is an amateur historian, because the blunders are piling up.’ The opinion piece by former soldier Niels Roelen in NRC was riddled with errors. If you want to condemn Israel, you need to get your facts straight. cidi.nl: Anyone who wants to condemn Israel must first know the facts

Flawed search terms

The bias is evident in the study itself, too. It starts with the search terms: ‘gaza OR hamas OR (israël AND (palestina OR bezetting OR idf OR netanyahu OR gazastrook))’ (Dutch for Palestine, occupation and Gaza Strip). According to the researchers, conditions were added to the search term Israël to filter out news about Israel unrelated to the war — football matches, the Eurovision Song Contest.

The question is whether this also caused articles about the aggression and discrimination Israelis face worldwide to slip through the net. How many articles about the Amsterdam Jew hunt (the attacks on Israeli football fans in November 2024) were lost as a result? Did articles about the intimidation of Eden Golan — Israel’s Eurovision contestant — still make it into the dataset?

And even if these search terms did pick up such news, the question remains why they lacked words that would yield articles about aggression against Israel. Why nothing about rockets? Attacks? Terror? Antisemitism? The search terms attached to Israel put the country in the dock from the outset.

The selected articles also turn out not all to be articles. The researchers appear to have made no distinction between columns, letters to the editor, blog posts, news articles, features and (editorial) commentary.

Israeli and Palestinian suffering

The researchers also assessed how much attention the media paid to Israeli and Palestinian suffering. They concluded that Palestinian suffering received far less attention than Israeli suffering. Anyone who follows the news closely knows that for years we have been flooded with Palestinian suffering, while Israeli suffering barely gets a mention. How could this study possibly reach that conclusion?

First, because of this: ‘…every mention of Israeli hostages in an Israel/Palestine context is regarded as Israeli suffering.’ That explains a lot. So every time Israeli hostages were coldly mentioned as bargaining chips in negotiations, the researchers counted it as ‘attention to Israeli suffering’. No doubt that made the tally climb nicely.

Second, they claimed that Palestinian victims received less attention than Israeli victims. In doing so, the researchers ignored the fact that a large proportion of the Palestinian dead were terrorists, and that a deliberately murdered civilian or hostage receives more attention than someone killed during the elimination of a terror base or rocket launcher.

And this study is a slow-motion train wreck, so there was a new development.

New revelations

Now that the full study is out, it turns out that Palestinian suffering received more attention than Israeli suffering all along. Even immediately after 7 October, as a chart on page 18 shows.

Tweet text: Yesterday: ‘Israel gets the most space in Dutch media.’ Anyone who actually reads the Erasmus report sees that the pro-Palestinian side in fact gets the most headlines about suffering. And those misleading headlines steer public opinion! Plus: TV talk shows and news bulletins weren’t even examined!

Anyone who carefully rereads the articles in Trouw and De Groene Amsterdammer can see how the researchers managed to paper over this reality. The authors justified their claims of ‘too little attention to Palestinian suffering’ by relating the number of headlines directly to the number of deaths. And since Hamas forced the far stronger Israel into a war and then hid behind the civilians of Gaza, far more people die on the Palestinian side than on the Israeli side. Scoreboard journalists then cite this as proof of Israel’s wickedness.

Incidentally, under the heading automated content analysis — in the methodological account: automated quantitative content analysis — the rules have been changed. Suddenly, Palestinian prisoners count as Palestinian suffering, whereas this was not the case in the methodological account. Which raises the question of which rules were actually applied. This is supposed to be scientific research, isn’t it?

Who gets to speak?

The next sub-question was: ‘Which sources are given a prominent place in the articles, and in what terms are their statements rendered?’ This sub-question has been revised too, and is now split into two long-winded sub-sub-questions that I will spare you here.

To answer the question about sources, the researchers looked only at direct quotes in the headline or the lead of articles. Incidentally, 40 per cent of these quotes turned out to be unusable, which raises the question of how representative the remaining quotes are.

There are two further problems. The first is how the sources were classified. The category Israeli covers not only the Israeli government but all Israeli sources, including anti-Netanyahu sources such as Haaretz and opposition politicians. Yet in the Trouw article, the researchers presented all Israeli sources as a single source in a chart, with this line above it: ‘For example, they regularly quoted Netanyahu and spokespeople for the Israeli army (see the chart below).’ That is misleading.

The second is that quotes from Israeli politicians are not always positive. Former Defence Minister Yoav Gallant has been misquoted countless times as though he made no distinction between Hamas terrorists and civilians, when he very clearly did make that distinction.

And isn’t it actually merciful not to quote certain Palestinian sources? Think of statements by Hamas leaders such as Ghazi Hamad: ‘We will repeat the October 7 attack time and again until Israel is annihilated.’ Or Ismail Haniyeh: ‘We need the blood of the women, children, and elderly in Gaza so it awakens the spirit of resolve within us.’

Euphemisms

The second half of the previous sub-question concerns the terms in which statements are rendered. The section on Language Choices states: ‘Euphemisms are softening or obscuring formulations that can weaken the emotional charge or gravity of an event.’

In Appendix 1 of the methodological account, they wrote that lost their lives or fell are euphemisms for murdered. But lost their lives is a normal term for the death of civilians during military operations, when those civilians are not the target. Likewise, fell is a standard term for fighters killed in combat. Murder means deliberately killing someone outside of any combat.

And once again, the final version holds surprises. The list of euphemisms has been changed. Whatever you do, don’t ask for logic: it now states that lost their lives, killed, liquidated and fell are all euphemisms for murdered. Sure, why not. Liquidated is now a euphemism for murdered. Is this still science?

Incidentally, in the Trouw article the researchers came up with this bizarre remark: ‘Journalists also described violence against Palestinians in softening terms almost three times as often. Palestinian victims “lost their lives” or “fell” in an “incident”, while Israeli victims were “slaughtered” or “murdered” in a “terrorist attack”.’

In effect, they were arguing here that Israel’s military operations should be described in the same terms as the mass murder Hamas committed on 7 October.

Genocide as dogma

In Appendix 2 of the methodological account — Coding Instructions: Genocide — the researchers explained that every time genocide was mentioned as fact, they classified it as DEED. When media wrote about the genocide accusation without passing judgement, the researchers classified it as DEBATE.

The full report shows that they were so keen to file articles under DEED that a single mention that could count as DEED could turn an entire article about DEBATE into DEED (page 22). And to pollute the statistics a little further, objecting to the term genocide also counted as DEED (page 24, on De Telegraaf). And it gets even less logical.

A correction Trouw published after reporting the rulings of the International Court of Justice (ICJ) incorrectly turns out to fall under DEBATE (page 11). Trouw had written that the ICJ had ruled that Israel was guilty of genocide in Gaza. In the correction, Trouw wrote: ‘This is incorrect. In early 2024, the court ruled in several interim orders that Israel must prevent genocide in Gaza. (…) The final ruling in the case against Israel is still to come.’ So in this study, this ICJ ruling counted as DEBATE, not as DEED.

Here, by contrast, is what did count as DEED: ‘We also coded it as DEED when an authoritative institution, through experts and often on the basis of research, had concluded that genocide was taking place.’ This was followed by a quote from the news site NU.nl, which has been speaking of genocide since December 2024 on the authority of Amnesty. From this combination, one might conclude that, according to these researchers, Amnesty is better placed to judge whether genocide is taking place than the International Court of Justice. Still with me?

Contradictions

And as if the shoddy study itself were not chaotic enough already, the researchers regularly contradicted themselves in the articles they wrote about it. On page 8 of their methodological account, the researchers wrote: ‘For the four journalists who left without a statement, the Gaza coverage weighed heavily. Eduard Cousin and Melvyn Ingleby belong to this group.’ So this states that Cousin left the NOS (the Dutch public broadcaster) without a statement.

Then this is how the article in De Groene Amsterdammer opened: ‘It is Friday morning, 16 May 2025, and Eduard Cousin types: “Dear colleagues, I am leaving the NOS.”’ Three paragraphs followed, setting out at length what his statement said about the NOS’s Gaza coverage. Incidentally, this Cousin is by no means a neutral journalist; he left to join the anti-Israel foundation The Rights Forum.

This chaos pales in comparison with some passages in the full report. Let’s hope it really is complete, because if a sequel follows, it will probably end in embarrassment again. Do you know what the final sentence of the full study is? ‘Even where there is a ruling that Hamas is committing genocide, we code it as OTHER, because here we are looking purely at Israeli violence.’ You might almost think there was some bias at work here. Almost.

Botched job

It is a travesty that Trouw and De Groene Amsterdammer gave this botched job such a platform. The same goes for the approving discussion on NPO Radio 1. Anyone who reads the methodological account can judge the standard of this study for themselves. And anyone who puts the full study next to it will feel an urge to lay wreaths at the nearest Dutch university.

Incidentally, that this ‘full study’ was a panicked rush job is also evident from all sorts of sloppiness: on page 3, an unfinished source reference ‘(@)’ still sits in the middle of the text; on pages 23 and 24, Israel is suddenly spelled ‘Israëll’; below the chart on page 23, chunks of text appear twice; and on page 27, international research ‘concludeerd’ (a misspelling, with a d, of the Dutch for ‘concludes’) the same as these bumbling researchers.

This is what happens when activists try to prove something that does not exist. It is staggering that Erasmus University Rotterdam facilitated this. One wonders whether Dr Nel Ruigrok, who supervised the study, still teaches her students anything resembling journalistic standards and values. One also wonders whether she supervised these beginners at all, or only stepped in, alarmed, after their articles and methodological account were published. Incidentally, Mediaforum stated in its programme notes that the Dutch journalists’ union NVJ also contributed to the study.

One could almost despair of the future of Dutch journalism, but there are a few glimmers of hope. At the end of the Trouw article are the responses of the media outlets examined. Even the NOS and NRC are critical of this study, and so far neither has published anything about it. That gives news consumers heart.

This article was originally published in Dutch.

You can support the work of Maaike van Charante at repelsteeltje.backme.org

Want to stay up to date with new articles and podcasts? Follow Maaike on Twitter.