On January 26, Ottawa-based think-tank Public Policy Forum (PPF) released a dire report on the state of news media in Canada —The Shattered Mirror: News, Democracy and Trust in the Digital Age — to largely negative criticism from pundits. Yet one issue often missed in the disparagement of the PPF report is its perpetuation of the notion of a sudden fake-news epidemic.
The 108-page, $270,000 report commissioned (and mostly funded) by the federal Liberal government paints a dark picture of a world where fake news had an outsized influence on the outcome of the U.S. election and now poses an imminent threat to Canada’s democracy as well. The report references “Trump” and “fake news” at least 13 and 35 times, respectively (depending how you count).
“It was the U.S. presidential race that served as the final punctuation mark on an annus horribilis for the news industry, as the dangers of filter bubbles and fake news manifested themselves on the giant platform sites, particularly Facebook, but also Google,” claims the report in its ominous introduction.
Further down, an Ipsos poll —commissioned and reported by BuzzFeed — is selectively cited to explain the scope of the fake-news phenomenon. The poll surveyed 3,015 Americans on both real and fake news headlines to see if voters were familiar with the election-related stories in question, and if they were, had they believed them. Each respondent was shown three fake-news headlines (randomly drawn from a selection of five) and three real-news headlines (randomly chosen from six).
The PPF report uses a bar graph, which shows the “percentage of respondents who thought the headline was accurate” for all five fake news stories tested. For instance, the graph shows 85 per cent of Trump voters (165 respondents) and 52 per cent of Clinton voters (106 respondents) had believed the fake story “FBI Agent Suspected in Hillary Email Leaks Found Dead in Apparent Murder-Suicide.”
However, the total sample size of respondents shown that headline was actually 1,809, of whom just 22 per cent said they had seen or heard about the story. Only the subset of people who indicated they were already familiar with it were asked to rate its veracity.
On average, just 16.8 per cent of American voters who responded to the poll had even heard of the fake news stories. The bar graph included in the PPF report looks only at this relatively small portion of respondents, yet makes no mention that it is a subset. Instead, the report claims: “In post-election research, BuzzFeed and Ipsos found that large majorities had believed such stories as Pope Francis endorsing Donald Trump.” The way the sentence is written and accompanying data is presented, a reader unfamiliar with the original poll would be left with the impression that large majorities of Americans had seen and believed fake-news headlines, when in fact the question was posed only to the small minority of respondents who had even heard of the fake stories.
When contacted for an explanation of the ambiguity of Shattered Mirror’s presentation of the fake-news data, PPF director of communications Carl Neustaedter stood by the report.
“We faithfully reproduced two BuzzFeed charts in The Shattered Mirror. They are both properly sourced, and they or the accompanying text summarize the procedure Buzzfeed/Ipsos used. Furthermore, the chart on page 59 also indicates the number of respondents to the questions, which helps the reader understand the sample sizes. We stand by our characterization and presentation of the BuzzFeed data, and that the fake-news phenomenon is a serious enough problem to be included in a report whose primary aim is to examine what threats to democracy exist or may be emerging in the current and fast-changing news media landscape.”
Neustaedter then referred me to BuzzFeed media editor Craig Silverman for further comment.
“You’re right that they did not specify that the people rating the headline accuracy were only those who said they recalled seeing the headline,” Silverman wrote in an email. “But I think they addressed that by listing the number of respondents for each headline. In that respect, they were transparent about the number of people giving those ratings, and didn’t give the impression that all 3,000 people in our survey had rated them. If they hadn’t given the respondent number, then I would have been concerned, but it seems like they are accurately presenting the data.”
The PPF report contains no mention of the 3,015 people who participated in the survey nor the 1,809 people shown each fake-news headline.
The report also includes a BuzzFeed line graph from an article that indicated the reach of fake news spiked in the last three months of the election, with the top 20 fake election stories outperforming the top 20 real ones in Facebook engagement in this time period. However, Facebook engagement (e.g., likes, comments, emoticons, shares, etc.) doesn’t necessarily translate to actual consumption of news, and some of the data used in this study was incorrect.
Further undermining the report’s dire warnings of rampant fake news taking over, a Northwestern University study found that fake news’s audience was ten times smaller than that of real news. The extensive study also concluded that, contrary to BuzzFeed’s determination, fake-news consumption remained at a constant low level throughout the election, while real-news consumption spiked in the last six weeks of the campaign. (It should be noted, however, that the study used a broader definition of “fake news” than did BuzzFeed, including satirical sites and highly partisan aggregators in addition to publishers of hoaxes.)
Before Shattered Mirror was released, Canadian Heritage Minister Mélanie Joly suggested that the government could play a role in combating fake news. Now she says she will be studying the report as she decides to overhaul the government’s approach to subsidizing Canadian content.
However, the minister’s press secretary, Pierre-Olivier Herbert, says in a phone call that the government is not interested in policing fake news. He also explains that The Shattered Mirror will not be given precedence over hundreds of other submissions in informing Joly on how to move forward on government funding of CanCon.
“We got hundreds and hundreds of submissions,” he says. The report is “going to be treated like any other submission.”
Prime Minister Justin Trudeau will be hosting the PPF’s annual Testimonial Dinner in April. When asked about a potential or perceived conflict of interest, the PPF’s Neustaedter responded: “As a non-partisan and independent organization, we have invited hosts of all political stripes over our 30 years, from Brad Wall (Saskatchewan Party) to Darrell Dexter (NDP) to Prime Minister Trudeau (Liberal), to help us celebrate public service. The host’s role is unpaid.”
