University of Michigan develops tool to measure 'iffy' news ahead of mid-term elections

Photo: MaxPixel

ANN ARBOR – 'Fake news,' the phenomenon, began in 2016 but the debate surrounding credible news stories is still alive and well.

The University of Michigan Center for Social Media Responsibility, housed in the U-M School of Information, has released a tool to help the media and members of the public to weed out fake news on social media using a metric called the Iffy Quotient, the center's first public assessment tool.

The Iffy Quotient is a Platform Health Metric, designed to determine the health of an overall platform. 

So how does it work?

Users can access a web-based dashboard that displays the Iffy Quotient for social media sites Facebook and Twitter. It is dated back to 2016 and will be continuously updated.

The metric draws data from Media Bias/Fact Checker and NewsWhip, two external sites which monitor media bias and social media engagement, respectively.

NewsWhip tracks site engagements on Facebook and Twitter from URLs on hundreds of thousands of sites daily.

Photo: Pexels


Iffy Quotient works by asking NewsWhip for the top 5,000 most high performing URLs on both social media platforms. The tool then checks to see if Media Bias/Fact Check has flagged any of those domain names. 

The Iffy Quotient divides the URLs into three categories based on which Media Bias/Fact Check list they appear on:

  • "Iffy" - if the site appears on the Conspiracy or Questionable Sources list
  • "OK" - if the site appears on any other list, such as Satire, Left-Bias or Right-Bias
  • "Unknown" - if the site does not appear on any list

Following extensive analysis of content during the 2016 U.S. presidential election, the Center for Social Media Responsibility concluded that the Iffy Quotient skyrocketed on Facebook and Twitter during the election.

Below are some of their findings:

  • On both Facebook and Twitter, the Iffy Quotient approximately doubled from January to November 2016.
  • On both Facebook and Twitter, the Iffy Quotient approximately doubled from January to November 2016.
  • The Iffy Quotient was higher at Facebook than Twitter in 2016 and into 2017.
  • The Facebook Iffy Quotient has declined steadily since early 2017 and has now returned to its early 2016 levels. 
  • The Twitter Iffy Quotient has not declined much and is still nearly twice its level in early 2016.

To summarize, Twitter remains a space for sharing iffy stories, while Facebook has done a better job at filtering out unreliable content.

"We at the School of Information are committed to meeting the intellectual and social challenges of this new era of unregulated public communication via social media," dean of the School of Information, Thomas Finholt, said in a statement. "At the Center for Social Media Responsibility, we are working directly with social media companies to produce the designs, systems and metrics to steer social media toward more beneficial discourse.

"Contributing to the accountability of social media platforms is one reason why the CSMR was created. This is an important area of focus for our faculty research. Our aim is to serve as a valued resource in the battle against misinformation."

While CSMR leaders say the tool is useful for determining reliable sources, it is not 100 percent accurate. For instance, they say sites marked as "iffy" can publish credible information. NewsWhip has also, on occasion, missed some popular URLs, and Media Bias/Fact Check sometimes has a delay in listing newer sites, since they can be created quickly.

Due to these limitations, the authors of the report warn not to over-interpret the Iffy Quotient.

Read the full report.

Like what you're reading? Sign up for our email newsletter here!

About the Author: