Is it verifiable? What many students often miss when pitching fact-checking story ideas

Understanding the process of verification could be difficult to many

Photo by The Climate Reality Project on Unsplash

Identifying fact-checkable claims is not always easy.

After all, a lot of information we come across in our daily lives is not something we can evaluate by its factualness (opinions, for example).

Even when some statements or social media posts are verifiable in theory, it does not mean finding and gathering evidence is feasible in reality.

In one of the assignments in my fact-checking course at the university, I ask my students to 1) find a piece of information that is potentially false or misleading and 2) think about how they can investigate it.

In the end, they need to use the following template and submit it as a story idea.

Claim: Write down in a couple of sentences what the information is, with links to the claim(s), photo(s), video(s), statement(s).

Spread: Demonstrate how widely the claim is being shared and believed already (e.g. engagement figures, strong reactions and comments). If it's not yet trending, describe how the claim is likely to spread quickly in the future.

Significance: Explain why it is important to verify the claim. What is the potential harm if it turns out to be false (or true)? What are the possible consequences if no one checks it? Evaluate how badly the claim(s) would influence people and society, if not investigated.

Initial assessment: Briefly write what you know so far and discuss the likelihood of the claim(s) being false (or misleading) or true. Explain what needs to be done by listing out methods and strategies that can be employed (you do not need to investigate it further at this point, but you must show you have a plan). Make sure to check if it has not been fact-checked by someone already before you pitch the story idea.

For trained journalists and educators, perhaps the above template may seem straightforward (a detailed explanation of this template can be found in our Toolkit).

However, in my experience, many students struggle to understand what the process of verification entails and what evidence is required to disprove (or prove) the claim.

In the ANNIE Connect study group this week on Friday (Sept. 15), I would like to share and discuss the pitfalls of teaching fact-checking with some examples, focusing on what some students find difficult.

By the way, thank you to those who responded to the poll in the previous newsletter. Since more people prefer the 10:30 to 11:30 (GMT+8) time frame, we decided to change the schedule.

You should have received (or will receive soon) an invitation with the revised time if you have registered for ANNIE Connect before, but just in case, here’s the registration link:

https://hku.zoom.us/meeting/register/tJ0rcuuvqzspGNBeY11cN6QZG8X6WIuVxhCO

Now, here is one example assignment to give you an idea:

Example of student submission

A student proposed to investigate the tweet below.

Claim: A Twitter post alleges that delivery-only restaurants in Korea have extremely poor hygiene conditions, providing photos and a video from Korean news outlet MBC as evidence. The visuals depict the unhygienic kitchen of a delivery-only restaurant in Busan, South Korea. The account owner also claims that many such restaurants have blocked windows.

Spread: Posted on March 19, the tweet has been retweeted over 1,400 times within two days and has received over 152k views. Given the growing concern surrounding hygiene issues, the tweet is likely to gain further attention and spread more widely.

Significance: The claim that delivery-only restaurants maintain poor hygiene conditions could severely impact the food delivery industry. If proven false, numerous delivery-only restaurant owners who maintain good hygienic standards could suffer significant profit losses, potentially leading to closures.

Initial assessment: The evidence presented focuses on a single restaurant, which is insufficient to support a general claim about the entire industry. To verify the claim, one could monitor the Korean news outlet that provided the video for further updates, or review annual or monthly assessments from the Korean hygiene management department to determine the percentage of delivery-only restaurants that meet hygiene standards.

My feedback

As you can tell, the above example was submitted by a student from Korea who was studying at our university, which is located in Hong Kong, making it even harder to fact-check the claims in the tweet.

The following is what I wrote as feedback:

You have discovered an intriguing claim that is certainly worth investigating. However, in its current form, your proposal focuses on a factor that is not entirely fact-checkable by a student in Hong Kong. While this could be excellent material for further journalistic investigation in Korea, it is not suitable for a fact-checking project in this course.

Here's why it's not fact-checkable.

Implied claim to be investigated: Many delivery-only restaurants in Busan have blocked their windows, presumably to conceal poor hygiene conditions, as demonstrated by the one mentioned in the MBC news report.

Assessment: Attempting to fact-check the windows of "many" restaurants in Busan from Hong Kong and investigate what the hidden kitchens behind them look like is not feasible. It is perhaps not realistic even for most people living in Busan as well, given the time it requires and potential troubles one might encounter with restaurant owners.

Your suggested two methods:

1. Wait for MBC to follow up and check other restaurants. — Yes, you may find out more but this is not a fact-checking methodology.

2. Check the records compiled by authorities. — You have not indicated if you have looked and made sure such data or records are available. But even if they exist, their accuracy in reflecting the real situation cannot be guaranteed.
   
   The existence of the MBC news report itself suggests that this issue has not been widely recognised by the public.
   
   If authorities were aware, it is likely that those restaurants would have been warned, and more news about this issue would have already been available in Korea.

This claim might require the type of investigation for you to go undercover, which is again not something a university student in Hong Kong can undertake.

I agree that the claim holds significance for people in Busan, especially if the number of unhygienic restaurants could be determined and there are indeed many.

However, for the reasons mentioned above, it is very unlikely that you can find that out.