Using impact evaluations to inform expansion of media literacy initiatives

In Ukraine, an increase in disinformation and propaganda has threatened democratic progress since the 2014 revolution. To address this issue, IREX developed a media literacy program called Learn to Discern (L2D) to teach citizens the latest techniques for identifying disinformation.

IREX’s Learn to Discern approach helps citizens recognize and resist disinformation, propaganda, and hate speech. Learn to Discern’s unique methodology builds practical skills for citizens of all ages through interactive training, videos, games, and other learning experiences.

IREX’s Learn to Discern approach helps citizens recognize and resist disinformation, propaganda, and hate speech. Learn to Discern’s unique methodology builds practical skills for citizens of all ages through interactive training, videos, games, and other learning experiences.

A cadre of 428 media literacy trainers taught 15,000 Ukrainian citizens, from high school teachers to library patrons, throughout the country. But given the flood of propaganda in Ukraine, did it actually help people separate fact from fiction?

About the program Learn to Discern (L2D) 

EVALUATING IMPACT

To assess the impact of Learn to Discern, IREX conducted an evaluation after the program finished to test whether participants retained the media literacy skills they learned. Encouragingly, trainees showed significant improvement: they were 13% more likely to detect false information and 25% more likely to check multiple sources of information versus a control group. The study was conducted a full 18 months after the trainings, suggesting that the effects were long-lasting.

The impact evaluation’s results helped IREX staff identify which elements of the L2D curriculum were essential to preserve as IREX streamlined the curriculum and rolled it out to more people. Since the study found that giving participants a sense of agency over what media and information they consumed was a critical factor in skill retention, the team created a simple, catchy, and scalable “name it to tame it” module that could be taught to a wide range of audiences. The exercise helps participants identify when they are having a strong emotional reaction and reactivate the logical part of their brains, putting them back in control of their own response to information.

SCALING WHAT WORKS

While the study’s results were encouraging, the pilot reached only 90,000 people in a country of 45 million. It was effective and staff began to brainstorm how to scale the approach to make more widespread change. The team looked for larger institutions in which to embed the curriculum. The IREX team is now working in partnership with the Ukrainian government, and the governments of the United States and United Kingdom, to integrate Learn to Discern and its critical-thinking skills into the high school curriculum in Ukraine. If successful, it will eventually be taught to millions of students a year, helping to make future generations less susceptible to disinformation and manipulation.

LESSONS FOR THE FIELD

The results of the evaluation underscore the value of running small pilot programs before scaling efforts to reach more people. The results of the impact evaluation of the L2D pilot gave the team confidence that the approach fundamentally worked. It also uncovered which aspects of the program were most effective, such as the module on avoiding emotional manipulation, so the team could retain those aspects when scaling up the initiative.

IREX first implemented Learn to Discern to counter highly sophisticated disinformation campaigns in Ukraine. It is now also conducting the program in Guatemala, Jordan, Serbia, Tunisia, and the United States.

The text is republished from IREX Annual Report: Data for Impact and Learning