Update: Public Editor’s Trajectory

If you’re following the Public Editor project, you may be wondering what we’ve been up to lately. It has been an eventful 2023 as we’ve been gearing up for 2024 elections in the EU and USA. This post won’t cover everything consequential about our recent and upcoming trajectory, but here are a few highlights:

Where we’ve been:

For the last year or so, the project has been focused on two major efforts. (1) We have been making a number of improvements to Public Editor’s software, both for newsreaders and for the annotators who evaluate news articles. (2) We’ve been engaging a number of partnerships, particularly in Europe where cultural, political, and regulatory environments are quite friendly to Public Editor. More details…

User Experience Updates
Based on multiple rounds of user research, we have updated the newsreader experience so that it is more legible to a broader range of users. While we initially thought it was cool and futuristic to navigate articles using an expanding credibility hallmark, too many of our newsreaders found it confusing. So, we’ve switched to a design that simply lists manipulations, reasoning errors, and cognitive biases in a categorized list in the right margin of each article. Readers can still click on highlighted text in the article to activate information about the specific error, but now the experience is more like reading a nutrition label (a ‘newstrition’ label as some of our colleagues call it). We have also added a dictionary of definitions for each error, and some additional examples, so newsreaders can better learn the general patterns they need to watch out for when reading content online.

We have also updated the annotator experience. First, we have made it prettier. Our earlier designs were functional, but had something of a retro vibe stylistically. Taking into account that the entire internet has become more beautiful as design talents and trends have improved, we decided to up our game and give annotators an experience befitting the decade.

We’ve also added more tools for onboarding and training annotators. Our initial tool-building was focused on competently creating the collective intelligence tools necessary to ensure that Public Editor delivers credible annotations in a rigorous, intricate, and scalable way. With that work solidly complete, we moved on to ensuring that thousands of Internet volunteers have the learning tools they need to understand the misinformation concepts – over 50 biases, fallacies, and manipulations we humans fool ourselves with – of the Public Editor system and learn to apply them through our fully new ‘Real-Time Feedback Trainer’ and Gold Standard evaluation tasks that ensure they are prepared to start working on daily news articles. We will be rolling out these and other features as part of a new Annotator Education Center in the coming months.

Partnerships
Public Editor has been fortunate to have some great partnerships from our inception, research and philanthropic organizations that empowered us to prototype, learn, build, and improve through 4 rounds of R&D and build a complex system of simpler software surfaces designed for newsreaders, students, research teams, data scientists, and online volunteers. As we prepare to launch at a nation scale, however, we have been busy forging partnerships with civil society organizations, research associations, student clubs and news companies that can help us expand into civil society to recruit thousands of volunteers and provide misinformation labeling services to millions of newsreaders.

With some exploratory trips to Europe, we found that the ground there is very fertile for Public Editor. Civil society organizations had already grown up organically to confront propaganda flowing into their information environments, far beyond the activity levels in the United States. The governments of Europe, too, are far more inclined to take actions protecting their populations from the harmful ‘attention economy’ dynamics that reward outrageous misinformation online, particularly on platforms. Perhaps the complexity of political and social life in Europe, with its more pluralistic federation of multiple nation-states speaking different languages is a factor. Perhaps, too, it matters that the major Internet companies who profit from recommendation algorithms feeding division are not European companies, and not deeply embedded in politics and philanthropy as they are in the US.

Whatever the reasons, the Europeans have been quite innovative when it comes to protecting their populations, instituting the GDPR over the past few years, and recently beginning the rollout of the Digital Services Act (DSA), which includes provisions to share platform data with researchers working to uncover the social harms of platforms’ algorithms and user experiences. Moreover, the DSA provides policy pathways supporting the adoption of tools like Public Editor. If researchers can show that interventions like those provided by Public Editor mitigate individual and social harms caused by the platforms, provisions of the DSA trigger regulatory action by the EU, including compelling the platforms to make the interventions available to the public. We believe Public Editor will prove useful for reducing the social harms of rampant misinformation, so we have been making many new friends in Europe who are interested in testing our tools and seeing them gain wider adoption. We will announce these partnerships in another moment soon.        
    

Where we’re going:

User Experience Updates
In addition to the updates described above, we will also soon be rolling out a dashboard for annotators that allows them to see their progress and impact. A more ‘gamified’ annotator experience will allow our volunteers to earn badges showing their proficiency in each category of annotation: Language, Probability, Reasoning, Quoted Sources, and Argument Relevance. When annotators earn all available badges, they will also earn a Certificate of Discernment from Goodly Labs and the Berkeley Institute for Data Science. Annotators will also get a leaderboard experience, allowing them to top the leaderboard for the week, month, or all time. Finally, and we are very excited about this, the Annotator Dashboard will allow users to see their impact. We can easily track how many labels each annotator applies, and how many articles they improve. And we can also track how many labeled articles are being read by Public Editor’s newsreaders. So, we can show annotators that their efforts labeling 200 articles, for example, have reached the eyeballs of 500,000 newsreaders.

This certainly beats the current reality where concerned citizens trying to combat misinformation simply post on one of the platforms and hope that some of their friends and family avoid being fooled –– an experience which is only slightly better than shouting into the void. Soon after we release the Annotator Dashboard, we will be working on Dashboards for journalists and news editors, so they can track the credibility of their own work, see their own ‘misinformation signatures’ (the ways their writing tends to exaggerate or mislead), and direct their own improvements. By partnering with journalist to give them better signals, we intend to help them join in the widespread effort to improve the quality of information that wins attention online. There will be more to come… but these are our next steps.   

Partnerships and Campaigns
We don’t want to give too much away, since we’ll have more to announce soon. But as many readers are aware: 2024 will witness EU-wide elections, and major elections in the USA, including for President. With AI content generators coming online, it will be easier than ever for propagandists to sow chaos and division with misleading content. Some are predicting, perhaps with too much alarm, that 2024 could end with democracies in shambles. Whether or not that is accurate, we intend to do all we can to empower the democratic citizens of the world with tools to stem the tide of misinformation online.

Stay tuned, and we will have more to report on this soon. Thank you for reading, and for tolerating our vagueness about some of these developments. In due time, we will have more specifics about the great (non-partisan) partners we are working with to stand-up a bulwark for democracy, and more specific information about how you can get involved. This campaign season does not have to be bleak. On the contrary, we the people of democracies around the world can take charge and demonstrate to ourselves that we are not helpless in the face of tech-amplified chaos. This can be our finest hour.