Meta’s decision, to replace centralized fact-checking teams with community-generated labeling by users has triggered a torrent of reactions . The changes are not without controversy. They raise questions about the effectiveness of Meta’s old policy of fact-checking and its new policy of community comments.
Platforms such as Meta’s Facebook or Instagram, which are used by billions of users worldwide, have a duty to protect their users from online harms, including hate speech, misinformation, and consumer fraud. combating the harms of online, given its scale, is a major challenge for society. In order to address these online harms, content moderation is important.
Three steps are involved in moderating content. First, scan online content (typically social media posts) to identify potentially harmful images or words. Second, the flagged material is assessed to determine if it violates any laws or platform terms of service. Third, you can intervene in any way. Interventions can include removing posts or adding warning labels.
Content moderation ranges from community-based moderation models to centralized models of content moderation such as Instagram. Both approaches have mixed results, according to research.
Does fact-checking work?
Meta’s content moderation policy used to rely on third-party fact checking organizations that brought problematic content into the Meta staff’s attention. Meta’s U.S. The fact-checking agencies were AFP USA (Check Your Fact), Factcheck.org, Lead Stories and PolitiFact.
Expert review is essential for fact-checking. It is not the cure-all, but it can reduce misinformation’s effects. The effectiveness of fact checking depends on how users view fact checkers, and whether they trust fact-checking organizations.
Crowdsourced content moderating
Mark Zuckerberg, CEO of Meta, announced that Meta’s content moderation would be based on a model similar to X (formerly Twitter). X’s Community Notes is a crowdsourced approach to fact-checking that allows users write notes informing others about potentially false posts.
The effectiveness of X style content moderation is mixed. In a large-scale research, there was little evidence to suggest that the introduction of community notes reduced engagement with misinformation on X. It appears that crowd-based initiatives may be too slow to reduce engagement with misinformation at the earliest and most viral stages of its spread.
Some platforms have seen success with badges and certifications. Community-provided labels might not be effective to reduce engagement with misinformation. This is especially true when they are not accompanied by training for platform users on labeling. Research shows that X’s Community Notes is subject to partisan bias.
The community-edited Wikipedia relies on the feedback of others and has a strong contributor system. A Wikipedia-style model requires strong mechanisms for community governance, as I’ve written previously. This will ensure that volunteers adhere to the same guidelines when authenticating and fact-checking posts. The system could be manipulated by people who vote up content that is interesting but unverified.
Renee DiResta, a researcher on misinformation, analyzes Meta’s new policy for content moderation.
Consumer harms and content moderation
The overall user experience can suffer if there are no motivated individuals willing to put in the effort to make the online environment safe and trustworthy.
Algorithms are used on social media platforms to increase engagement. Content moderation is important for consumer safety and product liability, but also because policies that encourage engagement may also cause harm.
Content moderation can have implications for companies that use Meta to advertise or connect with consumers. The issue of brand safety is also part of content moderation. Platforms must balance the desire to make social media environments safer with their desire for greater engagement.
There is AI everywhere
The growing amount of content produced by artificial intelligence will likely make it more difficult to moderate. AI detection tools have flaws, and the development of generative AI is challenging people’s abilities to distinguish between AI-generated and human-generated content.
OpenAI, for instance, launched in January 2023. This classifier was designed to distinguish between texts created by humans and AI. The company, however, discontinued the tool due to its poor accuracy in July 2023.
It is possible that a flood of inauthentic accounts, AI bots, will exploit algorithmic and user vulnerabilities to monetize harmful and false content. They could, for example, commit fraud or manipulate opinions to gain economic or political benefit.
ChatGPT, a tool that uses generative AI to generate large quantities of realistic social media content and profiles, makes it easy to create a high volume of . AI-generated content that is designed to engage may also be biased in terms of race and gender. Meta’s AI generated profiles was criticized by commentators, who called it ” AI generated slop.”
More than just moderation
No matter what type of content moderating is used, it will not be effective in reducing the belief of misinformation nor in limiting the spread of this misinformation.
Research shows that partnerships with researchers, platform audits, and citizens activists is important to ensure safe and trusted community spaces on social networks.
Anjana Susarla is funded by the National Institute of Health
