Skip to Main Content
PCMag editors select and review products independently. If you buy through affiliate links, we may earn commissions, which help support our testing.

Human Help Wanted: Why AI Is Terrible at Content Moderation

AI's frequent failure to understand context means that, for now, monitoring user-generated content on the web will remain a cat-and-mouse game that requires plenty of human labor.

July 10, 2019
Why AI Is Terrible at Content Moderation

Every day, Facebook's artificial intelligence algorithms tackle the enormous task of finding and removing millions of posts containing spam, hate speech, nudity, violence, and terrorist propaganda. And though the company has access to some of the world's most coveted talent and technology, it's struggling to find and remove toxic content fast enough.

Opinions In March, a shooter in New Zealand live-streamed the brutal killing of 51 people in two mosques on Facebook. But the social-media giant's algorithms failed to detect the gruesome video. It took Facebook an hour to take the video down, and even then, the company was hard-pressed to deal with users who reposted the video.

Facebook recently published figures on how often its AI algorithms successfully find problematic content. Though the report shows that the company has made tremendous advances in its years-long effort to automate content moderation, it also highlights contemporary AI's frequent failure to understand context.

Not Enough Data

Artificial neural networks and deep-learning technologies, at the bleeding edge of artificial intelligence, have helped automate tasks that were previously beyond the reach of computer software. Some of these tasks are speech recognition, image classification, and natural language processing (NLP).

In many cases, the precision of neural networks exceeds that of humans. For example, AI can predict breast cancer five years in advance. But deep learning also has limits. Namely, it needs to be "trained" on numerous examples before it can function optimally. If you want to create a neural network that detects adult content, for instance, you must first show it millions of annotated examples. Without quality training data, neural networks make dumb mistakes.

Artificial Intelligence AI

Last year, Tumblr declared that it would ban adult content on its website and use machine learning to flag posts that contained NSFW images. But a premature deployment of its AI model ended up blocking harmless content such as troll socks, LED jeans, and a picture of Joe Biden.

And in many cases, such as violent content, there aren't enough examples to train a reliable AI model. "Thankfully, we don't have a lot of examples of real people shooting other people," Yann LeCun, Facebook's chief artificial-intelligence scientist, told Bloomberg.

Neural networks also lack situational awareness. They make statistical comparisons only between new content and examples they've been shown. Even when trained on many examples, neural networks act erratically when faced with edge cases that look different from their training data.

Facebook's AI failed to detect the New Zealand massacre video because it was streamed from a first-person viewpoint and didn't resemble anything uploaded in the past. A person reviewing the video would immediately be aware of its violent content. But Facebook's neural networks, which only extract and compare patterns of pixels, dismissed it as safe.

Context and Intent

Facebook could have trained its AI on plenty of violent scenes from movies to enhance its moderation ability. But this would have only confused the AI, because it wouldn't be able to tell the difference between movies and real violence and would have blocked both.

This is because one of the most pressing problems facing neural networks is their inability to understand context and intent. Facebook CEO Mark Zuckerberg explained this in layman's terms in a call with analysts last year, on which he said, "It's much easier to make an AI system that can detect a nipple than it is to determine what is linguistically hate speech."

A well-trained neural network can be very good at detecting nudity. According to Facebook's figures, its AI can detect nudity with 96 percent accuracy. But it will struggle to tell the difference between safe nudity—say, breastfeeding or Renaissance art—and banned content such as sexual activity.

In 2016, Facebook removed a photo of the Vietnam War on the page of Norway's Prime Minister because it contained the image of a naked 9-year-old girl fleeing after a napalm attack. The company's algorithms flagged the iconic picture as child pornography; Facebook later apologized to the PM and restored the post.

In 2017, YouTube acknowledged that its AI mistakenly flagged videos posted by journalists and researchers as extremist content because it couldn't tell the difference between videos that promote extremism and reports on the topic.

connected social media

Things get even more complicated when AI has to deal with speech or text. Deep-learning algorithms are efficient at capturing and evaluating temporal consistency. That's why they're very good at recognizing speech, converting audio to text, and detecting spam. But they fall apart when they're tasked with detecting hate speech and harassment.

Those tasks require an AI to understand the nuances of human language, a problem that is hard to solve with ones and zeros. Hate speech can differ immensely across languages, and humans often disagree about what comprises bad activity.

According to Facebook's report, hate speech and harassment are two areas where its AI performs poorly. Last year, in testimony before the US Congress, Zuckerberg said it would take the company five to ten years to develop AI that can detect hate speech. But if the history of artificial intelligence is any indication, it will probably take longer.

Moderation Still Requires Humans

As Facebook and other social media networks work on their AI algorithms, humans will remain a big part of online content moderation. Facebook currently employs more than 20,000 people across the world to review user-generated content.

These people, who are often underpaid and must deal with disturbing images and content throughout the day, review the posts flagged by the AI algorithms to restore those that have been wrongly blocked and remove others that violate the site's policies. Their work will help to further train the AI and improve its accuracy. But there's no indication whether or when artificial intelligence will be able to independently moderate the billions of posts uploaded on Facebook and other social networks every day.

For the time being, content moderation will remain a cat-and-mouse game that will require a lot of human labor. In an interview with The New York Times, Facebook CTO Mike Schroepfer acknowledged that AI alone would not solve the company's toxic content problem.

"I do think there's an endgame here," Schroepfer said. But "I don't think it's 'everything's solved,' and we all pack up and go home."

Artificial Intelligence Develops Its Own Language
PCMag Logo Artificial Intelligence Develops Its Own Language

Get Our Best Stories!

Sign up for What's New Now to get our top stories delivered to your inbox every morning.

This newsletter may contain advertising, deals, or affiliate links. Subscribing to a newsletter indicates your consent to our Terms of Use and Privacy Policy. You may unsubscribe from the newsletters at any time.


Thanks for signing up!

Your subscription has been confirmed. Keep an eye on your inbox!

Sign up for other newsletters

TRENDING

About Ben Dickson

Ben Dickson

Ben Dickson is a software engineer and tech blogger. He writes about disruptive tech trends including artificial intelligence, virtual and augmented reality, blockchain, Internet of Things, and cybersecurity. Ben also runs the blog TechTalks. Follow him on Twitter and Facebook.

Read Ben's full bio

Read the latest from Ben Dickson