Facebook admits its AI moderators ‘aren’t perfect’ and vows to improve detection of terrorist videos in wake of New Zealand mosque shooting

  • Facebook said AI system wasn’t adequately trained to catch the gunman’s video
  • The firm admitted it’s more challenging to detect live video of terrorist attacks
  • Other content, like nudity and propaganda, are more easily spotted by its AI
  • Facebook is looking at ways to use AI to moderate live streams moving forward
  • e-mail



Facebook has acknowledged that its artificial intelligence systems weren’t prepared to catch the livestream video of the New Zealand mosque shooting, which left 50 people dead. 

In the wake of the tragic attack, many have questioned why Facebook’s AI, which works with human moderators to flag content that violates its policies, was unable to detect the shooter’s livestream. 

Now, in a new blog post, Facebook’s vice president of integrity Guy Rosen has provided additional details on how its AI systems are trained to detect these kinds of content. 

He also laid out how Facebook’s AI needs to be improved in order to detect videos like the Christchurch stream in the future. 

Scroll down for video 

Facebook has acknowledged that its artificial intelligence systems weren’t prepared to catch the livestream video of the New Zealand mosque shooting, which left 50 people dead


Facebook uses artificial intelligence and machine learning to detect objectionable material, while at the same time relying on the public to flag up content that violates its standards.

To report live video, a user must know to click on a small set of three gray dots on the right side of the post.

When you click on ‘report live video,’ you’re given a choice of objectionable content types to select from, including violence, bullying and harassment.

You’re also told to contact law enforcement in your area if someone is in immediate danger.

‘Many people have asked why artificial intelligence (AI) didn’t detect the video from last week’s attack automatically,’ Rosen explained. 

‘AI has made massive progress over the years and in many areas, which has enabled us to proactively detect the vast majority of the content we remove. 

‘But it’s not perfect,’ he added.  

Rosen explained that Facebook’s AI systems are based on ‘training data,’ which is made up of several thousands of different types of content. 

Facebook’s AI has been successful at taking down nudity, terrorist propaganda and some examples of graphic violence.  

‘However, this particular video did not trigger our automatic detection systems,’ Rosen said. 

‘To achieve that we will need to provide our systems with large volumes of data of this specific kind of content, something which is difficult as these events are thankfully rare.

  • Scientists develop radical robot translator that lets bees… SpaceX delays first ‘hop’ tests of Elon Musk’s Starhopper… The poisonous gas that could be a sign of alien life: Carbon… ‘Nuclear icebreaker’ ship being built by China weighs…

Share this article

‘Another challenge is to automatically discern this content from visually similar, innocuous content – for example if thousands of videos from live-streamed video games are flagged by our systems, our reviewers could miss the important real-world videos where we could alert first responders to get help on the ground,’ he added. 

Facebook will need to continue to rely on a mix of AI systems and human moderators in its efforts to remove hateful, violent and disturbing content, Rosen said. 

People gather around tributes at the Botanic Gardens in Christchurch. In the wake of the tragic attack, many have questioned why Facebook’s AI, which works with human moderators to flag content that violates its policies, was unable to detect the shooter’s livestream

He pointed to Facebook’s decision last year to double its human moderator workforce from 15,000 employees to 30,000 employees, while calling on users to continue reporting content that they believe violates the site’s policies. 

In the firm’s efforts to stop these kinds of content from spreading, it has been deploying new techniques, such as an ‘experimental audio-based technology’ to identify ‘variants of the video.’  

Facebook used this technology, which it had been building previously, to halt the spread of the Christchurch shooting video. 

The tech giant is also exploring how it can use AI to moderate livestreams. 

Rosen said Facebook is not currently considering adding a time delay to livestream videos, as it would only complicate the detection process. 

Many have suggested Facebook add a delay to broadcasts, like those that are used on some live TV channels. 

This combination of images shows logos for companies from left, Twitter, YouTube and Facebook. These Internet companies and others say they’re working to remove video footage filmed by a gunman in the New Zealand mosque shooting

‘There are millions of Live broadcasts daily, which means a delay would not help address the problem due to the sheer number of videos,’ he said.

‘More importantly, given the importance of user reports, adding a delay would only further slow down videos getting reported, reviewed and first responders being alerted to provide help on the ground.’ 

The update follows a previous post from Facebook where it said the gunman’s live 17-minute broadcast was viewed fewer than 200 times when it was live and the first user report didn’t come in until 12 minutes after it ended.  

All told, the video was viewed some 4,000 times before it was removed from Facebook. 

In the first 24 hours following the attack, Facebook said it took down more than 1.2 million videos of the incident ‘at upload, which were therefore prevented from being seen on our services.’ 

About 300,000 additional copies were removed after they were posted, Rosen said. 


Facebook has disclosed its rules and guidelines for deciding what its 2.2 billion users can post on the social network. 

The full guidelines can be read here. Below is a summary of what they say: 

1. Credible violence

Facebook says it considers the language, context and details in order to distinguish casual statements from content that constitutes a credible threat to public or personal safety.

2. Dangerous individuals and organisations

Facebook does not allow any organizations or individuals that are engaged in terrorist, organized hate, mass or serial murder, human trafficking, organized violence or criminal activity.

3. Promoting or publicising crime

Facebook says it prohibit people from promoting or publicizing violent crime, theft, and/or fraud. It does not allow people to depict criminal activity or admit to crimes they or their associates have committed. 

4. Coordinating harm

The social network says people can draw attention to harmful activity that they may witness or experience as long as they do not advocate for or coordinate harm. 

5. Regulated goods

The site prohibits attempts topurchase, sell, or trade non-medical drugs, pharmaceutical drugs, and marijuana as well as firearms. 

6. Suicide and self-injury

The rules for ‘credible violence’ apply for suicide and self-injury. 

7. Child nudity and sexual exploitation of children

Facebook does not allow content that sexually exploits or endangers children. When it becomes aware of apparent child exploitation, we report it to the National Center for Missing and Exploited Children (NCMEC).

8. Sexual exploitation of adults

The site removes images that depict incidents of sexual violence and intimate images shared without permission from the people pictured.

9. Bullying

Facebook removes content that purposefully targets private individuals with the intention of degrading or shaming them.

10. Harassment

Facebook’s harassment policy applies to both public and private individuals.

It says that context and intent matter, and that the site will allow people to share and re-share posts if it is clear that something was shared in order to condemn or draw attention to harassment.  

11. Privacy breaches and image privacy rights

Users should not post personal or confidential information about others without first getting their consent, says Facebook. 

12. Hate speech

Facebook does not allow hate speech on Facebook because it says it creates an environment of intimidation and exclusion and in some cases may promote real-world violence. 

13. Graphic violence

Facebook will remove content that glorifies violence or celebrates the suffering or humiliation of others.

It will, however, allow graphic content (with some limitations) to help people raise awareness about issues.

14. Adult nudity and sexual activity

The site restricts the display of nudity or sexual activity.

It will also default to removing sexual imagery to prevent the sharing of non-consensual or underage content.

15. Cruel and insensitive

Facebook says it has higher expectations for content that defined as cruel and insensitive.

It defines this as content that targets victims of serious physical or emotional harm. 

16. Spam

Facebook is trying to prevent false advertising, fraud and security breaches.

It does not allow people to use misleading or inaccurate information to artificially collect likes, followers or shares. 

17. Misrepresentation

Facebook will require people to connect on Facebook using the name that they go by in everyday life.

18. False news

Facebook says that there is also a fine line between false news and satire or opinion. 

For these reasons, it won’t remove false news from Facebook, but, instead, significantly reduce its distribution by showing it lower in News Feed.

19. Memorialisation

Facebook will memorialise accounts of people who have died by adding “Remembering” above the name on the person’s profile. 

The site will not remove, update or change anything about the profile or the account. 

20. Intellectual property

Facebook users own all of the content and information that they post on Facebook, and have control over how it is shared through your privacy and application settings. 

21. User requests

Facebook say they will comply with:

  • User requests for removal of their own account
  • Requests for removal of a deceased user’s account from a verified immediate family member or executor
  • Requests for removal of an incapacitated user’s account from an authorised representative

22. Additional protection of minors

Facebook complies with:

  • User requests for removal of an underage account
  • Government requests for removal of child abuse imagery depicting, for example:
  • Beating by an adult
  • Strangling or suffocating by an adult
  • Legal guardian requests for removal of attacks on unintentionally famous minors

Source: Read Full Article