Always have your eyes closed in photos? Facebook reveals AI that can tweak your snaps to make it look like they are open

  • Facebook engineers have developed a way to fix photos were users have blinked
  • A machine learning system studies photos of users with open and closed eyes, collecting traits like skin color, eye shape and more to replicate them 
  • It’s unclear whether Facebook plans to apply the tech to its sites and apps

Facebook has developed a way to salvage photos where you’ve accidentally blinked.

A team of Facebook engineers utilized a novel technology called ‘in-painting’, where a program fills in a space with what it thinks should be there. 

In this case, it replaces closed eyes with open ones, but in a way that appears strikingly realistic. 

Scroll down for video

Facebook has found a way to salvage photos where you’ve accidentally blinked. The team used a machine learning network to study photos where your eyes are open to replace closed ones

HOW DOES IT WORK?  

Facebook has developed a way to fix photos where you’ve accidentally blinked. 

A team of engineers used a machine learning system, called a General Adversarial Network. 

The GAN looked at photos of users with open eyes and learned to recreate them on photos where their eyes were closed. 

Another technology, called in-painting, studies a person’s skin tone, eye shape and iris color, in order to replicate it.

The effects are strikingly realistic. 

Researchers supplied users with two photos — an edited photo of a user with their eyes open and an undoctored image. 

Respondents said the AI-edited image was the ‘real’ image 54% of the time, according to Facebook engineers.  

The engineers trained a machine learning system, called a Generative Adversarial Network (GAN), to automatically retouch closed eyes.

Two neural networks are trained to recognize a person’s eyes and recreate them by referencing a collection of images of the user where they’re not blinking.

In the reference images, the neural networks use in-painting to collect information on the user’s skin tone, eye shape and iris color. 

In-painting has been used to replicate other elements of a photo, such as missing backgrounds in nature photography, Facebook noted in the paper, which was published this week. 

They hoped that the same technique could be applied to pictures of people, but in a way that the machine learning network could learn what the human face ‘should’ look like, without compromising a person’s unique face structure.    


  • Who do YOU think wins? IBM’s ‘unsettling’ AI supercomputer…


    Tortured and mutilated before death? 1,500-year-old…


    Yet another reason not to update to iOS 11.4! Users are…


    What a Tesla in Autopilot mode ‘sees’ in real-time:…

Share this article

Neural networks are trained to recognize a person’s eyes and recreate them by referencing a collection of images of the user where they’re not blinking. In the reference images, the neural networks use in-painting to collect information on the user’s skin tone, eye shape and iris color

‘There is little doubt that realistic face retouching algorithms are a growing research topic within the computer vision and machine learning communities,’ the researchers wrote. 

Retouching algorithms have been widely used for things like fixing red eye, removing blemishes, manipulating facial features and creating synthetic makeup, they said.  

‘However, humans are very sensitive to small errors in facial structure, especially if those faces are our own or are well-known to us,’ the researchers continued.  

‘Moreover, the so-called ”uncanny valley” is a difficult impediment to cross when manipulating facial features.’ 

To generate realistic photos, the researchers trained the GAN on two million photos. 

Roughly 200,000 individuals supplied photos for the study, with one or more photos of each person with their eyes open. 

The researchers noted that the in-painting technique isn’t always successful. The tech stumbled when it encountered glasses or certain hairstyles in a photo

Facebook compared its technology to Adobe’s ‘Open Closed Eyes’ feature in Photoshop Elements. Adobe’s feature is in the third column, while Facebook’s is in the fourth column

A separate dataset included 100,000 celebrity photos.  

To verify that the system worked, Facebook engineers asked participants to look at a set of images, which contained one reference image and a real image, and decide which were not in-painted. 

The participants said the AI-generated photo was the ‘real’ image 54% of the time.  

However, the researchers noted that the in-painting technique isn’t always successful.

The technology stumbled when it encountered glasses or certain hairstyles in a photo. 

It’s unclear when or if Facebook will release the technology for use on its social media platforms. However, researchers said the in-painting system could have other uses

It also has trouble analyzing faces that are shown at extreme angles and photos with a lot of noise. 

Researchers said it wouldn’t be able to produce results without a reference photo of the user with open eyes. 

But it generates much more realistic results than similar technologies developed by companies like Adobe and Pixelmator. 

Facebook compared its technology to Adobe’s ‘Open Closed Eyes’ feature in Photoshop Elements and the differences are striking. 

It’s unclear if or when Facebook plans to release the technology for use on its social media platforms. 

The firm will likely have to answer some questions around how it will use the technology and whether it will require users’ consent when it comes to photos uploaded to the site. 

Researchers said there may be other applications for the in-painting technique, such as applying it to ‘other in-painting tasks, such as filling in missing regions from a natural but uniquely identifiable scene.’

Source: Read Full Article