Researchers at Facebook have developed a new automatic photo filter able to open closed eyes in photgraphs with very impressive results.
Although similar automated processes are already available such as the ‘Open Closed Eyes’ tool in Adobe Photoshop Elements 2018, this new technique produces compelling results where previous efforts tend to stand out immediately as fake.
In the example below we see ‘before’ and ‘after’ samples comparing the results of the new method (column d) with Adobe Photoshop Elements (column c).
As you can see, the results in column c are far from convincing, whereas those in column (d) look entirely believable.
The new technique, published by Brian Dolhansky and Cristian Canton Ferrer, is a fresh take on a much-studied process known as ‘in-painting’ where missing components or holes in a photograph are automatically filled in with new pixels to create a convincing facsimile of an original image.
In this case the algorithm learns what the subjects eyes usually look like from an open-eyed image (column a in the example above) and uses that information to reconstruct a new pair of eyes which blend seamlessly into the new image, taking into account any changes in expression, facial orientation or gaze direction as well as other differences in the overall look of the picture, such as color, exposure and shadows.
Facebook’s new method uses a form of machine learning known as an “Exemplar Generative Adversarial Network” (ExGAN). The Generative Adversarial Network essentially pits two machine learning processes against each other, the first trying to produce convincing images while the second attempts to spot these fakes among a selection of real photos.
However, while this technique alone may paint in a pair of eyes which look real, it won’t produce eyes that the person in the photo would recognise as their own. This is where the ‘Exemplar’ comes in. By examining example photos of the person in the image, the ExGAN can generate new eyes that match those in the example photos, thereby retaining the person’s identity and any unique features of their eyes.
The result is a new pair of eyes which match the real eyes of the person in the photo while looking in the right direction and matching all the other photographic qualities of the image.
While it’s not immediately clear how this technology would be used effectively on Facebook, it could be useful when editing group shots. A great group photo can often be ruined by that one person who happened to blink at the wrong time, but the technique would allow the photographer to re-open their eyes without having to copy them manually from another image.
Source:-forbes