New Algorithms Better in Masked Face Recognition

This post is also available in: עברית (Hebrew)

Software developers have made demonstrable progress at recognizing masked faces, according to the findings of a new study of face recognition technology created after the onset of the COVID-19 pandemic. 

The research report of the US National Institute of Standards and Technology (NIST)’s first study that measures the performance of face recognition algorithms developed following the arrival of the pandemic. 

The new study adds the performance of 65 newly submitted algorithms to those that were tested on masked faces in the previous round, offering cumulative results for 152 total algorithms. 

A previous report from July explored the effect of masked faces on algorithms submitted before March 2020, indicating that software available before the pandemic often had more trouble with masked faces.

“Some newer algorithms from developers performed significantly better than their predecessors. In some cases, error rates decreased by as much as a factor of 10 between their pre- and post-COVID algorithms,” said NIST’s Mei Ngan, one of the study’s authors. 

Using the same set of 6.2 million images as it had previously, the team again tested the algorithms’ ability to perform “one-to-one” matching, in which a photo is compared with a different photo of the same person — a function commonly used to unlock a smartphone. And as with the July report, the images had mask shapes digitally applied, rather than showing people wearing actual masks. 

Among the report’s findings: When both the new image and the stored image are of masked faces, error rates run higher. With a couple of notable exceptions, when the face was occluded in both photos, false match rates ran 10 to 100 times higher than if the original saved image showed an uncovered face. Smartphones often use one-to-one matching for security, and it would be far more likely for a stranger to successfully unlock a phone if the saved image was of a masked person.

The more of a face a mask covers, the higher the algorithm’s error rate tends to be. Continuing a trend from the July 2020 report, round mask shapes — which cover only the mouth and nose — generated fewer errors than wide ones that stretch across the cheeks, and those covering the nose generated more errors than those that did not.  

Mask colors affect the error rate. The new study explored the effects of two new mask colors — red and white — as well as the black and light blue masks the July study tested. While there were exceptions, the red and black masks tended to yield higher error rates than the other colors did, according to homelandsecuritynewswire.com.