Forensic Facial Identification – Surprising Research

This post is also available in: עברית (Hebrew)

Experts at recognizing faces often play a crucial role in criminal cases. A photo from a security camera can mean prison or freedom for a defendant — and testimony from highly trained forensic face examiners informs the jury whether that image actually depicts the accused. Just how good are facial recognition experts? A recent study by the US National Institute of Standards and Technology NIST measured the accuracy of forensic face examiners and the science behind their work for the first time.

The study combines forensic science with psychology and computer vision research. The team provided at least one revelation that surprised even the researchers: Trained human beings perform best with a computer as a partner, not another person.

The study also examines the best technology as well, comparing the accuracy of state-of-the-art face recognition algorithms to human experts.

Their result from this classic confrontation of human versus machine? Neither gets the best results alone. Maximum accuracy was achieved with a collaboration between the two.

Technology has also advanced. “If we had done this study three years ago, the best computer algorithm’s performance would have been comparable to an average untrained student,” said NIST electronic engineer P. Jonathon Phillips. “Nowadays, state-of-the-art algorithms perform as well as a highly trained professional.”

One of the findings, according to nist.gov, was unsurprising but significant to the justice system: The trained professionals did significantly better than the untrained control groups. This result established the superior ability of the trained examiners, thus providing for the first time a scientific basis for their testimony in court.

What raised the team’s collective eyebrows regarded the performance of multiple examiners. The team discovered that combining the opinions of multiple forensic face examiners did not bring the most accurate results.

“Our data show that the best results come from a single facial examiner working with a single top-performing algorithm,” Phillips said. “While combining two human examiners does improve accuracy, it’s not as good as combining one examiner and the best algorithm.”

Combining examiners and AI is not currently used in real-world forensic casework. While this study did not explicitly test this fusion of examiners and AI in such an operational forensic environment, results provide a roadmap for improving the accuracy of face identification in future systems.

“If combining decisions from two sources increases accuracy, then this method demonstrates the existence of different strategies,” Phillips said. “But it does not explain how the strategies are different.”

The study was published in the Proceedings of the National Academy of Sciences.