This post is also available in:
עברית (Hebrew)
Despite the growing presence of facial recognition in public spaces, new research suggests that the performance of these systems in real-world settings may be far below what official benchmarks indicate.
The technology is now widely used in airports, public security operations, and commercial settings. However, researchers from the University of Oxford have raised concerns that current performance metrics, such as those from the U.S. National Institute of Standards and Technology’s (NIST) Facial Recognition Technology Evaluation (FRTE), may paint an overly optimistic picture.
The issue, according to the researchers, lies in the gap between lab-based testing and field conditions. While evaluations like FRTE report accuracy rates as high as 99.95% for top-tier systems, these results are derived from highly controlled environments—far from the unpredictability of public spaces.
In practice, facial recognition has shown significant shortcomings. Misidentifications have led to wrongful stops and arrests, including cases in both Detroit and London where individuals were incorrectly flagged by live facial recognition systems. Such incidents have raised longstanding concerns about the technology’s reliability, especially when it comes to recognizing people of color, women, and older individuals.
One of the key points raised by the researchers is that real-world conditions—poor lighting, crowded scenes, weather interference—are not replicated in the benchmark datasets. These evaluations often rely on high-quality, front-facing images that do not represent the challenges encountered in everyday deployments.
Additionally, the datasets used for evaluation frequently lack demographic diversity, which can lead to disproportionate error rates across different population groups. This imbalance contributes to a higher likelihood of false positives, especially in high-stakes environments like policing.
Despite these concerns, facial recognition technology continues to expand. In the UK, for example, the Home Office recently announced the deployment of additional live facial recognition units across multiple police forces.
The researchers stress the need for independent, real-world assessments of facial recognition systems before relying on them for critical public safety decisions. Without this shift, they argue, policymakers risk making deployment decisions based on figures that don’t reflect actual performance on the ground.
