AR/VR Technologies As Threat Modeling Tools

threat modeling

This post is also available in: עברית (Hebrew)

Augmented reality and virtual reality have the potential to serve as threat modeling tools that could enhance security professionals’ understanding of the threat landscape both in the industrial and defense fields.

Virtual reality creates a virtual world for users to interact with. It is more about what users feel or experience in that world than how they connect with it. Augmented reality, on the other hand, blends elements of a virtual world with the real world, so it is more about how the user interacts with those various elements and the components of the real world.

Furthermore, according to, virtual reality is well-suited for the gaming, training and simulation sectors, while augmented reality is better positioned in the commercial, industrial, educational and medical sectors.

While virtual reality offers only one plausible form factor so far — that of a wearable helmet or goggles, augmented reality has at least five different form factors in the commercial sector alone and offers one form factor designed specifically for mobile technology.

Virtual reality can be used to simulate catastrophic disasters in industrial sectors without putting users in physical danger, destroying company or industrial property, or harming the physical environment. It could make an excellent threat modeling tool for physical threats.

In the industrial sector, for example, virtual reality might be used to simulate a nuclear reactor meltdown, a large fire in a high-rise office building, a coal mine collapse or a major infrastructure failure. Similarly, in the civil sector, it can be applied to replicate a citywide blackout, emergency response in disaster zones, and earthquake and aftershock damage. It can even be used to train astronauts to work on the International Space Station.

In the defense sector, virtual reality may be used to simulate terrorist attacks on U.S. or foreign soil, train special forces for overseas missions where the precise physical layouts of buildings are other structures may be unknown, conduct urban warfare exercises and test advanced aircraft.

Unlike augmented reality, virtual reality allows a headset to be integrated with a drone. There are even drone races in which operators are required to fly the devices through structures of which they have no prior knowledge.

Because augmented reality is primarily designed to enhance how users interact with the world around them, there are various potential uses for this technology in all facets of threat modeling. Here are just a few:

  • Overlaying power lines, water pipes and gas line assets onto a standard map image for field workers in the energy and utilities sector;
  • Nuclear reactor safety inspections and walkthroughs;
  • Structural inspections of all types in factories;
  • Structural inspections of civil structures;
  • Implementation of heads-up displays in all types of automobiles, including self-driving cars, to assist automotive engineers with their work on traffic patterns and algorithms;
  • Visual data modeling for data scientists, statisticians, mathematicians and risk management professionals.

Augmented reality could become advanced enough to allow information security professional to visualize the physical paths an adversary might take through a network, building, city block or industrial facility. It could also provide penetration testers with three-dimensional virtual threat models of applications, software and solution blueprints.

They also introduce new threats. The implementation of augmented reality in a highly classified environment, industrial control facility, nuclear power station, military base, or any research and development lab that supports defense research could lead to disaster in the wrong hands. An adversary capable of compromising the application program interface (API) that connects augmented reality software to an automobile’s heads-up display could remotely crash the car. A threat actor could also breach augmented reality glasses to access what amounts to a livestream of personal data without the wearer’s knowledge.

In addition, a malicious insider could wear augmented reality glasses to work to gather intelligence, commit industrial espionage or upload anything the device picks up to a personal cloud.

Augmented reality is going to be a unique technology for information security professionals to contend with because of its significant potential to be both helpful and extremely harmful.