This post is also available in: heעברית (Hebrew)

Artificial intelligence coupled with cloud computing and Internet of Things (IoT) technology can be a powerful measure to achieve public safety. Oil company Royal Dutch Shell has recently partnered with Microsoft to try an artificial intelligence technology that’ll help save lives by detecting certain behaviors.

Imagine a man lighting a cigarette while he’s waiting at the pump for his car to finish fueling, unaware that with one move he could cause a fire or explosion. An onsite video camera captures the scene, and a device inside the station running Microsoft Azure IoT Edge can now use artificial intelligence tools to pick out that behavior as a potential safety risk.  

It’s a first line of defense on the “intelligent edge,” where data is quickly processed close to where it’s collected, without accessing the cloud, and simple machine learning algorithms can dispense with anything that’s not of interest, as blogs.microsoft.com explains. They can also be trained to look for other high-risk incidents: people driving recklessly, theft, improper fueling.

Questionable frames are immediately uploaded to the Microsoft Azure cloud, which can power more sophisticated deep learning AI models. These can identify that the man is smoking and raise an instantaneous alert on an onsite dashboard, so the station manager can take action to shut down the pump before any harm is done.

A pilot project is now in action at two gas stations in Thailand and Singapore.

The key to the quick turnaround is edge computing, which processes data near its site of origin rather than in the cloud. In this example, which uses Microsoft’s Azure IoT Edge platform, the footage analysis happens onsite at the gas stations. Only frames that raise any red flags move on to the cloud for advanced processing.

“Each of our retail locations has maybe six cameras and captures something in the region of 200 megabytes per second of data,” Daniel Jeavons, Shell’s general manager for data science, told Microsoft. “If you try to load all that into the cloud, that quickly becomes vastly unmanageable at scale. The intelligent edge allows us to be selective about the data we pass up to the cloud.”