Top Secret Data Moved to Cloud

This post is also available in: עברית (Hebrew)

The US National Security Agency (NSA) has been systematically moving most of the mission data it collects, analyzes and stores into a classified cloud computing environment. The Intelligence Community GovCloud is a single integrated “big data fusion environment” that allows analysts to rapidly “connect the dots” across all NSA’s data sources, according to the NSA’s Chief Information Officer Greg Smithberger.

The system is built on the same commercial hardware you might see in data centers owned by Facebook, Amazon or other industry titans. But that hardware is blended with NSA-developed custom software, exotic processing, high-performance computing and other unique NSA intellectual property.

The NSA wanted its data, including signals intelligence and other foreign surveillance and intelligence information it ingests from multiple repositories around the globe, to be kept on a single data lake analysts from the NSA and other IC agencies can run queries against.

The IC GovCloud environment accelerates the analytic work humans can do by employing machine learning and algorithms, Smithberger said.

Data ingested by NSA has been meta-tagged with bits of information, including where it came from and who is authorized to see it, which ensures analysts only immerse themselves in intelligence they’re cleared to see.

“This environment allows us to run analytic tools and do machine-assisted data fusion and big data analytics, and apply a lot of automation to facilitate and accelerate what humans would like to do, and get the machines to do it for them,” Smithberger told nextgov.com. Analysts can “interactively ask questions” of the data in the cloud environment.

While the IC GovCloud is NSA’s creation – and centrally funded by the NSA – its basic services are available to the 16 other agencies that comprise the IC, including the Central Intelligence Agency and National Geospatial-Intelligence Agency.

IC GovCloud is one of two major cloud initiatives across the IC. Four years ago, the CIA awarded a $600 million contract to Amazon Web Services to develop a commercial cloud environment for the IC agencies. Today, the Amazon-developed C2S provides utility computing services to the IC.