Big Data Role in Intelligence Missions

big data

This post is also available in: עברית (Hebrew)

The operational world that interests governments the most nowadays is the intelligence world. Unlike the “national/military” intelligence of the past, the current intelligence realm focuses on bringing information from the civilian world for the purposes of combat and terror prevention. States have been willing to pay considerable sums and invest a lot of efforts in order to detect and prevent an action by a lone terrorist – as in the case of the Manchester attack in the UK.

The intelligence world is principally divided into three parts – general prevention (Mass Collection), focused prevention (ticking time bomb), and investigation in retrospect (forensics).

The technologies suitable for each stage are different, the scope of population’s privacy violation changes according to the threat severity, the various working methods, etc.

Only recently we have been exposed to the leaking regarding tools used by the NSA. Large corporations act in hypocrisy when they ask intelligence services to reveal their knowledge regarding vulnerabilities in the companies’ products. They forget that the reason why intelligence services are looking for these vulnerabilities is, in fact, the attempt to catch criminals and terrorists that use the corporations’ platforms – for criminal activities and even terrorism.

The general primary information gathering level employs big data technologies. A state’s ability to connect to information sources which deal with petabytes of streaming information per day (credit companies, telecom, applications such as Linkedin, Twitter, Facebook) is available, the gap is in the capability to contain the information, understand it, and draw quality conclusions from its analysis (business Intelligence – BI). Among the companies in this sphere are TA9, Palantier from Detica Group, Rayzone, Verint, Elta, and more.

When dealing with the level of focused intelligence, the work is target oriented. There are specific tools available for the detection of the target, receiving information from and about it, and execute a focused prevention. In addition to technology, human intelligence (HUMINT) is vastly used – by the integration of intelligence work methods with advanced technologies. Technologies such as Trojan horses, wi-fi traps, GPS systems, cellular interception, etc. Here, too, we can see large corporations, yet most of the “tactical” tools are supplied by small companies, while the larger ones integrate the systemic solutions they supply.

At the level of investigation in retrospect (forensics), again we use customized tools, including ones for investigation management and big data fusion, yet this time the data analyzed is past data and not live information, the data was already gathered and exists in databases somewhere. Here, too, we can see the same players as in the first general mass collection level.

The companies’ websites reflect the innovation in this sector, based on unique technologies for information gathering and interception – systems such as Vegas, a unique strategic data interception system which does not depend on collaboration with Internet providers (ISP), and Echo, a virtual global SIGINT system.