This post is also available in: עברית (Hebrew)
Zvi Brunner, Vice President of Technology at Team Group
The field of defense is the first to adopt new and advanced technologies. Cyber systems, for example, are getting a substantial push due to the fact that military and defense bodies are investing vast resources in developing systems in the field, an investment that is of course beneficial to organizations in the private sector. Along with cyber, there are two main growing fields: Big Data and the Internet of Things (IoT). Both of these “hot” fields and their combination are on their way to revolutionize defense organizations.
The modern battlefield includes a wide variety of advanced tools, including different kinds of sensors that provide a lot of information which can determine the course of the battle – as long as all the data arrives in time to be incorporated in a system providing a complete picture. The sensors themselves keep getting more advanced by computerized power inside the sensors. The challenge is to collect all this large amount of data in a logical way and in real-time – including correlation between the data in order to help commander make decisions based on the larger picture. This challenge can be achieved using a modern IoT infrastructure integrated in advanced analytics engines and with a Big Data platform, to handle the information streaming in real-time.
BIg Data systems has the ability to analyze large amounts of data and offer fast execution due to their ability to use computerized systems’ memory on the one hand, and save the information for long periods of time in “lakes of information” on the other. Smart systems are also based on computerized tool with Machine Learning capabilities, allowing for fast conclusions on the current situation, as well about predictive occurrences, partly by comparing complex current data with vast historical information.
The architecture of Big Data systems also offers the capability of performing Scale Out for the servers and softwares operating in them, thus allowing to put together very busy and complex systems, as well as redundant ones, with no single point of failure.
For example, in case of sensors that detect movement next to a security fence, a real-time procedure which takes into account all the details of different information from a variety of sensors along the fence, allowing to reach more accurate conclusions in real-time, with no need to alert the whole perimeter. In this case, the sensors that detect a movement near the fence operate as smart end-points which are capable of initially processing the information along with filtering and synchronizing it with high levels of details. Furthermore, there is an option to integrate data mining systems via SQL (ODBC\JDBC) mechanisms and to use various data mining and display tools such as Cognos, Tableau, Webfocus, which provide a visual picture of the situation – an option that hadn’t existed in the past.
Another example: In order to manage a modern battlefield, the Control & Command infrastructures analyze a lot of data arriving from different sources, such as sensors, human intelligence, VCR, etc., meant to provide an up-to-date picture at any given time. The more sources of information being analyzed in real-time, the better battlefield commanders can manage the battle in a more intelligent and accurate fashion.
However, the combination between the two fields – IoT and Big Data – revolutionary as it might be, also presents many challenges, since the two fields are especially dynamic and change in a rapid pace, as well as due to the need to adjust them to the special needs of security services such as a designated communication systems and a secured environment.
How can these challenges be met?
- Using Open Source – New tools are being changed and upgraded and new products are entering the market frequently. As such, and in order to keep up with the speed of these changes, many companies have changed their perception and are releasing their open source product to the community, to be publicly used.
- Implementing new generation storage systems – Until a few years ago, storage systems have used magnetic storage, which often proved as a bottleneck. Today storage companies offer high performance, flash-based solutions. These storage infrastructures allow to sample incoming data at a certain point in time, meaning they run terabytes of data quickly and in real time. Alternatively, there are systems that use the computer’s internal memory to perform speedy process or in-memory analytics such as the popular Spark by Apache, based on open source.
- Accumulating knowledge and sharing knowledge – By exchanging information with bodies and organizations that have already entered the field as well as with bodies who are already specialized and hold a base of knowledge, it is possible to move forward much faster and learn faster by way of exchanging knowledge. This is a varied discipline requiring extensive knowledge in many different fields – software, hardware, information security, storage, etc. Team Group, which is varied by definition, and provides solutions for a wide range of different fields, as well as a line of specialists – successfully handles the challenge of collecting data from different sources in processing it for its clients. Team Group is already successfully implementing projects in the field of Big Data for security bodies as well as for large businesses in the private sector.