Powerful Big Data Solutions for Military and Commercial Uses

This post is also available in: עברית (Hebrew)

Some 90% of the data available today has been created in only a few years. From devices at the edge to large data centers, the world is awash in data – being stored, indexed and accessed. Intel has been selected by DARPA, a U.S. Department of Defense agency, to collaborate on the development of a powerful new data-handling and computing platform that will leverage machine learning and other artificial intelligence (AI) techniques, according to the company’s announcement on its website. Intel will work as one of the hardware architecture research performers for the DARPA’s HIVE program (Hierarchical Identify Verify & Exploit), with a joint research program valued at more than $100 million during a 4½-year effort.

DARPA’s Microsystems Technology Office (MTO) created the HIVE program to develop new technologies to realize 1,000x performance-per-watt gains in the ability to handle graph analytics.

“The HIVE program is an exemplary prototype for how to engage the U.S. commercial industry, leverage their design expertise, and enhance U.S. competitiveness, while also enhancing national security,” DARPA website cites William Chappell, director of MTO. “By forming a team with members in both the commercial and defense sectors, we hope to forge new R&D pathways that can deliver unprecedented levels of hardware specialization. That can be a boost for commercial players but also can advance our military electronics supply to make sure the national defense infrastructure is empowered with the best capabilities in the world.”

Unlike traditional analytics that are tools to study “one to one” or “one to many” relationships, graph analytics can use algorithms to construct and process the world’s data organized in a “many to many” relationship – moving from immediate connections to multiple layers of indirect relationships. While some graphs are small and easy to visualize – such as a family tree – many graphs are vast and constantly changing, and they represent significant complex semantics – such as the evolving search list of every user on the planet for Amazon sales or Apple iTunes.

“By mid-2021, the goal of HIVE is to provide a 16-node demonstration platform showcasing 1,000x performance-per-watt improvement over today’s best-in-class hardware and software for graph analytics workloads,” said Dhiraj Mallick, vice president of the Data Center Group and general manager of the Innovation Pathfinding and Architecture Group at Intel.