Big Data – What Are The Next Challenges?

Big Data – What Are The Next Challenges?

big data

This post is also available in: heעברית (Hebrew)

The big data realm is developing rapidly and confronting new challenges. RISELab, a research lab opened recently at the University of California, Berkeley, focuses on systems that provide Real-time Intelligence with Secure Execution (RISE). The new computer lab, the successor of the renowned AMPLab, strives to “move beyond big data analytics into a more immersive world,” where “sensors are everywhere, AI is real, and the world is programmable”.

One example cited by infoworld.com: Managing the data infrastructure around “small, autonomous aerial vehicles,” whether unmanned drones or flying cars, where the data has to be processed securely at high speed”.

One of the challenges is security, but not the conventional focus on access controls. Rather, it involves concepts like “homomorphic” encryption, where encrypted data can be worked without first having to decrypt it.

Prof. Joseph Hellerstein, a database systems veteran and lab principal investigator, elaborated on the shift in technology trends. “Ten years ago, a huge acceleration in data growth ushered in the era of Big Data and practical machine learning. The next commoditization shock is underway with the proliferation of data-centric devices. Billions of networked sensors—in cellphones, cameras, cars and buildings—instrument the world around us.  We’re seeing rapid growth in programmable devices that can take action as well. The next big challenge in data-centric computing is to make it easy to close the loop between sensing and acting, via new platforms for real-time intelligent decision-making.” according to Berkeley’s website.

Some projects have already started to emerge at the lab, one of them is Ground – a context management system for data lakes. It provides a mechanism that “enables users to reason about what data they have, where that data is flowing to and from, who is using the data, when the data changed, and why and how the data is changing.”

Data aggregation has moved away from strict, data-warehouse-style governance and toward “very open and flexible data lakes,” but that makes it “hard to track how the data came to be.” In some ways, he pointed out, knowing who changed a given set of data and how it was changed can be more important than the data itself. Ground provides a common API and meta model for track such information, and it works with many data repositories.

Learn more about data lakes and all other aspects of big data within the homeland security realm at the forthcoming Big Data for HLS Conference and Exhibition, organized by iHLS. The event will take place on February 13th, 2017 at the Lago Conference Center, Rishon LeZion. it will host some of the leading experts in the field and showcase cutting-edge technologies.

Sponsorship/booth

Agenda