While big data is already a big business, it’s still in its early stages.
In 2013, information technology spending driven by big data functional demands will total $34 billion. By 2016, the industry-wide impact driven by big data will be $232 billion, estimates Gartner Inc., a technology research and advisory firm.
“A lot of the work up until now has been focused on how to collect, store and retrieve the huge flood of data,” said Charles Gillespie, founder of Semantic Research Inc.
His San Diego-based company, founded in 2001, is focused on the next step: how to help humans make sense of all that data so they can profit from it and make better, faster and more confident decisions.
“We call it the last meter of Big Data: It's what happens between the computer and the human that determines how valuable all that big data is or isn't,” he said.
Experts in the space say “big data” is an overused term, referring generally to the problem of lots of data and a lack of capability to keep up with it -- a problem that’s common across various industries and sectors.
“Businesses have trouble keeping up, and the government has access to, and produces so much data that it is struggling to cope,” Gillespie said.
His analytical platform, Semantica, creates an adaptive knowledge environment in which data from almost any source can be consumed, turned into large, visual graphs of knowledge, and queried, filtered and manipulated so that the user can make sense of and understand whatever problem they are trying to address.
“We automate a lot of this functionality so that building networks of interconnected data is fast, easy and effective,” he said.
The platform is being used for counter-drug, counter-terrorism, anti-gang and social network analysis, as well as other classified applications.
Users also need to be able to look at the data in many different ways, including on maps and played back over time to help them understand the data in time and space.
In one of the programs the company supports, it was able to automate the process of analyzing suspected drug trafficking airplanes.
“So what used to take between 12 and 24 hours resulted in detailed analysis of huge volumes of data from many different sources in less than six minutes,” he said.
That means planes flying the air can be analyzed, tracked and seized by the time they land.
The ability to automatically fuse data from many different sources turns big data into actionable intelligence, he said.
“The results include seizure of over 30 aircraft, $2.6 billion and numerous indictments and arrests,” he said.
Another company in town, Packetsled, is also cashing in on the big data space.
“The size and complexity of data that is representative of security events is ever increasing,” said Matt Harrigan, president and CEO of Packetsled. “There needs to be a new set or breed of tools designed to address that data and make sense of it.”
Following 24 months of “hardcore research,” his company came up with a next-generation security analytics platform called Toboggan.
The program identifies bad actors and malicious traffic in real-time, giving security analysts the ability to immediately respond to incidents.
For instance, if a very large company or agency is experiencing a cyber attack on multiple disparate networks or locations, its technology can isolate the nature of the attack and contain the event.
“We identify attacks by many of their attributes, such as global location, protocol, and actor behavior,” said Harrigan, whose five-person office is actively hiring additional engineering team members.
He spoke in April about the convergence of big data and security at the annual C4ISR Symposium in San Diego.
“Attacks by nation states and cybercriminals are persistent and constantly evolving. If the enterprise and public sector do not have the visibility tools they need, the assets in these organizations are at substantially greater risk,” Harrigan said. “We're hoping to make a positive impact on this critical area.”
Toboggan has been doing private beta testing for the past three months with large Internet backbone carriers. Once in full launch mode, he thinks the public sector will also catch on.
“Specific areas in the DoD, like intelligence, would benefit greatly from the offering we have,” Harrigan said.
Due to the government’s high priority placed on cybersecurity and big data, companies like his should fare well through the defense budget storm. There’s still a level of uncertainty, however.
“We have had conversations that may have gone differently, maybe quicker, if there wasn’t sequestration in place. Everyone is feeling it -- even the security industry,” he said.