Businesses are collecting massive amounts of data annually, but do they really know how to make good use of it? A recent study from NewVantage Partners finds the answer is: not really.
Nearly three-quarters of organizations in the study (73%) say business adoption of big data and artificial intelligence initiative is a challenge. And just 38% say they’ve created a data-driven organization.
What are the roadblocks to making better use out of all of the data we’re collecting? We asked our IDG Influencer community of experts about the biggest challenges to extracting business value from data, and how organizations are using cloud analytics to overcome those issues. Here’s a summary of their insights.
Data complexity can be daunting
With organizations collecting and hoarding so much data, the sheer complexity of finding value is the biggest challenge, according to Gene De Libero (@GeneDeLibero), Chief Strategy Officer and Head of Consulting with GeekHive.
“There's way too much data and not enough time to create business-critical insights,” he said. “Add to that the challenge of surfacing the right data and tools for the job, and you have a scalability and agility problem.”
Further adding to the challenge are the variety of data types and the lack of integration among different data silos across the organization. “Large enterprises have many disparate data sources, and it’s difficult to see how all the moving parts of an organization are working together if they’re in different places,” said Jo Peterson (@digitalcloudgal), VP of the Cloud Services Practice at Clarify360.
The scale and complexity of the data often make it hard for IT leaders simply to get a clear picture of what data is available to use to advance business objectives.
“One challenge I see in many organizations is the presence of dark data,” said Jason James (@itlinchpin), CIO at Net Health. “That is to say, data that lives in the proverbial shadows and is seemingly stored but unused or underutilized. In fact, they may not understand what is truly being captured or stored. If data is the new oil, then many organizations may be sitting on large, untapped reserves.”
These underutilized data reserves make it critical for organizations to invest in the tools to normalize data across disparate sources, said Isaac Sacolick (@nyike), President of StarCIO and author of Driving Digital.
“Databases and data lakes are more like data vaults,” he said. “Business people know there's value in the vault and often make deposits, but they don't have adequate tools to know when, where, and how to get insights from it.”
Scott Schober (@ScottBVS), President and CEO of Berkeley Varitronics Systems, agrees that making data accessible is an important step toward deriving value from it. “Since most large enterprises have many differing data sources, it can be challenging even for internal divisions to leverage various resources and personnel to work together throughout the organization,” he said.
Integration challenges extend beyond technical issues to data governance, said Frank Cutitta (@fcutitta), CEO of HealthTech Decisions Lab. “Some of the greatest battles in the enterprise are related to who owns the data, and more importantly who controls the messaging derived from the insights,” he said.
Quality data, quality insights
Beyond complexity, data quality is also a challenge. Business value extraction is only as good as the data provided, according to Dave Evans (@DaveTheFuturist), CIO & VP of Technology at The Computer History Museum.
“Quality data depends upon the sources of the data,” he said. “‘Garbage in, garbage out’ is apropos here. With a data deluge, how can organizations ensure the data is quality data? More work is now needed to extract real value, which points to the quality and capability of the tools.”
But even with the right data, many organizations still struggle to use the data to truly capture business value. “Many don't first clearly understand the business problem they are solving for,” said Tim Crawford (@tcrawford), CIO Strategic Advisor with AVOA.
That’s where the skills gap comes into play. Data science skillsets are in significant demand, but limited supply.
“The biggest challenge to extracting business value from data is having the professional ability to decipher its true impact,” said Jeff Cutler (@JeffCutler), a technology journalist. “If you’re not sure what data says about your operations, profitability, or even your reach with specific audiences, you’re working with fewer tools than your competitors.”
That need reflects another skills gap: the “translation layer” connecting raw data with business objectives and outcomes.
“It’s a challenge to work with subject matter experts that understand the context,” said Mark Sangster (@mbsangster), VP and Industry Security Advocate at eSentire. “In this case, it’s a matter of commitment and providing resources.”
Pairing data scientists and business users is critical to identifying what Cutitta calls actionable data input. “The exercise must start with what data you need to have for competitive advantage rather than trying to find an unknown needle in the haystack,” he said.
Business objectives should always be the starting point when devising an analytics plan, said Peterson. “Real value from data is derived by defining specific business questions that need answers and then creating a strategy to identify the correct data set that can provide those answers,” she said.
Cloud analytics deliver clarity
To address challenges around storing, analyzing, and extracting insights from ever-larger volumes of data, more organizations are turning to the cloud.
“Analytics is the perfect workload for the cloud,” said Herain Oberoi (@herain), Director of Product Marketing, Databases, Analytics, and Blockchain at Amazon Web Services (AWS). “You need lots of data, so you need storage. You need to process that data, so you need compute. And you need the ability to spin up and spin down resources to run your analytics on an as-needed basis. At its essence, cloud is about low-cost storage, compute, and elasticity.”
Moving data to the cloud is an important step toward integrating disparate sources of data. “Our clients are using cloud analytics to break down data silos across the enterprise and foster more effective collaboration and more agile outcomes,” said De Libero.
A centralized, cloud-based data lake also will help organizations make data accessible to everyone across the business. “Self-service cloud analytics can help overcome these challenges by democratizing data analytics for business users,” said Will Kelly (@willkelly), a technical writer.
Democratizing data can unlock new opportunities for innovation across all parts of the business. “Cloud analytics can also enable new business initiatives by taking advantage of previously unused data to gain new business insights,” said Peterson.
For example, “Big data has grown not just in the scale but also in depth, allowing sales teams to better understand and even predict customer needs,” said Schober.
Cloud-based analytics can also help security teams find signals in the noise, said Kayne McGladrey (@kaynemcgladrey), Cybersecurity Strategist at Ascent Solutions.
“Where cloud analytics shine is in detecting a repeated series of risky actions by an individual user account [that signal] a business email compromise followed by a ransomware attack,” he said. “Cloud analytics allow organizations to detect and prevent these and other attacks not only at scale but also faster than traditional investigative techniques.”
Those are just a few examples of how a modern, cloud-based data infrastructure can increase the pace of insights and innovation.
“By allowing technology to do the heavy lifting,” said Cutler, “you can examine data quickly and respond to business challenges in time to actually make a difference.”