Is big data really the way of the future?

Big data initiatives are proving to be very beneficial for many organisations and hold a lot of potential for the future - if the organisation is willing to do more work, according to a new study by CompTIA.

According to the report, 72% of companies that have self-reported launching some form of big data initiative say that their results have exceeded expectations.

Yet while early returns may be encouraging, the study also reveals that much more work must be done to harness and make use of data.

"The amount of data crossing the wires and airwaves is mind-boggling," says Seth Robinson, CompTIA senior director technology analysis.

"So while individual pieces of a holistic data solution may be improving, these individual pieces are not yet integrated in a way that drives ideal results," Robinson.

Approximately three-quarters of organisations surveyed by CompTIA say that their business would be stronger if they could harness all of their data.

Additionally, 75% of companies feel that they should be more aware of data privacy, while 73% feel they need better real-time analysis.

Companies cite several factors for the increased importance of data:

• 63% rely on data for day-to-day operations
• 61% cited sensitivity around data privacy
• 60% use data to better understand customers
• 59% rely on data to measure business objectives
• 56% say they store data outside the company

"This reflects a theme consistent throughout much of our research," says Robinson. "Technology is a more powerful tool for every facet of the organisation and business lines are weighing in on technology matters as they use these tools to drive their objectives," he says.

Across the board, companies see data of all types growing in volume, led by customer data, email and instant messages, log files and documents.

They're also dealing with fragmented and siloed data: 45% of companies say that a high degree of their data is fragmented, and another 42% say their data fragmentation is moderate.

For organisations seeking to move from the basic to the more advanced, Robinson says they should take measured steps at each of the three stages of data usage: collection and storage, processing and organisation and analysis and visualisation.

By doing so, companies will be better prepared to assess new data technology options, evaluate potential partners for data initiatives and be positioned to more fully realise the potential of big data, he says.

As with most new technology undertakings, having the right skills can be a major hurdle. Big data is no exception, says CompTIA.

About half of the companies surveyed say they currently have the appropriate level of big data skill, while the other half see skills gaps in areas such as real-time analytics, relational databases and data security, the report shows.

Organisations also express a willingness to work with third parties for help with their data initiatives.

More than one-third of companies currently work with an IT firm for their data needs, those these engagements tend to be somewhat simplistic (data storage and data backup, for example).

But as companies become more aggressive with their data initiatives, IT channel firms may find opportunities to offer comprehensive end-to-end services around data, CompTIA says.

CompTIA's Big Data Insights and Opportunities report is based on an online survey of 402 business and IT professionals conducted in September and October 2015.