Five areas to watch in big data

As our lives have become increasingly online, our ever-expanding digital footprint combined with the explosion of information heralded by the internet has led to big data taking a high priority on the agenda for business.

The ability then to harness this wealth of data and convert it into deeper insights about customers, is a very valuable commodity and furthermore, a very real source of competitive advantage.

How do businesses do this? Big data tools are an easy, effective and efficient way to help businesses analyse large amounts of digital information and more than ever, how organisations approach the challenges of big data will be key to their success.

Here are five areas in big data business leaders should be watching in 2015.

1. Democratisation of big data through cloud

In the past year alone, there has been incredible growth in the use of cloud-based data analytics services. Even smaller organisations that once thought advanced data analytics were out of their realm of possibility can now begin managing and analysing both structured as well as unstructured data, quickly and cost-effectively.

Essentially, the cloud will offer a greater array of choices for organisations, no matter their size, to hit their desired big data benefit / price point trade-off, as well as lower the bar for companies looking to experiment with big data, particularly unstructured data.

2. Growth of unstructured data

Unstructured data volumes – things like human information, machine sensor data, Internet of Things (IoT) data, and business data – continue to grow at a mind-boggling rate in 2015.

According to industry commentators, IoT data, excluding PCs, tablets and smartphones, will grow to 26 billion connected devices by 2020. Countries like China, Australia and Korea are contributing to this trend, and New Zealand is following suit.

Regional organisations will increasingly seek solutions that can tie structured and unstructured data sources together virtually and generate connected intelligence from social media and video analytics. This will give greater context to the structured data that most organisations have come to rely on.

3. Predictive analytics becomes the norm

Predictive analytics will evolve beyond the next "cool thing," to a "you better have it or else" business tool.

Business processes must be geared toward action at the speed of insight, which will make the re-engineering for big data critical. There is no value in identifying what customers are doing every minute of the day if you can't respond predictively and proactively. By the time you've extracted, transformed and loaded the data in some warehouse, it's too late.

Organisations will re-engineer their big data environments to enable information streams to be accessed, analysed and shared in real time. This will be the key to increasing revenue, improving knowledge worker productivity, and lowering costs.

4. Changing IT operations

Companies who 'get' big data will apply big data principles via small "proof of concept" teams, often much closer to the customer interface than the current IT-focused business intelligence (BI) teams. In many cases marketing or sales principals should lead these teams, not IT professionals. Once proven, the systems should be then scaled and applied to internal IT operations. Such an approach instils a "customer-first" dynamic and reduces the risk of getting sidelined by the complexities of current Operational Data Pools. In short the focus needs to be on enhancing customer value, not on optimising IT operations (and costs).

Big data analytics will also play a role in identifying IT security threats, which are constantly growing and evolving. It will also deliver connected intelligence to organisations, generating insights which drive innovation and critical business advantage. This process will reinvigorate the traditional service desk. The transformation to a big data service desk will bring businesses the capabilities to deliver service anywhere.

5. Big data education

Today's universities can't seem to train data scientists fast enough. Many in the industry view data scientists – those individuals with engineering and business skills, as well as the statistical savviness – as the key to analysing and deriving value from big data.

That said, the current lack of "big data talent" should not hold businesses up in terms of launching initiatives. Instead, the key will be empowering business analysts of today with tools they already know.

The idea of a "data scientist" may very well be played out in a couple of years. I see that the "Data Savvy Business Person" will likely emerge as the new rock star.

However it's important to note, if a company assigns big data to its existing BI teams, they're almost guaranteed to fail. The tech, thinking and approaches that lead to success at BI level require a whole new approach for big data.

In addition, the availability of data analytics in the cloud presents a huge opportunity for developers, and we expect the big data developer community to increasingly emerge as a hotbed of innovation. Cloud-based big data services represent decades of significant intellectual property around managing, accessing, and analysing a wide range of data, including unstructured information that developers can now use like they have used Amazon Web Services or open source systems. Developers have only just begun to exploit the value of big data, especially unstructured data, and this trend will only accelerate in the next few years.

To find out more about HP's Big Data storage and analysis systems, please contactDavid Eaton.