Why big data should be built into every stage of the IT journey

The risks of avoiding big data in enterprise organisations could end up costing more, or it could result in a failed project, according to enterprise application provider Micro Focus.

Organisations will begin to realise that big data is critical to organisational operations and decision-making, and that it should be included at every stage, says the company's ANZ managing director, Peter Fuller.

"This is not too dissimilar to what we previously experienced with security. Security was once treated as an afterthought, retrofitted to solutions and projects as businesses finally realised how significant the threat of cybercriminals really was. Now, organisations are demanding that security be baked into everything they buy and everything they develop right from the start. That's the next stage of evolution for big data.

When is big data valuable? Micro Focus says value is derived when big data offers actionable insights. These could be customer-based insights, to cyber attack predictions, to which equipment needs to be serviced. If businesses act on predictive insights, it could lead to increased customer lifetime value, lower operating costs, and a reduction in operational risks.

But it comes with a caveat – some businesses may not be able to collect and store big data - or analyse it and act on insights.

Fuller says big data has potential to derive insights from data sourced from diverse platforms like social media, purchase history, and online interactions with the business.

"However, extracting and analysing that data doesn't happen automatically; it requires businesses to have a big data platform that includes a mechanism to gather that data from various sources, collating it in a single location, then analysing it. Only then can the data yield the insights that create real value," he explains.

According to research from Vertica, as many as 40% of organisations that embark on big data projects only have a data warehouse as their infrastructure. Fewer than half of those organisations use analytics, and 44% say there are 'pockets' of advanced analytics use.

Micro Focus says these findings suggest that many organisations are disorganised when it comes to big data. These organisations are still likely to derive value from their big data initiatives.

However, there is significant untapped potential for these businesses to streak ahead if they were to become more sophisticated in their big data approach.

"For organisations to become more data-driven, they need to have a mature approach to managing big data. This requires a combination of technology solutions and big data skills, which most organisations don't yet have in-house. Working with a partner that can offer a big data platform, software, and services can help organisations accelerate their big data journey and avoid the potential pitfalls along the way," Fuller concludes.