Big data in little New Zealand

Big data is emerging as big business, spawning something of an acquisition frenzy among information management companies.

The usual suspects have stumped up cash and lots of it, among them IBM, HP, Oracle and SAP, spending upwards of a combined US$10bn in recent years filling out their portfolios to vie for superiority in what is but a nascent market.

Also getting in on the action is information management company EMC; TechDay sat down with Phill Patton, country manager of EMC New Zealand, to gauge the applicability of big data in a small country.

“The organisations for which big data is relevant in this country include such obvious ones as those in telecoms and banks, and government,” he says.

However, Patton also contends that there are plenty more for which big data can provide results that can improve business performance. Yet for these organisations, making use of big data infrastructure depends on how it is delivered. More on that in a moment.

What’s that again?

Let’s take a step back to consider just what big data is. As the name implies, it is the discipline of dealing with (up until several years ago) improbably large sets of data. These data sets are, by and large, the product of an increasingly connected world.

From relatively straightforward business cases like those in retail, banking and government – where daily commerce produces millions of electronic records – to new ones like those surrounding the Large Hadron Collider or the Mars rover, enormous volumes of data are increasingly characteristic of our time.

Making sense of the totality of these data sets is what big data is all about. It is far more than simply storing it; instead, big data is more about using it. For businesses, that means (by and large) making money out of it; for scientific pursuits, it means discovering new knowledge. Or, as Gartner would have it, ‘Big Data are high-volume, high-velocity, and/or high-variety information assets that require new forms of processing to enable enhanced decision making, insight discovery and process optimisation.’

Important to note is that the term ‘big data’ does not define data sets by their volume; what is big data for one company might be a couple of petabytes, for another organisation, several exabytes.

Back to Patton. “Doing big data right means using it to generate income. A lot of that is around the predictive side of analytics; analysing vast quantities of data at speed and pushing the results of analysis to the right people so they can use make better business decisions.” In other words, the outcome of big data is a (relatively) small report, fast.

EMC likes to call it ‘volume, velocity and variety’, where a lot if information is processed quickly to deliver (you guessed it) a variety of insights. Patton calls these ‘wow moments’; back in the good old says, Archimedes settled for ‘Eureka!’

Expanding limited use cases

While Patton makes mention of several opportunities for EMC’s big data solutions (which incorporate technology from its 2010 acquisition of Greenplum) in government and ‘big industry’, what is likely more interesting is its push to the middle market. Interesting, because the greater majority of Kiwi companies, even the big ones, count as mid-size organisations.

“One of the advantages of Greenplum is that it is ready for multi-tenanting and the cloud, right now. While it’s relatively early days, we are working with service providers to provide big data analytics on a ‘per case’ basis; in other words, Big Data as a Service, in the cloud,” Patton reveals.

It’s a tantalising prospect; without having to invest in the considerable infrastructure necessary to do the big data number-crunch, local businesses can gain access to the important part: those reports and insights which can improve performance.

“If we can do that on a true ‘software as a service’ model, with models designed to produce cyclical outputs to meet the needs of clients, it broadens the market for big data considerably,” Patton concludes.