IT Brief New Zealand logo
Technology news for New Zealand's largest enterprises
Story image

Is distributed analytics the solution to a connected world of data?

Mon 22 Aug 2016
FYI, this story is more than a year old

It’s hardly breaking news to suggest that monolithic approaches to data management and analytics are not fit for purpose in a connected, big data world. The interesting question is how to extend the data and content architecture, and the accompanying analytics, to encompass new and unfamiliar sources without throwing away millions of dollars of existing investment. Step forward distributed analytics, a new way of thinking about how to extend capabilities out into the data landscape by providing appropriate data management and analytics “in the moment” while only transporting the data and insights back to the core that are necessary.

A concert of machine learning, analytical agents, and the cloud

I am not going to claim that distributed analytics is going to happen overnight, nor even that all the technology required to achieve the vision is available yet, at least not in an enterprise-friendly package. However, I believe that as a collection of design and approach principles it has some of the most solid foundations for how enterprises (across the industries) should start planning for a much more connected, data-dependent world.

Briefly, the idea that already difficult to manage data warehouses and inflexible data governance processes can be adapted to incorporate much greater amounts of data, at much higher speeds, is largely discredited. While it might be theoretically possible, the costs associated with applying a legacy approach to a relatively new and growing problem would be too high. The conundrum is that legacy has cost enterprises a small fortune in investment, a fortune that is still far from paying its returns, and core data-related business processes such as financial management and regulatory reporting require a heavily governed approach to ensure accuracy (if not always timeliness). Some argue that it is time to throw away these technologies and start afresh with data lakes and the cloud, among other options. I argue it’s possible to have both: a locked-down core of technology matched to much more flexible technologies that augment it – something I referred to as the elastic architecture in our research agenda for this year.

But what does that mean? The – slightly disappointing – answer is that no two distributed analytics solutions will look the same, but they will share characteristics. In the short term, the data lake will become a standard among most organisations. By its very nature, it should be a mix of both data landing zone and longer-term data storage, as well as, increasingly, home to some pretty complex data science-led analysis. Likely best located in the cloud (public, private, or – much more likely – a hybrid deployment that helps span the bridge between on- and off-premises), the data lake could be considered a buffer between a relatively well-organised internal data architecture and the more chaotic world beyond. This would, in essence, augment existing investments in information management technologies and provide the route to “bring the data home” if deemed necessary.

Longer term, I see a future in spreading analytic capabilities outside that core, beyond the warehouse and the data lake and out toward the machines and devices generating the vast amounts of data that are causing the “problem.” I talk in terms of analytical agents, packages of software loaded locally that don’t rely on heavy compute or memory. Using local resources backed up by machine learning-driven optimisation running at the centre, these analytical agents would only send the data back to the core that is necessary for things like exceptional events, improving the machine learning algorithms and regulatory reporting. Immediate proximity means that optimisation of physical processes could happen in near real time, without relying on the transport of vast quantities of data from and to the edge.

Orchestrating these capabilities is only just the beginning, and the technology is still emerging and maturing, but as an approach that looks to the future without abandoning existing requirements and the investments that support them, distributed analytics provides practical steps forward for the enterprise.

Article by Tom Pringle, Ovum analyst

Related stories
Top stories
Story image
Tech job moves
Tech job moves - Bitdefender, Cohesity, Fortinet & MODIFI
We round up all job appointments from June 27-30, 2022, in one place to keep you updated with the latest from across the tech industries.
Story image
Digital
Ivanti puts spotlight on power of employee digital experiences
The report revealed that 49% of employees are frustrated by the tech and tools their organisation provides and 64% believe this impacts morale.
Story image
Artificial Intelligence
Juniper study reveals top AI trends in APAC region
Juniper's research shows an increase in enterprise artificial intelligence adoption over the last 12 months is yielding tangible benefits to organisations.
Story image
Infrastructure
New VMware offerings improve cloud infrastructure management
VMware has unveiled VMware vSphere+ and VMware vSAN+ to help organisations bring benefits of the cloud to existing on-prem infrastructure.
Story image
Robotics
Evonik relies on Getac F110 tablet to control autonomous robot
The aim of the project is to evaluate the practicality of an automated robotic maintenance and inspection solution in the chemical industry.
Story image
MSP
Video: 10 Minute IT Jams - An update from CyberArk
Olly Stimpson joins us today to discuss the importance of MSP programmes and how MSP partners are experiencing success with CyberArk.
Story image
Cybersecurity
Blasé attitudes to cybersecurity by business a national risk
The largely unregulated state of cybersecurity in NZ, and consequential ambivalence of most businesses, risk hurting the country's trading prospects.
Story image
Hybrid Cloud
Advent One acquires Layer 8 Networks, complements hybrid cloud offering
The acquisition comes at a time of surging demand in hybrid cloud, network virtualisation and network security.
Story image
Mergers and Acquisitions
SAS acquires Kamakura to propel risk technology innovation
Underscoring SAS growth in the domain-specific solutions space, the acquisition will enable SAS to greatly enhance the breadth of its risk solutions portfolio. 
Story image
Microsoft
SAS wins Microsoft ISV 2022 Partner of the Year award
"We formed the SAS and Microsoft strategic partnership with a shared goal of making it easier for customers to drive better decisions in the cloud."
Story image
Cybersecurity
Video: 10 Minute IT Jams - An update from CrowdStrike
Scott Jarkoff joins us today to discuss current trends in the cyber threat landscape, and the reporting work CrowdStrike is doing to prevent further cyber harm.
Story image
Artificial Intelligence
Accenture shares the benefits of supply chain visibility
It's clear that gaining better visibility into the supply chain will help organisations avoid excess costs, inefficiencies, and complexity to ultimately improve their bottom line.
Story image
Cybersecurity
Zero trust security adoption rises 27% in just two years
A survey of WAN managers has revealed that multi-factor authentication and single sign-on are the top zero trust features implemented.
Story image
Cloudian
Cloudian, Vertica to deliver on-premise data warehouse platform
"We’re enabling our customers to capitalise on a leading object storage platform and maximise the value of their digital assets.”
Story image
Identity and Access Management
Ping Identity named a Leader in Access Management
Ping Identity has been named a leader in the 2022 KuppingerCole Leadership Compass report for Access Management. 
Story image
Hybrid workforce
Why hybrid working is here to stay and how to ace it
Citrix's new report reveals hybrid workers are more productive and engaged at work than their office and completely remote counterparts.
Story image
Oracle Cloud
Commvault, Oracle to deliver Metallic Data Management as a Service
"We are excited to partner with Commvault and enable our customers to restore and recover their most mission-critical cloud data."
Story image
Cybersecurity
Unknown connections: How safe is public WiFi in Aotearoa?
If it's not your own household WiFi, then who has control of your data and is your connection actually safe?
Story image
Financial results
Margins & revenues up at New Zealand arm of Acer Computer
We look at the local financial statements of Taiwanese manufacturer Acer Computer Inc.
Story image
Research
New study reveals 51% of employees using unauthorised apps
The research shows that 92% of employees and managers in large enterprises want full control over applications, but they don't have it.
Story image
Cryptocurrency
NOWPayments launches new service to analyse cryptocurrency fees
NOWPayments has launched a new network fee optimisation solution that analyses current network fees and picks the most profitable option out of the client's payout wallets.
Story image
Digital Transformation
What CISOs think about cyber security, visibility and cloud
Seeking to uncover the minds of CISOs and CIOs across Asia Pacific, my company recently asked Frost & Sullivan to take a snapshot of cloud adoption behaviour in the region.
Story image
Metaverse
How the metaverse will change the future of the supply chain
The metaverse is set to significantly change the way we live and work, so what problems can it solve in supply chain management?
Story image
Electrical
Up to $2.4 million shortfall in the collapse of IndeServe
We delve into the liquidators first report on long-standing networking service provider IndeServes collapse.
Story image
Entelar
How TruSens air purifiers can create healthier workspaces
The pandemic has heightened our awareness of our own and others’ health, and made us all much more conscious of the environments we work in.
Story image
Citrix
The best ways to attract young talent during labour shortages
New research from Citrix reveals hybrid working and ventures into the metaverse are top of mind for Gen Z workers.
Story image
Microsoft
ASI Solutions named finalist of Microsoft Surface Partner of the Year
"ASI Solutions has a strong Microsoft focus, building value by helping customers maximise investment in modern workplace solutions."
Story image
eCommerce
Robust digital warehouse management crucial in Asia-Pacific
Thanks to a network of “cloud” stores, grocery and food delivery providers such as Foodpanda can arrange for these commonly requested items to get packed up and sent over in almost no time.
Story image
Finance
Airwallex launches global payment services in New Zealand
The launch will enable businesses in New Zealand to tap into Airwallex's global payments services, offering an alternative to traditional banks.
Story image
Voice recognition
Renesas and Cyberon expand services with voice recognition
“We are honoured to collaborate with Renesas to simplify the development of embedded voice recognition functions."
Story image
Great Resignation
New SAP study uncovers impact of 'the great resignation'
Coined in 2021, the phrase 'the great resignation' refers to millions of employees globally leaving their jobs. The phenomenon is real and impacting SMEs.
Story image
Dark web
Cybercrime in Aotearoa: How does New Zealand law define it?
‘Cybercrime’ is a term we hear all the time, but what exactly is it, and how does New Zealand define it in legal terms?
Story image
Cybersecurity
Why is NZ lagging behind the world in cybersecurity?
A recent report by TUANZ has revealed that we are ranked 56th in the world when it comes to cybersecurity - a look into why we're so behind and what needs to be done.
Story image
Microsoft
Digital innovation could shape the future of NZ - Microsoft
With cloud technologies available to more people around the world than ever before, it is not only businesses who will benefit from using them.
Story image
Public Cloud
Public cloud services revenues top $400 billion in 2021
"For the next several years, leading cloud providers will play a critical role in helping enterprises navigate the current storms of disruption."
Story image
API
Industry-first comprehensive risk-based API security enhances protection
Application Programming Interfaces (APIs) have become a crucial part of operating web and mobile application businesses and are causing significant economic growth in the digital sector.
Story image
Vendor
Forescout reveals top vulnerabilities impacting OT vendors
Forescout’s Vedere Labs has disclosed OT: ICEFALL, naming 56 vulnerabilities affecting devices from 10 operational technology vendors.
Story image
Revenue
Datacom announces revenue of $1.45 billion, fall in profit
Growing market pressures and border closures saw Datacom place increased focus on talent development initiatives for both existing and future employees.
Story image
Airwallex
How Airwallex helps businesses achieve globalisation success
As markets continue to shift, businesses need to be able to provide the same quality of service for customers regardless of where they are located around the world.
Story image
Hybrid workforce
How organisations can prepare for a post-pandemic workforce
The so-called 'new normal' office looks different to how it did pre-pandemic, and organisations need to take steps to better manage their post-pandemic workforce. 
Story image
Infrastructure
Oracle Cloud Infrastructure expands distributed cloud services
“Distributed cloud is the next evolution of cloud computing, and provides customers with more flexibility and control in how they deploy cloud resources."
Story image
Web Development
Whitecliffe fosters careers for the future of tech
Do you want a career in Information Technology, Networking, Web Development, Software Development, or are you looking to upskill?
Story image
Data ownership
Brands must reclaim trust by empowering data ownership
According to Twilio's new State of Personalisation Report 2022, 62% of consumers expect personalisation from brands, and yet only 40% trust brands to use their data responsibly and keep it safe.
Story image
Samsung
Monitors are an excellent incentive for getting employees back
The pandemic has taught us that hybrid working is a lot easier than we would’ve thought, so how can the office be made to feel as comfortable as home? The answer could be staring you in the face right now.