IT Brief New Zealand logo
Technology news for New Zealand's largest enterprises
Story image

Q&A: Cloudera on building the perfect hybrid cloud architecture

By Sara Barker
Mon 13 Sep 2021

Cloudera's APJ cloud director Stevie Walsh chats to us about hybrid cloud and the democratisation of data.

One of the most common pieces of advice to organisations is that an entirely on-prem IT infrastructure is seldom workable for most organisations’ IT needs, so the hybrid infrastructure is the best of both worlds. Is there a trick (or perhaps several tricks) to making the most of all these environments?

Many organisations take a well-intentioned leap into a hybrid cloud environment without taking the entire digital transformation journey into account - essentially putting the cart before the horse. 

They’re on the cloud, but their data could be spread across multiple clouds. There are also situations where they must split their data between the cloud and on-premise, which makes it even more challenging to access, manage and secure.

All clouds have their benefits, however, there are common challenges that need to be addressed to make the most of today’s modern cloud environments. These include:

  • Greater amounts of data coming from multiple sources 
  • Data capture at the Edge, where it must be ingested and analysed in real-time
  • Multiple systems that can be problematic to integrate
  • Concerns related to data security and accessibility of data within an organisation
  • Multiple public and provide clouds and data warehouses within a single enterprise customer

It’s also important to understand which workloads are best suited for each deployment – the public cloud is ideal for intermittent, seasonal, ‘batchy’ or unknown (newly developed workloads) whereas a private cloud is best for highly utilised workloads. Hybrid cloud environments can help organisations support increasingly remote workforces, improving scalability and control while maintaining business continuity.

Having a single platform for both public and private provides organisations with the possibility to choose the right destination for workloads or launch, develop or sandbox new workloads in the cloud and then move them on-prem.

In many cases, data needs to be used by both analysts and less technical users, as well as machine learning or automated systems. Does data need to be ‘re-engineered to cater for all these different users and applications?

 A decentralised structure allows for greater productivity benefits from data and analytics efforts. Customers use data to accelerate innovation efforts by combining existing technologies in new ways, or “upcycling” existing technologies by applying them to a different domain to solve a different problem.

Data democratisation is the idea of empowering every employee (not just the data scientists) with the training and tools to be data-savvy. Compared to the IT team, business user groups like HR, marketing or product development have a more in-depth understanding of the data collection context and will be able to come up with domain-specific actionable insights. 

For example, the marketing team in a financial services organisation may need data modelling to develop a recommendation engine for lead capture. Within the same company, the customer support team might be on using data modelling to understand and reduce customer churn. 

The roadblock, however, is often legacy data systems that don’t have the high-speed processing power to support the instantaneous delivery of ensuring the right data reaches the right people at the right time. 

Applying Machine learning (ML) is another area of significant untapped potential – businesses that have adopted a successful ML strategy are outperforming their competitors by over 9%.

To help make it faster and simpler for businesses to unlock greater ML capabilities, we recently released Applied ML Prototypes (AMPs). Everyone is trying to do more with their data and AMPs enable data teams to go from idea to a fully working ML use case in a fraction of the time.

Enterprise data clouds help organisations use analytics at every stage of the data lifecycle, working at speed to extract the true value from their data. They also enable users to control who has access to data in a secure manner that maintains data integrity. 

Speaking broadly about the democratisation of data, what does this mean in a general enterprise/business context? 

With data playing an increasingly strategic role across all industries, data democratisation is something that should no longer be considered optional. Providing employees with access to data ensures they can gain better insights to make informed decisions quickly. Benefits can come in the form of financial or operational improvements such as increased operational efficiencies, new revenue streams, increasing average spend per customer, resilience and cost savings. To get started on data democracy, businesses need to look at structure, tools and training. 

One of the barriers to data democracy is, ironically, organisational structure. It is common for businesses to have centralised data analyst teams. It is also common for data to be kept in disparate silos, and data teams need time to extract and consolidate the relevant data. Such structures can often delay the decision-making process. Removing these data silos ensure more people can access the data, empowering them to make smarter, data-driven decisions.

As businesses push data and analytics out to more people within their organisation, it’s important they manage risk around sensitive information and also consider how to remain compliant with governmental regulations related to data privacy and locality. There are various strategies to achieve these goals, but the key is simplifying the security and governance end to end.

Being able to deliver consistent security and governance across all clouds - public or private rather than multiple security solutions stitched together reduces operational complexity, risk, and ongoing costs. 

How can data democratisation be applied to different industry sectors?

Locally, within A/NZ, telco providers are leveraging customer experience analytics to build a 360-degree view of their customer journey across all channels and lines of business. From this, they’re building prediction models to identify a customer base for personalised offers, even during this uncertain time.

When it comes to the in-person retail experience, technology and widespread access to data is proving a game-changer. Businesses can now combine the power of connected technology with a physical presence, placing beacons and sensors in stores to track purchase paths of customers by identifying ‘hot’ and ‘cold’ spots throughout the store. In addition, they can utilise facial recognition technologies to measure customer moods and sentiments.

On the supply chain front, retailers can use IoT devices to track goods, for instance, embedding devices within refrigerated trucks carrying perishable foods to measure temperature and humidity.

There are times when organisations need timely and accurate data to make quick decisions, such as in emergency response situations. How does data democratisation fit into these situations and these decision-making processes?

In an emergency response situation, the need for speed is mission-critical, to make sense of data accurately and quickly. Organisations need to be able to ingest and process data in real-time. Decisions can then be made faster and with confidence, often as a situation is unfolding.

For example, to mitigate the impacts of power outages, utility companies are now using advanced data analytics and machine learning to monitor and manage capacity, build predictive capacity models, identify bottlenecks, and prioritise and plan network expansion decisions. They’re also using advanced analytics to predict network capacity, mitigate business disruption and effectively ‘keep the lights on’. 

Another of our Australian based customers is in the financial services industry. For them, having a single pane of glass view across all their data is critical to support their fraud detection programme, mitigating risk for themselves and their customers. Existing in-house tools were inadequate in managing their data workloads considering the increasing scale of data clusters and demand. The company harnessed our platform to process large volumes of data in real-time and deliver analytical insights on transactions for reporting. This in turn enabled the customer to detect issues and irregularities, proactively avoiding fraudulent transactions.

Is there anything else you would like to add?

Data is a strategic asset that warrants its own strategy in the form of an enterprise data strategy. It’s very easy for organisations to lose sight of their initial motivation of migrating to the cloud. What I hear all too often is, “we’re going cloud – why are we doing this again?”. Sticking to strategy, or being able to steer the ship as needed, is critical especially when it comes to scaling data and ensuring strong governance and security. 

Equally important is separating data strategy from cloud and infrastructure strategy to give customers choice and flexibility. Delivering all these elements relies on the right set of tools to start with, consideration of end-to-end data lifecycle and building complex data pipelines with a single platform.

Related stories
Top stories
Story image
Amazon Web Services / AWS
Zscaler, AWS accelerate onramp to the cloud with zero trust
Zscaler has announced an extension to its relationship with Amazon Web Services, as well as innovations built on Zscaler's Zero Trust architecture.
Story image
Aqua Security, CIS create software supply chain security guide
Aqua Securityand the Center for Internet Security have together released the industry’s first formal guidelines for software supply chain security.
Story image
Threat actors ramp up their social engineering attacks
As people get better at identifying potential threats in their inbox, threat actors must evolve their methods. Their new M.O? Social engineering.
Story image
Commerce Commission
ComCom puts electronics sector on notice over resale price maintenance
The Commerce Commission has concluded an investigation into allegations that television manufacturers were engaging in illegal resale price maintenance.
Story image
Microsoft expands APAC Enabler Mentorship Program
"Mentors are the key to success for every professional. A good mentor is a coach, a guide, as well as a vocal advocate."
Story image
10 Minute IT Jams
Video: 10 Minute IT Jams - An update from Rimini Street
Today we welcome back Daniel Benad, who is the GVP & regional GM for Oceania at Rimini Street.
Story image
N4L, Spark, Chorus partner for Hyperfibre school upgrade
Networks for Learning (N4L) has partnered with Spark and Chorus to upgrade Wellington College to Hyperfibre, fostering stronger outcomes for students and teachers.
Story image
Digital Transformation
Apptio adds portfolio enhancements to promote digital strategy
"While digitalisation creates opportunities, it also makes budgeting far more complex, leading many companies to waste substantial funds."
Story image
Network Security
Netskope announces zero trust network access updates
Customers can now apply zero trust principles across a range of hybrid work security needs, including SaaS, IaaS, private applications, and endpoint devices.
Story image
The best ways to attract young talent during labour shortages
New research from Citrix reveals hybrid working and ventures into the metaverse are top of mind for Gen Z workers.
Story image
Rapid7 report examines use of double extortion ransomware attacks
New insight into how attackers think when carrying out cyber attacks, along with further analysis of the disclosure layer of double extortion ransomware attacks, has come to light.
Story image
Disaster Recovery
Eaton provides business continuity with disaster avoidance application
Many SMBs often struggle with problems relating to UPS and IT assets due to the sheer amount of time and resources required to run effectively. 
Story image
Dark web
Cybercrime in Aotearoa: How does New Zealand law define it?
‘Cybercrime’ is a term we hear all the time, but what exactly is it, and how does New Zealand define it in legal terms?
Story image
Jamf updates healthcare IT to protect data on Apple devices
Jamf has rolled out new functionality to help healthcare and IT teams protect patient data and streamline clinical access for their Apple fleet.
Story image
Workday winning on culture and family focus
This family-first approach sees all employees receive access to family-wide private healthcare cover, as well as income protection and life insurance policies.
Story image
VMware wins Google Cloud partner award for infrastructure modernisation
The cloud computing and virtualisation company was recognised for its achievements as part of the Google Cloud ecosystem.
Story image
Contact Centre
Customer service agents don't want to return to contact centres
A new report has revealed that 85% of customer service agents want to work full-time at home and not return to contact centre offices.
Story image
F5 Networks
Telstra, F5 team up to bolster services and solutions
“This partnership demonstrates our ongoing investment into APAC as we continue delivering high value services and solutions to our partners and customers."
Story image
Industry-first comprehensive risk-based API security enhances protection
Application Programming Interfaces (APIs) have become a crucial part of operating web and mobile application businesses and are causing significant economic growth in the digital sector.
Story image
Artificial Intelligence
Salesforce harnesses automated solutions with new developments
Salesforce has launched Sales Cloud Unlimited, a new feature to help accelerate productivity with AI and automation.
Story image
Significant security concerns resulting from open source software ubiquity
"The risk is real, and the industry must work closely together in order to move away from poor open source or software supply chain security practices."
Story image
Orbital Insight
Orbital Insight solution set to drive better data-driven decisions
The company says the new Site Intelligence solution will provide granular visibility, behaviour analytics and deep insights about customers and competitors at any location. 
Story image
Airwallex, Xero extend partnership with easier invoice payments
Airwallex has extended its long-term partnership with Xero by releasing a new payment link integration for Xero invoices that will make receiving them easier and faster for Australian businesses.
Story image
Robotic Process Automation / RPA
rapidMATION helps Coates achieve success with landmark RPA solution
A strong Robotic Process Automation solution (RPA) can help solve many complex issues that businesses face daily. 
Story image
The rise of digital gifting in the workplace
The name itself does most of the explaining; it’s a gift you receive virtually. But a misconception about digital gifts is that they need to be redeemed virtually as well. 
Story image
How TruSens air purifiers can create healthier workspaces
The pandemic has heightened our awareness of our own and others’ health, and made us all much more conscious of the environments we work in.
Story image
Employers look to hire inexperienced coders due to skills shortage
"Even inexperienced workers without prior qualifications or experience had managed to pivot to new roles in coding as long as they are willing to upskill."
Story image
Greater API usage raises concerns for protection - report
Radware has released its 2022 State of API Security report, which shows a rise in APIs, with 92% of the organisations surveyed significantly or somewhat increasing their usage.
Story image
Why is NZ lagging behind the world in cybersecurity?
A recent report by TUANZ has revealed that we are ranked 56th in the world when it comes to cybersecurity - a look into why we're so behind and what needs to be done.
Story image
Cloudflare outage in 19 data centers worldwide due to own error
Cloudflare says its outage for 19 of its data centers yesterday was because of a change in a long-running project to increase resilience in its busiest locations.
Story image
Ready for anything with the PagerDuty Operations Cloud
In a world of digital everything, teams face increasing complexity. Ever-growing dependencies across systems and processes put customer and employee experience, not to mention revenue, at risk.
Story image
Tech job moves
Tech job moves - Boomi, Limepay, Thales, VMware & Zoom
We round up all job appointments from June 6-16, 2022, in one place to keep you updated with the latest from across the tech industries.
Story image
Consumers want personalisation, but don't trust brands with their data
Customers expect personalisation during every brand interaction but they don't trust brands to keep their personal data secure and to use it responsibly. 
Story image
Ingram Micro launches vendor-backed security program
Ingram Micro has unveiled a new program intended to give resellers the effective offerings their customers need to stay safe in the evolving threat landscape.
Story image
Digital Transformation
Cybersecurity priorities for digital leaders navigating digital transformation
In recent years, Asia-Pacific has especially been a hotspot for cyberattacks, and as we continue into 2022, it’s evident that the problem is becoming more significant.
Story image
Video: 10 Minute IT Jams - An update from Tricentis
Tricentis provides software testing automation, and software quality assurance products for enterprise software.
Story image
Hybrid workforce
Why hybrid working is here to stay and how to ace it
Citrix's new report reveals hybrid workers are more productive and engaged at work than their office and completely remote counterparts.
Story image
Sternum joins NXP, collaborates on IoT security and observability
Sternum has announced it has joined the software partner community of NXP Semiconductors, a manufacturer of and large marketplace for embedded controllers.
Story image
IT and security team collaboration crucial to data security
Many IT and security decision makers are not collaborating as effectively as possible to address growing cyber threats.
Story image
Secure access service edge / SASE
Cloudflare adds new capabilities to zero trust SASE platform
New features for Cloudflare One include email security protection, data loss prevention tools, cloud access security broker, and private network discovery.
Story image
Volpara, Microsoft project to detect cardiovascular issues
Volpara Health Technologies is working with Microsoft on a research and development project to speed up creating a product that detects and quantifies breast arterial calcifications (BACs).
Story image
Sealord partners with Infor to improve sustainability
Sealord has chosen Infor as a strategic partner to implement an operational cloud-based platform that provides day-one functionality and sustainability gains.
Story image
Canstar finds Flick Electric NZ’s favourite provider
Canstar’s annual research to find New Zealand’s favourite electricity provider reveals Flick Electric has come out on top.
Story image
Open source
DataStax secures US$115 million to fund database expansion
DataStax has secured US$115 million in funding, which it will use to develop and expand its Astra DB multi-cloud database and Astra Streaming service globally.