IT Brief New Zealand logo
Technology news for New Zealand's largest enterprises
Partner content
Story image

Modern data integration: A must-have for the data-driven enterprise

By Contributor
Thu 21 May 2020
FYI, this story is more than a year old

Article by Qlik Vice president APAC data integration division, Byron Yeung.

Agility and real-time insights have quickly become the keystone of the data-driven enterprise. And for good reason, according to Gartner, by 2022 companies will be valued on their information portfolios.

It goes without saying that being able to move data at the speed of change makes the difference between a successful business and one that falls short.

As organisations continue their digital transformation journeys, adopting modern technologies such as the cloud and data lakes, it’s important they also embrace a modern approach to data integration.

In 2019, IDC predicted the sum of the world’s data will grow to 175 zettabytes in 2025—a 61% compounded annual growth rate. Considering this exponential growth of data, for many businesses modern data integration may seem too steep a mountain to climb—but in reality, the climb isn’t as tough as it may appear.

By collecting and interpreting multiple datasets, modern data integration eliminates information silos, democratising data access and providing a consistent view to users. This in turn helps create agile, integrated data environments that enable companies to respond faster to change, better leverage new technologies and develop innovative products and services. 

Enter DataOps

Modern data integration has become a catalyst for a new breed of data operations (DataOps). DataOps is an emerging set of practices and technologies for building data and analytics pipelines to meet business needs quickly. As these pipelines become more complex and development teams grow in size, organisations need better collaboration and development processes to govern the flow of data from one step of the data lifecycle to the next—for example, from data ingestion and transformation to analysis and reporting.

DataOps was born from the need for a new methodology that would encompass the adoption of modern technologies and the teams using the data. It leverages real-time integration technologies, such as change data capture (CDC) and streaming data pipelines, to ensure data is readily available for use, typically in a self-service model or integrated with current systems. 

As a result, DataOps streamlines how data owners, database administrators, data engineers and data consumers interact, as they all use data to improve decision-making and achieve business goals. 

Bringing IT and the business together

DataOps allows organisations to gain insights—and ultimately, take action—quicker than before by continuously processing new data, monitoring performance and producing real-time insights. However, users should have full visibility of the data they’re using, including when, where and by whom it has been modified, making data catalogues the backbone of DataOps.

After data has been created, extracted, transformed and integrated, a data catalogue informs users of available datasets and metadata on a specific topic, providing assistance in locating data required to build analytics. Essentially, a data catalogue is an inventory of data that enables users to glean accurate and trustworthy critical business insights. It holds information on the datasets, offering quality assessment scores for critical factors relevant to users—for example, whether the data is clean, if it’s being used by other teams, and so on. 

A leading global pharmaceutical company, Astellas Pharma US, realised its data warehouse simply couldn’t keep up with the business’ data needs. In order to stay on the leading edge of an industry built on innovation, it invested in Qlik Data Catalyst to accelerate and simplify data functions. In less than 12 weeks, they deployed a governed data platform, moving data closer to users through a secure, managed data lake. This gave teams immediate self-service, on-demand access. The platform automatically loads, encrypts, organises and validates prepared datasets, enabling the analysis of 100 billion records from 50+ data sources. The results speak for themselves; Astellas:

•    Achieved 30-fold increase in total projects the marketing analytics team could deliver
•    Reduced time required for marketing analysts to get new datasets for analytic research projects from six months to one hour
•    Eliminated 50% of the IT system costs associated with the data management platform
•    Gave 60+ analysts self-service access to data, expanding their ability to find the data they need and collaborate, interpret, enhance and apply data to high-value business challenges

More than ever, organisations need to build on modern data infrastructure and leverage DataOps to achieve a more holistic and agile approach to data. It is now easier than ever to transform raw data into a governed, analytics-aware information resource with both speed and scale. Qlik strives to do just that with its Data Integration Platform

A modern data management solution should drive more insights and value from data, enabling better decision-making and transformation across the entire organisation. By leveraging modern data integration, businesses can gain a competitive advantage to build an agile and future-ready enterprise.

Related stories
Top stories
Story image
Employment
Tech job moves - Forcepoint, Malwarebytes, SolarWinds & VMware
We round up all job appointments from May 13-20, 2022, in one place to keep you updated with the latest from across the tech industries.
Story image
Migration
Let’s clear the cloud visibility haze with app awareness
Increasingly, organisations are heading for the cloud, initiating new born-in-the-cloud architectures and migrating existing applications via ‘lift and shift’ or refactoring.
Story image
Remote Working
Successful digital transformation in the hybrid work era is about embracing shifting goalposts
As organisations embraced remote working, many discovered they lacked the infrastructure needed to support history’s first global load test of remote work capabilities.
Story image
Vectra AI
Understanding the weight on security leader’s shoulders, and how to shift it
Millions of dollars of government funding and internal budgets are being funnelled into cybersecurity to build resilience against sophisticated threats, indicating how serious this issue has become.
Story image
Cybersecurity
More than 40% of banks worried about cloud security - report
Publicis Sapient's new report finds security and the lack of cloud skills and internal understanding of business benefits are big obstacles for banks moving to the cloud.
Story image
Artificial Intelligence
SAS announces new products amid cloud portfolio success
Analytics and AI company SAS is deepening its broad industry portfolio with offerings that support life sciences, energy, and martech.
Story image
Collaboration
Meadows exits Ingram Micro New Zealand on a good note
We look at the resignation of Danny Meadows from Ingram Micro's New Zealand Unified Communications & Collaboration business unit.
Story image
Apricorn
Data backup plans inadequate, data still at risk - study
The Apricorn 2022 Global IT Security Survey revealed that while the majority organisations have data backup plans in place, data for many are at risk.
Story image
SaaS
Maintaining secure systems with expectations of flexible work
Most office workers feel they've proved they can work successfully from home, and as much as employers try, things aren't going back to the way they were anytime soon.
Story image
Digital Transformation
The impact of COVID-19 on healthcare environments and care delivery
The COVID-19 pandemic has revolutionised the healthcare industry while overcoming staff shortages, social distancing requirements, and lockdowns.
Story image
Microsoft
New Relic enters multi-year partnership with Microsoft Azure
New Relic has announced a strategic partnership with Microsoft to help enterprises accelerate cloud migration and multi-cloud initiatives. 
Story image
Sustainability
Legrand unveils Nexpand, a data center cabinet platform
Legrand has unveiled a new data center cabinet platform, Nexpand, to offer the necessary scalability and future-proof architecture for digital transformation.
Story image
Apple
Apple previews new features for users with disabilities
Apple says new software features that offer users with disabilities new tools for navigation, health and communication, are set to come out later this year.
Story image
Lightspeed
Lightspeed launches all-in-one marketing platform in A/NZ
ECommerce provider, Lightspeed has launched a new all-in-one marketing solution, Lightspeed Marketing & Loyalty in Australia and New Zealand.
Story image
Cybersecurity
A10 Networks finds over 15 million DDoS weapons in 2021
A10 Networks notes that in the 2H 2021 reporting period, its security research team tracked more than 15.4 million Distributed Denial-of-Service (DDoS) weapons.
Story image
Fonterra
Fonterra to use automated guided vehicles by Dematic
New Zealand's dairy giant Fonterra is upgrading its manufacturing facility in Edendale with new automated guided vehicles (AGVs) from Dematic.
Story image
Data Center
Preventing downtime costs and damage with Distributed Infrastructure Management
Distributed Infrastructure Management (DIM) can often be a lifeline for many enterprises that work with highly critical ICT infrastructure and power sources.
Story image
Digital Transformation
Trading up: It's time to swap core systems for flexible digital applications
This year will see more oranisations planning and commencing high tech renovations that will shake up the way they operate.
Story image
Malware
New vulnerabilities found in Nuspire’s Q1 2022 Threat Report
“Threat actors are quickly adjusting their tactics and these exploits tend to get industry attention, but the threat posed by older and attacks still persists."
Story image
Digital Transformation
The Huawei APAC conference kicks off with digital transformation
More than 1500 people from across APAC have gathered for the Huawei APAC Digital Innovation Congress to explore the future of digital innovation.
Softiron
For every 10PB of storage run on HyperDrive vs. comparable alternatives, an estimated 6,656 tonnes of CO₂ are saved by reduced energy consumption alone over its lifespan. That’s the equivalent of taking nearly 1,500 cars off the road for a year.
Link image
Story image
Sift
Sift shares crucial advice for preventing serious ATO breaches
Are you or your business struggling with Account Takeover Fraud (ATO)? One of the latest ebooks from Sift can provide readers with the tools and expertise to help launch them into the new era of account security.
Story image
Application Security
What are the DDoS attack trend predictions for 2022?
Mitigation and recovery are vital to ensuring brand reputation remains solid in the face of a Distributed Denial of Service (DDoS) attack and that business growth and innovation can continue.
Story image
Sisense
Data and analytics could be key to higher selling prices in APAC
Sisense's latest report has found that almost half of data professionals in APAC think customised data and analytics can create better selling prices for their products.
Story image
Ransomware
A third of companies paying ransom don’t recover data - report
Veeam's report finds 76% of businesses who are victims of cyberattacks paid the ransom to recover data, but a third were still unable to get their information back.
Story image
Excel
Could your Excel practices be harming your business?
While Excel has been the de-facto standard for budgeting, planning, and forecasting, is it alone, enough to support organisations in the global marketplace that’s facing rapid changes due to digital transformation?
Story image
ChildFund
ChildFund launches new campaign to protect children online
ChildFund says WEB Safe & Wise aims to protect children from sexual exploitation and abuse online while also empowering them to become digitally savvy. 
Exabeam
Find out how a behavioural analytics-driven approach can transform security operations with the new Exabeam commissioned Forrester study.
Link image
Story image
Commerce Commission
ComCom appeals $2.25 million fine in Vodafone FibreX case
The Commerce Commission has filed an appeal in the High Court against a $2.25 million fine imposed on Vodafone NZ for its offending under the Fair Trading Act.
Booster
Booster Innovation Fund. A fund of Kiwi ingenuity – for Kiwi investors.
Link image
Story image
Amazon Web Services / AWS
RedShield leverages AWS to scale cybersecurity services
"Working with AWS gives RedShield the ability to mitigate significant application layer DDoS attacks, helping leaders adopt best practices and security architectures."
Story image
Attain
Revenue operations is taking centre stage
As the business world continues to evolve, new demands need to be met to keep up with the ever-changing landscape. 
Story image
SaaS
Rubrik Security Cloud marks 'next frontier' in cybersecurity
"The next frontier in cybersecurity pairs the investments in infrastructure security with data security giving companies security from the point of data."
Darktrace
Threat actors are exploiting weaknesses in interconnected IT/OT ecosystems. Darktrace illuminates your entire business and takes targeted action to stop emerging attacks.
Link image
Story image
Infrastructure
Nutanix study reveals financial services sector lagging with multicloud adoption
Nutanix has released new research that reveals the financial services sector is lagging behind when it comes to multicloud adoption.
Story image
Microsoft
PwC NZ unveils new Cloud Security Operations Center
PwC New Zealand has unveiled its new Cloud Security Operations Center for the entire Microsoft technology stack.
Story image
Digital Transformation
Pluralsight and Ingram Micro Cloud team up on cloud initiative
Pluralsight has teamed with Ingram Micro Cloud to build upon cloud competence and maturity internally, and externally support partners’ capabilities.
Story image
Cybersecurity
NCSC advisory highlights poor security configurations
The GCSB's National Cyber Security Centre (NCSC) has released a cyber security advisory identifying commonly exploited controls and practices.
Story image
Ransomware
Cybersecurity starts with education
In 2021, 80% of Australian organisations responding to the Sophos State of Ransomware study reported being hit by ransomware. 
Story image
Sustainability
Power at the edge: the role of data centers in sustainability
The Singaporean moratorium on new data center projects was recently lifted, with one of the conditions being an increased focus on power efficiency and sustainability.
Story image
Sustainability
Grasping the opportunity to rethink the metrics of a sustainable data centre
A data centre traditionally has two distinct operations teams: the Facility Operations team, and the IT Operations team. Collaboration between them is the key to defining, measuring, and delivering long-term efficiency and sustainability improvements.
Story image
Artificial Intelligence
AI-based email security platform Abnormal Security valued at $4B
"A new breed of cybersecurity solutions that leverage AI is required to change the game and stop the rising threat of sophisticated and targeted email attacks."
Story image
Artificial Intelligence
CFOs using digital workers and AI to prevent unnecessary loss
New technology is now allowing CFOs to use digital workers to automate their accounting processes, making it easier for them to avoid unnecessary losses.
Story image
Wireless Nation
Wireless Nation, N4L provide 4G network to remote NZ schools
Wireless Nation and Network for Learning (N4L) have rolled out the Rural Connectivity Group’s (RCG) new 4G network to better connect three Chatham Islands schools.