IT Brief New Zealand logo
Technology news for New Zealand's largest enterprises
Partner content
Story image

Interview: SolarWinds explores databases & DataOps in the cloud-first era

By Sara Barker
Fri 5 Nov 2021

Database management has progressed in leaps and bounds as organisations look to cloud migration, fuelling the birth of DataOps and automation that have drastically changed the way organisations collect, use, and manage seemingly endless tides of data every day. 

We spoke to SolarWinds head geek Thomas LaRock to discuss the world of database administrators (DBAs), the possibilities within DataOps, and how SolarWinds is helping organisations navigate challenges and opportunities in their journeys.

While the definition of DBA can mean many things to many organisations, the general overview is the DBA helps organisations create or maintain databases according to their data collection and management requirements. 

But as LaRock points out, those who work across DBA roles can have vastly different responsibilities depending on the nature of their organisation.

"Some may focus on maintenance tasks, others do more development work, and there are also site engineers who used to fall under old preconceptions of a DBA. These different roles can create challenges."

The challenges lie in two areas: the level of expectations from end users, and what role a DBA has in their organisation. DBAs are also using technologies that change rapidly from day to day. On top of that, DBAs and database developers are also navigating a global pandemic where priorities are constantly changing.

LaRock likens the concept of a database to a black box—often, organisations don't know what's going on inside it. 

"You know when something's gone wrong because there's a feeling that something should be faster than it is. The feeling is usually vague and often undefined, so a DBA needs to figure out why something doesn't work to a user's expectations. Somebody will just call you and say, this needs to be faster, and you have to start extracting information to find out exactly what the problem is."

Issues within databases are inevitable, particularly in today's world where organisations are creating, collecting, and processing millions of data pieces every day. From this inevitable rise of data came DataOps, which provides a framework for data management in databases and other places.

"DataOps is not just automation, but there is also a feedback loop. Automation, in this sense, is more of a data engineering function and a pipeline. Data needs to move in and out of systems, but it also needs to be automated."

Imagine a system in which a database fails on one particular row or piece of data. That failure stops the entire system, and it requires a manual process to sort out what went wrong.

"For example, a DBA in a financial services organisation might receive files from banks, overnight. These files will then get input into a system, but if there is one little garbage character in there, the whole thing might be abandoned. Then it’s back to square one as the DBA would show up in the morning, rerun the log, and figure out where the data problem is." 

The goal of automation is to minimise human involvement. Instead, human interaction should focus on observability—how the system is doing, what kind of error rates there are, if the system is flowing, and anything that needs manual reprocessing. That's essentially the automation part of DataOps.

"DataOps is a way of automating all of that because there's no real reason the human has to get that involved, especially for one record—toss it to the side, for now, let everything else process. Let's come back and figure out why that one record is wrong. Everything else continues in the background. DataOps provides a constant feedback loop about things that go wrong, if users have problems, or if a pipeline has a higher error rate than normal. So DataOps must have a feedback loop built-in."

While DataOps and DBA vary depending on individual business needs, there is one thing every organisation requires: a data catalogue. This is essentially an inventory of all datasets that enables organisations to find out, describe, and organise their data. There are different data catalogues for various business purposes.

"I think companies would be surprised to learn they’re collecting and likely paying for data feeds that contain a lot of the same information," says LaRock.

Think of financial services, for example. The word 'cash' can mean different things to different parts of the business. If there are 10 different definitions, creating 10 columns in a table based on each definition would be wasteful.

"You have to build this proper data catalogue to get started because before you start building data pipelines, you want to know more about the data you're even working with, what's required, and what you want compared to what you need." 

"The first step to DataOps is building out that data catalogue and understanding that you might have a massive amount of data to deal with. If there are thousands of systems in a business, pick one that’s mission-critical or perhaps a couple that are linked. Once you're done, figure out where the next system is and if there are any overlap points. Is there something you can streamline between those two feeds?"

The SolarWinds portfolio of database solutions includes Database Performance Analyzer, SQL Sentry, Database Performance Monitor, and many other solutions.

"Essentially, these solutions cater for everything from high-level to deep-dive, either as-a-service or self-hosted by our customers."

Database Performance Analyzer is designed for database query performance monitoring, analysis, and tuning across platforms like Oracle, Azure SQL Database, AWS RDS, Microsoft SQL Server, PostgreSQL, and MySQL.

DBA xPress is a free tool that can help organisations compare and synchronise data across databases, analyse dependencies and connections, browse SQL Server database content, and much more.

"DBA xPress enables users to find out where a piece of data is across their systems. It can also show different systems related to each other, which is essential when building a data catalogue and mapping data around their enterprise database," says LaRock.

"Of course, when you start building that catalogue out, you start with understanding where your data is and where the overlaps are with, that's going to help you when you start thinking about how you need to migrate your systems to the cloud. 

"Migration could be as simple as a shift to cloud, or you could talk about a re-engineering of your system as you migrate. Both of these require an understanding of the data being shifted, and that's where a tool like DBA xPress comes in.

"On top of that, you could add performance tools that compare performance before and after migrations. Many companies stumble on this point when they're migrating to the cloud or starting DataOps, so it's important to have the right products and solutions to help data professionals get the job done."

Learn more about SolarWinds here.

Related stories
Top stories
Story image
A third of companies paying ransom don’t recover data - report
Veeam's report finds 76% of businesses who are victims of cyberattacks paid the ransom to recover data, but a third were still unable to get their information back.
Story image
Apple previews new features for users with disabilities
Apple says new software features that offer users with disabilities new tools for navigation, health and communication, are set to come out later this year.
Story image
Meadows exits Ingram Micro New Zealand on a good note
We look at the resignation of Danny Meadows from Ingram Micro's New Zealand Unified Communications & Collaboration business unit.
Story image
Fonterra to use automated guided vehicles by Dematic
New Zealand's dairy giant Fonterra is upgrading its manufacturing facility in Edendale with new automated guided vehicles (AGVs) from Dematic.
Story image
Zendesk announces new conversational CRM solutions
“The last few years have made it obvious that digital is the front door, convenience is paramount and relationships are anchored in conversations."
Story image
Managed service providers: effective scoping to avoid costly vendor pitfalls
Managed security services are outsourced services focusing on the security and resilience of business networks.
Story image
Veryfi announces Mobile Receipt Capture for D2C marketing apps
Veryfi has announced a new enhancement to its portfolio, with Mobile Receipt Capture for direct-to-consumer marketing apps.
Story image
Video: 10 Minute IT Jams - An update from IronNet
Michael Ehrlich joins us today to discuss the history of IronNet and the crucial role the company plays in the cyber defence space.
Story image
Artificial Intelligence
Clear Dynamics closes $35M funding round, invests in global growth
The funding is a major milestone and speaks to Clear Dynamics’ vision for AI-enabled ‘composable’ enterprise software, the company states.
Story image
Power at the edge: the role of data centers in sustainability
The Singaporean moratorium on new data center projects was recently lifted, with one of the conditions being an increased focus on power efficiency and sustainability.
Story image
Absolute Software expands Secure Access product offering
Absolute Software is enhancing its Secure Access product portfolio, enabling minimised risk exposure and optimised user experiences in the hybrid working environment.
Story image
Cloudflare launches instant serverless database for dev teams
"Today we’re announcing our first serverless database which we expect will quickly become one of the largest databases in the world."
Story image
Artificial Intelligence
AI-based email security platform Abnormal Security valued at $4B
"A new breed of cybersecurity solutions that leverage AI is required to change the game and stop the rising threat of sophisticated and targeted email attacks."
Story image
Application Security
What are the DDoS attack trend predictions for 2022?
Mitigation and recovery are vital to ensuring brand reputation remains solid in the face of a Distributed Denial of Service (DDoS) attack and that business growth and innovation can continue.
Story image
The 'A-B-C' of effective application security
Software applications have been a key tool for businesses for decades, but the way they are designed and operated has changed during the past few years.
Story image
Power / Energy
Keysight Technologies introduces new next-gen DPT solution
Keysight Technologies has announced its new next-generation Double-Pulse Tester (DPT) with the PD1550A Advanced Dynamic Power Device Analyser.
Story image
Fortinet's Security Fabric hits new record for integrations
The Fortinet Security Fabric has surpassed 500 technology integrations with more than 300 Fabric-Ready Technology Alliance Partners.
Story image
Adyen expands partnership with Afterpay as BNPL payments increase
Adyen has expanded its partnership with AfterPay allowing more of Adyen’s merchants in more countries worldwide to use the BNPL provider.
Story image
Artificial Intelligence
SAS launches human-focused responsible innovation initiative
SAS has launched a responsible innovation initiative, furthering its commitment to equity and putting people first.
Story image
Commerce Commission
ComCom welcomes new marketing codes for the telecom industry
The Commerce Commission is welcoming the creation of new marketing codes for the telecommunications industry.
Story image
Sift shares crucial advice for preventing serious ATO breaches
Are you or your business struggling with Account Takeover Fraud (ATO)? One of the latest ebooks from Sift can provide readers with the tools and expertise to help launch them into the new era of account security.
Find out how a behavioural analytics-driven approach can transform security operations with the new Exabeam commissioned Forrester study.
Link image
Story image
Ivanti and Lookout bring zero trust security to hybrid work
Ivanti and Lookout have joined forces to help organisations accelerate cloud adoption and mature their zero trust security posture in the everywhere workplace.
Story image
Cybersecurity starts with education
In 2021, 80% of Australian organisations responding to the Sophos State of Ransomware study reported being hit by ransomware. 
Story image
Could your Excel practices be harming your business?
While Excel has been the de-facto standard for budgeting, planning, and forecasting, is it alone, enough to support organisations in the global marketplace that’s facing rapid changes due to digital transformation?
Story image
SAS Viya on Microsoft Azure to deliver 204% return - study
The Forrester Total Economic Impact study finds SAS Viya on Microsoft Azure brings a 204% return on investment over three years.
Story image
IT budget
$20m boost for digital technologies announced
The government is spending an extra $20m over four years on its plan to transform the digital technologies industry.
Story image
Public Cloud
Cloud adoption still a work in progress, NetApp finds
NetApp has announced the results of the annual Cloud Infrastructure Report based on a survey of public cloud business and IT decision makers.
Story image
A10 Networks finds over 15 million DDoS weapons in 2021
A10 Networks notes that in the 2H 2021 reporting period, its security research team tracked more than 15.4 million Distributed Denial-of-Service (DDoS) weapons.
Story image
Hands-on review: STM laptop bags
The advent of hybrid working has meant we need laptop bags. We got our hands on two of the most popular laptop bags from STM.
Story image
BeyondTrust integrates Password Safe solution with SailPoint
BeyondTrust has announced the integration of BeyondTrust Password Safe with SailPoint identity security offerings.
Story image
Prophecy International migrates VMware environment with Oracle Cloud Solution
The Adelaide-based global provider is using the solution to eliminate the need to re-write applications, therefore allowing the company to enhance its business operations.
Story image
Hawaiki Cable
BW Digital completes acquisition of Hawaiki Submarine Cable
BW Digital has completed its full acquisition of Hawaiki Submarine Cable, with all applicable regulatory filings and approvals now received.
Story image
Data Center
Preventing downtime costs and damage with Distributed Infrastructure Management
Distributed Infrastructure Management (DIM) can often be a lifeline for many enterprises that work with highly critical ICT infrastructure and power sources.
Story image
Grasping the opportunity to rethink the metrics of a sustainable data centre
A data centre traditionally has two distinct operations teams: the Facility Operations team, and the IT Operations team. Collaboration between them is the key to defining, measuring, and delivering long-term efficiency and sustainability improvements.
Story image
Talend introduces new data health solutions for businesses
Talend has announced its latest version of Talend Data Fabric, with the release of Talend Trust Score enabling data teams to establish a foundation for data health.
Story image
Artificial Intelligence
ANU and Seeing Machines to use AI to improve driver safety
The Australian National University and Seeing Machines have won a grant to develop AI systems monitor human behaviour while driving.
Story image
Commerce Commission
ComCom appeals $2.25 million fine in Vodafone FibreX case
The Commerce Commission has filed an appeal in the High Court against a $2.25 million fine imposed on Vodafone NZ for its offending under the Fair Trading Act.
Threat actors are exploiting weaknesses in interconnected IT/OT ecosystems. Darktrace illuminates your entire business and takes targeted action to stop emerging attacks.
Link image
For every 10PB of storage run on HyperDrive vs. comparable alternatives, an estimated 6,656 tonnes of CO₂ are saved by reduced energy consumption alone over its lifespan. That’s the equivalent of taking nearly 1,500 cars off the road for a year.
Link image
Story image
Workato unveils enhancements to enterprise automation platform
"The extra layer of protection with EKM, zero-logging, and hourly key rotation gives customers a lot more visibility and control over more sensitive data."
Story image
Digital Transformation
Unlocking the next digital frontier for educational institutions
Understanding where to invest in technology can be challenging for education institutions, especially after the COVID-19 disruptions.
Story image
Google reveals new safety and security measures for users
Google's new measures include automatic two step verification, virtual cards and making it easier to remove contact information on Google Search results.
Booster Innovation Fund. A fund of Kiwi ingenuity – for Kiwi investors.
Link image