IT Brief New Zealand logo
Technology news for New Zealand's largest enterprises
Story image

Monitoring has a place in your test-and-learn culture

By Contributor
Tue 28 Jan 2020
FYI, this story is more than a year old

Article by ThousandEyes A/NZ regional sales manager Will Barrera

Experimentation in the enterprise is being embraced, but how does one work out what difference it makes?
Dynamism is an accepted - even desirable - part of workplace culture. And as more companies embrace agile ways of working, experimentation and ‘test-and-learn’ methodologies, it will become even more baked into what we consider to be business-as-usual.
Though test-and-learn and its derivatives are often treated as a new ways of thinking, they are far from it. A 1923 book, Scientific Advertising by Claude Hopkins, notes that “almost any question can be answered cheaply, quickly and finally, by a test campaign. And that’s the way to answer them – not by arguments around a table.”
Harvard Business Review notes these methodologies have taken root in part because they can be used by almost anyone to achieve results quickly.
“Take one action with one group of customers, take a different action (or often no action at all) with a control group, and then compare the results,” it advises.
“The outcomes are simple to analyse, the data are easily interpreted, and causality is usually clear.
“The test-and-learn approach is also remarkably powerful. Feedback from even a handful of experiments can yield immediate and dramatic improvements in profits.”
The UNHCR’s Innovation Service concurs: “Another way to think about the costs of experiments, is to consider how much does not experimenting cost,” it counsels.
“Too often projects are rolled out without much trialling. Without testing ideas, products or services, we might end up having large (untested) projects that fail to deliver. What are the costs of failed projects?”
This is all well and good. But products and services that are targeted for innovation and experimentation today are often complex, consisting of a large number of interdependent components.
Before users reach your site or app, for example, they have to traverse a web of complex dependencies. Starting with their own ISP connection, a user request is routed over the internet, and may pass through content delivery networks (CDN), DDoS mitigation and other security services on its way to the destination server. These are all components and pieces that end users need to interact with, unbeknownst to them, before they even get to where they are going.
Each hop in the journey can add milliseconds of delay and impact the performance the user sees, as well as the end-to-end experience. That inevitably means many organisations spend a lot of time trying to optimise each component involved in the hope that it will execute its part of the process in the most efficient amount of time.
In a complex environment such as this, it can be difficult to determine whether a test-and-learn or experiment in one part of the system is actually having an effect.
One way organisations are doing that measurement is through intelligent network monitoring.
While monitoring is sometimes thought of as an operations tool, in reality its usage today is much broader. When you're considering taking on a new deployment architecture, or a new platform or a new service provider, you can use monitoring to understand the benefits: whether the new technology and vendor is delivering what they're promising to. That could help to create a business case on the feasibility of using the technology on a broader scale.
For example, if you are evaluating a new content delivery network (CDN), you want to have active monitoring in place that can clearly identify to you that you're getting the benefits that you think you are from that new technology or architecture.
We recently worked with a large taxation services company that started to use a CDN for the first time. It was hypothesized that this would result in much faster webpage loading times, but after implementation there was no large or obvious improvement.
One of the reasons for this was the site was already fairly streamlined and loaded fairly quickly. But despite the fact that overall page load times didn't seem to change, the company saw clear benefits in “time to first byte” - another important responsiveness metric.
Before the cutover to the CDN, this was 281 milliseconds; after the cutover, it dropped to under 100 milliseconds.
Importantly, this was only apparent because the company used intelligent network monitoring to visualise how its experiments impacted all aspects of the customer experience.
It also means they now know that if they do some significant optimisation on the application side, then they're going to be able to leverage all the benefits of the CDN, and they can clearly see that and document it, because they already know that their time to first byte has been dramatically improved.
Test-and-learn will continue to be part of technology culture for the foreseeable future. Finding better ways to visualise and track effectiveness will be key for those that embrace it.

Related stories
Top stories
Story image
Manhattan Associates
Shortening the click-to-customer cycle through smart technologies
Speed of delivery without accuracy is a dealbreaker for consumers. How can retailers operating in an omnichannel environment overcome the challenge of click-to-customer cycle times.
Story image
Vectra AI
Vectra’s inaugural Partner of the Year Awards revealed
APAC companies Baidam, Firmus, ShellSoft and Macnica have been recognised in Vectra AI's inaugural Partner of the Year Awards.
Story image
Cybersecurity prompts upgrade for 1.3 billion electricity meters
ABI Research finds Advanced Metering Infrastructure (AMI) and cybersecurity concerns are prompting the upgrade of 1.3 billion electricity meters by 2027.
Story image
Women in Technology
Huawei webinar emphasises the importance of women in tech
Industry findings by Coursera discussed as part of a webinar jointly organised by Huawei and Reuters Events found 6% more women enrolled in tech courses this year than in 2021.
Story image
Tech job moves - Forcepoint, Malwarebytes, SolarWinds & VMware
We round up all job appointments from May 13-20, 2022, in one place to keep you updated with the latest from across the tech industries.
Story image
Wireless Nation
Wireless Nation, N4L provide 4G network to remote NZ schools
Wireless Nation and Network for Learning (N4L) have rolled out the Rural Connectivity Group’s (RCG) new 4G network to better connect three Chatham Islands schools.
Story image
Customer experience
Research unveils precarious customer loyalty for retailers
New research has found customers are reassessing established brand loyalties as their priorities and behaviours shift.
Story image
New Relic enters multi-year partnership with Microsoft Azure
New Relic has announced a strategic partnership with Microsoft to help enterprises accelerate cloud migration and multi-cloud initiatives. 
Story image
Digital Marketing
Similarweb acquires SEO and rank tracking company Rank Ranger
Digital intelligence company Similarweb, which specialises in analysing web traffic, has acquired Rank Ranger, a market leader in SEO and rank tracking.
Story image
Data Center
Preventing downtime costs and damage with Distributed Infrastructure Management
Distributed Infrastructure Management (DIM) can often be a lifeline for many enterprises that work with highly critical ICT infrastructure and power sources.
Story image
Alarming surge in Conti Ransomware Group activity - report
A new report has identified a 7.6 per cent increase in the number of vulnerabilities tied to ransomware in Q1 2022.
Story image
Cradlepoint expands its Cellular Intelligence capabilities
Cradlepoint has announced additional Cellular Intelligence capabilities with its NetCloud service.
Threat actors are exploiting weaknesses in interconnected IT/OT ecosystems. Darktrace illuminates your entire business and takes targeted action to stop emerging attacks.
Link image
Story image
New vulnerabilities found in Nuspire’s Q1 2022 Threat Report
“Threat actors are quickly adjusting their tactics and these exploits tend to get industry attention, but the threat posed by older and attacks still persists."
Story image
Nutanix study reveals financial services sector lagging with multicloud adoption
Nutanix has released new research that reveals the financial services sector is lagging behind when it comes to multicloud adoption.
Booster Innovation Fund. A fund of Kiwi ingenuity – for Kiwi investors.
Link image
Story image
Artificial Intelligence
SAS announces new products amid cloud portfolio success
Analytics and AI company SAS is deepening its broad industry portfolio with offerings that support life sciences, energy, and martech.
Story image
Sift shares crucial advice for preventing serious ATO breaches
Are you or your business struggling with Account Takeover Fraud (ATO)? One of the latest ebooks from Sift can provide readers with the tools and expertise to help launch them into the new era of account security.
Story image
Digital Transformation
Trading up: It's time to swap core systems for flexible digital applications
This year will see more oranisations planning and commencing high tech renovations that will shake up the way they operate.
Story image
Could your Excel practices be harming your business?
While Excel has been the de-facto standard for budgeting, planning, and forecasting, is it alone, enough to support organisations in the global marketplace that’s facing rapid changes due to digital transformation?
Story image
Comcast to use ThreatQuotient for cybersecurity operations
Comcast, the parent company of NBC Universal and SKY Group, has chosen ThreatQ Platform and ThreatQ Investigations to meet their cybersecurity needs.
Story image
Revenue operations is taking centre stage
As the business world continues to evolve, new demands need to be met to keep up with the ever-changing landscape. 
Story image
A10 Networks finds over 15 million DDoS weapons in 2021
A10 Networks notes that in the 2H 2021 reporting period, its security research team tracked more than 15.4 million Distributed Denial-of-Service (DDoS) weapons.
Story image
Customer experience
Gartner recognises Okta for abilities in Access Management
Okta has announced it has been recognised as a Customers' Choice for the fourth time in a row in the Gartner Peer Insights "Voice of the Customer" report.
Story image
Rubrik Security Cloud marks 'next frontier' in cybersecurity
"The next frontier in cybersecurity pairs the investments in infrastructure security with data security giving companies security from the point of data."
Story image
Maintaining secure systems with expectations of flexible work
Most office workers feel they've proved they can work successfully from home, and as much as employers try, things aren't going back to the way they were anytime soon.
Story image
Legrand unveils Nexpand, a data center cabinet platform
Legrand has unveiled a new data center cabinet platform, Nexpand, to offer the necessary scalability and future-proof architecture for digital transformation.
Story image
Remote Working
Successful digital transformation in the hybrid work era is about embracing shifting goalposts
As organisations embraced remote working, many discovered they lacked the infrastructure needed to support history’s first global load test of remote work capabilities.
Story image
Digital Transformation
Pluralsight and Ingram Micro Cloud team up on cloud initiative
Pluralsight has teamed with Ingram Micro Cloud to build upon cloud competence and maturity internally, and externally support partners’ capabilities.
Story image
Data backup plans inadequate, data still at risk - study
The Apricorn 2022 Global IT Security Survey revealed that while the majority organisations have data backup plans in place, data for many are at risk.
Story image
SPS network now available to CrescoData eCommerce customers
CrescoData, a Pitney Bowes Company and PaaS business in the commerce space, says its customers can now connect to the SPS Commerce Retail Network.
Story image
NCSC advisory highlights poor security configurations
The GCSB's National Cyber Security Centre (NCSC) has released a cyber security advisory identifying commonly exploited controls and practices.
Story image
Data Protection
Barracuda launches new capabilities for API Protection
"Every business needs this type of critical protection against API vulnerabilities and automated bot attacks," Barracuda says.
Story image
Qualys updates Cloud Platform solution with rapid remediation
The new update is designed to enable organisations to fix asset misconfigurations, patch OS and third-party applications, and deploy custom software.
Find out how a behavioural analytics-driven approach can transform security operations with the new Exabeam commissioned Forrester study.
Link image
Story image
Amazon Web Services / AWS
RedShield leverages AWS to scale cybersecurity services
"Working with AWS gives RedShield the ability to mitigate significant application layer DDoS attacks, helping leaders adopt best practices and security architectures."
Story image
Cloud Security
Aqua Security createa unified scanner for cloud native security
“By integrating more cloud native scanning targets into Trivy, such as Kubernetes, we are simplifying cloud native security."
Story image
Cybersecurity starts with education
In 2021, 80% of Australian organisations responding to the Sophos State of Ransomware study reported being hit by ransomware. 
Story image
Trojan cyber attacks hitting SMBs harder than ever - Kaspersky
In 2022 the number of Trojan-PSW detections increased by almost a quarter compared to the same period in 2021 to reach 4,003,323.
Story image
Fastly acquires Glitch, enables faster developer innovation
"This acquisition brings together two of the worlds best ecosystems for application development into a single, seamless developer experience."
Story image
New Relic
New Relic launches vulnerability management platform
New Relic has introduced New Relic Vulnerability Management to help organisations find and address security risks faster and with greater precision.
Story image
Silver Peak
The path to an adaptive, modern network
Managing and securing the network looks different than it did just two years ago—especially given that most of these networks are made up of multi-generations of infrastructure stitched together over time.
Story image
Application Security
What are the DDoS attack trend predictions for 2022?
Mitigation and recovery are vital to ensuring brand reputation remains solid in the face of a Distributed Denial of Service (DDoS) attack and that business growth and innovation can continue.
Story image
More than 40% of banks worried about cloud security - report
Publicis Sapient's new report finds security and the lack of cloud skills and internal understanding of business benefits are big obstacles for banks moving to the cloud.