New Relic unpacks top observability trends and forecasts

New Relic, the observability company, published the 2022 Observability Forecast report, which captures insights into the current state of observability and its growth potential.

As IT and application environments increasingly move toward complex, cloud-based microservices, the research found technology professionals have bold plans to ramp up observability capabilities to get ahead of issues that could impact customer experience and application security.

At least three-quarters of respondents surveyed in Australia and New Zealand (ANZ) said C-suite executives in their organisations are advocates of observability, and more than three quarters (87%) saw observability as a key enabler for achieving core business goals, which implies that observability has become a board-level imperative, the researchers state.

The report reveals the technologies they believed will drive further need for observability and the benefits of adopting an observability practice. It also highlights that across ANZ, the key driving factors cited for using observability were cost-cutting and tool consolidation.

According to the research, ANZ organisations today monitor their technology stacks with a patchwork of tools. At the same time, respondents indicated they longed for simplicity, integration, seamlessness, and more efficient ways to complete high-value projects.

Moreover, as organisations race to embrace technologies like blockchain, edge computing, and 5G to deliver optimal customer experiences, observability supports more manageable deployment to help drive innovation, uptime, and reliability.

Overall, the 2022 Observability Forecast found:

• Only 30% had achieved full-stack observability by the reports definition (the ability to see everything in the tech stack that could affect the customer experience). Just 8% had a mature observability practice by the reports definition.
• More than a quarter (26%) said they still primarily detect outages manually or from complaints, and most (88%) used four or more tools to monitor the health of their systems.
• Almost half (43%) said they experience high-business-impact outages once per week or more, and 32% said they take more than an hour to resolve those outages.
• Just 10% said their telemetry data is entirely unified (in one place), and only 14% said the visualisation or dashboarding of that data is entirely unified.
• Half said they prefer a single, consolidated observability platform.

Peter Marelas, Chief Architect APJ at New Relic, says, "Respondents surveyed in ANZ predicted their organisations will most need observability for artificial intelligence (AI), the Internet of Things (IoT), and blockchain in the next three years. ANZ is home to a melting pot of different cultures and observability maturity levels.

"Despite many differences, findings in the2022 Observability Forecast are echoing what we are hearing from best-of-breed businesses in the field: that shifting left by embedding observability into all parts of the software lifecycle is a key contributor to engineering success."

Among the report's key takeaways, the data supports a strong correlation between achieving/prioritising full-stack observability and experiencing fewer outages, improved outage detection rates, and improved resolution.

For example, 33% of respondents surveyed in ANZ who indicated that they had already prioritised/achieved full-stack observability were also less likely to experience the most frequent high-business-impact outages (once per week or more), compared with the 43% who had not.

In addition, 67% who said they had already prioritised/achieved full-stack observability also said it takes less than 30 minutes to detect high-business-impact outages, compared with the 39% who had not.

The research implies that the ideal state of observability is one where engineering teams monitor the entire tech stack in all stages of the software development lifecycle, employ mature observability practice characteristics, and have unified telemetry data and a unified dashboard or visualisation of that data ideally in a single, consolidated platform.

Half said they prefer a single, consolidated observability platform, yet none were using one tool for observability. They said some of the main challenges preventing them from prioritising/achieving full-stack observability are a lack of personnel or budget, un-instrumented systems, and too many monitoring tools.

According to the 2022 Observability Forecast, developers and engineers seek solutions that will make their lives better and easier.

When New Relic and ETR asked practitioners surveyed in ANZ themselves how observability helps developers and engineers the most, they found:

• Four in 10 (40%) believed observability increases their productivity and enables them to find and resolve issues faster.
• Almost half (46%) said observability enables cross-team collaboration, and 33% said it improves their skillset or hireability.
• Almost a third (29%) felt that it enables less guesswork when managing complicated and distributed tech stacks.

When asked about the top trends driving observability needs at their organisations, respondents surveyed in ANZ said risk mitigation, cloud-native application architectures, adoption of open-source technologies, and customer experience were among the highest drivers.

Challenges aside, they saw observability bottom-line benefits and expected to deploy additional observability capabilities including AIOps, alerts, and serverless monitoring in the next three years (the report focuses on 17 capabilities in all). Just 2% indicated that their organisations have all 17 observability capabilities deployed.

By 2025, the majority expected to deploy capabilities like APM, MLOps, synthetic monitoring, and serverless monitoring, with the majority indicating they would have 8699% of the 17 observability capabilities deployed. This finding suggests that most ANZ organisations will have robust observability practices in place by 2025.

As they pursue aggressive observability capability deployment plans, 69% of respondents surveyed in ANZ expected to maintain or increase their observability budgets next year. Nearly half (47%), including 48% of C-suite executives, expected observability budgets to increase over the next year. This includes 13% of all respondents and 19% of C-suite executives surveyed in ANZ who expected to increase budgets significantly and the market opportunity is sizeable.

Looking ahead, they foresaw their organisations needing observability for a variety of trending technologies, including AI, 5G, and Web3. ANZ C-suite executives anticipated needing observability most for IoT (55%), AI (45%), blockchain (45%), and edge computing (39%) in the next three years.

The largest study of its kind, the second annual Observability Forecast from New Relic and technology market research firm ETR had 1,614 respondents globally, including 65% practitioners day-to-day users of observability tools and 35% IT decision-makers across 14 countries to understand their current use of observability tools and approaches, as well as their perspectives on the future of observability.