- Summary:
- A survey carried out for New Relic on the future of observability underlines how rapidly this new approach to software engineering is being adopted - what's behind it?
Adoption of observability is surging, according to the results of a new survey published today by observability vendor New Relic. Not exactly surprising, huh? Nevertheless the scale of change recorded by this online survey of 1,300 IT decision makers across North America, Europe and Asia is testament to the rapid rise of observability. Three-quarters of respondents expect to increase their observability spend in the next year, a quarter "significantly," while 50% are in the process of implementing an observability practice, for completion within the next year.
The survey, carried out for New Relic by CITE Research, quantifies trends that were already visible, as Greg Perotto, SVP Corporate Marketing at New Relic, explains:
We've seen other research that suggests that that timeframe for digital transformation, because of the pandemic, has been accelerated by as many as three or four years, in terms of people trying to just move faster to be able to serve their customers. And frankly, because of that, it's condensed software development timeframes and cycles, it's burdened data pipelines. And so we wanted to understand how much impact that's having on the observability space.
Those trends have been accelerated by two converging factors, as Buddy Brewer, New Relic's Global Vice-President and General Manager, explains:
Digital transformation and cloud adoption ... was just continuing to accelerate and hitting this sort of peak, with many of our customers struggling with how to continue to manage their software amid the shift to containerization and adopting Kubernetes. These were all trends that predated the pandemic. It was already hard as an operator of software.
Then the pandemic happened, which ratcheted up the expectations, as everyone was driven home and then online. And so the pressure, both the technology pressure, as well as now the social pressure from all the people using all of the software [and] expecting it to work, have really collided. It's driven, I think, a lot of what we see reflected in this report.
Change in the nature of software engineering teams
The term observability was coined to describe a holistic approach to analyzing metrics in real-time across the full IT stack, in contrast to the old approach to monitoring, which was more siloed and backwards-looking, with dashboards and alerts built around known problems. The hangover from that more siloed approach is evident in the survey, with almost a third of respondents (31%) grappling with six or more different systems to gain end-to-end observability, while the same proportion have to deal with from two to five.
Adoption of observability also corresponds with a change in the nature of software engineering teams, which used to be focused on specific aspects of the stack, such as front-end, servers or infrastructure. Nowadays, they typically identify with a specific set of functionality that's delivered to customers, and take responsibility for every element of the stack that goes behind it. This is driving the move to more integrated observability tooling, as Brewer explains:
More and more folks are finding themselves responsible for everything. But they've been challenged by trying to use all of these different tools, and then cobble them together in their head with massive swivel-chair problems and flipping back and forth between tabs and all of that. So there's a real demand, an imperative, that we're seeing from the people who are building software to be able to look at it all holistically across the stack.
Similarly, observability is seen as equally important across all four stages of software development, through plan, build, deploy and run, with 80-90% of respondents describing it as either extremely or very important at each stage. But there's yet more that organizations can do with the observability data they're collecting, says Brewer, who observes that, while four out of five C-suite executives fully support more spending on observability, only a small fraction of them say they currently consume the data. That underlines the potential to do more, as he explains:
We're really just scratching at the surface of the utility of this observability data. Both in terms of where it can be used in the software development lifecycle, so that people can use this data when they're actually building software [and] understand where the most common points are today that are throwing off errors, for example, so they can fix those things before they become problems at scale later. It also points to an opportunity for that data to be used by people who aren't just the engineers in the company ...
Once you've made that investment and you have that treasure trove of data that's telling you everything about what's going on in your software — not just using it when the software is on fire, but actually pulling that everywhere else in the software development lifecycle, including planning and the actual construction phases ...
Our customers who do that, they they find that it just transforms their idea of the value that they get out of observability.
My take
I've likened the surge in takeup of observability to the shift in finance teams towards more continuous analysis of financial data. In both cases — and this can be seen in other aspects of business operations too — it's indicative of a shift to operating on the basis of real-time data analysis rather than historic reporting. Therefore it's part and parcel of the ongoing digitalization of business operations, which has been accelerated by the need to adapt rapidly to the sudden changes of the past year.
You might nevertheless argue that there's a fair bit of hype around observability at the moment. It will be interesting to compare this year's results with what New Relic finds from the next iteration of this survey, which will now be done annually. The finding that so many organizations are rushing ahead with adoption is certainly a useful marketing hook that I'm sure New Relic is very pleased with. But the underlying trends are real enough, and the survey reflects strong awareness among IT leaders of what they must do to respond.
Disclosure - New Relic is a diginomica premier partner at time of writing
0 Commentaires