Honeycomb Sees Sweet Reward in Unifying Dev, Ops

“DevOps” is so easy to say. But the simple phrase masks an uneasy tension between development and operations that can lead to poor communication and ultimately poor management of increasingly complex tech stacks. Addressing that DevOps tension and communication gap also happens to be challenge that the founders of an observability startup called Honeycomb aim to solve.

Honeycomb’s story starts circa 2012, when two engineers, Christine Yen and Charity Majors, found themselves working together at Parse, which developed a back-end for mobile apps. Yen was a developer while Majors was more inclined towards the infrastructure. A friendship blossomed between them, despite differences in jobs and natural talents.

“She was an ops and infra engineer who I’m sure popped out of the womb carrying a pager. I was a product engineer who wanted to build cool experiences for our customers,” Yen said. “I was terrified of production and, to be frank, broke a bunch of Charity’s infrastructure.”

Even back in 2012, gaps in observability were obvious. Getting information out of logs was challenging, and it only got harder as the software stack evolved from monolithic applications running on-prem towards microservices running in the cloud–let alone today’s serverless and stateless apps running in K8S clusters.

Parse was acquired by Facebook in 2013, which exposed Yen and Majors to a novel logging and metrics tool called Scuba. Built upon an in-memory time-series database, Scuba suddenly allowed the engineers to visualize a lot more of the “nitty gritty” data, such as app IDs, SDK versions, and app versions, that were previously off limits. That opened Yen and Majors’ eyes to a new world of observability detail and performance troubleshooting.

“Everything was so fast you could churn through tons and tons and tons of data, spit out a chart, and then say ‘Drill into this one app and this one SDK version,’” Yen said. “Because we were a platform, we wouldn’t know if there was a Russian dating app that would launch, that would use a new endpoint in an unexpected way and take down our Mongo cluster. But it happened all the time, and so we had to figure out how to get ahead of that.”

Emboldened by the Scuba experience, Yen and Majors left Facebook and founded Honeycomb in 2016. The pair had an idea for a new kind of observability tool that could express operations concepts in a developers’ language, and (hopefully) bridge the gap between Dev and Ops, thereby bringing back some semblance of simplicity–or at least understandability.

“What used to be these pretty understandable worlds that could be understood at a high level where we all had Rails apps on five beefy EC2 instances, and they were your pets, right? You give them names–cute things–and you get very stressed when one goes down,” said Yen, who is Honeycomb’s CEO. “And now it’s the pets versus cattle. Now we have instances we’re cycling through in the cloud. We are shifting from these five app servers and monoliths to microservices on Kubernetes pods that are constantly getting recycled.”

Honeycomb aims to make complex observability data readily consumable by developers (Image courtesy Honeycomb)

At a technical level, Yen and Majors, Honeycomb’s CTO, made a few decisions in 2016 that set it on its path. It decided there was no good reason to split up the Holy Trinity of observability data–logs, metrics, and traces–as some observability companies do. By storing logs, metrics, and traces together, it would be easier to correlate details found in them, which would accelerate the remediation of problems.

The second big architectural decision the company made was it selected an column-oriented database in which to store all this observability data. This made Honeycomb more like Snowflake than Datadog, Yen said.

“We didn’t invent column stores, and we’ll be the first to tell you that there’s nothing, honestly, all that special about column stores,” she told BigDATAwire in an interview at re:Invent 2024 last week. “But we were the first people to really start to question, should we maybe not have logs over here and metrics over here forever? Should we maybe try to find ways [to keep them together], especially if they’re talking about the same thing that happened in an application?”

Instead of spending time and effort trying to join these data types together so they can be analyzed, Honeycomb’s design naturally stores them together, using the OpenTelemetry (OTEL) format. That not only eliminates the need for data integration, but it helps with the next capability that Honeycomb is known for: Providing context to the complex metadata it collects, which in turn helps turn the technical infrastructure jargon that operations folks speak into the business language that developers speak.

“It meant that previously Charity would be like ‘Christine, what did you do to the write throughput on my Cassandra instance?’ And I’d be like, let me try to figure it out. I’m really scared,” Yen said. “It turned those conversations into, ‘Hey, Christine, I saw elevated write throughput on my Cassandra instances. So I looked, and it seems that that elevated traffic is driven by this app on this endpoint. What’s going on?’

“And I’d be like, Oh, that’s a regression,” she continued. “Now let me go look, because now it’s in my world. Now I can go try to reproduce it in a test. Now I can go look at the business logic. Now I understand why it matters, because I know which apps you’re talking about, and that’s the way that it changed how we work together as an engineering team.”

That approach seems to be resonating, as the San Francisco-based company has already signed 800 paying customers, including companies like Vanguard, Fender Guitars, and Jack Henry & Associates. The company, which has raised about $147 million through a Series D round, also last week picked up a 2024 BigDATAwire Readers’ Choice Award in the category of Best Big Data Product: DataOps and Observability.

Honeycomb isn’t the only observability company that’s shaking up the status quo these days. Companies that have moved their IT operations to public clouds are straining under mountains of logs, metrics, and traces, as well as the massive costs of storing and processing all that data, and they’re open to new ideas. The incumbent observability vendors still have momentum, but a new guard is emerging in this space, including startups like Honeycomb.

Honeycomb is actively pursuing customers who are tired of the business practices of some of its big observability tool competitors and are looking for something new to tackle the big storage and processing challenges of today, Yen said.

“You should be questioning what you’re doing with your tools to begin with, and maybe if what you’re doing with your tools is the same thing you’ve been doing for the last 20 years with telemetry, with APM, logging, and monitoring tools, perhaps the way we approach building software is worth a revisit,” she said. “I think maybe that’s why your readers like us, because we’re here to question.”

Honeycomb Taps Observability Buzz to Raise $50M Series D

VCs Open Up the Checkbook for Observability Startups

The post Honeycomb Sees Sweet Reward in Unifying Dev, Ops appeared first on BigDATAwire.