Log Management Used to be an Afterthought for Engineers

Please welcome LogDNA Co-founder Chris Nguyen to Hacker Noon! LogDNA is log management system that will provide deep insights into your business’ production environment, and a marketplace partner of our weekly sponsor, Manifold.

Manifold is the marketplace for independent developer services. You can learn more about their partners by checking out this interview with JawsDB Founder John Willman. For Hacker Noon readers, Manifold is offering $10 off any service with code HACKERNOON2018.

David: Let’s talk numbers, what is the scale of your business? And what metrics do you use to gauge company progress?

Chris: It’s been one hell of a journey. Since launching in February 2016, we now work with over 2,000 customers who love LogDNA. Our customers varies in sizes, from 1 GB per day of log ingestion to 10 TB a day. Keep in mind that log traffic is not always steady. We’ve had customers automatically spin up thousands of spot instances that all start authenticating and logging simultaneously. It’s not just about total numbers, but also about adapting to fluctuating load.

We measure progress based on a number of indicators such as new sign ups, product engagement, customer growth, and monthly storage.

How has Log Management evolved over the last 5 years?

Log management used to be an afterthought to many engineering teams. There was often a lack of adherence to best practices as customers would simply write log data to a random file on that server. To debug, they would then SSH into each of these machines and grep these files to view log lines. This process is typically time consuming and relies on specific infrastructure knowledge to act on. Not to mention the security implications of access to production servers.

In terms of scale, the volume of generated log data is growing exponentially. When adoption of cloud hosting started picking up, we saw the introduction of early logging aggregation platforms. These players educated the market about the importance of log management and focused primarily on the aggregation and searching of log data.

When we launched 2 years ago, we took a different approach to logging. Instead of solely emphasizing capability, we closely studied logging use cases and used this data to inform the design, experience, and interface of LogDNA. We essentially re-imagined the typical flow of interactions, and created what we believe is the next evolution of log management.

For example, when debugging an issue and a log line of interest is identified, we consider two very important points:

• What information is required to determine if this log line is the source of an issue?
• If it is not the culprit, where do you go next?

These considerations strongly influenced the development of a feature we refer to as the context menu. Each line can be expanded to reveal a set of parsed metadata that can be used to pivot to other lines sharing a similar property or set of properties. Simply clicking on a field inside this context menu can either populate your search query or even jump straight to another set of log lines containing the property of interest.

Not having to manually dredge up data or how to form a specific query enables incredibly natural and rapid debugging flows.

What are the technical advantages versus other log management systems?

While our product is continuously built on the use cases and feedback from our customers, here are just some technical advantages that our product offers:

• Blazing fast search speed from a responsive distributed database
• High availability by eliminating single points of failure
• Dynamic load capacity and metered billing powered by quick scaling
• Hosting provider agnostic deployments lower costs for our customers
• Rapid development iteration and testing enable high feature velocity
• Low ingestion latency stems from our tail-heavy logging focus
• Developer ownership of individual repositories drive responsible coding practices
• Flexible pipeline management results in no data caps for our customers

Why do customers trust LogDNA?

We work with a lot of larger customers like Instacart, OpenAI, Segment where we have earn their trust to be their logging system of choice. Majority of our customers have come from a competing product and it’s our job to earn their business by showing great value right away. We are big believers in compliance and currently HIPAA, SOC2, with PCI and GDPR around the corner. Its earning the respect and business of large enterprises with a security compliance focus that allowed us to truly gain trust.

In addition to securing their trust, we also earn and keep customers because of the seamless experience that we offer. We listen to our users and act on their feedback, ruthlessly optimize for speed, carefully design our UI/UX, and offer fair pricing. By way of analogy, we are the iPhone of logging in an industry full of antiquated BlackBerries.

In November, you raised $7m from Initialized Capital (Alexis Ohanian& Garry Tan ). Those two have amazing experience and post/write some great startup stuff. What do they bring to the table beyond the money?

We’ve been fortunate to have worked with the Initialized Capital team since the inception of our company (pre-LogDNA, pre-Y Combinator days). They witnessed the journey since the beginning, and what they bring to the table is great sound advice.

Having been Y Combinator partners in the past, both Garry Tan and Alexis Ohanian have seen all ends of the spectrum when it comes to hyper growth, as well as mistakes companies make along the way. I love the fact that they were founders themselves, they understand all the challenges and opportunities we encounter month to month.

And how have you started allocating the money?

After raising our Series A, we’ve been hiring the right leaders and team members to fill in new positions in sales, marketing, and engineering. 5 months ago we had fewer than 10 people working at LogDNA. We’re now at 30 people and growing. This funding has significantly increased our product and sales velocity and allow us to scale faster than ever before.

LogDNA was in YC 15. I saw you pivoted from marketing platform to log management system. Was the pivot the primary benefit? How else did YC change your growth trajectory?

During our time at Y Combinator, we had an email personalization tool for ecommerce companies. We wanted to help independent ecommerce stores to send Amazon-like personalized messages. After graduating YC, we realized our internal logging system was a pretty compelling product to what was current available on the market. We decided to re-focus our energy and launch LogDNA in February 2016. YC allowed us to truly understand product market fit, unit economics, growth strategies and constantly listening to customers to build something they truly want.

You’ve had entrepreneurial ventures across many different industries — Recruitment (co-founder JobLoft — Acquired), Dating (GM Cupid, Acquired), Social Commerce (co-founder TeamSave — Strategic partner with eBay Classifieds), and DevOps (LogDNA). What is it that attracts you to business you must work on?

Because we never started off in DevOps, we approach opportunities through a different lens based on our eclectic background in recruitment, dating, and commerce. We take what we’ve learned and apply it to the way we problem solve, always with the attempt to offer a solution that is innovative and effective.

I’ve been fortunate to have the same co-founder/CTO for all our companies. He’s the Ying to my Yang when it comes to executing a new business. I love taking on the challenge of tackling new industries, but it helps tremendously to bring a fresh perspective to resolve obstacles along the way.

I would analyze the market size of the opportunity, observe the current landscape and determine any feature gaps that exist. We enjoy pursuing ideas that we’ve experience personal pain points with… like LogDNA.

What is your company’s biggest threat?

Companies rarely fail due to competitors, but instead fall to their own worst enemy: themselves. Early company failures are often the result of poor execution, interpersonal conflicts, or a bad product-market fit. How we execute as a team is paramount to the evolution of our product, our sales strategy, and even our support plan. We have to love LogDNA ourselves if we expect our customers to love our product as well. At the end of the day, all our success metrics are all about gauging one very important thing: customer happiness, and that is what we strive for in our work.

Why did you decide to partner with Manifold?

Already knowing the founding team, I instantly saw the big vision they wanted to accomplish. We are honored to be partners of Manifold and looking forward to growing with their team.

What does LogDNA look like as a company in 5 years?

In 5 years, we want to be the logging system of choice for all software-related teams and organizations.

If you weren’t building LogDNA what would you be doing?

Having 3 nieces who are all under the age of 10, I’d love to be in an environment that allows me to inspire and mentor the next generation of founders. Having worked in many industries, I think I’d be a decent investor/mentor.

What’s easiest way to uncover the value of LogDNA?

Try us out for a free 14 day trial!

This is story is brought to you by Hacker Noon’s weekly sponsor, Manifold. Find, manage and share all of your developer services with 1 account on Manifold. Use code HACKERNOON2018 to get $10 off any service.

Log Management Used to be an Afterthought for Engineers was originally published in Hacker Noon on Medium, where people are continuing the conversation by highlighting and responding to this story.