Logs are boring… and expensive

There is a solution! And it’s not a commercial product!

In this article I discuss issues plaguing the metrics & logs ingestion landscape. How incumbent vendors are benefiting from a lack of cohesiveness, and how OTel has arrived to save the day!

As a DevOps practitioner I’ve come to love logs in all their mundane, burdensome glory. The application doesn’t actually need them. They just take up space and bandwidth, they are constantly spraying sensitive data all over the place like a broken firehose, and extracting meaningful data from them is challenging.

Logs come in all different shapes and sizes, they don’t seem to easily fit anywhere without a bit of wrangling. If you ask 5 teams what their logging standard is, you’ll likely get 5 different standards. If you ask them what systems ingest and monitor those logs, you’ll probably get another five.

However, they are absolutely essential to operations of any scale for security incident alerting and response, performance detection and optimization, and error reporting and debugging.

It’s a Minefield

Don’t believe me? Here are a some products and protocols you might encounter at a conference or Google search.

ElasticSearch | Kibana | Sensu | Graphite | Logstash | NewRelic | DataDog | SumoLogic | Kinesis | CloudTrail | Splunk | Prometheus | Jaeger

Every one of these products promises to gather important telemetry data from your running application.

There is a lot of functionality overlap with different approaches and architectures. They all:

• cost an arm and a leg to either set up / operate or rent.

• need some degree of customization

• require specialized knowledge

• have different philosophies to harvest logs from your systems

• result in sticky vendor-lock