beginner · Module 1.5
Centralized Logging with EFK / Loki
Learning Objectives
- Compare the index-everything model (Elasticsearch/EFK) with Loki's label-index + chunk model.
- Explain why structured JSON logs with a trace_id are essential for correlation.
- Reason about log volume and cost at OTT match scale.
Key Concepts
EFK vs Loki
EFK (Elasticsearch-Fluentd/Fluent Bit-Kibana) indexes full log content for rich search at high storage cost. Loki indexes only labels and stores compressed chunks, trading query flexibility for dramatically lower cost.
Structured logs + trace_id
Emitting JSON logs carrying trace_id and span_id lets you jump from a failing trace directly to its exact log lines, closing the detect -> localize -> read loop.
Volume at scale
At 250K RPS, logging every request line is prohibitively expensive; teams sample verbose logs, keep errors at full fidelity, and rely on metrics/traces for the aggregate view.
Info
Loki's cost model favors low-cardinality labels for the same reason Prometheus does — labels are the index.
Warning
Free-text logs without a trace_id force manual timestamp correlation across services, which is hopeless during a fast-moving incident.
Knowledge Check
1. How does Loki reduce cost compared to Elasticsearch/EFK?
2. What makes structured JSON logs with a trace_id valuable during triage?
3. Why do OTT teams sample verbose logs at 250K RPS?