intermediate · Module 2.2
OTel Collector Pipeline & Architecture
Learning Objectives
- Explain the Collector's receiver -> processor -> exporter pipeline and the service.pipelines wiring.
- Justify why memory_limiter and batch belong on every production pipeline path.
- Design an agent (DaemonSet) + gateway (Deployment) topology for an OTT fleet.
- Predict the linter warnings a pipeline missing memory_limiter or batch should raise.
Key Concepts
Receiver -> processor -> exporter
A Collector pipeline ingests via receivers (otlp), transforms via ordered processors (memory_limiter, batch, attributes), and ships via exporters (otlp, prometheus). The service.pipelines block wires which components form each traces/metrics/logs path.
memory_limiter first, batch before export
memory_limiter must be the FIRST processor so it can shed load and protect the Collector from OOM during a match-traffic spike. batch should sit just before the exporter to amortize network round-trips. Every production path needs both.
Agent + gateway topology
Run a lightweight agent Collector as a DaemonSet next to each node (cheap local batching, tag enrichment) that forwards to a horizontally scaled gateway Deployment doing heavier processing (tail sampling, routing) before export.
Why the linter matters
The OTel Builder sandbox lints your graph: a path with no memory_limiter risks OOM, a path with no batch wastes network, an unconnected node is dead config, and a receiver with no reachable exporter drops data silently.
Warning
Processor ORDER matters. Putting batch before memory_limiter means you buffer memory you may not have — memory_limiter must run first to shed load.
Insight
Tail sampling belongs on the gateway, not the agent: only the gateway sees all spans of a trace, which is required to make a whole-trace keep/drop decision.
Info
generateYaml() in the sandbox is deterministic — canonical key order and stable lists — so the same graph always yields byte-identical YAML.
Production Configs & Runbooks
Production configs and runbooks are a Pro feature.
Knowledge Check
1. Which processor MUST be first in a production Collector pipeline, and why?
2. Where should the batch processor sit in the pipeline?
3. Why must tail sampling run on the gateway Collector rather than the per-node agent?
4. The OTel Builder linter flags a receiver whose data can never reach any exporter. Which warning is that?