Observability bills get loud when teams treat telemetry as "ship everything, sort later." Storage vendors charge per byte ingested and per GB retained, so every chatty health check, every container restart loop, every debug log left on in production becomes a line item. The good news: most waste is removable at collection time, before a single byte hits your backend. This guide walks through five places to cut noise — at the Collector, in your naming, at ingest, and in your review cadence.
Filter known noise at the collector
The OpenTelemetry Collector's filter processor and tail sampling processor are your first line of defense. Drop health-check spans (GET /health, GET /readyz) before they leave the cluster. Drop DEBUG-level logs from services where the production log level is INFO. Drop high-volume low-value spans like inner-loop loop iterations that someone instrumented for one debugging session and forgot.
For trace sampling specifically, head sampling (drop at the SDK) is cheapest but blind to outcome. Tail sampling at the Collector lets you keep error traces, slow traces, and a random fraction of the rest. See our sampling strategies post for concrete tail_sampling rules.
Batch before export
The Collector's batch processor reduces network overhead and smooths export spikes. Pair it with the memory_limiter processor so the Collector fails predictably under pressure instead of OOM-killing itself and losing the in-flight buffer. A typical Collector pipeline looks like: receivers → memory_limiter → filter → batch → exporter. Don't reorder this — batch before memory_limiter means the limiter can't shed load before the batch grows.
Normalize service names
Service names with deploy hashes, pod IDs, or random suffixes create fake cardinality across every metric, log, and trace. Within a week, your billing view shows 200 "checkout-7f3a9"-style services that should have been one service with a stable name. Set service.name at SDK startup from a hard-coded value (or a stable env var), not from HOSTNAME in Kubernetes. Use service.version and deployment.environment for variant separation. More on naming hygiene.
Use Data Shaper for ingest-time cleanup
Some noise can only be recognized after parsing the log body — for example, "this stack trace appears 4000 times an hour and means the same thing." Filtering at the Collector level requires deploying a config change for every new pattern, which is painful. Telemtra's Data Shaper lets you drop, mask, or rewrite logs as they arrive at ingest, with rule changes that take effect immediately and no Collector redeploy. Use this for PII redaction, repetitive error messages, and noisy access logs.
Review top fields and patterns regularly
Even with filters in place, new noise sources appear with every deploy. Make it a monthly habit to review your top log groups and your highest-cardinality metric attributes. In Telemtra, the Log Insights view groups logs by structured fields and surfaces the top contributors by volume. Three minutes once a month often catches the next noisy source before it becomes a quota problem.
Related reading
- OpenTelemetry sampling: head, tail, what to keep
- OpenTelemetry Collector receivers: a practical guide
- Data Shaper docs — ingest-time log processing.