Cardinality and Cardinality Explosion

Cardinality is the number of unique time series a metric can produce. A metric like http_requests_total with labels method and status has low cardinality — maybe a few dozen combinations. Add a label like user_id or the raw request path, and the same metric can generate millions of distinct series, because every unique combination of label values creates its own stored time series.

Cardinality explosion is what happens when that number grows uncontrollably, usually because a label carries an effectively unbounded set of values. It is the single most common way to take down a metrics backend in production.

Why it matters

• Backends run out of memory. Prometheus and similar TSDBs keep an in-memory index of every active series so they can answer queries quickly. Each new series adds a fixed overhead to that index; enough of them and the process OOMs or grinds to a halt well before disk space runs out.
• Queries get slow or fail. A PromQL query that has to scan millions of series to compute a sum by (service) gets dramatically slower as cardinality rises, and can time out entirely, defeating the purpose of having the metric at all.
• Managed platforms bill by series. Datadog, New Relic, and other commercial backends typically charge per custom metric or per unique time series, so a cardinality mistake in code can show up directly as an unexpected bill, sometimes overnight.
• It’s easy to introduce accidentally. A single line of code adding .WithLabel("user_id", id) or .WithLabel("url", rawPath) can turn a healthy metric into a liability, and the damage often isn’t visible until the backend is already struggling.

Common causes

• Unbounded identifiers as labels. User IDs, session IDs, order IDs, and similar values are unique per entity by design — putting them in a label multiplies series count by the number of entities that have ever existed.
• Raw, unparameterized URLs. Using the full request path (/users/12345/orders/98765) instead of a route template (/users/:id/orders/:id) turns a handful of endpoints into an effectively infinite label space.
• High-precision timestamps or free-text fields as labels. Anything a human typed, or any value with fine-grained precision, tends to be unique per occurrence rather than a useful grouping dimension.
• Combinatorial growth across multiple labels. Cardinality multiplies across labels on the same metric — five labels with 100 values each is not 500 series, it’s up to 100^5 in the worst case.

Mitigation strategies

• Label hygiene at instrumentation time. Before adding a label, ask whether you’ll ever want to aggregate by it. If the answer is no, or if its value space is unbounded, it doesn’t belong as a label — put it in a log line or trace span attribute instead.
• Normalize high-variance values. Convert raw URLs to route templates, bucket continuous values (latency, size) into ranges, and drop or truncate free-text fields before they reach the metrics client.
• Use recording rules and pre-aggregation. Recording rules let you precompute an expensive, high-cardinality query into a smaller derived metric on a schedule, so dashboards and alerts query the cheap aggregate instead of the raw series.
• Reach for exemplars instead of high-cardinality labels. When you need to jump from a spike in a histogram bucket to a specific slow request, exemplars attach a trace ID to individual observations without turning that ID into a label — Prometheus and OpenTelemetry both support this pattern, keeping the metric’s cardinality bounded while still linking to traces for the outlier case.
• Enforce limits at the collection layer. Options like Prometheus’s sample_limit per scrape target, or cardinality limits in an OpenTelemetry Collector pipeline, catch runaway series before they reach the backend.

Cardinality problems are cheap to prevent and expensive to fix after the fact, since by the time they’re visible in dashboards or bills, the offending series are often already baked into months of stored data. Treating label design as a deliberate decision, not an afterthought, is the most effective defense.