Metrics
Computing SLOs and Error Budgets from Metrics
The PromQL patterns for turning request-rate and latency metrics into a working SLI, SLO, and burn-rate alert.
Last updated
A service level objective is only as good as the metric feeding it. This article covers the practical, metrics-layer side of that: which raw counters and histograms turn into a valid SLI, and how to query them. For the full definitions of SLI, SLO, SLA, and how error budgets are used organizationally, see SRE Fundamentals — this article assumes that vocabulary and focuses on the query patterns.
Why the metrics layer matters here
- The SLI is a query, not a concept. “99.9% of requests succeed” is only meaningful once it’s backed by a specific PromQL (or equivalent) expression that both engineering and the business agree measures the right thing.
- Small query mistakes silently break the SLO. Counting client-cancelled requests as failures, measuring at the wrong aggregation level, or using an average instead of a percentile for latency can all make an SLO report a number that doesn’t match what users experience.
- Burn-rate alerting depends entirely on the underlying metric’s resolution and labels. A success-rate metric without enough label granularity to isolate a single failing dependency makes burn-rate alerts accurate but not actionable.
PromQL patterns for SLIs
- Request success rate, the standard availability SLI, is typically computed as
sum(rate(http_requests_total{code!~"5.."}[5m])) / sum(rate(http_requests_total[5m]))— the proportion of successful requests over a rolling window, built directly on the counters described in metric types. - Latency percentiles, the standard performance SLI, come from a histogram via
histogram_quantile()(or an equivalent function on other backends), since percentiles capture tail latency in a way an average cannot — a p99 SLO of “300ms” is a very different promise than an average of 300ms. - Burn-rate alerts compare the current rate of error-budget consumption against multiple time windows at once — for example a fast 1-hour window and a slower 6-hour confirming window — so both sudden outages and slow leaks get caught before the budget is fully gone, rather than alerting only after the SLO is already breached.
- Measurement window and event boundaries have to be decided in the query, not just in policy. Whether a client-cancelled request counts as a failure, and whether the window is a rolling 28 or 30 days, changes what the PromQL expression actually returns — settle this before treating the number as authoritative.
Most metrics platforms — Prometheus with recording rules, or commercial tools like Datadog and New Relic with built-in SLO features — now offer dedicated support for defining an SLO, tracking its error budget in real time, and alerting on burn rate, which has made this pattern a default rather than a bespoke build for most teams adopting it.
Related tools
The CNCF-graduated metrics collection and alerting system built around a pull model and its own query language, PromQL.
The open-source dashboarding and visualization platform that has become the default front end for metrics, logs, traces, and profiles alike.
A unified commercial observability platform covering infrastructure metrics, APM/distributed tracing, log management, and continuous profiling.
A managed, all-in-one observability platform combining metrics, traces, logs, and AI-driven analysis under a single usage-based pricing model.