Correlating Continuous Profiles with Traces

Tracing tells you which service, which request, and which span was slow. Continuous profiling tells you which function was consuming CPU or memory during some window of time. On their own, closing the loop between them means manually cross-referencing timestamps — correlating profiles with traces automates that, linking a specific slow span directly to the exact flame graph produced while it was executing.

Why it matters

• It closes the gap between “what” and “why.” A trace can show a span that took 900ms with no further detail about what the code was actually doing during that time; a profile shows CPU and memory activity but has no inherent concept of “this request.” Correlating the two turns “this endpoint is slow” into “this specific function, in this specific request, was the cost.”
• It avoids guesswork during incident response. Without correlation, an engineer investigating a slow trace has to guess at the right time window, pod, and service to go look up in a separate profiling UI. Direct linking removes that manual step and the risk of looking at the wrong window entirely.
• It makes rare, request-specific slowness attributable. Aggregate, fleet-wide profiles are great for finding a hot path across many requests, but they average away outliers. Span-linked profiling can isolate the profile data specific to one slow request rather than an average across thousands of fast ones, which is often the only way to explain a long-tail latency spike.
• It ties code-level cost back to the request that a user or SLO actually cares about, rather than leaving profiling as a purely infrastructure- or host-level exercise disconnected from request-level traces.

How it works

• Two complementary mechanisms exist: span profiles and time-window correlation. Span profiles work by tagging profile samples with the active trace and span ID at the moment they’re captured — this requires a language-specific integration that has visibility into both the tracing context and the profiler at the same time, so it’s the deeper but more setup-intensive option. Time-window correlation instead uses the span’s start time, duration, and service/pod metadata to query a separate profiling backend for the matching profile after the fact — it needs no per-span tagging, but is coarser since it just matches on time and location rather than sampling exactly during that span.
• Grafana’s “traces to profiles” feature is a widely used implementation of this pattern: a Tempo (or other tracing backend) data source is linked to a Pyroscope data source in Grafana, so clicking a span opens the corresponding profile query. Its deeper variant, “span profiles,” requires a language-specific integration package to attach trace and span IDs directly to samples, giving resource attribution down to the specific span rather than just the surrounding time window.
• OpenTelemetry is standardizing this at the data-model level. OpenTelemetry’s profiling signal (which reached public alpha in 2026, joining traces, metrics, and logs as a core signal) defines an OTLP profiles format that round-trips with pprof and explicitly supports attaching trace ID and span ID links to individual profile samples — meaning correlation becomes a property of the data itself rather than something bolted on by a specific vendor’s UI.
• Vendor implementations vary in depth. Datadog’s continuous profiler links flame graphs to specific traces and endpoints (marketed as code-level hotspot attribution tied to APM traces); other APM vendors offer similar trace-to-profile linking. The common thread is always the same: capture enough shared context (a trace/span ID, or at minimum a precise time window and workload identity) at the point profiles and traces are collected so the two signals can be joined later.

As the OpenTelemetry profiling signal matures, this correlation is likely to become less dependent on a single vendor’s integration and more of a standard property available across any OTLP-compatible backend — following the same trajectory that made OpenTelemetry the common substrate for traces, metrics, and logs.