Seeing AI Ops with Observability and Telemetry, Previewed

Observability and telemetry are the dashboard for seeing AI operations. You measure what is happening across three signals (metrics, logs, traces), but you keep only what changes a decision and sample the rest. With observability costs rising 40-48% a year, the key is cardinality control: never put unbounded labels (like user IDs) on metrics. This blog also instruments its publishing funnel, pass rate, unique visits, dwell, and average latency (about 10ms), while capping and sampling events to avoid overload.

In plain terms: telemetry is a car dashboard. You could put a sensor on every part, but you watch only speed, fuel, and warning lights to actually drive.

First, the terms. Telemetry is the measurement signals (numbers and records) a system emits about itself. Observability is the ability to infer, from the outside, "what is happening inside" by combining those signals. Cardinality is the number of distinct values (label combinations) a metric can take; when it explodes, cost explodes. Below we unpack the three signals and cardinality with diagrams.

What do you measure, and where?#

The three signals play different roles: metrics for trends, logs for events, traces for paths. Metrics are aggregate time series for SLOs and trends, logs are individual events for debugging and audit, and traces stitch the segments a request passed through to expose bottlenecks. The standard today is OpenTelemetry: instrument the code once and swap backends (Prometheus, Grafana, and others) without touching the app. The point is to connect signals with shared context, not collect them in isolation.

See how the three signals split the work: one request is recorded three different ways.

Why is "measure everything" a trap?#

Because cardinality silently burns the budget. Every label you add to a metric creates one time series per combination: 10 regions x 1M users x 5 devices makes 50 million unique series. So unbounded values like user IDs, request IDs, and UUIDs belong in logs or traces, not metric labels. Traces use tail sampling to cut volume 80-90% while keeping errors and slow paths, with 5-10% as a baseline sample. The principle is not to observe less but to observe smarter: every byte should earn its storage cost.

Seeing how one label multiplies into millions makes the danger concrete.

What does AI (LLM) ops need on top?#

On top of the three signals come prompts, tokens, cost, and quality. LLM observability tracks prompt/response, token usage, latency, cost attribution, and hallucination/drift detection. The key split is monitoring (real-time numbers like latency, tokens, cost) versus evaluation (quality such as accuracy, hallucination, relevance). The market is growing fast for it: LLM observability platforms are forecast to grow from 1.97 billion dollars in 2025 to 2.69 billion in 2026, a 36.3% annual rate. The trick is wiring tracing, evaluation, and alerting into one feedback loop.

Wiring those three into a single loop looks like this.

How do you start?#

Small, core only.

• Instrument with OpenTelemetry and keep unbounded identifiers in logs or traces, not metrics.
• Start traces at a 5-10% tail sample while preserving errors and slow paths.
• Put only decision-changing metrics (north-star, pass rate, latency) on the dashboard and trim the rest.

Reference links

• OpenTelemetry (observability standard)
• Prometheus (metrics)
• Grafana (dashboards)
• W3C Trace Context (trace propagation)
• Langfuse (LLM observability)

Note: cost and market figures are public 2025-2026 reports and surveys and vary by scope (observability budget guidance is 15-25%). Measure exact cost and effect on your own system. Signals must be consistently named and connected for AI to correlate them. Reviewed quarterly.