Overview

OpenTelemetry tracing, metrics and log correlation for Vercube applications

The Telemetry module turns every request into an OpenTelemetry span, records the standard HTTP metrics, and stamps each log line with the trace it belongs to. It speaks the OpenTelemetry API rather than a Vercube-specific one, so any OTLP backend works: Jaeger, Grafana Tempo, Honeycomb, Datadog, or a collector of your own.

By default it is on in development and off in production, because tracing every request costs real throughput and a production deployment should turn it on together with an exporter.

Installation

$ pnpm add @vercube/telemetry

That is the whole installation. This is the only package in the framework that depends on OpenTelemetry, and it ships the tracer and meter providers, the samplers and the in-memory test providers with it. Exporting to an OTLP collector is the one thing that needs more, because the exporter pulls a protobuf stack an application doing purely local tracing has no use for:

$ pnpm add @opentelemetry/exporter-trace-otlp-http

Getting started

Register the plugin in your configuration:

vercube.config.ts
import { defineConfig } from '@vercube/core';
import { TelemetryPlugin } from '@vercube/telemetry';

export default defineConfig({
  telemetry: true,
  plugins: [TelemetryPlugin],
});

That alone gives you W3C trace context propagation and log correlation. Nothing is recorded yet, because the OpenTelemetry API is a no-op until a tracer provider exists. Start one in your entry file:

src/index.ts
import { startNodeTelemetry } from '@vercube/telemetry/sdk';

await startNodeTelemetry({
  serviceName: 'checkout',
  endpoint: 'http://localhost:4318',
});

Call it before createApp() so application bootstrap is traced too.

What you get

Every request produces a SERVER span named after its route, carrying the stable HTTP semantic conventions:

GET /users/:id
  http.request.method     GET
  http.route              /users/:id
  http.response.status_code 200
  url.path                /users/42
  server.address          api.example.com
  vercube.controller      UsersController
  vercube.handler         byId

Alongside it:

  • http.server.request.duration, a histogram keyed by method, route and status. The URL is deliberately not an attribute, because one time series per URL is how a metrics backend gets destroyed.
  • Process gauges: heap usage, resident set size, CPU utilisation, event loop delay and utilisation.
  • Container construction replayed as a trace, so application startup shows up as an ordinary waterfall.

Instrumented modules add their own spans automatically. @vercube/storage traces each operation, @vercube/cache traces lookups and counts hits and misses, @vercube/ws traces messages, @vercube/queue traces published and processed jobs, and @vercube/auth records each authentication decision. They appear nested under the request that caused them without any wiring on your side.

They all do it through @vercube/telemetry/instrument, which is public: see Instrumentation to do the same in a library of your own.

Trace context

Incoming traceparent and tracestate headers are read, so a request that arrives from another service continues the same trace. Going the other way, inject the current context into an outgoing call:

import { Inject } from '@vercube/di';
import { Telemetry } from '@vercube/telemetry';

class InvoiceClient {
  @Inject(Telemetry)
  private gTelemetry!: Telemetry;

  public async fetchInvoice(id: string) {
    const headers = new Headers();
    this.gTelemetry.inject(headers);

    return fetch(`https://billing.internal/invoices/${id}`, { headers });
  }
}

Trace context also travels through @vercube/queue: a published job carries traceparent in its headers, and the worker that runs it continues that trace, in whatever process it happens to be.

Tracing your own work

import { Inject } from '@vercube/di';
import { Telemetry } from '@vercube/telemetry';

class InvoiceService {
  @Inject(Telemetry)
  private gTelemetry!: Telemetry;

  public refund(id: string) {
    return this.gTelemetry.span('invoice.refund', (span) => {
      span.setAttribute('invoice.id', id);
      return this.process(id);
    });
  }
}

span() returns whatever your function returns, unchanged. A synchronous function stays synchronous.

Telemetry is the token for application code. A reusable library is better off with @vercube/telemetry/instrument, which needs no container.

Logs

With telemetry active, every log line carries traceId and spanId, and the per-request wide event is keyed by the trace id instead of a random one. A backend that ingests both can jump from a log line to the span that produced it.

Shipping logs to the same collector takes one option:

vercube.config.ts
export default defineConfig({
  telemetry: {
    logs: true,
    endpoint: 'http://localhost:4318',
  },
  plugins: [TelemetryPlugin],
});

Configuration

vercube.config.ts
export default defineConfig({
  telemetry: {
    enabled: true,
    serviceName: 'checkout',
    sampler: { ratio: 0.1 },
    exclude: ['/health'],
    spans: {
      handler: true,
      middleware: false,
      di: true,
      bodies: false,
      headers: false,
    },
    propagation: true,
    metrics: true,
    logs: false,
  },
});

exclude is worth knowing about: a health check probed once a second is traffic about your application rather than through it, and left in, it dominates the request histogram.

spans.bodies attaches request and response bodies to the span as events. It is off by default and meant for development: it clones every message and holds the span open until the read finishes, and bodies routinely contain personal data.

spans.headers does the same for headers. Authorization, Cookie and the other credential-bearing names are always withheld whatever you configure; headers: { redact: ['x-internal-token'] } adds your own. Query parameters whose name reads like a credential, access_token and api_key among them, are withheld from url.query unconditionally.

Enabling devtools turns body and header capture on for you, but only outside production. Both ride on the server span, so every exporter sharing the pipeline sees them; in production you have to ask for them here explicitly.

Cost

With telemetry: false the framework's instrumentation points are a single null check, and the allocation-free route fast path still returns its response synchronously. Turning telemetry on without registering a provider costs slightly more, because the OpenTelemetry API still creates non-recording spans.

On a serverless platform, call the flush before your invocation returns. The Vercube adapters in @vercube/serverless already do.

Previous

API

The Telemetry token and the package's entry points

Next