---
name: opentelemetry-instrumentation
description: "Instrument services with OpenTelemetry across traces, metrics, logs, and the new Profiles signal, plus Collector pipelines. Full-stack OTel: SDK setup with auto/manual instrumentation, span context propagation (W3C traceparent/baggage), OTLP exporters over gRPC/HTTP, and the alpha Profiles signal correlated to traces via exemplars. Build Collector pipelines (OTLP/Prometheus receivers, batch/memory_limiter/tail_sampling processors, exporters), apply semantic conventions for resource and HTTP/DB attributes, and deploy via the Operator and OTel Collector DaemonSet on Kubernetes."
license: MIT
---

# OpenTelemetry Instrumentation

End-to-end OpenTelemetry help for engineers wiring up traces, metrics, logs, and the new Profiles signal — across application SDKs and the Collector. Use it when you're standing up observability from scratch, migrating off a vendor agent, or debugging why spans aren't showing up in your backend.

## When to use it

- A service emits no traces, or traces break across an HTTP/gRPC boundary (broken context propagation).
- You're choosing between auto-instrumentation and hand-written spans, or both.
- You need a Collector config with sampling, batching, and multiple exporters.
- You want continuous CPU profiles correlated to slow spans.

## What it does

On the SDK side it sets up the `TracerProvider`, `MeterProvider`, and `LoggerProvider`, wires a `BatchSpanProcessor` to an `OTLPSpanExporter`, and configures `Resource` attributes (`service.name`, `service.version`, `deployment.environment`). It applies auto-instrumentation — `opentelemetry-instrument` for Python, the Java agent JAR via `-javaagent`, `@opentelemetry/auto-instrumentations-node` for Node — and shows where to drop manual spans with `tracer.start_as_current_span()`.

Context propagation uses the W3C `TraceContext` (`traceparent`/`tracestate`) and `Baggage` propagators. Exporters speak OTLP over gRPC (4317) or HTTP/protobuf (4318), tunable with `OTEL_EXPORTER_OTLP_ENDPOINT` and `OTEL_EXPORTER_OTLP_PROTOCOL`.

For the Collector it builds `receivers`/`processors`/`exporters`/`pipelines`: OTLP and Prometheus receivers, `memory_limiter` (place it first), `batch`, `tail_sampling`, and `resourcedetection`. The alpha Profiles signal is exported via OTLP and linked to traces through exemplars so you can jump from a span to the CPU profile that produced it.

## Example

```bash
curl -fsSL --create-dirs -o ~/.claude/skills/opentelemetry-instrumentation/SKILL.md https://claudeskil.com/skills/opentelemetry-instrumentation/SKILL.md

# Python auto-instrumentation, no code changes
pip install opentelemetry-distro opentelemetry-exporter-otlp
opentelemetry-bootstrap -a install
OTEL_SERVICE_NAME=checkout \
OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4317 \
OTEL_TRACES_SAMPLER=parentbased_traceidratio \
OTEL_TRACES_SAMPLER_ARG=0.1 \
  opentelemetry-instrument python app.py
```

```yaml
processors:
  memory_limiter: { check_interval: 1s, limit_mib: 512 }
  batch: {}
service:
  pipelines:
    traces: { receivers: [otlp], processors: [memory_limiter, batch], exporters: [otlphttp] }
```

## Tips & gotchas

- Order matters: `memory_limiter` must come before `batch` in every pipeline, or it can't shed load under pressure.
- `tail_sampling` requires all spans of a trace to reach one Collector instance — use a trace-ID-aware load balancer (the `loadbalancing` exporter) in front of a sampling tier.
- Sampler config belongs at the SDK; setting `OTEL_TRACES_SAMPLER` and a Collector `probabilistic_sampler` both will double-drop.
- Profiles is alpha — pin Collector/SDK versions; the schema still shifts.

## FAQ

**gRPC or HTTP exporter?** gRPC (4317) is lower overhead; use HTTP/protobuf (4318) when proxies or service meshes mangle gRPC streams.

**Auto and manual together?** Yes. Auto-instrumentation creates the root spans; your manual `start_as_current_span` calls nest as children automatically via the active context.

**Why are traces split into separate traces?** Almost always a missing propagator — ensure both sides use the W3C `tracecontext` propagator and forward the `traceparent` header.