Python (OpenTelemetry)
Instrument FastAPI, Flask, or any WSGI/ASGI Python application with OpenTelemetry and export traces, metrics, and logs to TracePath.
There is no application code to write. The OTel agent patches your framework at interpreter startup, so endpoints, status codes, child spans, exceptions, logs, and HTTP metrics all arrive from an unmodified app.
Django uses this same path but needs two extra steps of its own. See the Django guide.
Requirements
- Python 3.10 or newer. Current
opentelemetry-sdkreleases declarerequires-python >= 3.10. - A TracePath project created with framework OpenTelemetry, and that project's token.
Install
pip install opentelemetry-distro opentelemetry-exporter-otlp
opentelemetry-bootstrap -a installopentelemetry-bootstrap reads the packages already installed in the environment and installs the matching instrumentation for each one: FastAPI, Flask, Starlette, requests, httpx, urllib3, sqlite3, psycopg, and the standard logging module, among others. It only sees what is installed when you run it, so run it again after you add a dependency.
Install both packages. opentelemetry-sdk on its own records spans and throws them away, because it ships no OTLP exporter and no opentelemetry-instrument command.
Configure
Set these in the process environment: the container env: block, a systemd Environment= line, a Procfile, or export in the shell that starts the server. opentelemetry-instrument reads them before your application is imported, so a .env file that your settings module loads at import time is too late.
export OTEL_SERVICE_NAME=my-python-service
export OTEL_RESOURCE_ATTRIBUTES="service.version=1.0.0"
export OTEL_EXPORTER_OTLP_ENDPOINT=https://ingest.tracepath.dev/api/otel
export OTEL_EXPORTER_OTLP_PROTOCOL=http/protobuf
export OTEL_EXPORTER_OTLP_HEADERS="Authorization=Bearer your-project-token"
export OTEL_TRACES_EXPORTER=otlp
export OTEL_METRICS_EXPORTER=otlp
export OTEL_LOGS_EXPORTER=otlp
# Python only. Without these, logger.info(...) never reaches TracePath.
export OTEL_PYTHON_LOG_CORRELATION=true
export OTEL_PYTHON_LOG_LEVEL=info
OTEL_EXPORTER_OTLP_PROTOCOL=http/protobufis not optional. WithOTEL_TRACES_EXPORTER=otlpand no protocol set, the Python SDK resolves the exporter to gRPC, and TracePath has no gRPC listener. The app starts, serves traffic, and exits 0 while every export goes into a void. The gRPC exporter does log the failure (StatusCode.UNAVAILABLE ... Expected SETTINGS frame as the first frame), but you only see it once something has put a console handler back on the root logger, which is whatOTEL_PYTHON_LOG_CORRELATION=truebelow does. Without that variable the failure is silent.
Three more things about that block:
OTEL_EXPORTER_OTLP_ENDPOINTtakes the base URL. The SDK appends/v1/traces,/v1/metrics, and/v1/logsitself, which is exactly how TracePath's ingest paths are laid out. A full signal path here produces/v1/traces/v1/traces.OTEL_SERVICE_NAMEbecomes the Server Name on every endpoint, task, and issue.- There is no
OTEL_SERVICE_VERSIONvariable. Version travels inOTEL_RESOURCE_ATTRIBUTES, and without it every endpoint row carries an empty App Version, so release comparison does not work.
Run
Put opentelemetry-instrument in front of whatever starts your process.
# FastAPI, Starlette, any ASGI app
opentelemetry-instrument uvicorn app:app --port 8000
# Flask development server
opentelemetry-instrument flask --app wsgi run --port 8000
# Any WSGI server, same shape
opentelemetry-instrument gunicorn wsgi:app --bind 0.0.0.0:8000
# A worker, cron job, or one-shot script
opentelemetry-instrument python worker.pyStarting the server directly, without the opentelemetry-instrument prefix, sends nothing at all. That is the single most common reason a Python integration stays silent.
For short-lived scripts, the last batch flushes on exit either way. opentelemetry-instrument registers a shutdown hook, and a hand-built provider registers its own atexit hook by default. Only an abrupt end such as os._exit or SIGKILL loses the batch.
What Arrives With No Application Code
| TracePath | Where it comes from |
|---|---|
| Endpoints, grouped by route pattern | opentelemetry-instrumentation-fastapi / -flask / -starlette set http.route from the matched route |
| Status codes, including 500 on an unhandled exception | The ASGI and WSGI instrumentation |
| Child Spans for database and outgoing HTTP calls | -sqlite3, -psycopg, -requests, -httpx, -urllib3 |
| Issues with the full Python traceback | Unhandled exceptions, recorded as exception span events |
| Logs, linked to the trace that produced them | -logging bridges the standard logging module |
| HTTP Metrics | http.server.duration, http.server.active_requests, http.server.response.size, http.client.duration |
Verify Endpoint Grouping First
Hit one parametrized route three times with different ids, then open Endpoints. You must see one row with a count of 3, not three rows.
| Framework | Route in your code | Endpoint row |
|---|---|---|
| FastAPI | @app.get("/users/{user_id}") | GET /users/{user_id} |
| Flask | @app.get("/orders/<order_id>") | GET /orders/<order_id> |
The pattern syntax is the framework's own, and TracePath stores whatever http.route carried. Those two rows look different and both are correct.
Three rows with literal ids in them (GET /users/101, GET /users/102) means http.route is not being set, so TracePath fell back to the concrete url.path. Check that the app really started under opentelemetry-instrument and that opentelemetry-bootstrap -a install ran after your web framework was installed.
Exceptions
An unhandled exception needs no configuration. FastAPI, Starlette, and Flask all respond 500 on their own, the ASGI/WSGI instrumentation records the exception event, and TracePath shows the Issue with the full traceback and the endpoint row with status 500.
@app.get("/boom")
def boom():
raise ValueError("exploded on purpose")Only catch-and-handle paths need code. record_exception fills in exception.type, exception.message, and exception.stacktrace for you, and set_status takes a Status object rather than a dict:
from opentelemetry import trace
from opentelemetry.trace import Status, StatusCode
span = trace.get_current_span()
try:
charge_card()
except PaymentError as error:
span.record_exception(error)
span.set_status(Status(StatusCode.ERROR, str(error)))
raiseIf you swallow the error and still return 200, the endpoint row is recorded as a success and the Issue arrives with no failing request beside it. Return 500 whenever you record an exception.
Logs
opentelemetry-bootstrap -a install installs opentelemetry-instrumentation-logging, which attaches the OTel handler to the root logger. Every logger.info(...) and logger.error(...) in your code, your framework, and your libraries is then forwarded to TracePath and stamped with the active trace and span ids.
Two Python behaviours decide what you actually see:
- Python's root logger defaults to
WARNING. A logger that inherits that level drops its own INFO records before the OTel handler ever runs, so only WARN, ERROR, and CRITICAL reach TracePath. Since most application logging islogger.infoon an inheriting logger, the practical result is "my logs do not show up" with no error anywhere. Libraries that set their own level, such as Werkzeug at INFO, are unaffected and keep arriving. - The OTel handler becomes the root logger's only handler. The root logger normally has none, so Python falls back to printing WARNING and above on stderr. Adding the OTel handler ends that fallback: your records go to TracePath and stop appearing on stdout. On a server whose runbook is
docker logs, that reads as the app having stopped logging.
OTEL_PYTHON_LOG_CORRELATION=true fixes both. It calls logging.basicConfig(), which restores a console handler and sets the root level, and it stamps otelTraceID / otelSpanID onto every record so your own log lines carry the trace id too:
export OTEL_PYTHON_LOG_CORRELATION=true
export OTEL_PYTHON_LOG_LEVEL=infoOTEL_PYTHON_LOG_LEVEL on its own does nothing. It is only read inside the correlation branch, so without OTEL_PYTHON_LOG_CORRELATION=true the root logger stays at WARNING. With correlation on and no level set, the level defaults to info. Setting both is the readable form.
The equivalent in code, if you would rather not use the variables, is logging.basicConfig(level=logging.INFO) early in your app.
With both set, a handler like this lands in TracePath and on the endpoint's Logs card:
import logging
logger = logging.getLogger(__name__)
@app.get("/users/{user_id}")
def get_user(user_id: str):
logger.info("looking up user %s", user_id)
return {"id": user_id}Structured attributes go through extra, and dotted keys match the OTel semantic conventions:
logger.info("order received", extra={"order.id": "ord_123"})Do not set
OTEL_PYTHON_LOGGING_AUTO_INSTRUMENTATION_ENABLED=true. On current versions the bridge is on by default, and that variable switches it to the SDK's deprecated handler. The instrumentation then logs a duplicate-handler warning, which is itself ingested as a log record.
Background Tasks
A cron job or queue worker becomes a Task only when its root span has SpanKind.CONSUMER. A root span with the default INTERNAL kind and no HTTP attributes is discarded on ingest, and TracePath still answers 200 OK, so a plain start_as_current_span("my-job") gets you silence.
In Python the kind is a keyword argument, not an options object:
from opentelemetry import trace
from opentelemetry.trace import SpanKind, Status, StatusCode
tracer = trace.get_tracer("my-app")
def run_scheduled_job():
with tracer.start_as_current_span(
"cleanup-expired-sessions", kind=SpanKind.CONSUMER
) as span:
try:
do_work()
span.set_status(Status(StatusCode.OK))
except Exception as error:
span.record_exception(error)
span.set_status(Status(StatusCode.ERROR, str(error)))
raiseRun it as opentelemetry-instrument python worker.py. The span name is the task name and the grouping key, so keep it stable and put job ids, batch sizes, and timestamps in span attributes instead.
Custom Metrics
Under opentelemetry-instrument the meter provider is already configured, so ask the global API for a meter and record:
from opentelemetry import metrics
meter = metrics.get_meter("my-app")
sessions_cleaned = meter.create_counter(
"app.sessions_cleaned",
unit="1",
description="expired sessions removed by the cleanup job",
)
sessions_cleaned.add(7, {"job": "cleanup-expired-sessions"})
checkout_latency = meter.create_histogram("app.checkout.latency", unit="ms")
checkout_latency.record(12.5, {"tier": "pro"})The counter appears in metric discovery as app.sessions_cleaned, with job as a tag you can group by. A histogram arrives split in two, app.checkout.latency.avg and app.checkout.latency.count, which is how TracePath stores every histogram. See Metrics.
Metrics export every 60 seconds by default. That is the SDK default for
OTEL_METRIC_EXPORT_INTERVAL, so a working pipeline still looks empty for the first minute. SetOTEL_METRIC_EXPORT_INTERVAL=10000while you are verifying the integration.
Child Spans and Database Calls
Database and outgoing HTTP calls appear as child spans on the endpoint's waterfall, named after the SQL statement or the request. One detail costs people an afternoon:
The DB-API instrumentation traces cursor calls.
connection.cursor().execute(...)produces a child span named after the SQL. Theconnection.execute(...)shortcut bypasses the traced cursor proxy and produces no span at all. If your queries are missing from the waterfall while your outgoing HTTP calls are there, that is usually why.
For business logic that no instrumentation covers, start your own child span. It stays attached to the request because it runs inside the active context:
from opentelemetry import trace
tracer = trace.get_tracer("my-app")
with tracer.start_as_current_span("rebuild-price-index"):
rebuild()Verify It Worked
Telemetry is batched, so give traces about 10 seconds and metrics up to a minute. Set the dashboard time picker to the last 15 minutes, then check:
- Endpoints. Three requests to a parametrized route give one row with count 3. Three rows means
http.routeis missing. - Issues. A route that raises shows up with its Python traceback, and its endpoint row reports status 500.
- Spans. Open that endpoint and confirm your database and outgoing HTTP calls are in the waterfall.
- Logs. A
logger.info(...)from inside a handler appears on the endpoint's Logs card, on the This Trace tab. If only WARN and above are there,OTEL_PYTHON_LOG_CORRELATIONis not set. - Tasks. A job run under
opentelemetry-instrumentwithkind=SpanKind.CONSUMERappears by its span name. - Metrics.
http.server.duration.avgand your custom metric names appear in metric discovery.
Nothing at all in any of the six, with the app running and no error printed? Check OTEL_EXPORTER_OTLP_PROTOCOL=http/protobuf first. Then set OTEL_PYTHON_LOG_CORRELATION=true and restart. Exporter failures are logged at WARNING and ERROR, but the OTel handler is the only handler on the root logger until something calls logging.basicConfig(), so a broken exporter swallows its own error report. Correlation restores the console handler and the failures appear on stderr. OTEL_LOG_LEVEL does not help here: opentelemetry-python declares the variable but no released package reads it.