> ## Documentation Index
> Fetch the complete documentation index at: https://agentrail.co/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Python SDK reference

> Every option of agentrail-python, notes for ASGI, WSGI and worker processes, how to add it to an existing OpenTelemetry setup, how delivery works, and what each capture error means.

This page is the reference for `agentrail-python`. To install it, start with [Install the Python SDK](https://agentrail.co/docs/guides/journeys-python.md). It needs Python 3.10 or later.

## Options

`init` and `create_processors` take the same `Options`, as keyword arguments. Invalid options, identities and milestone labels raise `ConfigurationError`. The SDK never reads environment variables itself, so check yours before calling them.

| Option | Required | What it does |
| - | - | - |
| `collector` | Yes | `CollectorOptions(endpoint=..., secret=...)`: the collector's address, ending in `/ingest/otlp`, and your server source's secret. |
| `hash_key` | Yes | Your workspace's hash key. Identities and operation references are hashed with it before they leave your server. |
| `service_name` | Yes | A stable name for this service, such as `orders-api`. |
| `service_instance_id` | No | This process's id. A random id by default. |
| `outbound_origins` | No | The exact origins, such as `https://inventory.example.com`, that receive your trace context. None by default. |
| `ignore_incoming_request` | No | `(IncomingRequest) -> bool`. A request it returns `True` for is not recorded. See [Leave out your own servers](https://agentrail.co/docs/guides/journeys-own-servers.md). |
| `incoming_location` | No | `(IncomingRequest) -> IPLocation \| None`. An approximate location for each request. See [Add an approximate location](https://agentrail.co/docs/guides/journeys-python.md#add-an-approximate-location). |
| `logs` | No | `LogsOptions(include_fields=(), exclude_uncorrelated=False)`. See [Logs](#logs). |
| `queue` | No | `QueueOptions(...)`. See [Delivery and shutdown](#delivery-and-shutdown). |
| `propagator` | No | Your existing propagator, used for work outside recorded requests. |
| `on_error` | No | `(CaptureError) -> None`. Told of every capture problem. Writes to standard error by default. |

The hooks receive an `IncomingRequest` with `method`, `target` (the path and query as sent), `scheme`, `host`, `headers` (lowercase names; a repeated header's values joined with `", "`), `client` (the address the server saw, not a forwarded header) and `raw` (the ASGI scope or WSGI environ).

## How `init` starts

`init` installs a tracer provider, a logger provider and a propagator for the process. It refuses to replace a provider that is already installed, such as one from Sentry, Datadog or your own setup: use [`create_processors`](#an-existing-opentelemetry-setup) instead. Calling `init` again with the same `Options` object returns the same client. With `init`, what is queued is also sent when the interpreter exits.

## ASGI and WSGI

* **ASGI** (FastAPI, Starlette, Quart, Litestar, Django ASGI): each request records its route template, such as `/orders/{order_id}/confirm`.
* **WSGI** (Flask, Django WSGI, Falcon, Pyramid, Bottle): requests are recorded without a route template, because the framework clears it before the middleware can read it. A streamed response runs in its request, so what it records while streaming joins that request.
* **Exact paths under WSGI.** A signed request from the edge is checked against the exact path and query it was sent with. Gunicorn, uWSGI and Werkzeug pass that on (`RAW_URI` or `REQUEST_URI`). Under a server that does not, the path is rebuilt from the decoded one, and a signed request to a path with percent-encoding may start a trace of its own.

Install the middleware once, around the part of your application you want recorded. Installed after your authentication middleware, authentication itself is not recorded. Use it in place of any other incoming HTTP instrumentation, so each request has one entry.

## Async code, threads and worker processes

* The current request follows your code through `await`, asyncio tasks and `asyncio.to_thread`.
* A thread you start yourself does not inherit it. Run its work with `contextvars.copy_context().run(...)`.
* **Worker processes.** If your server loads the application before it forks its workers, as Gunicorn's `--preload` does, each worker starts its own delivery. A worker sends only what it recorded itself.

## Outgoing calls

`transport(agentrail, base)` wraps your own `httpx.HTTPTransport`, for proxies, retries or certificates, and `async_transport` does the same for `httpx.AsyncClient`. `adapter(agentrail, **kwargs)` takes the same arguments as `requests`' `HTTPAdapter`. A `requests` call's step ends when `send` returns, so with `stream=True` it does not wait for the body.

An origin in `outbound_origins` is a lowercase scheme and host, with its port when it is not the default. The address a call connects to decides, not a `Host` header, and each redirect is checked again. Calls made outside a request or milestone pass through untouched.

## Requests from the edge

With [`agentrail-edge`](https://agentrail.co/docs/guides/journeys-edge) in front of your service, the edge signs each request it forwards. A request with a valid signature continues the edge's trace, so your server's steps join the visit the edge recorded. A direct call, or one whose signature fails, starts a trace of its own, and a trace context the caller sent is kept only as a link.

The signature covers the method, the exact path and query, and the time it was made, and is accepted for 60 seconds. So give the edge and every service the same workspace hash key, keep your servers' clocks synchronized, and rewrite a path or query on the edge before it signs. A recorded call never forwards `traceparent`, `tracestate`, `baggage` or `sentry-trace` headers it did not set itself.

## Identity and milestones

`identify` works inside a request or milestone. Outside one, it reports `identity_outside_request` and records nothing. The first identity in a request stands: repeating it does nothing, and a different one is recorded as its own step.

A milestone's name and kind are 1 to 256 characters. A milestone's exception is recorded by its class name only, and raised unchanged. Outside a request, a milestone starts a trace of its own.

## Logs

The `AgentrailHandler` sits beside your other logging handlers. By default, a log sends its severity, event name, the exception's class name and the request and span it was written in.

Message text and other fields are sent only when you name them, as in `LogsOptions(include_fields=["message", "order_status"])`. `message` is the formatted message. Only named top-level fields with text, number, boolean or null values are sent, in the order named, as a JSON object of at most 4,096 characters; a long value is shortened to fit, and a field that cannot fit is left out. Set `exclude_uncorrelated=True` to send only logs written inside a request or milestone. Never name fields that hold credentials, identities or customer content.

You can also record a log directly with `agentrail.log(...)`, or add `processors.log_processor` to your own `LoggerProvider`.

## MCP

The MCP integration works with the official `mcp` SDK 2.x and its `MCPServer`. Each call is recorded as an MCP step with its registered name, method, the client's name and version, the protocol version and whether it failed. A result with `isError` marks the step as failed and is returned unchanged.

Served behind the ASGI middleware, as `agentrail.asgi(server.streamable_http_app(stateless_http=True))`, each call nests under its HTTP request. This is the tested setup. A call that runs outside a recorded request, over stdio for example, starts a trace of its own. What a client says about itself is never a verified identity.

## An existing OpenTelemetry setup

If your process already owns a tracer provider, build Agentrail's parts into it when you construct it. A provider that is already running cannot recover the spans its sampler dropped.

```python theme={null}
from opentelemetry import propagate, trace
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.sampling import ParentBased, TraceIdRatioBased

import agentrail_python as ar

processors = ar.create_processors(
    ar.Options(..., propagator=propagate.get_global_textmap())
)
provider = TracerProvider(
    sampler=processors.sampler(ParentBased(TraceIdRatioBased(0.1))),
    resource=your_resource,
)
provider.add_span_processor(your_processor)
provider.add_span_processor(processors.span_processor)
trace.set_tracer_provider(provider)
propagate.set_global_textmap(processors.propagator)
your_logger_provider.add_log_record_processor(processors.log_processor)

agentrail = processors.bind(provider)  # once
```

* `create_processors` registers nothing globally, and never changes what your other processors and exporters receive.
* Pass your existing propagator as `propagator`, and register the returned `processors.propagator`. It hands work outside recorded requests to yours.
* The sampler keeps the spans of a recorded request your sampler would drop, as record-only, without changing your sampled flag. An exporter that sends only sampled spans behaves as before.
* `agentrail.shutdown()` never shuts down your provider.

This release is built on OpenTelemetry Python 1.45.

## Delivery and shutdown

Records wait in a bounded queue in memory, one for spans and one for logs, each sent by its own background thread. Nothing is written to disk.

| `QueueOptions` | Default | What it limits |
| - | - | - |
| `max_records` | 2048 | Records held per signal, the batch being sent included |
| `batch_size` | 128 | Records sent in one call |
| `scheduled_delay` | 1 s | Time between scheduled sends |
| `export_timeout` | 5 s | Time for one send, retries included |
| `flush_timeout` | 10 s | Time `flush()` may take |
| `shutdown_timeout` | 10 s | Time `shutdown()` may take |

One batch is sent at a time. A send the collector can retry is retried within `export_timeout`; a refusal, or a batch still failing, is dropped and reported.

* Stop accepting requests and let them finish, then call `agentrail.shutdown()`, or `await asyncio.to_thread(agentrail.shutdown)` from async code. It sends what is queued once, and is safe to call twice.
* `agentrail.flush()` waits until every record accepted so far has been tried. It does not mean every record arrived.
* `agentrail.diagnostics()` reports the records accepted, sent, dropped and still waiting, for spans and for logs.

## Capture errors

A capture problem never changes your application's response. Each goes to `on_error` as a `CaptureError` with a `code`, a `signal` (`traces`, `logs` or `context`) and a `count` of records, and never holds customer data. The callback runs outside capture, so what it logs is never recorded.

| Code | What happened |
| - | - |
| `queue_full` | The queue was full, and records were dropped. |
| `export_failed` | The collector refused a batch, or its retries ran out. |
| `deadline` | A send, `flush` or `shutdown` ran out of time. |
| `closed` | A record arrived after `shutdown`. |
| `handoff_refused` | A signed request from the edge did not verify, so a new trace started. |
| `entry_not_recording` | A sampler dropped a request that should have been recorded. |
| `location_failed` | `incoming_location` raised or returned an invalid location. |
| `ignore_failed` | `ignore_incoming_request` raised. The request was recorded. |
| `identity_outside_request` | `identify` was called outside a request or milestone. |
| `invalid_performance` | Workflow or plugin evidence was invalid and was left out. |

## What is sent

* Each request's and outgoing call's method, address without its query (except Agentrail's own `h` markers), route template, status, user agent and error class name.
* Supporting operations carry only their place in the trace, their timing, status and error class name.
* Hashed identities, milestone names and kinds, workflow and plugin evidence, and MCP client details.
* The service name and instance, the SDK's name and version, and the hash key's id, never the key.

Bodies, query values, headers, cookies, arguments, results, exception messages, raw identities, prompt text and resource addresses are never sent. Your service names, routes, milestone names and the fields you name are yours to choose, so keep them free of personal data. For every source, see [What Agentrail records](https://agentrail.co/docs/guides/journeys-data.md).


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.