LLM Tracing Setup Generator

Generate OpenTelemetry tracing for OpenAI, Anthropic, LangChain, LlamaIndex or Vercel AI SDK apps, with env vars for Langfuse, Phoenix, Honeycomb or Datadog.

The interactive tool loads here when JavaScript runs.

About this tool

What is the LLM Tracing Setup Generator?

Pick your language, LLM SDK or framework, tracing backend and privacy options, and this generator writes the install command, the environment variables and the setup file that make every model call show up as an OpenTelemetry span. It covers the OpenAI and Anthropic SDKs, LangChain and LangGraph, LlamaIndex, the Vercel AI SDK and hand-written agent loops, in Python or TypeScript.

It also produces a manual span example for an agent run (one invoke_agent span, a chat span per model call and an execute_tool span per tool call) and a reference table of the gen_ai.* attributes with their current stability status. Everything is generated in your browser from templates; no API keys are asked for and nothing is sent anywhere.

  • Python and TypeScript setup for six frameworks
  • OpenTelemetry GenAI, OpenInference and OpenLLMetry options with automatic fallback
  • Env vars for six backends, including sampling and environment
  • Content capture switch per library
  • OpenTelemetry Collector config that masks emails and API keys
  • Agent loop span example and a gen_ai.* attribute table
Read the guide: Tracing LLM and Agent Apps with OpenTelemetry, Plus Audit Logs

Updated

How to use the LLM Tracing Setup Generator

  1. Choose Python or TypeScript and the SDK or framework your app calls.
  2. Pick an instrumentation library. If the one you choose does not cover that framework, the generator switches and says why.
  3. Choose where traces go: a local OTLP collector, Langfuse, Arize Phoenix, Honeycomb, Grafana Cloud or Datadog.
  4. Set the service name, environment and sampling rate, and decide whether to capture prompt and response content and mask emails and keys.
  5. Copy the install command, add the environment variables to your deployment and import the setup file before any LLM client is created.
  6. Open the Agent spans tab if you run your own agent loop, and the reference tab to check attribute names.

Worked examples

Python OpenAI app to Langfuse

Input

Python, OpenAI SDK, OpenTelemetry GenAI, Langfuse, 25% sampling, content off

Output

pip install opentelemetry-sdk opentelemetry-exporter-otlp-proto-http opentelemetry-instrumentation-genai-openai, OTEL_EXPORTER_OTLP_ENDPOINT=https://cloud.langfuse.com/api/public/otel, OTEL_TRACES_SAMPLER_ARG=0.25

Sampling and the service name live in environment variables, so the same code runs in every environment.

TypeScript LangChain to Honeycomb

Input

TypeScript, LangChain, OpenTelemetry GenAI

Output

Switches to @arizeai/openinference-instrumentation-langchain with manuallyInstrument(CallbackManagerModule)

OpenTelemetry JS has an official instrumentation only for the OpenAI SDK, so the generator picks one that exists.

Background and insight

OpenTelemetry defines semantic conventions for generative AI so that traces from different libraries look alike: the span name is the operation and model (chat gpt-4.1), and attributes such as gen_ai.provider.name, gen_ai.request.model, gen_ai.usage.input_tokens and gen_ai.response.finish_reasons carry the details. In 2026 these conventions moved from the main semantic-conventions repository to their own semantic-conventions-genai repository, and every gen_ai.* attribute is still in Development status, which means names can change. One rename already caught many teams out: gen_ai.system became gen_ai.provider.name in version 1.37.0.

Three families of instrumentation libraries emit these spans. The OpenTelemetry project publishes opentelemetry-instrumentation-genai-openai, -anthropic, -langchain and -llama-index for Python (they replaced opentelemetry-instrumentation-openai-v2) and @opentelemetry/instrumentation-openai for Node. Arize maintains OpenInference, which adds its own attributes that Phoenix reads, and Traceloop maintains OpenLLMetry. Prompt and response text is opt-in in the OpenTelemetry libraries: Python uses OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT with no_content, span_only, event_only or span_and_event, while the Node OpenAI instrumentation takes true or false. OpenLLMetry records content unless TRACELOOP_TRACE_CONTENT=false.

Common use cases

  • Debugging a slow or wrong answer

    See each model call, its token counts and finish reason, and which tool call returned bad data.

  • Cost and latency dashboards

    Group gen_ai.usage tokens and span durations by model, route or customer.

  • Adding tracing to an agent

    Wrap a custom loop in invoke_agent, chat and execute_tool spans so runs read as a tree.

  • Switching observability vendors

    Keep the OpenTelemetry code and change only the OTLP endpoint and headers.

Tips and limitations

  • Keep content capture off in production unless you have a reason and a retention plan. Token counts, models and timings answer most questions.
  • Pin instrumentation versions. The GenAI conventions are still in Development, so attribute names can shift between releases.
  • Two packages called opentelemetry-instrumentation-anthropic exist in spirit: the official one is opentelemetry-instrumentation-genai-anthropic, the other is Traceloop's. Check the publisher.
  • Import the setup file before creating LLM clients. In Node ESM apps, start with node --import ./instrumentation.js.
  • Short scripts and serverless functions must flush spans before exit, or the last batch is lost.
  • Regex masking in the collector catches common email and key formats only. Treat it as a safety net, not a guarantee.

Frequently asked questions

What replaced gen_ai.system in OpenTelemetry?

gen_ai.provider.name, from semantic conventions 1.37.0. Some older libraries still send gen_ai.system, and backends such as Datadog expect the newer name.

How do I stop prompts and responses being stored in my traces?

For the OpenTelemetry Python libraries set OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT=no_content (the default); for OpenInference set OPENINFERENCE_HIDE_INPUTS and OPENINFERENCE_HIDE_OUTPUTS to true; for OpenLLMetry set TRACELOOP_TRACE_CONTENT=false.

Is opentelemetry-instrumentation-openai-v2 still the package to use?

It has been superseded by opentelemetry-instrumentation-genai-openai in the opentelemetry-python-genai repository. The generator uses the new package and import path.

Which tracing backends accept these spans?

Anything that speaks OTLP. The generator has ready settings for a local collector, Langfuse, Arize Phoenix, Honeycomb, Grafana Cloud and Datadog LLM Observability.

How should spans be nested for an AI agent?

One invoke_agent span per run; inside it a chat span for each model call and an execute_tool span for each tool call, as siblings in the order they happen.

Does tracing slow down my LLM calls?

Barely. Spans are batched and exported in the background, and the model call itself takes far longer. Capturing large prompts adds memory and export size, not latency to the call.

Does this tool need my API keys?

No. It writes placeholders such as <HONEYCOMB_API_KEY> for you to fill in your own deployment. Nothing you type leaves the browser.

Is there an in-depth guide to the LLM Tracing Setup Generator?

Yes. “Tracing LLM and Agent Apps with OpenTelemetry, Plus Audit Logs” covers the background, how it works, common pitfalls and worked examples. Read the full guide.

Related Tools You May Like

Free, in-browser tools that pair well with this one.