Agents don’t fail the way chatbots do. They call the wrong tool, silently retry three times, or burn through your token budget mid-conversation, and a normal log file won’t tell you why. This guide sets up real AI agent observability that shows every model call, tool invocation, retrieval step, retry, and error your agent produces.

This guide isn’t about tracing one simple LLM call. It’s about a real agent that calls several tools one after another, sometimes fails along the way, and only makes sense when you can see how each step connects to the one before it.

Why AI Agent Observability Matters for Multi-Step Agents

A single LLM call includes prompt in, response out. An agent is not. It calls a lookup tool, gets a small result, retries, calls a second tool, then feeds both results back into the model for a final answer.

If any one of those steps breaks, a plain-text log shows you a stack trace with no context about what the agent was actually doing at that moment.

This is the problem AI agent observability solves. Instead of guessing, you open a trace and see the whole run laid out, including which tool ran, what it was given, how long it took, whether it failed, and what happened just before and after.

With good AI agent observability, you’re not stuck digging for hours anymore. The proof of what went wrong is already there, laid out in order with timestamps, so finding the problem takes minutes instead.

What You Need Before Starting

You’ll need two machines for this AI Agent observability setup. You can run the in one server while testing, but it should be separate in production:

• A telemetry server: any small Linux VPS to run Arize Phoenix.
• An agent runtime: a GPU server if you’re self-hosting a model, or any VPS if you’re calling an API like OpenAI.
• Docker and Docker Compose are installed on the telemetry server.
• Python 3.10+ running on the agent runtime.
• An OpenAI API key or any provider.

Keeping telemetry on its own machine matters once your AI agent observability setup goes into production. Your agent shouldn’t have to share CPU or memory with tracing, and if the agent crashes, you don’t want your trace history to go down with it.

How OpenTelemetry, OpenInference, and Phoenix Fit Together

Three tools work together here:

• OpenTelemetry (OTel) is the open standard that generates and sends traces from any application.
• OpenInference builds on top of it with naming rules made specifically for LLM and agent data, labels like input.value, llm.token_count.total, and openinference.span.kind, which marks whether a trace is an LLM call, a tool call, or a retriever.
• Arize Phoenix is the dashboard that receives these traces and displays them as a clear trace tree you can actually read.

In practice, your code creates traces, OpenInference conventions describe what’s inside them, and Phoenix is where you go to actually look at the result.

Step 1. Deploy Arize Phoenix

For quick local testing, you can run Phoenix with two Docker commands on your telemetry server:

docker pull arizephoenix/phoenix:latest
docker run -d --name phoenix --restart unless-stopped \
  -p 6006:6006 -p 4317:4317 \
  arizephoenix/phoenix:latest

Port 6006 serves the web UI and the OTLP HTTP collector, and 4317 handles OTLP gRPC traffic. Open http://telemetry-server-ip:6006; you should see an empty Phoenix dashboard, ready to collect traces.

This is fine for testing, but it has no persistent storage, authentication, or TLS. For a full setup, you can check this guide on how to self-host Arize Phoenix with Docker, which covers Postgres-backed storage, HTTPS via Caddy, and firewall rules, all things a real AI agent observability setup needs.

Step 2. Set Up the Agent Environment

On your agent runtime, create a dedicated folder and a clean virtual environment so this project doesn’t clash with anything else on the server:

mkdir -p ~/agent-tracing && cd ~/agent-tracing
python3 -m venv venv
source venv/bin/activate

Install everything the agent and its tracing layer need:

pip install --upgrade pip
pip install openai arize-phoenix-otel openinference-instrumentation-openai \
    opentelemetry-sdk opentelemetry-exporter-otlp tenacity

Verify everything works correctly:

python -c "import phoenix.otel; import openai; print('OK')"

If that prints OK, your environment is ready for the actual AI agent observability wiring in the next steps.

Now point the agent at your Phoenix server using environment variables, so nothing is hardcoded into your Python files:

export PHOENIX_COLLECTOR_ENDPOINT="http://<telemetry-server-ip>:4317"
export OPENAI_API_KEY="sk-your-key-here"

Step 3. Register the Tracer

Still in the virtual environment shell, create a file that configures OpenTelemetry once, and import it everywhere else:

nano ~/agent-tracing/tracing.py

from phoenix.otel import register
from openinference.instrumentation.openai import OpenAIInstrumentor

tracer_provider = register(
    project_name="multi-step-agent",
    auto_instrument=True,
)

OpenAIInstrumentor().instrument(tracer_provider=tracer_provider)
tracer = tracer_provider.get_tracer(__name__)

register() reads PHOENIX_COLLECTOR_ENDPOINT and builds the OTLP exporter automatically.

OpenAIInstrumentor connects into every OpenAI call so prompts, completions, model name, and token counts are captured without extra code.

The tracer object you exported here is what wraps your custom tool functions. It’s where real AI agent observability actually gets built.

Step 4. Build a Multi-Step Agent

Still in the virtual environment shell, create the agent file:

nano ~/agent-tracing/agent.py

This agent looks up an order, retries if needed, then calls a refund tool that depends on the first tool’s result, a multi-step and dependent chain.

import time
import random
import json
from tenacity import retry, stop_after_attempt, wait_exponential
from opentelemetry import trace
from opentelemetry.trace import Status, StatusCode
from openai import OpenAI
from tracing import tracer

client = OpenAI()


@retry(stop=stop_after_attempt(3), wait=wait_exponential(min=1, max=4))
def lookup_order(order_id: str) -> dict:
    with tracer.start_as_current_span(
        "tool.lookup_order",
        attributes={
            "openinference.span.kind": "TOOL",
            "tool.name": "lookup_order",
            "input.value": order_id,
        },
    ) as span:
        span.add_event("attempt_start")
        start = time.time()

        if random.random() < 0.4:
            span.record_exception(ConnectionError("order service timeout"))
            span.set_status(Status(StatusCode.ERROR))
            raise ConnectionError("order service timeout")

        result = {"order_id": order_id, "status": "delivered", "amount": 49.99}
        span.set_attribute("output.value", json.dumps(result))
        span.set_attribute("tool.latency_ms", (time.time() - start) * 1000)
        return result


def process_refund(order: dict) -> dict:
    with tracer.start_as_current_span(
        "tool.process_refund",
        attributes={
            "openinference.span.kind": "TOOL",
            "tool.name": "process_refund",
            "input.value": json.dumps(order),
        },
    ) as span:
        refund = {"order_id": order["order_id"], "refunded_amount": order["amount"]}
        span.set_attribute("output.value", json.dumps(refund))
        return refund


def run_agent(user_message: str):
    with tracer.start_as_current_span(
        "agent.run",
        attributes={"openinference.span.kind": "AGENT", "input.value": user_message},
    ) as agent_span:
        response = client.chat.completions.create(
            model="gpt-4o-mini",
            messages=[
                {"role": "system", "content": "You handle refund requests. Extract the order ID."},
                {"role": "user", "content": user_message},
            ],
        )
        agent_span.set_attribute(
            "llm.token_count.total", response.usage.total_tokens
        )

        order_id = "ORD-1001"  # extracted from the model response in a real agent
        order = lookup_order(order_id)
        refund = process_refund(order)

        agent_span.set_attribute("output.value", json.dumps(refund))
        return refund


if __name__ == "__main__":
    result = run_agent("I need a refund for my last order")
    print(result)

Once you are done, save and close the file.

Then, run the script:

python agent.py

Step 5. Read the Traces in Phoenix

After running python agent.py, open your Phoenix URL:

http://<telemetry-server-ip>:6006

Click into the multi-step-agent project. You’ll see a trace tree per agent run:

• agent.run at the top, with total duration and the user’s input.
• tool.lookup_order nested underneath, repeated if it retried.
• tool.process_refund nested after it.
• Any red traces marking a failed attempt, with the exception message attached.

Click on any individual trace to see its full attributes panel, including inputs, outputs, latency, token counts, and, if it failed, the exact error and stack trace. This one screen is what most of your AI agent observability debugging will actually look like on a normal day.

Debugging Slow or Incorrect Tool Selection

Two problems come up constantly once agents go live:

• The agent picks the wrong tool,
• Or it’s just slow.

Phoenix helps with both.

If the agent picked the wrong tool: Open the agent.run trace and look at the input.value (the exact prompt sent to the model) next to the child tool traces that actually got called. If the model called process_refund before lookup_order ever succeeded, you’ll see it immediately in the tree order; no need to reconstruct the sequence from logs.

If the agent was slow: Sort your traces by duration in Phoenix and open the slowest ones. The timing breakdown tells you where the delay came from: a long model reply (check the token count, since longer completions take longer), a tool that had to retry (you’ll see it repeated), or a slow outside API (check that tool’s recorded latency).

A few extra tips that help once you’re past the first working trace:

• Filter traces in Phoenix by openinference.span.kind = TOOL to see only tool calls across every agent run.
• Add a session.id attribute to agent.run so you can group multiple turns of the same conversation.
• Set openinference.span.kind = RETRIEVER on any RAG lookup step, so retrieval shows up as its own category alongside tool calls.

Once your local test works, move the setup to real servers. If you’re running your own model with vLLM, Ollama, etc., put the agent on a GPU server so both the model and its tool calls stay fast. PerLod’s AI Hosting and GPU servers are built for this kind of work. Keep Phoenix on a separate, smaller VPS like before, since trace data keeps growing and shouldn’t eat the RAM or disk your model needs.

Splitting things this way is the standard setup for serious AI agent observability, because a sudden spike in traces never slows down your agent’s actual responses.

Conclusion

You now have a pipeline that catches everything your agent does, its model calls, tool calls, retries, speed, token use, errors, and how each step connects to the next.

That’s the full trace you need for real AI agent observability, and since it’s all built on open standards like OpenTelemetry and OpenInference, you’re not locked into any single vendor. From here, all that’s left is tracing each new tool you add and watching the trace grow along with your agent.

We hope you enjoy this guide.

For the full OpenInference semantic conventions used in the trace attributes above, see the OpenInference specification on GitHub.

FAQs