Tracing and monitoring

Tracing is a critical feature in the aiXplain platform that enables developers to inspect and debug the internal workings of agents and team agents. Tracing allows visibility into execution flows, memory states, tool usage, role-based actions, and intermediate outputs. This document describes how to use tracing effectively to understand and improve agent behavior.

Why use tracing?

Tracing is essential for:

• Debugging incorrect or unexpected outputs
• Auditing agent decisions and reasoning steps
• Optimizing agent performance and cost
• Explaining agent actions to stakeholders or end-users
• Monitoring for regressions or failures in production

Team Agent vs. Agent

Tracing single agents and team agents follows a similar structure, but there are key distinctions based on the design and behavior of each. Below are the main aspects you'll notice when reviewing their traces:

Aspect	Team Agent	Agent
Agent architecture	Multiple specialized agents with distinct roles	One agent handling the entire task
Workflow structure	Complex orchestrated workflow with handoffs	Linear execution flow with no handoffs
Quality assurance	Built-in inspection and feedback correction loops	No internal validation mechanisms
Resource efficiency	Higher resource usage (API calls, runtime, credits)	Lower resource usage, faster execution
Output depth	More comprehensive, nuanced results	Simpler, more direct results
Error Recovery	Can detect and correct errors mid-execution	No mid-process error correction
Execution complexity	Multi-step process with planning and coordination	Simple one-step execution
Instrumentation	Detailed execution tracking and performance metrics	Minimal execution metadata
Task specialization	Different agents optimized for specific subtasks	One agent performing all aspects of the task
System overhead	Additional coordination and communication overhead	No coordination overhead

Understanding team agent execution traces

This guide helps developers interpret the structure and meaning of a team agent's execution trace, retrieved using .run() followed by inspecting response.data.

Example use case

query = "Identify potential leads in the EdTech industry for 'AI for Personalized Learning'..."
response = team_agent.run(query)
print(response.data)

The trace gives you:

• A full view of all member agents invoked (Lead Finder, Lead Analyzer, etc.)
• How long each took
• How much each cost
• Which tools were used
• A step-by-step chain of reasoning

Use this trace to:

• Debug unexpected outputs
• Optimize performance and credit usage
• Compare agent plans vs. actual execution
• Generate user-facing activity logs

---

Top-level structure

Each team agent trace includes the following top-level fields:

{
  "input": "...",
  "output": "...",
  "feedback": "...",
  "session_id": "...",
  "intermediate_steps": [...],
  "runTime": 0,
  "usedCredits": 0,
  "executionStats": {...},
  "plan": [...]
}

---

Field-by-field explanation

input

The original query input to the team agent, typically passed as a string but wrapped into JSON internally. Example:

{
  "input": "Identify potential leads in the EdTech industry for 'AI for Personalized Learning'...",
  "chat_history": [],
  "outputFormat": "text"
}

output

The final result generated by the team agent. This is the direct answer to the user query.

---

feedback

Currently a placeholder for user feedback or agent self-assessment. May be empty ("").

---

session_id

A UUID representing the session. Useful for traceability, debugging, and correlating logs.

---

intermediate_steps

A list of execution steps taken by each agent in the system.

Each item contains:

• agent: Name of the agent that executed this step
• input: What the agent received
• output: What the agent produced
• tool_steps: Details of tools used (can be null or an array)
• thought: Optional planning or reasoning step
• runTime: Time taken in seconds
• usedCredits: Cost of execution in credits
• apiCalls: Number of model/tool API calls
• task: (Optional) The named task the step was solving

Example snippet:

{
  "agent": "Lead Finder",
  "input": "Lead Finder,...",
  "output": "Here is a list of potential leads...",
  "tool_steps": [...],
  "thought": null,
  "runTime": 13.836,
  "usedCredits": 0.09058,
  "apiCalls": 5,
  "task": null
}

---

runTime

The total wall-clock time of the entire execution (in seconds).

---

usedCredits

Total cost of execution in credits. This may be 0 in some SDK configurations but is available in executionStats.credits.

---

executionStats

Detailed execution summary. Key fields:

{
  "status": "SUCCESS",
  "apiCalls": 22,
  "credits": 0.3318,
  "runtime": 57.086,
  "apiCallBreakdown": {...},
  "runtimeBreakdown": {...},
  "creditBreakdown": {...},
  "assetsUsed": [...],
  "sessionId": "...",
  "environment": "prod",
  "timeStamp": "2025-04-05 06:32:34.679979"
}

apiCallBreakdown

How many API calls each agent/tool made.

{
  "orchestrator": 3,
  "Lead Finder": 11,
  "inspector_0": 3,
  ...
}

runtimeBreakdown

Runtime per agent/tool in seconds.

{
  "orchestrator": 8.881,
  "Lead Finder": 33.827,
  ...
}

creditBreakdown

Cost (credits) incurred per agent/tool.

{
  "Lead Finder": 0.26206,
  "Lead Analyzer": 0.00928,
  ...
}

---

plan

Shows the predefined steps that the team agent was expected to follow. Each item includes:

{
  "step": "Task: find leads...",
  "worker": "FINISH",
  "init_worker": "Lead Finder"
}

This helps in understanding how the agent decomposed the task and which agent was assigned each part.

---

Use cases for developers

• Debugging tool behavior: Trace failures or unexpected output via intermediate_steps.
• Latency analysis: Use executionStats.runTime and per-step durations to identify bottlenecks.
• Usage tracking: Leverage apiCalls, toolCount, and usedCredits for reporting and optimization.
• QA feedback loop: feedback field enables closing the loop for agent performance evaluation.
• Session continuity: sessionId ties the trace to a broader chat or API session for context and personalization.