> ## Documentation Index
> Fetch the complete documentation index at: https://docs.plaisolutions.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Agent Analytics

> Track conversations, executions, and performance metrics for your agents

If an agent's success rate drops or a user reports a wrong answer, Agent Analytics lets you drill into the exact conversation and execution log to see what happened and why, rather than guessing from aggregate numbers alone.

***

## What is Agent Analytics?

**Agent Analytics** is an **agent-level dashboard** accessible in agent settings. It shows:

* **Conversations** - All interactions with the agent
* **Execution Log** - Detailed trace of each execution
* **Performance Metrics** - Response time, success rate, etc.
* **Filters** - Search by date, member, status, etc.
* **Configuration Insights** - How the agent is performing

<Tip>
  Agent Analytics is INSIDE agent settings. For project-wide consumption and cost tracking, see [Monitoring](../concepts/monitoring.mdx).
</Tip>

***

## Accessing Agent Analytics

<Steps>
  <Step title="Go to Your Agent">
    Navigate to the agent in your project
  </Step>

  <Step title="Open Settings">
    Click the agent settings or configuration area
  </Step>

  <Step title="Find Analytics Tab">
    Look for "Analytics" or "Performance" tab
  </Step>

  <Step title="View Data">
    See conversations, logs, and metrics
  </Step>
</Steps>

***

## Conversations

### What Are Conversations?

A **conversation** is an interaction with your agent. It includes:

* Input from user/caller
* Agent's processing
* Agent's output/response
* Metadata (timestamp, duration, success status)

### Conversation List

Agent: Support Bot — recent conversations:

1. **Sep 7, 2:45 PM**
   * Input: "How do I reset my password?"
   * Status: Success ✅
   * Duration: 1.2 seconds
   * Tokens: 145 input + 67 output
   * Member: [alice@company.com](mailto:alice@company.com)
2. **Sep 7, 2:30 PM**
   * Input: "What are your hours?"
   * Status: Success ✅
   * Duration: 0.8 seconds
   * Tokens: 98 input + 45 output
   * Member: [bob@company.com](mailto:bob@company.com)
3. **Sep 7, 2:15 PM**
   * Input: "I need to talk to someone"
   * Status: Success ✅
   * Duration: 1.5 seconds
   * Tokens: 156 input + 89 output
   * Member: [charlie@company.com](mailto:charlie@company.com)

### Viewing Conversation Details

Click on a conversation to see:

**Conversation Details**

* **Input**: "How do I reset my password?"
* **Output (Agent Response)**: "You can reset your password by:
  1. Going to Settings
  2. Click 'Security'
  3. Choose 'Change Password'
  4. Follow the email confirmation link"
* **Metadata**:
  * Started: Sep 7, 2:45 PM
  * Completed: Sep 7, 2:45:01 PM
  * Duration: 1.2 seconds
  * Status: Success ✅
  * Input Tokens: 145
  * Output Tokens: 67
  * Model: Claude 3.5 Sonnet
  * Member: [alice@company.com](mailto:alice@company.com)

***

## Execution Logs

### What's in the Execution Log?

The **execution log** shows the detailed steps the agent took:

Execution log for Conversation #1234:

1. **Parse Input** (0.1s)
   * Input received: "How do I reset my password?"
   * Detected intent: password\_reset
   * Confidence: 0.98
2. **Search Datasources** (0.3s)
   * Searching "FAQ Database"
   * Found: 3 matching results
   * Selected: Top match (confidence: 0.95)
3. **Generate Response** (0.8s)
   * Model: Claude 3.5 Sonnet
   * Input tokens: 145
   * Output tokens: 67
   * Generated response
   * Applied guardrails: ✓
4. **Return Result** (0.02s)
   * Status: Success
   * Total duration: 1.22s

### Understanding Log Levels

| Level       | Meaning       | Info Shown                 |
| ----------- | ------------- | -------------------------- |
| **TRACE**   | Most detailed | Every tiny operation       |
| **DEBUG**   | Detailed      | Variable values, decisions |
| **INFO**    | Standard      | Main steps, results        |
| **WARNING** | Issues        | Potential problems         |
| **ERROR**   | Failures      | What went wrong            |

***

## Filtering Conversations

### Available Filters

<Tabs>
  <Tab title="By Date">
    * Today
    * Last 7 days
    * Last 30 days
    * Custom date range
  </Tab>

  <Tab title="By Member">
    * All members
    * Specific organization member
    * Multiple members (select multiple)
  </Tab>

  <Tab title="By Status">
    * All
    * Success
    * Failed
    * Timeout
    * Partial (incomplete)
  </Tab>

  <Tab title="By Performance">
    * All
    * Fast (`<1s`)
    * Normal (1-5s)
    * Slow (>5s)
    * Errors only
  </Tab>
</Tabs>

### Filter Examples

* **Example 1: Find all failures from last week**
  * Date: Last 7 days
  * Status: Failed
  * Result: 3 conversations
* **Example 2: Check specific member's usage**
  * Member: [alice@company.com](mailto:alice@company.com)
  * Date: Last 30 days
  * Result: 487 conversations
* **Example 3: Find slow responses**
  * Performance: Slow (>5s)
  * Date: Last 24 hours
  * Result: 12 conversations (need optimization)

***

## Performance Metrics

### Key Metrics Displayed

| Metric                  | What It Shows                   | Healthy Range     |
| ----------------------- | ------------------------------- | ----------------- |
| **Avg Response Time**   | How fast agent responds         | 0.5-2s            |
| **Success Rate**        | % of successful conversations   | >95%              |
| **Error Rate**          | % of failed conversations       | `<5%`             |
| **Avg Tokens/Conv**     | Resource usage per conversation | Project-dependent |
| **Total Conversations** | Usage volume                    | Project-dependent |

### Metrics Example

Agent: Support Bot (Last 7 days)

* **Performance Summary**:
  * Total Conversations: 1,245
  * Success Rate: 97.8% ✅
  * Error Rate: 2.2%
  * Avg Response Time: 1.3 seconds
  * Median Response Time: 0.9 seconds
  * Slowest Response: 15.2 seconds
  * Fastest Response: 0.2 seconds
  * Avg Tokens per Conversation: 245
  * Total Tokens Used: 305,025
* **Breakdown by Hour**:
  * Peak usage: 2-3 PM (234 conversations)
  * Off-peak: 3-4 AM (12 conversations)
  * Busiest day: Wednesday (201 conversations)

***

## Troubleshooting with Analytics

### Scenario: High Error Rate

**Problem**: Agent error rate 8% (should be `<5%`)

1. Go to Agent Analytics
2. Filter: Status = Failed
3. Review failed conversations
4. Identify pattern:
   * Most failures between 2-3 PM
   * Error type: "API Timeout"
   * External service overloaded
5. Solution: Add retry logic or use backup service

### Scenario: Slow Responses

**Problem**: Avg response time 4.2s (should be `<2s`)

1. Filter: Performance = Slow (>5s)
2. Review execution logs of slow conversations
3. Identified bottleneck:
   * Step 2: Datasource search taking 2.1s
   * Datasource too large
4. Solution:
   * Archive old data from datasource
   * Or use filters to narrow search
   * Retest - now 1.2s average

### Scenario: Unexpected Cost Increase

**Problem**: Agent using 50% more tokens than before

1. Filter: Date = Last 7 days (vs previous week)
2. Check metrics:
   * More conversations? No, about the same
   * Longer responses? Yes, avg 2x longer
   * Agent configuration changed
3. Investigation:
   * System prompt made more verbose
   * Datasource context increased
   * Each request now uses more tokens
4. Solution: Simplify prompt, reduce context

***

## Exporting Data

### Export Conversations

Export for external analysis:

```
Format: CSV

Conversation ID, Timestamp, Member, Input, Output, Duration, Tokens, Status
1234, 2024-09-07 14:45:00, alice@company.com, "How to reset?", "Steps are...", 1.2, 212, Success
1235, 2024-09-07 14:30:00, bob@company.com, "Help needed", "I can help with...", 0.8, 143, Success
1236, 2024-09-07 14:15:00, charlie@company.com, "Talk to person", "Transferring you...", 1.5, 245, Success
```

### Export Execution Logs

Get detailed logs for debugging:

```
Format: JSON or CSV

Includes all execution steps, timing, tokens, decisions, etc.
Useful for:
- Debugging issues
- Performance analysis
- Compliance/audit
- Machine learning training
```

### Automated Reports

Set up recurring reports:

* Daily performance summary (email)
* Weekly usage trends (PDF)
* Monthly cost breakdown (spreadsheet)
* Custom reports on schedule

***

## Member-Based Filtering

### Organization Member Access

Analytics can be filtered by which organization member used the agent:

Organization: Acme Corp

* **Members**:
  * [alice@company.com](mailto:alice@company.com)
  * [bob@company.com](mailto:bob@company.com)
  * [charlie@company.com](mailto:charlie@company.com)
  * [diana@company.com](mailto:diana@company.com)
* **Filter by member to see**:
  * Which conversations each member triggered
  * Each member's usage patterns
  * Peak usage times by person
  * Performance metrics per member
* **Example**:
  * alice: 245 conversations (19% of total)
  * bob: 189 conversations (15%)
  * charlie: 567 conversations (46%)
  * diana: 244 conversations (20%)

### Use Cases

* **Auditing** - Track who used agent and when
* **Compliance** - Verify authorized access only
* **Billing** - Charge back costs per team
* **Performance** - Compare agent usage across teams

***

## API Access

For programmatic access to analytics:

### Query Conversations via API

```
GET /api/agents/{agentId}/conversations
```

Query params:

* `start_date`: 2024-09-01
* `end_date`: 2024-09-07
* `member_id`: [alice@company.com](mailto:alice@company.com)
* `status`: success
* `limit`: 100

### Query Execution Logs via API

```
GET /api/conversations/{conversationId}/logs
```

Returns:

* All execution steps
* Timestamps
* Token usage
* Errors/warnings
* Full trace

### Get Metrics via API

```
GET /api/agents/{agentId}/metrics
```

Query params:

* `start_date`: 2024-09-01
* `end_date`: 2024-09-07
* `granularity`: hourly|daily|weekly

Returns:

* Success rate
* Avg response time
* Error rates
* Token usage
* Conversation count

***

## Best Practices

<CardGroup cols={2}>
  <Card title="Check Weekly" icon="calendar-week">
    Review analytics weekly to catch issues early
  </Card>

  <Card title="Export Key Data" icon="download">
    Export metrics for trending and compliance
  </Card>

  <Card title="Set Baselines" icon="target">
    Know normal performance to spot anomalies
  </Card>

  <Card title="Audit Access" icon="audit">
    Filter by member to verify appropriate usage
  </Card>
</CardGroup>

### Analytics Checklist

* ✅ Check success rate regularly
* ✅ Monitor response times
* ✅ Review error logs for issues
* ✅ Export reports for stakeholders
* ✅ Compare with previous periods
* ✅ Identify optimization opportunities
* ✅ Audit member access
* ✅ Track token usage trends

***

## Key Takeaways

✅ **Conversations** - Individual interactions with agents\
✅ **Execution Logs** - Detailed steps agent took\
✅ **Filters** - Slice data by date, member, status\
✅ **Metrics** - Performance indicators\
✅ **API** - Programmatic access to analytics

***

## Related Topics

* **[Monitoring](../concepts/monitoring.mdx)** - Project-wide consumption and costs
* **[Agents, Datasources & Tools](../concepts/agents-datasources-and-tools.mdx)** - How to build agents
* **[Security & Compliance](../concepts/security-and-compliance.mdx)** - Audit logs and access control

***

## Next Steps

* **[View Your Agent Analytics](#)** - Check your agent's conversations
* **[Optimize Performance](../concepts/monitoring.mdx#reducing-consumption)** - Use analytics to improve agents
* **[Export Reports](#)** - Share analytics with your team