What Monitoring Shows
Monitoring is a project-level dashboard that tracks:- Token Consumption - Input and output tokens by LLM model
- Cost Breakdown - Costs by provider (Claude, GPT-4, Perplexity, etc.)
- Service Usage - Resource consumption by tool type In Roadmap
- Trends - Usage patterns over time In Roadmap
- Budget Tracking - Spending vs budget limits In Roadmap
Cost Tracking
Understanding Your Costs
Cost structure:- LLM Services (Claude, GPT-4, etc.)
- Based on tokens consumed
- Different rates for input vs output
- Rates vary by model
- Tool Services (Web Search, Code Execution, etc.)
- Based on usage (searches, executions, calls)
- Per-unit pricing or flat fees
- Varies by service
- External Services (via API tools)
- Billing by external provider
- PLai may charge platform fee
- Check service docs for pricing
Cost Per Agent/Workflow
Track which agents or workflows are most expensive: Estimated Cost Breakdown:Reducing Consumption
Optimize Token Usage
Savings: 77% reduction
Best Practices
Concise Prompts
Every token costs. Be brief and specific.
Smart Context
Include only relevant information, not full datasets.
Cheaper Models
Use Haiku for simple tasks, Sonnet for complex.
Batch Processing
Process multiple requests together when possible.
Example Optimization
Email Support Bot:- Problem:
- Current: 500 requests/day
- Avg input tokens: 1,500 per request
- Daily token cost: 3/1M)
- Optimization:
- Reduce context from 1,500 to 500 tokens
- New daily token cost: 3/1M)
- Monthly savings: $45 (66% reduction)
Troubleshooting
Unexpected cost increase
Unexpected cost increase
Steps:
- Check which model/service increased
- Look at token consumption
- Review recent agent/workflow changes
- Optimize expensive components
- Set alerts to catch next time
Cost not updating
Cost not updating
Check:
- Monitor refreshes every 5-15 minutes
- Some costs finalized at end of day/week
- Different providers have different billing cycles
- Wait a few hours for full update
Budget alert not working
Budget alert not working
Verify:
- Budget is set in project settings
- Alert threshold configured correctly
- Notifications enabled
- Check spam folder for alert emails
Need detailed usage breakdown
Need detailed usage breakdown
Use:
- Agent Analytics for per-agent data
- API endpoints for programmatic access
- Export data for custom analysis
Monitoring Best Practices
Review Weekly
Spend 5 minutes weekly checking consumption
Set Budgets
Know your spending limits before issues arise
Enable Alerts
Get notified of cost spikes immediately
Optimize Early
Donβt wait for problems. Optimize as you build.
Monitoring Checklist
- β Check project monitoring dashboard daily
- β Review cost trends weekly
- β Set realistic budget limits
- β Configure alert thresholds
- β Identify expensive agents/workflows
- β Optimize high-consumption components
- β Export reports for stakeholders
- β Archive data per retention policy
Key Takeaways
β Monitor tracks consumption - Tokens and service usageβ Understand costs - Different models and services cost differently
β Set budgets & alerts - Avoid surprise bills
β Optimize early - Reduce consumption before it becomes expensive
β Track trends - Understand patterns and plan capacity
Related Topics
- Agent Analytics - Conversations, executions, performance per agent
- Security & Compliance - Audit logs and access tracking
- Organizations & Projects - Project-level management
- API - Programmatic access to detailed metrics and analytics