How to Monitor an Agent?

Once your AI agent is published and being used, ongoing monitoring is essential to ensure optimal performance and user satisfaction. Shipable provides comprehensive tools to track usage, analyze conversations, and continuously improve your agents.

Accessing Analytics

To view performance data for your agents:

Navigate to the Analytics section in your Shipable dashboard
Select the specific agent you want to monitor
Choose your preferred time range for data analysis

Key Metrics to Track

Usage Statistics

Monitor how frequently your agent is being used:

Total Conversations: Track the overall number of interactions
Active Users: See how many unique users are engaging with your agent
Session Duration: Understand how long users typically interact with your agent
Peak Usage Times: Identify when your agent receives the most traffic

Performance Metrics

Evaluate how well your agent is performing:

Response Time: Measure how quickly your agent responds to queries
Completion Rate: Track the percentage of conversations that reach a successful conclusion
Tool Usage: See which connected tools are being utilized most frequently
Knowledge Base Hits: Identify which parts of your knowledge base are most referenced

User Satisfaction

Gauge how users feel about their interactions:

Feedback Scores: Review user ratings if you’ve enabled feedback collection
Abandonment Rate: Track how often users leave conversations before completion
Follow-up Questions: Measure how frequently users need to ask for clarification

Reviewing Conversations

Shipable allows you to review actual agent-user interactions:

Navigate to the Conversations section
Browse through conversation history
Use filters to find specific types of interactions
Analyze both successful conversations and potential problem areas

Screenshot 2025-05-26 at 8.18.41 AM.png

What to Look For

When reviewing conversations, pay attention to:

Misunderstandings: Instances where the agent didn’t properly interpret the user’s intent
Knowledge Gaps: Questions the agent couldn’t answer adequately
Successful Patterns: Interactions that led to positive outcomes
Common Queries: Frequently asked questions that might warrant specialized handling

Improving Your Agent

Knowledge Base Enhancements

Identify information gaps from unanswered questions
Add new documents or data to address these gaps
Refine existing content to improve relevance and accuracy
Remove or update outdated information

Instruction Refinement

Adjust your agent’s instructions based on conversation analysis
Provide more specific guidelines for handling common scenarios
Update the agent’s persona if the tone isn’t resonating with users
Add examples of ideal responses for frequently asked questions

That’s it. You're ready to test your first agent in minutes with Shipable.