Understanding Critical Signals

Critical Signals and their definitions

Completion Rate: Measures how often users complete their conversations with the bot.
Sentiment Value: Assesses user emotions during interactions, indicating whether they are expressing positive or negative feelings.
User Sentiment: Measures the overall emotional tone of user interactions, indicating satisfaction or frustration. This insight helps improve user experience by identifying areas that need enhancement to increase positive sentiment.
Helpfulness: An AI-enriched metric that evaluates whether the bot's responses are genuinely helpful in addressing user queries.
Effort: Tracks the efficiency of the resolution process, measuring how long or how many interaction turns it takes to resolve a user's issue.

Completion of User's Requests	User Sentiment	Helpfulness	Effort
Full Completion: Whether all the user's requests were fully met.	Positive Indicators: Identifying expressions of satisfaction, gratitude, or relief.	Accuracy: Checking if the provided information was correct and pertinent to the user's requests.	Response Time: Measuring the speed at which the user's requests were addressed.
Partial Completion: Identifying if only some aspects of the user's requests were addressed.	Negative Indicators: Noting any complaints, frustration, or dissatisfaction.	Clarity: Evaluating how clearly the information was communicated.	Interaction Efficiency: Assessing the number of exchanges needed to resolve the requests.

AI Leveraged Signals

These critical signals are enhanced by our AI models, providing insights into the following areas:

Health Score: A comprehensive indicator of overall bot performance.
Top Satisfaction Drivers: Key factors contributing to positive user experiences.
Top Dissatisfaction Drivers: Common reasons behind user frustration or dissatisfaction.

Tracking Areas for Improvement

Transfer Rate: calculates the percentage of conversations where users are escalated to live agent support.
Transfer Rate Drivers: Identifies categories and reasons that cause users to transfer to agents, highlighting the biggest issues needing attention.

Tracking What's Working

Self-Service Rate: Measures how many conversations are successfully handled by the bot without human intervention, including early drop-offs.
Top Self-Service Drivers: Highlights the top use cases that lead to successful self-service interactions.

Conclusion

The health score can increase or decrease based on these critical signals. By monitoring them through the bot health dashboard, you can effectively track and improve your bot's performance.