If you’ve noticed that chatbots are becoming common and popular, it’s not just you. The chatbot industry is booming. Not only are fewer customer service interactions handled by humans, experts predict that the chatbot market will grow to be worth more than $1.3 billion by 2024. Chatbots are one of the most cost-effective ways to deliver customer service quickly and impactfully.

If you’re currently using or thinking about setting up a chatbot as part of your business contact center, the good news is, there are a ton of KPIs to help you determine its effectiveness. We’ve outlined the key metrics you need to track to ensure your chatbot is performing optimally.

Why Chatbot Analytics Matter

Chatbot success metrics are important because they give you a wealth of data about your bot and your customers. You should monitor how customers are interacting with your chatbot to make sure you’re continuously improving their experience, meeting the goals you’ve set, and getting a good ROI. Set up a KPI dashboard and start tracking the relevant metrics for your business. So, which one to choose? Here is a list to get you started.

Types of Chatbot Metrics

1. User Metrics

Total Number of Users

Measure the number of people who have used or are using your chatbot. If these numbers seem lower than expected, there might be a technical issue with your bot, or it’s not easily accessible enough on your site for users to engage with.

Engaged Users

They are regular users of your chatbot, whether that’s daily, weekly, or monthly. If you’ve built your chatbot with the intention of capturing repeated user interaction, this is especially important to measure.

Number of New Users

Is your chatbot gaining or losing popularity? Find out by looking at the number of new users your chatbot gets over a specific period of time. If the number of new users are going down, you might need to reconfigure your chatbot to be more helpful.

Returning Users

Returning users are neither new nor “engaged.” These are users who came back to your chatbot after using it, but are not yet using it at regular intervals.

Chat Volume or Sessions

Chat sessions are the total number of times your chatbot was used over a period of time (sessions started, completed, or abandoned). If this number seems low for a specific time you’re looking at, that’s a sign that you may need to tweak it so that more people engage with it.

Goal Completion Rate

This number measures how successful your chatbot is at completing its defined goal. For example, if your chatbot’s goal is to reduce your customer service team’s time spent answering certain questions, the goal completion rate would reflect how well your chatbot performs.

Number of Bot Sessions Initiated

This denotes the number of times someone started a chat with your chatbot. Compare it against the completed and abandoned sessions to gauge the effectiveness of your chatbot. If you notice a pattern of sessions getting abandoned midway through, you might want to consider if they’re getting the assistance they need.

Average Daily Sessions

This metric shows your chatbot’s average amount of sessions per day. Comparing this to other daily metrics, like average daily traffic to your site, provides a gist on what percentage of users are using your chatbot on any given day.

2. Conversation Metrics

Total Conversations

This refers to how many conversations your chatbot handles in a day. As your business grows, this number will also need to increase accordingly to ensure your customer satisfaction rate is high.

Human vs Chatbot Interaction

Some customer queries would need human intervention, and you need to take that into account as well. Ensure to keep a tab on the amount of human interactions vs. chatbot interactions to gauge if your chatbot has managed to reduce the number of human interactions or not.

Conversation Duration

Track how long a conversation with your chatbot lasted for. If this number is high, it could mean that your chatbot is not fast enough at answering questions, or that your customers are asking multiple questions.

Interaction Rate

The interaction rate is the average number of messages exchanged during each conversation with your chatbot. This is a key metric for understanding overall engagement.

Average Chat Time

This measures the average duration of a chatbot session. It’s not necessarily a bad thing if this metric is high—it actually may indicate that your chatbot is very engaging, rather than inefficient.

Response Time

It’s the time taken for your chatbot to respond to a question or comment. Ideally, you want this number to be on the lower end since your customers are using your chatbot with the expectation that they’ll receive a quick response.

Non-Response Rate (Fallback Rate)

There are some chances your chatbot won’t be able to interpret or answer certain questions. The non-response rate measures this. If this rate is high, you may need to add more content or reassess your chatbot’s natural language processing abilities. On the higher end, bots that have low engagement see 60-65% non-response rates. On the higher end, better bots may see 10-20% non-response rates.

3. Customer Satisfaction Metrics

Retention Rate

This is the percentage of users that have used your chatbot multiple times over a given period of time. Try comparing this to how often your customers in your industry tend to contact you to see how your chatbot is fitting into the equation.

User Satisfaction Rate

Your chatbot might ask something along the lines of, “how satisfied were you with your service today?” once your customers are done using it. This metric needs special attention for improving your retention rate. A study by Dimensional Research found that 39% of customers will avoid a company for two years after a bad customer service experience.

User Sentiment

Thanks to artificial intelligence, your chatbot may be able to measure user sentiment whenever people interact with it. This helps you understand if your chatbot is hurting or improving the customer experience.

4. Quantitative KPIs

Activation Rate

Activation rate refers to how many customers engaged with more than one question. For example, a customer might use your chatbot to view their statement, and then pay their bill after being prompted to. That means two different actions were activated within the same session by the same customer.

Use Rate by Open Sessions

This one is the number of chatbot sessions that are happening simultaneously at any given time. However, in order to capture this accurately, you’ll also need to look at it in conjunction with the average number of open sessions in a given time period.

Usage Distribution by Hours

This measures how many times your chatbot is being used during each hour of the day. You can use this metric to schedule more staff during peak usage hours.

Questions per Conversation

Ideally, your chatbot helps answer customer questions as quickly and efficiently as possible. The questions per conversation metric refers to how many questions a customer needs to ask in order to get the answer they’re looking for. The lower this number, the more efficient your chatbot is at addressing the questions it’s given.

5. Qualitative KPIs

Ticket Deflection Rate

Deflection rate is calculated by taking the total number of chatbot conversations divided by the number of conversations that were transferred to a customer support agent. If your deflection rate is high, you may need to better equip your chatbot to answer customer questions.

User Feedback

This KPI is directly tied to user satisfaction rate. The feedback that your users provide will help you calculate a satisfaction rate or score, which will show you how to improve your service.

Bounce Rate

The bounce rate measures how many conversations fail to get pushed forward because your chatbot isn’t equipped to handle them. For example, someone may ask your chatbot for a specific product tutorial that your chatbot isn’t programmed to provide or recognize. If you see that your bounce rate is high, you should reevaluate its content.

Self-Service Rate

The self-service rate is the number of customers who were able to get the assistance they needed through your chatbot without having to speak to a live customer support representative. This is another way to measure your chatbot’s effectiveness at reducing your customer care reps’ volume and the overall ROI for whichever chatbot provider you’re using.

To help contextualize this further, this is very similar to a call center’s first call resolution rate (FCR).

Most Frequently Asked Questions

Look at customer journeys closely to identify the questions asked most often and how your chatbot is addressing them. That way, you can make improvements that are relevant to what your customers are looking to do.

Conclusion

There are a lot of different metrics you can look at to see how your chatbot is doing. Even if you don’t have the bandwidth to track every chatbot analytics metric, identifying the most relevant ones for your business will ensure you’re making smarter decisions.

Remember as your business evolves, so should your chatbot. As an extension of your customer engagement strategy, your chatbot should be updated any time your business launches new features or goes through a brand revamp,

27 KPIs to measure chatbot effectiveness