Chatbot Analytics: The 7 Most Important Metrics To Track Success

Like with all customer contact channels, you need to track and manage chatbot performance.

This process will provide valuable insights into your chatbot is doing well and the areas needing improvement. 

But which key metrics are most important when it comes to customer service chatbots?

In this article, we'll explore the top seven chatbot metrics you should be tracking in your chatbot analytics. We'll cover:

• Performance analytics
• Cost analytics
• The next step to chatbot success

Chatbot performance analytics

To measure the success of your chatbot, you need to track five key chatbot metrics. 

But remember that success means different things for different companies.

To get the most out of your chatbot performance analytics, make sure you set clear goals and KPIs to compare your monthly data against.

1. Deflection

This metric refers to the percentage of customer queries that are handled by your chatbot without any human intervention.

It’s a clear indication of how well your bot is doing its job. 

In most customer service scenarios, the role of a chatbot is to reduce the demand on live agents.

The bot handles queries from start to finish, only handing over to an agent if it can’t resolve the request. 

If your chatbot analytics tools show a low level of deflection, it means your bot is not adequately handling a good portion of customer queries.

This could mean it’s actually creating more work for agents, rather than helping them.

Firstly, as chatbots work as an additional contact channel for customers, they tend to increase the number of interactions.

This is great for customer engagement, but if chatbot conversations aren't successfully deflected, it’s just more queries for your human agents to deal with.

Secondly, if your bot can’t answer simple questions, active users will grow frustrated.

When they are transferred to a live agent, the agent has to deal with their frustration as well as their initial query.

This makes it a lot harder to ensure the customer enjoys a  good customer experience. 

Of course, your chatbot analytics won’t show a 100% deflection rate, and this should never be the aim.

Bots are not designed to replace human-driven customer service - they’re just another tool to help improve the overall customer experience. 

A good level of deflection will depend on your company and the type of customer queries you receive.

Studies have shown that aiming to deflect anywhere between 40% to 80% of queries is a realistic goal. 

Talkative customer Healthspan has even managed to reach a 88% deflection rate with our generative AI-powered chatbot.

Many chatbots are programmed to handle common customer requests by building pre-defined conversation paths known as "decision trees".

Others work by incorporating advanced AI technologies like conversational AI, natural language processing (NLP), and/or generative AI.

2. Unrecognized customer queries

Another important metric to keep an eye on in your chatbot analytics is the percentage of customer inputs that fall outside your chatbot’s prescribed workflow.

In layman terms: what percentage of customer questions does your chatbot not understand? 

If a customer asks a question you have not trained your chatbot to answer, it won’t understand the query and won’t be able to answer it.

In this case, it will need to transfer the customer to a live agent. 

You will never be able to predict every single question a customer can ask you, but the more questions you prepare your chatbot for, the better.

Many chatbots are programmed to handle common customer requests by building pre-defined conversation paths known as "decision trees".  

If this is the type of chatbot you choose, try to map out all likely chatbot conversations and cover as many eventualities as possible. 

With Talkative, you can achieve this with help of our user-friendly chatbot builder.

Other chatbot solutions work by incorporating advanced AI technologies like conversational AI, natural language processing (NLP), and/or generative AI.

For an AI-powered system, the best thing to do is integrate the bot with your company knowledge base.

With Talkative, this involves creating knowledge bases using web pages from your company site and file-based content.

The chatbot can then learn from your knowledge base datasets and answer countless question about your brand, product, and services.

From there, our AI knowledge gap report can be leveraged to monitor unrecognized customer queries.

This report works by generating a list of all the questions active users have raised with your chatbot, plus whether or not the bot was able to answer them using your knowledge base.

Armed with this information, you’ll be able to optimise and expand your knowledge base dataset, improving the accuracy and performance of your chatbot over time.

‍

3. Average handling time (AHT)

‍A good chatbot should help your agents’ workflows and provide efficient customer service.

This means it should reduce your average interaction handling time. 

One of the driving factors for companies adopting chatbots is the speed in which they can answer customer questions.

Today, a timely response is one of the most valued aspects of customer service. Chatbots answer this demand. 

When it comes to FAQs, bots will always be faster than agents.

They can handle multiple requests at once, meaning no chat queues, and they can provide immediate responses. 

Even your most efficient agent will be slower than a chatbot in an FAQ scenario.

This is why your average handling time should be reduced after introducing a chatbot.

A reduction in AHT is another chatbot analytic that shows your bot is doing its job well.

It’s taking care of common queries quickly and efficiently, while also freeing up agents to deal with more complex issues. 

4. Customer satisfaction scores (CSAT)

CSAT score is a key metric to keep track of on your chatbot analytics dashboard.

After all, every business wants satisfied and loyal customers.

The easiest way to measure user metrics like customer satisfaction is to get feedback.

This can be as simple as asking customers to click a ‘thumbs up’ or ‘thumbs down’ button after a chatbot interaction, or to rate the experience out of 10. 

For more detailed reviews, you can offer customers feedback forms or surveys to fill out and submit post-interaction.

By asking a few simple questions, you can find out what active users like and dislike about their chatbot experience, giving you a clearer picture of where you can improve. 

That said, it’s all well and good measuring overall customer satisfaction with your chatbot, but it’s more useful if you have other data to compare it with.

As such, we recommend tracking two CSAT-related user metrics.

First, you should track chatbot CSAT scores vs human agent CSAT scores.

If CSAT scores are a lot lower for your bot compared to live agents, your bot needs improving.

If it’s a lot higher, encourage more engagement with your bot (and maybe consider retraining your agents!).

Second, you should compare your current scores to CSAT levels before implementing a chatbot.

Chatbots are supposed to improve overall customer satisfaction - if your satisfaction rates are lower after introducing your bot, it's not serving its purpose.

5. Lead generation

Is your chatbot generating leads? To know this you need to include lead generation in your chatbot analytics. 

For sales and ecommerce chatbots, this is one of the most important metrics to track as these bots should be capturing high quality leads for your business.

If they’re not, it probably means your bot is not adequately engaging customers. 

If your bot is principally for customer support rather than sales, lead generation might be less important for you.

However, customer data collection (such as names, contact details, and personal preferences) is always useful for targeted marketing campaigns, meaning lead generation can still be worth keeping an eye on. 

What constitutes a lead can be defined by your business.

Is it simply collecting a customer’s name and contact details?

Or do you want a bit more information to qualify the lead, such as age range, product interest, company and role (for B2B)? 

When you have defined what makes a lead, you can set up tagging in your chatbot analytics.

With this in place, when a chatbot interaction fulfils the parameters of a lead, it will be tagged as an inbound lead. 

To really know if your chatbot is succeeding at generating more leads, compare the number of inbound leads generated month by month before and after you introduced the bot.

Ideally, you want to see a significant increase in leads after the chatbot has been put in place. 

Cost chatbot analytics

When researching the right bot for you, there are important cost-related chatbot analytics you should take into account.

Price is always a factor when introducing new tech and features to your company, but time-related costs are equally important. 

6. Development and configuration cost

Unless you have the expertise in-house to build your own chatbot, you'll likely need to look to a third-party to both develop and set up a bot on your website.

In this case, pricing always needs to be carefully considered. 

Price will vary largely based on the complexity of the chatbot solution.

For example, a rule-based chatbot that automates FAQs will cost a lot less than an AI-powered virtual assistant bot.

The easiest way to ensure you are getting the best deal for you is to compare like for like.

Know what you want your bot to do and request pricing quotes from multiple companies. Remember to get the cost for:‍

1. Developing the bot
2. Configuring bot on website e.g. customising the UX and chat widget (if required)
3. Integrating bot into existing workflows e.g. CRMs and chat systems
   

It’s also good to know how much customer support you’ll get with the price. Are you on your own after configuration or will you get continued expert support? 

7. Time to go live

Another important factor to consider when choosing a chatbot company is the time it will take to develop your bot and get it up and running. 

Much like price, this will be influenced by the complexity of your chatbot, but ideally you want a quick turn around.  

A standard FAQ chatbot shouldn’t take long to develop and configure. With Talkative, for example, we can build the bot and get it live within a few days. 

Consider how long you’re prepared to wait for your new chatbot and ask companies for an accurate timescale of development through to deployment. 

The quicker your bot is up and running, the sooner it can start helping agents, improving the customer experience, and generating leads. 

The takeaway

To sum up, the most critical chatbot analytics to track are:

1. Deflection
2. Unrecognised customer queries
3. Average handling time
4. Customer satisfaction
5. Lead generation
6. Deployment and configuration cost
7. Time to go live (development to deployment timescale) 
   

By tracking these chatbot metrics and setting clear goals, you can take your chatbot performance to the next level.

But if you want to truly get the most out of your chatbot, you also need it to be powered by the right technology.

That’s where Talkative comes in.

With our scalable and flexible chatbot solution, you can:

• Choose between an intent/rule-based bot, AI, or a combined approach (an AI chatbot with rule-based fall-back for maximum efficiency).
• Integrate our GenAI chatbot with your own AI knowledge base to create virtual assistants that are experts in your brand, products, and services.
• Meet and serve customers across your website, app, and messaging channels.
• Seamlessly escalate to human agents when needed.
• Easily track chatbot analytics and leverage AI-driven reporting.
• Build multiple chatbots in-house (if you prefer to take the wheel with bot design).
• Automate customer-specific queries with chatbot fulfilment.

In addition to chatbots and AI solutions, we offer a suite of customer contact channels and capabilities - including live chat, web calling, video chat, cobrowse, messaging, and more.

Want to learn more? Book a demo with Talkative today, and check out our interactive product tour.