The 7 Most Important Chatbot Analytics To Track Success

November 21, 2022
chatbot with graph, clock, magnifying glass

Like all customer contact channels, you need to track certain metrics to measure the performance of your chatbot.

These will indicate what your bot is doing well and the areas where it can be improved. 

But which metrics are most important when it comes to customer service chatbots?

In this article, we'll look at the top seven metrics you should be tracking in your chatbot analytics. We'll cover:

  • Performance analytics
  • Cost analytics
  • The next step too chatbot success
Chatbot graphic

Performance analytics

To measure the success of your chatbot, you need to track five key performance metrics. 

But remember that success means different things for different companies.

To get the most out of your chatbot performance analytics, make sure you set clear goals and KPIs to compare your monthly data against.

1. Deflection

This metric refers to the percentage of customer queries that are handled by your chatbot without any human intervention.

It’s a clear indication of how well your bot is doing its job. 

In most customer service scenarios, the role of a chatbot is to reduce the demand on live agents.

The bot handles FAQs from start to finish, only handing over to an agent if it can’t deal with the customer request. 

If your chatbot analytics show a low level of deflection, it means your bot is not adequately handling a good portion of customer queries.

This could mean it’s actually creating more work for agents, rather than helping them.

Firstly, as chatbots work as an additional contact channel for customers, they tend to increase the number of interactions.

This is great for customer engagement, but if your chatbot isn’t handling a lot of these additional interactions, it’s just more queries for your agents to deal with.

Secondly, if your bot can’t answer simple questions, customers will grow frustrated.

When they are transferred to a live agent, the agent has to deal with their frustration as well as their initial query.

This makes it a lot harder to ensure the customer enjoys a  good customer experience. 

Of course, your chatbot analytics won’t show a 100% deflection rate, and this should never be the aim.

Bots are not designed to replace human-driven customer service - they’re just another tool to help improve the overall customer experience. 

A good level of deflection will depend on your company and the type of customer queries you receive.

Studies have shown that aiming to deflect anywhere between 40% to 80% of queries is a realistic goal. 

Many chatbots are programmed to handle common customer requests by building pre-defined conversation paths known as "decision trees".

Others work by incorporating advanced AI technologies like conversational AI, natural language processing (NLP), and/or generative AI.

When set up correctly, your chatbot will be able to handle these specific queries from start to finish, automating your FAQ customer service.

If a customer has a complicated or unusual issue, your chatbot should be programmed to hand over to a live agent. 

Customer contact agent smiling

2. Unrecognized customer queries

Another important metric to keep an eye on in your chatbot analytics is the percentage of customer inputs that fall outside your chatbot’s prescribed workflow.

In layman terms: what percentage of customer questions does your chatbot not understand? 

When you build a chatbot, the aim is to design complete, human-like conversations.

Let’s look at an example. One of your most common customer queries is: ‘I can’t log into my account’, so you build an appropriate conversation path for your bot.

As long as the customer follows one of your pre-built conversation paths, the chatbot can handle the customer request. 

If a customer asks a question you have not trained your chatbot to answer, it won’t understand the query and won’t be able to answer it.

In this case, it will need to transfer the customer to a live agent. 

You will never be able to predict every single question a customer can ask you, but the more questions you prepare your chatbot for, the better.

Try to map out all likely customer conversations and cover as many eventualities as possible. 

Combined with the deflection metric we discussed above, these two chatbot analytics will help you understand how in-sync your bot is with your customers.

The aim should be to get it efficiently answering all of your FAQs. 

An example of a customer and chatbot conversation

3. Average handling time (AHT)

A good chatbot should help your agents’ workflows and provide efficient customer service.

This means it should reduce your average interaction handling time. 

One of the driving factors for companies adopting chatbots is the speed in which they can answer customer questions.

Today, a timely response is one of the most valued aspects of customer service. Chatbots answer this demand. 

When it comes to FAQs, bots will always be faster than agents.

They can handle multiple requests at once, meaning no chat queues and, without the need to type or ‘think’, they can provide immediate responses. 

Even your most efficient agent will be slower than a chatbot in an FAQ scenario.

This is why your average handling time should be reduced after introducing a chatbot.

A reduction in AHT is another chatbot analytic that shows your bot is doing its job well.

It’s taking care of common queries quickly and efficiently, while also freeing up agents to deal with more complex issues. 

Average handling time is a metric that any good chat and chatbot analytics system will track automatically.

You can track bot handling times and live chat handling times separately, but to truly know if your bot is making a difference, we recommend tracking an overall chat enquiries AHT too.   

Chatbot chatting with several customers

4. Customer satisfaction (CSAT)

Customer satisfaction is a key metric to keep track of in your chatbot analytics.

After all, every business wants satisfied and loyal customers.

The easiest way to measure customer satisfaction with your chatbot is to get feedback.

This can be as simple as asking customers to click a ‘thumbs up’ or ‘thumbs down’ button after an interaction, or to rate the experience out of 10. 

For more detailed feedback you can give customers a short post-conversation questionnaire.

By asking a few simple questions, you can find out what customers like and dislike about their experience with the bot, giving you a clearer picture of where you can improve. 

That said, it’s all well and good measuring overall customer satisfaction with your chatbot, but it’s more useful if you have other data to compare it with.

As a result, we recommend tracking two customer satisfaction related metrics in your chatbot analytics.

First, you should track chatbot customer satisfaction vs agent customer satisfaction.

If CSAT scores are a lot lower for your bot compared to live agents, your bot needs improving.

If it’s a lot higher, encourage more engagement with your bot (and maybe consider retraining your agents!).

Secondly, you should also track your customer satisfaction levels from before and after introducing your chatbot.

Chatbots are supposed to improve overall customer satisfaction - if your satisfaction rates are lower after introducing your bot, it is not serving its purpose.

A customer hand and a chatbot hand reaching to interact

5. Lead generation

Is your chatbot generating leads? To know this you need to include lead generation in your chatbot analytics. 

For sales and ecommerce chatbots, this is one of the most important metrics to track as these bots should be capturing high quality leads for your business.

If they’re not, it probably means your bot is not adequately engaging customers. 

If your bot is principally for customer support rather than sales, lead generation might be less important for you.

However, customer data collection (such as names, contact details, and personal preferences) is always useful for targeted marketing campaigns, meaning lead generation can still be worth keeping an eye on. 

What constitutes a lead can be defined by your business.

Is it simply collecting a customer’s name and contact details?

Or do you want a bit more information to qualify the lead, such as age range, product interest, company and role (for B2B)? 

When you have defined what makes a lead, you can set up tagging in your chatbot analytics.

With this in place, when a chatbot interaction fulfils the parameters of a lead, it will be tagged as an inbound lead. 

To really know if your chatbot is succeeding at generating more leads, compare the number of inbound leads generated month by month before and after you introduced the bot.

Ideally, you want to see a significant increase in leads after the chatbot has been put in place. 

A magnet attracting customer interactions

Cost analytics

When researching the right bot for you, there are important cost-related chatbot analytics you should take into account.

Price is always a factor when introducing new tech and features to your company, but time-related costs are equally important. 

6. Development and configuration cost

Unless you have the expertise in-house to build your own chatbot, you will need to look to a third-party to both develop and set up a bot on your website.

In this case, pricing always needs to be carefully considered. 

Price will vary largely based on the complexity of the chatbot solution.

For example, a rule-based bot that caters for a handful of FAQs will cost a lot less than a free text input AI chatbot. 

The easiest way to ensure you are getting the best deal for you is to compare like for like.

Know what you want your bot to do and request pricing quotes from multiple companies. Remember to get the cost for:

  1. Developing the bot
  2. Configuring bot on website e.g. customising the UX and chat widget (if required)
  3. Integrating bot into existing workflows e.g. CRMs and chat systems

It’s also good to know how much support you’ll get with the price. Are you on your own after configuration or will you get continued expert support? 

A happy contact center agent on a laptop

7. Time to go live

Another important factor to consider when choosing a chatbot company is the time it will take to develop your bot and get it up and running. 

Much like price, this will be influenced by the complexity of your chatbot, but ideally you want a quick turn around.  

A standard FAQ chatbot shouldn’t take long to develop and configure. With Talkative, for example, we can build the bot and get it live within a few days. 

Consider how long you’re prepared to wait for your new chatbot and ask companies for an accurate timescale of development through to deployment. 

The quicker your bot is up and running, the sooner it can start helping agents, improving the customer experience, and generating leads. 

Customers surrounded by timer icons

The takeaway

To sum up, the most important chatbot analytics to track are:

  1. Deflection
  2. Unrecognised customer queries
  3. Average handling time
  4. Customer satisfaction
  5. Lead generation
  6. Deployment and configuration cost
  7. Time to go live (development to deployment timescale) 

By tracking these metrics and setting clear goals, you can take your chatbot's performance to the next level.

But if you want to truly get the most out of your chatbot, you also need it to be powered by the right technology.

That’s where Talkative comes in.

With our scalable and flexible chatbot solution, you can:

  • Choose between an intent/rule-based system, AI, or a combined approach (an AI chatbot with rule-based fall-back for maximum efficiency).
  • Integrate our GenAI chatbot with your own AI knowledge base to create virtual assistants that are experts in your brand, products, and services.
  • Meet and serve customers across your website, app, and messaging channels.
  • Seamlessly escalate to human agents when needed.
  • Leverage AI-driven analytics and reporting.
  • Build multiple chatbots in-house (if you prefer to take the wheel with bot design).
  • Automate customer-specific queries with chatbot fulfilment.

In addition to chatbots and AI solutions, we offer a suite of customer contact channels and capabilities - including live chat, web calling, video chat, cobrowse, messaging, and more.

Want to learn more? Book a demo with Talkative today, and check out our interactive product tour.

Engage more customers in just three minutes

Get your weekly three minute read on making every customer interaction both personable and profitable. Sign up to the Talkative newsletter now.

Smiling customer on a smartphone

Ready for the future of customer service?

Download The 2024 Inner Circle Guide to Chatbots & Conversational AI

Thank you! Your submission has been received!
Oops! Something went wrong while submitting the form.