AI Knowledge Base Test Runs

November 11, 2024

Save time and improve chatbot performance with automated knowledge base testing.

Knowledge Base Test Runs are now available to all Talkative customers.

With this new feature, you can quickly and easily test your chatbot’s knowledge base against a list of questions.

Previously, testing chatbot performance was a time-consuming task, requiring you to think of and test each question manually.

Now, you can automate this process.

It means you can save time, assess knowledge base quality with ease, and instantly identify gaps or errors in chatbot responses.

‍

Optimise chatbot performance with ease

Knowledge Base Test Runs offer a fast, efficient way to evaluate chatbot accuracy and reliability.

The feature allows you to automate the creation and execution of test cases, significantly reducing manual workload.

It works by running a series of questions against your knowledge base and generating a response for each one (as your chatbot would).

In turn, you can review how well your chatbot retrieves and delivers answers from your knowledge bases.

Flexible testing tailored to your needs

You can access Knowledge Base Test Runs within the settings of the knowledge base you want to test.

There, you’ll find options to either initiate a new test run or review previous ones.

If you have AI Insight Reports enabled, the list of questions for a test run can be auto-generated based on these reports, ensuring they align closely with common customer queries.

Alternatively, you can start with a blank slate and manually enter custom questions tailored to specific areas of the knowledge base.

You can also choose which large language model to use for each test run, allowing you to compare the responses generated by different LLMs.

This flexibility means you can adapt each test run based on your current needs and knowledge base content.

Once you launch a test, responses will populate in just a few seconds, indicating whether answers were found or if further updates are needed.

If you do update a knowledge base post-test, you can then re-run previous tests to check that the changes you’ve made have positively impacted performance.

Overall, this feature provides consistent, standardised testing across different knowledge bases and LLMs, empowering you to assess and improve chatbot performance over time.