OvationCXM Product Updates logo
Back to Homepage

Product Updates

Subscribe to Updates

Labels

  • All Posts
  • CX Manager
  • Knowledge
  • Bots
  • Journeys
  • Messenger
  • Mobile SDK
  • Dashboards & Reporting
  • Integrations & APIs
  • Settings & Admin

Jump to Month

  • May 2025
  • April 2025
  • March 2025
  • February 2025
  • January 2025
  • December 2024
  • November 2024
  • October 2024
  • September 2024
  • August 2024
  • July 2024
  • June 2024
  • May 2024
  • April 2024
  • March 2024
  • February 2024
  • January 2024
  • December 2023
  • November 2023
  • October 2023
  • September 2023
  • August 2023
  • June 2023
  • May 2023
  • April 2023
  • March 2023
  • February 2023
  • January 2023
  • December 2022
  • November 2022
  • October 2022
  • September 2022
  • August 2022
  • July 2022
  • June 2022
  • May 2022
  • April 2022
  • March 2022
  • February 2022
  • January 2022
  • December 2021
  • November 2021
  • October 2021
  • September 2021
  • August 2021
  • July 2021
  • June 2021
  • May 2021
  • April 2021
  • March 2021
BotsDashboards & Reporting
2 years ago

Analytics for Virtual Assistants

Hello, OvationCXM world!

We are happy to announce to you our Analytics Platform for Virtual Assistants. Now it is easier to understand and evaluate your Virtual Assistant's users, usage, and performance with our interactive dashboard that comes with a whole lot of valuable visual insights.

Aim:

To help you better understand how your Virtual Assistants are performing, whether the conversation flows are working as intended and which ones need improvement.

Analytics available now:

Usage Analytics:

  • Added the Charts on Assistant Hit-Rates and Resolutions-Statuses to provide insights on how the assistant is handling the conversations.
    • The charts will show the proportion of the reasons responsible for hits and misses on the assistant.
    • It will also drill down to present the resolution status of the issues handled by the assistant.
  • Added the Idle Misses and Response Misses tables to drill down on exact user messages, assistant responses, and associated flow to understand where Idle misses or response misses are happening.

User Analytics:

  • Added the Number of Users graphs to show the number of existing and new users who accessed your assistant.
  • Added the Returning Users graphs to show how much percentage of previous users returned on that day to your assistant.
  • Added the Retention Matrix to better understand the user behavior and present the pattern of how many prior users are returning for the following 12 days.

Conversation Analytics:

  • Added the Intents by the number of Utterances charts to present the intents that the user's messages matched most frequently.
  • Added the Intents by the number of Conversations charts to show the intents that were involved in the most conversations.
  • Added the Top Intents by High Confidence charts to provide insight into how confident your top-performing intents are at matching utterances. See which intents are the most confident.
  • Added the Top Intents by Low Confidence charts to provide insight into the intents with the lowest confidence scores.

Training Analytics:

  • Added the Stats Summary to provide the composition of keywords added and associated with each knowledge base, intent, entities, Q&As, and triggering phrases for training.
  • Added the Performance Summary to better evaluate and understand the performance of intents during Model Training. 
    • The graphs and charts will provide insights on Accuracy, Precision, Recall, and F1 score for intent during the training, helping you understand which intents need improvement to better train the assistant.
    • The Insights will also drill down on true positives, true negatives, false positives, and false negatives to provide insights on what exactly needs improvement and how to improve those intents.
  • Added the Training History graphs to present the performance history of each training of the assistant to help monitor the assistant's improvement/degradation over the course of different deployment iterations.
  • Added the Confusion Matrix to report on the number of confusions occurring between every two intents. The higher the confusion number, the more likely they are overlapping; hence recommended to either modify or delete one of the confused intents.
  • Added the Confusion Report on exactly which phrases in predicted intents are causing the overlap with actual intent to provide insights on which phrases for these intents need to be changed.

Rollout Plan:

We will be adding the Virtual Assistants Analytics Dashboard for partners with Virtual Assistants and Dashboards Access. Please email product@ovationcxm.com if you would like access to the Virtual Assistants Analytics Dashboard.

Avatar of authorVaibhav Gupta