Kore.ai Bots Platform allows you to get real-time metrics and conversation flows from its Dashboard. Following are the sections in this option:
- Bot Summary to get a bird’s eye view of all the bot configuration and capabilities in a place. See here for details.
- From the Usage Metrics section view real-time and standard dashboards on your bot’s performance. See here for details.
- The Containment Metrics section provides insights into how effectively customers are engaging with the bot vs transferring to agents or dropping offs. . See here for details.
- The Custom Dashboard allows you to design your own reports and dashboards. See here for details.
Bot Summary
Bot Summary can be accessed from the Build tab in the top menu and is the default page displayed when you open any bot. This page gives a detailed summary of the Bot configuration. This includes various snippets of bot definition as widgets. Clicking on any of these widgets takes you to the corresponding module.
These snippets give an overview of the:
- Tasks added to the bot (list of Linked Bots in case of a Universal Bot)
- Knowledge Graph configured including the summary like the number of paths, terms, synonyms, etc.
- Small Talk groups along with the number of conversations added
- Digital Skills added in terms of the number of Forms, Widgets, and Panels
- Natural Language – utterances, patterns, and synonyms
- Channels enabled
- Activity Logs – visible only to the owner of the bot;
- Team working on the Bot – visible only to the owner of the bot;
- Events Handlers, Environment Variables, and Content Variables configured
- Interruption Behavior, Authorization Profiles configured
- Debug Logs and any other settings configured for this Bot.
Clicking any of the above-mentioned cards takes you to the corresponding page.
You can use this page to Import Bot and Export Bot and add New Tasks and New Group (Small Talk) too.
Usage Metrics
From the Analyze tab on the top menu, you can access the Usage Metrics page. This page is your one-stop bot activity and bot user overview. It includes both interactive and non-interactive sessions. See here for more details on sessions.
Realtime Status
The first section on the Usage Metrics page is Real-time Status. This gives an overview of your Bot usage at a given point in time. It is set to refresh every 30 secs, which can be customized from 10secs to 10 mins using the dropdown next to the Refresh button.
The Realtime Status includes:
- Active Users – number of users interacting with the Bot at that given moment in time.
- Active Channels – clicking the card expands the list of active channels along with the number of users using the channel.
- Active Agent Sessions – sessions that are in progress at a given instance.
The Refresh button can be used to get the latest data.
Other usage metrics give an idea regarding the bot’s performance in terms of the frequency of user interaction, channels of interaction, tasks performed, etc.
- Chat Session Records: By default, Bot usage metrics based on usage of the bot during the past 24 hours are displayed. You can customize the metrics displayed by the time period and bot selection. Filter criteria can also be applied, see below for details.
- An overview of the number of Successful Tasks being completed, the number of Conversation Sessions, and the number of Alerts Sent are displayed.
- A graphic illustration of the Messages and Conversation Sessions. This includes
- Messages – the number of messages exchanged between the user and Bot;
- Conversation Sessions – the number of conversation sessions initiated with the bot ie the exchange between user and Bot without change of intent even after a specified idle time See here for details on the session definition.
- Details of:
- the number of Successful Tasks in the selected period of time;
- the pie chart comparing Intent Recognition vs Failed Tasks;
- Name and Count of successful tasks executions, with an option to export as CSV;
- Channels of interaction with the number of messages exchanged;
- Number sessions in Agent Transfer.
Containment Metrics
Notes:
- Containment metrics are only available for conversation sessions created after the 8.0 release, i.e post-November 1, 2020.
- Containment metrics are generated only for conversations in a closed state, see here for more on conversation session durations.
Containment Metrics provides insights on whether the calls were successfully answered by the virtual assistant or if the calls were landed with agents.
The metrics are categorized into:
- Self-service Conversations – These include sessions where a user has completed an interaction successfully. This will include all the sessions that are neither ‘agent transfer’ sessions nor ‘drop-off’ sessions;
- Drop-offs – These include sessions where the user exits the chat session abruptly. This will include all the sessions where the session has ended in any of the following scenarios:
- The user was in the middle of a task execution;
- The last task ended as a Task Failure;
- The last interaction/message resulted in an ‘intent not identified’ scenario;
- Non-interactive sessions;
- No response from the bot.
- Agent Transfer Conversations – These include sessions where the user is transferred to an agent using the Agent Transfer node at any point in the session.
Each category gives a bird’s eye view of:
- The number of conversation sessions that have started in the selected time period;
- The percentage increase or decrease when compared to the previous time period.
The following table gives the details of the metrics displayed.
Widget | Description | Self Service | Agent Transfer | Drop Offs |
---|---|---|---|---|
Totals | Percentage of conversation sessions in the selected time period – hourly if 24hr period is selected, daily for other intervals. | Self Service Conversations | Conversations that were transferred to an agent in the selected time period – hourly if a 24hr period is selected, daily for other intervals. | Conversations where the user dropped off in the selected time period – hourly if a 24hr period is selected, daily for other intervals. |
Insights |
Percentage of conversations against buckets of:
|
Record of all self-service conversations within the session. The range for the duration, number of messages, and tasks can be set as per your need. |
Record of all conversations within the session before Agent Transfer. The range for the duration, number of messages, and tasks can be set as per your need. |
Record of all conversations within the session before Drop Off. The range for the duration, number of messages, and tasks can be set as per your need. |
Conversation Flows | The sequence of tasks and the count |
Top conversation flows that were executed in Self-Service conversations. For each flow, execution of the flow along with relevant details is displayed with the entire transcript displayed when clicked upon. |
Top conversation flows that were executed in Agent Transfer For each flow, execution of the flow along with relevant details is displayed with the entire transcript displayed when clicked upon. |
Top conversation flows that resulted in user drop off. For each flow, execution of the flow along with relevant details is displayed with the entire transcript displayed when clicked upon. |
Tasks | Name of the task and the count. | N/A |
The total number of failed tasks. For each task, additional information like User Name, Failure Point, Failure Reason, Date & Time, and Channel would be given with the entire transcript displayed when clicked upon. |
The total number of failed tasks. For each task, additional information like User Name, Failure Point, Failure Reason, Date & Time, and Channel would be given with the entire transcript displayed when clicked upon. |
Intents | The cluster of similar intents/utterances and the count. | N/A |
The total number of times that this utterance was unidentified. For each utterance, additional information like User Name, Utterance, Date & Time, and Channel would be given with the entire transcript displayed when clicked upon. |
The total number of times that this utterance was unidentified. For each utterance, additional information like User Name, Utterance, Date & Time, and Channel would be given with the entire transcript displayed when clicked upon. |
Filter Criteria
A filter can be applied to display the required Metrics.
- Select the Time Period:
- 7 Days – Data aggregated over the past seven days is displayed.
- 24 Hours – Only data aggregated during the immediately preceding 24 hours is displayed. This is the default setting.
- Custom – Click to display the Custom Date Range dialog that you can use to define a specific date period. On the left side, select or enter a start date, and then on the right side, select or enter the end data for displaying activity data. Selection up to a maximum of 90 days preceding the current date is allowed.
- Using the Filter option, select:
- Metrics over a specified Channel.
- Metrics for a specified Language.
- The conversation session types can be either one or both (default setting) of the following:
- interactive – the normal conversation between the user and the bot;
- non-interactive – sessions where bot messages are presented to the user without any message from the user
See here for details on conversation session types.
- based upon the meta tags added at a message, user, or session level. Multiple tags can be selected wherein the ‘AND’ condition is applied.
Tag-based
You can filter the dashboard using meta tags added at a message, user, or session level. Multiple tags can be selected under the filter criteria, an ‘and‘ condition will be applied across multiple fields selected for filtering.
The following table gives the widget-wise applicability of meta tags on the dashboard.
Widget Type | Session Tags | User Tags | Message Tags |
Realtime Widgets | Not Applicable | Not Applicable | Not Applicable |
Successful Tasks | Applicable | Applicable | Not Applicable |
Alerts Sent | Applicable | Applicable | Not Applicable |
Sessions | Applicable | Applicable | Not Applicable |
Messages & Conversation Sessions | Applicable | Applicable | Applicable for Chats |
Successful Tasks | Applicable | Applicable | Not Applicable |
Intent Recognized vs. Failed | Applicable | Applicable | Not Applicable |
Top Tasks | Applicable | Applicable | Not Applicable |
Top Channels | Applicable | Applicable | Not Applicable |
Agent Transfer | Applicable | Applicable | Not Applicable |