Quality Analysis with Open AI
- XCALLY Knowledge (Unlicensed)
- Product Knowledge
What’s about
Starting from version 3.40.0 you will see the new GUI to improve the User Experience of Voice Recordings section
With the New Client Experience, on Voice Recordings, it is possible to apply a Quality Analysis using the AI features.
You can perform transcription, sentiment analysis on voice channel recorded calls using AI tools powered by OpenAI.
A voice recording can have one or more Transcriptions or Sentiment Analyses. To launch Transcribe and Sentiment Analysis with OpenAI, you can select the voice recording and use the icons on the top right. There is also a specific button to export files in .csv format.
It’s possible to select more than one Voice Recordings and then launch these actions, but consider that each Voice Recording can be analysed according to the specific permissions, therefore, not all the actions are enabled for each Voice Recording.
Click on the three dots menu next to each audio recording to see which actions are enabled
It is also possible to see all the actions' results relative to a Voice Recording, by editing a specific recording. From this interface, you can see the details of the Voice Recording, which cannot be modified. It’s only possible to define a rating, while from the topbar you can decide to play audio, download file, run transcribe/sentiment/post call analytics.
In edit page, you can see buttons/sections Transcribe/Sentiment Analysis/Post Call Analytics only if you have enabled the features of Settings → General → Quality Analysis
Requirements
This feature is available from version 3.35.0 and to use it, you need to enable on your license Quality Analysis Add-on
You need to have a valid OpenAI account
If you don’t have the New Client Experience, you can still see most of the features described below. However you might find some limitations
If you want to launch Transcription you need to enable split voice recordings in settings section
For Quality Analysis, a Redis container and these environment variables has been added starting from Version 3.35.0 to installation script .env
So for all installations (new and existing ones, for which the update script should be executed) the variable values must be:
XC_QA_QUEUE_WORKERS=10 # Quality Analysis SECTION
XC_QA_REDIS_PORT=21000 # timings redis port
XC_QA_REDIS_DB=0 # timings redis db
XC_QA_REDIS_USERNAME=
XC_QA_REDIS_PASSWORD=
XC_QA_REMOVE_FAILED_JOBS_AFTER=3600 # 1 hour
XC_QA_REMOVE_COMPLETED_JOBS_AFTER=604800 # 7 daysLet’s view Troubleshooting paragraph if you want to verify if required redis container is launched
Video Tutorial
Consider that in video tutorial we explore all quality analysis features.
Post call analytics and QA Categories are available only with AWS cloud provider.
XCALLY Configuration
From XCALLY Tools, configure the Cloud Provider. Click on the Add button to create a new Open AI Account:
insert a name
choose OpenAI as Service
copy and paste the ApiKey of your OpenAI account. See more info here
How it works on old Experience
By using OpenAI as provider, you can launch the transcription feature. Starting from XCALLY version 3.50.0, sentiment analysis is also available.
Transcribe
If you want to launch Transcribe by default with OpenAI, you need to choose on Settings → General → Quality Analysis your account on Transcribe configuration.
Then on Voice Recordings you can run transcribe
A modal will open with the default Account set on settings section
You always have the default Account but you can change it with another account if for example you have multilingual recordings and you know that with some languages AWS Transcription is best performing. In contrast, other language transcriptions work better with OpenAI.
If you select a different account from the default one, this will not be changed in the settings section
Click on Start transcription to get the result:
Transcript: Call transcript. Open AI’s Whisper uses machine-learning models to convert speech to text.
Sentiment Analysis
Sentiment analysis can be run on the latest transcription produced. This feature inspects the call transcript text and returns an inference of the prevailing sentiment (POSITIVE, NEUTRAL, MIXED, or NEGATIVE, expressed in percentage).
The sentiment with OpenAI is based on the new features of OpenAI API responses.
The output of the score calculation function is not deterministic, because it is subject to interpretation by the AI.
Therefore, running the sentiment on the same transcription multiple times may result in slightly different scores.
Please make sure the transcription is completed before launching the sentiment analysis.
You can run the sentiment analysis feature in the Voice Recordings section as described below and, from XCALLY version 3.50.0, in the Cally Square → Recordings section as well.
If you want to launch sentiment analysis by default with OpenAI, you need to choose on Settings → General → Quality Analysis the related account on Sentiment Analysis configuration.
Click on the three dots and on Run Sentiment Analysis.
A modal will open with the default Account set on settings section, in this case OpenAI account.
You always have a default account selected, but you can switch to a different one if needed—for example, when working with multilingual recordings. In some cases, AWS Transcription may deliver better results for certain languages, while others might perform better with OpenAI.
Keep in mind that changing the account here will not update the default account in the settings section.
After you click on Start analysis, wait until the Sentiment Analysis value becomes Completed.
By opening sentiment analysis tab, you can view the result with the Main sentiment (POSITIVE, NEUTRAL, MIXED, or NEGATIVE) and Percentages of each sentiment
How it works on New Client Experience
By using OpenAI as provider, you can launch the transcription feature. Starting from XCALLY version 3.50.0, sentiment analysis is also available.
Transcribe
Open AI’s Whisper uses machine-learning models to convert speech to text.
You can launch the transcription feature in the Voice → Recordings section as described below and, from XCALLY version 3.50.0, in the Cally Square → Recordings section as well.
If you want to launch Transcribe by default with OpenAI, you need to choose on Settings → General → Quality Analysis the related account on Transcribe configuration.
Click on the three dots and on Run Transcribe.
A modal will open with the default Account set on settings section, in this case OpenAI account.
If you choose OpenAI, you don’t need to indicate the language, because it has automatic recognition.
You always have a default account selected, but you can switch to a different one if needed—for example, when working with multilingual recordings. In some cases, AWS Transcription may deliver better results for certain languages, while others might perform better with OpenAI.
Keep in mind that changing the account here will not update the default account in the settings section.
After you click on Start transcription, wait until the Transcribe value becomes Completed:
Then, click on the three dots → Edit → Transcribe:
By clicking on the 3 dots button → View, it is possible to see the result:
Two audio bars, indicating the User and Customer audio channels (enabling split voice recordings in settings section).
The agent channel has downward values, while the client's audio shows upward bands.Buttons to play, rewind, forward audio or download it
Conversation transcription, with indication of role, begin at and duration time reported in seconds
A red dot to identify the part of conversation played
In Transcribe tab, you can see the details of each transcription, with indication of Created at (default descending sorting and filter from calendar), Status (with checkbox menu), Service (with checkbox menu to choose Amazon AWS or OpenAI), Language, while by clicking on 3 dots button you can View the transcription.
Status can be:
New: the job has just been created and is waiting for processing
UploadingData: the file is being uploaded to the provider's server
InProgress: the provider is processing the uploaded file
Unknown: unknown status (due to some error)
Completed: processing completed successfully
Failed: processing failed (in this case, if the provider gives an error message, a warning icon is placed next to the status (with a tooltip showing that message)
Sentiment Analysis
Sentiment analysis can be run on the latest transcription produced. This feature inspects the call transcript text and returns an inference of the prevailing sentiment (POSITIVE, NEUTRAL, MIXED, or NEGATIVE, expressed in percentage).
The sentiment with OpenAI is based on the new features of OpenAI API responses.
The output of the score calculation function is not deterministic, because it is subject to interpretation by the AI.
Therefore, running the sentiment on the same transcription multiple times may result in slightly different scores.
Please make sure the transcription is completed before launching the sentiment analysis.
You can run the sentiment analysis feature in the Voice Recordings section as described below and, from XCALLY version 3.50.0, in the Cally Square → Recordings section as well.
If you want to launch sentiment analysis by default with OpenAI, you need to choose on Settings → General → Quality Analysis the related account on Sentiment Analysis configuration.
Click on the three dots and on Run Sentiment Analysis.
A modal will open with the default Account set on settings section, in this case OpenAI account.
You always have a default account selected, but you can switch to a different one if needed—for example, when working with multilingual recordings. In some cases, AWS Transcription may deliver better results for certain languages, while others might perform better with OpenAI.
Keep in mind that changing the account here will not update the default account in the settings section.
After you click on Start analysis, wait until the Sentiment Analysis value becomes Completed:
By clicking on the 3 dots button → View, it is possible to see the result:
The list of sentiment analysis shows the following columns:
Created at (default descending sorting and filter from calendar)
Status: Completed, New, Failed or blank field if not launched
Service: OpenAI or AWS
Main Sentiment (
POSITIVE,NEUTRAL,MIXED, orNEGATIVE)Percentages of each sentiment
Language