Quality Analysis with Open AI
- XCALLY Knowledge (Unlicensed)
- Product Knowledge
Overview
Starting with version 3.40.0, a redesigned graphical user interface (GUI) has been introduced to enhance the user experience in the Voice Recordings section.
This update is part of the New Client Experience.
In the New Client Experience, users can perform Quality Analysis on voice recordings using AI-powered tools provided by OpenAI. The following analysis features are available:
Transcription – Converts audio recordings into text.
Sentiment Analysis – Identifies emotional tone within the conversation.
Each voice recording may support multiple transcriptions and sentiment analyses, depending on user permissions.
To launch Transcribe or Sentiment Analysis with OpenAI:
Navigate to the Voice Recordings section.
Select one or more recordings.
Use the action icons in the top-right corner of the interface to run:
Transcribe
Sentiment Analysis
To export data, use the Export button to download results in
.csvformat.
You can select multiple voice recordings and run analysis actions (Transcribe or Sentiment Analysis) simultaneously.
Note: The availability of each action depends on the specific permissions associated with each recording. Not all actions may be enabled for every recording.
To view the available actions for a specific recording click the three-dot menu next to the audio file and only the actions you have permission for will be shown.
To review the analysis results for a specific recording:
Click Edit on the desired voice recording.
The Edit Voice Recording page displays:
Recording details (read-only)
Playback controls (play, rewind, forward)
Options to download the file or launch AI actions
You can assign a rating to the recording from this interface.
The Transcribe and Sentiment Analysis tabs and buttons will only appear if these features are enabled on your license and under: Settings → General → Quality Analysis
Requirements
To enable Quality Analysis features in XCALLY, follow the steps below to properly configure your OpenAI account and system environment.
This feature is available starting from version 3.35.0
Request the XCALLY Quality Analysis add-on license.
Create a valid OpenAI account
To run Transcription, make sure the Split Voice Recordings option is enabled:
Go to Settings → General.
Enable Audio Split for Voice Recordings.
If you are not using the New Client Experience, most of the features described below are still accessible, though some limitations may apply.
Starting from XCALLY version 3.35.0, the installation script .env requires specific environment variables to support Quality Analysis via Redis container. Add the following entries to your
.envfile during installation or update:
XC_QA_QUEUE_WORKERS=10 # Quality Analysis SECTION
XC_QA_REDIS_PORT=21000 # timings redis port
XC_QA_REDIS_DB=0 # timings redis db
XC_QA_REDIS_USERNAME=
XC_QA_REDIS_PASSWORD=
XC_QA_REMOVE_FAILED_JOBS_AFTER=3600 # 1 hour
XC_QA_REMOVE_COMPLETED_JOBS_AFTER=604800 # 7 daysRefer to the Troubleshooting section to verify whether the required Redis container is running correctly.
Video Tutorial
The video tutorial provides a comprehensive walkthrough of all Quality Analysis features.
Please be aware that Post-Call Analytics and QA Categories are available exclusively when using Amazon AWS as the cloud provider.
XCALLY Configuration
To enable Quality Analysis features using OpenAI, follow these steps to configure the cloud provider in XCALLY.
Navigate to XCALLY Tools to add a Cloud Provider Account:
Click the Add button to create a new cloud provider account.
Fill in the required fields:
Name – A custom label for the account (e.g. OpenAI Account).
Service – Select OpenAI from the dropdown.
Copy and Paste the API key from your OpenAI account. For more information, refer to the documentation at this link.
Quality Analysis on Old Experience
Using OpenAI as the provider, you can run the transcription feature. Starting from XCALLY version 3.50.0, sentiment analysis is also supported.
Transcribe
To use OpenAI as the default provider for transcriptions:
Go to Settings → General → Quality Analysis.
In the Transcribe configuration section, select your OpenAI account as the default.
Navigate to the Voice Recordings section. Click on the Transcribe icon for the desired call.
A modal window will open with the default account pre-selected (as defined in settings).
You can change the selected account in the modal (e.g., to use a different provider for better language support). Changing the account here will not affect the global default account in the settings.
Click Start transcription. Once transcription is complete, open the Edit Voice Recording page to access the transcript.
Transcription: The conversation is transcribed. Open AI’s Whisper uses machine-learning models to convert speech to text.
Sentiment Analysis
Sentiment Analysis evaluates the emotional tone of a conversation based on the call transcript. It classifies prevailing sentiment into one of the following categories, with corresponding confidence levels expressed in percentage:
Positive – Generally optimistic or favorable tone
Negative – Generally unfavorable or critical tone
Mixed – Contains both positive and negative sentiment
Neutral – Objective or emotionless tone
The sentiment with OpenAI is based on the new features of OpenAI API responses.
The output of the sentiment scoring function is not deterministic, as it relies on AI interpretation.
As a result, running sentiment analysis multiple times on the same transcription may produce slightly different scores.
Sentiment Analysis can only be performed on completed transcriptions. Ensure the transcription process is finished before launching the analysis.
You can initiate Sentiment Analysis from the following sections:
Voice Recordings
Cally Square → Recordings (available from version 3.50.0)
To use OpenAI by default for Sentiment Analysis:
Navigate to Settings → General → Quality Analysis.
Under the Sentiment Analysis section, select your OpenAI account.
In the Voice Recordings list, click the three-dot menu beside a recording and select Run Sentiment Analysis
A modal will appear with the default account:
You can change the selected account in the modal (e.g., to use a different provider for better language support). Changing the account here will not affect the global default account in the settings.
After you click on Start analysis, wait until the Sentiment Analysis value becomes Completed.
Once the analysis is complete, navigate to the Edit Voice Recording page and open the Sentiment Analysis tab. You will see:
Sentiment (Primary Result): The overall sentiment determined with the highest confidence (
POSITIVE,NEUTRAL,MIXED, orNEGATIVE) .Confidence Scores: A breakdown of confidence levels (%) for each sentiment category.
Quality Analysis on New Client Experience
When using OpenAI as your provider, you can run the transcription feature. Starting from XCALLY version 3.50.0, sentiment analysis is also available.
These are accessible from:
Voice Recordings
Cally Square → Recordings (available from version 3.50.0)
Transcribe
Open AI’s Whisper uses machine-learning models to convert speech to text.
To use OpenAI as the default provider for transcriptions:
Go to Settings → General → Quality Analysis.
In the Transcribe configuration section, select your OpenAI account as the default.
To run a transcription:
Go to Voice → Recordings.
Click the three dots next to a recording and select Run Transcribe.
A modal opens showing the default account, in this case OpenAI account.
If you select OpenAI as the provider, specifying the language is not required, as it supports automatic language detection.
You can change the selected account in the modal (e.g., to use a different provider for better language support). Changing the account here will not affect the global default account in the settings.
Click Start transcription and wait until the Transcribe value becomes Completed:
Click the three dots → View to see the transcription:
Two audio bars, indicating the User and Customer audio channels (enabling split voice recordings in settings section). Dual audio waveforms (agent = downward; customer = upward).
Controls to play, rewind, fast-forward and download audio
Transcribed conversation with:
speaker role,
timestamp (begin at and duration time reported in seconds)
A red dot to identify the part of conversation played
In Transcribe tab, you can see the details of each transcription:
Created at (default descending sorting and filter from calendar)
Status (with checkbox menu)
New: job created, waiting to process
UploadingData: file being uploaded
InProgress: transcription runnin
Unknown: error or undefined status
Completed: transcription successful
Failed: transcription failed (warning icon shows error message, if available)
Service: Amazon AWS or OpenAI
Language, selected language code
View the transcription.
Sentiment Analysis
Sentiment analysis can only run after a transcription is completed. The sentiment with OpenAI is based on the new features of OpenAI API responses.
Click the three dots on a recording and choose Run Sentiment Analysis.
In the modal, the default OpenAI account appears.
Click Start to initiate analysis and wait until the Sentiment Analysis value becomes Completed.
The output of the sentiment scoring function is not deterministic, as it relies on AI interpretation.
As a result, running sentiment analysis multiple times on the same transcription may produce slightly different scores.
By clicking on the 3 dots button → View, it is possible to see the result:
The list of sentiment analysis shows the following columns:
Created at (default descending sorting and filter from calendar)
Status: Completed, New, Failed or blank field if not launched
Service: OpenAI or AWS
Main Sentiment (
POSITIVE,NEUTRAL,MIXED, orNEGATIVE)Percentages of each sentiment
Language