/
Live Call Translator

Live Call Translator

Overview

Live Call Translator enables real-time transcription and translation of spoken content of the customer. Through a text channel, the agent can write messages in their own language, which are translated into the customer’s language and synthesized into speech.

This removes language barriers during conversations and eliminates the need for agents to have specific language skills, ensuring smoother and more efficient communication.

Please note that a dedicated license is required to use the Live Call Translator plugin. For further details, refer to the price list or contact your sales account.

Discover here the agent experience.

Video Demo

Requirements

  • Live Call Translator is compatible with WebRTC and Windows Phonebar (not supported with external phones or mobile app)

  • Open Channel Accounts, by entering the following URL in the Reply URL field: http://localhost:5000/api/createPlayTTS

  • On Voice Queue you need to allow recordings and use WAV format for records. To do this, select wav in the Recording Format field in the advanced section of the queue settings

  • On Outbound Route, allow recordings and use WAV format for records. To do this, select wav in the Recording Format field in the settings section of the outbound route

  • Enable audio split for voice recordings, going on Settings → General → Global and switch on this setting

  • At least one plugin-enabled agent in the desired queue: to do this you can edit the desired agent and open Permissions tab. At the bottom enable the plugin Live Call Translator. Moreover add the agent to the queue

  • Create a Context: Add a new context in Voice section and enter the name confbridge-context. This name cannot be changed.

  • Python Version 3 (instructions to install it below)

  • Have credentials for at least one of the following providers: Google, Amazon AWS, OpenAI. In the next section, we will see in detail which credentials are needed for each provider

  • Create an Internal Route:

    • Phone Number: _X.

    • Context: confbridge-context

    • In the Actions tab of the route enter two Custom applications: Set and ConfBridge actions.

      • Application Name: Set, Arguments: CONFBRIDGE(user,quiet)=yes

      • Application Name: ConfBridge, Arguments: ${EXTEN}

The below configuration is not mandatory, but useful if you want to enable automatic interaction acceptance by the agent:

Staff → Agents → three-dot menu of the desired agent → Edit Agent → Other Channels → Enable Openchannel auto answer

Installation

To install Live Call Translator:

  1. Access the machine via ssh

  2. Run the following command with the root user to install the package for creating python virtual environments: apt-get install python3-venv

  3. Upload the Live Call Translator zip archive through the admin interface on XCALLY Motion, from the AppZone → Plugins section.

  4. After the upload is complete, you can find the Live Call Translator plugin row. Click on the three dots button menu and select Install plugin.

The Live Call Translator plugin is now installed, and you can proceed with the requirements check and configuration steps described below.


Use these commands to install Python 3:

sudo apt update sudo apt install python3 python3-pip python3-venv

 

 

Plugin configuration

Under Plugins → Live Call Translator → General, you need to enter the URL of the server in use and the Motion Apikey. Then you find the tabs of the supported providers (Google, AWS, OpenAI) where you need to enter the required data for configuration, as described in the paragraph below.

To make the plugin work properly, at least one provider must be correctly configured and entered in the general configurations of the transcription, translation and TTS (text-to-speech) services.

For some providers, requesting an API key may not be enough — they may also require enabling specific services such as transcription, translation, and text-to-speech directly from their portal.

Cloud Provider configuration

Google

  1. Open your Google Cloud Service accounts, choose a project and create a new service account.

  2. Click on the email related to the service account that was created and go to Keys section.

  3. Create a new key.

    1. You can select key type among JSON and P12 types.

    2. By selecting JSON, the system downloads a JSON file on the computer with information to insert on the configuration. In fact by opening the JSON file you can copy from here (without apices) the Client email and Private Key

image-20240725-125846.png

  1. Enter this information in the respective fields within the plugin interface API Transcription, Translation and Text-To-Speech services.

  2. Return to the Google Cloud Console home page and search the API services Speech-to-Text, Translation, Transcription and click on enable.

Amazon AWS

To configure this provider you need the below information, which you can obtain from your Amazon AWS account:

  • Access Key ID

  • Secret Access Key

You will also have to enter a region.

Please note that the transcription service is only available in certain regions. Here you can see the regions available for transcription.

About AWS pricing, you can explore official documentation:
TTS AWS: AWS Polly pricing which usally uses standard or neural TTS model
Transcribe AWS: AWS Transcribe

OpenAI

For OpenAI configuration, you only need the API key.
From your account, go to API section → API keys → Create new secret key, then copy and paste the generated API key into the plugin configuration.

Services configurations

Transcript

Under Plugins → Live call translator → Transcript tab you must define:

  • Provider: the provider you would like to use:

    • Amazon AWS:

      • you can choose only a language.

    • Google:

      • you can choose a maximum of 5 possible input languages to be transcribed. By reducing the number of languages, transcription speed and quality may be improved.

    • OpenAI:

      • automatic language recognition available.

      • you can choose a specific language, improving the speed and quality of transcription.

      • you must specify an input language for the caller, because if the caller has not yet spoken and you want to synthesize a message, the system needs a target language to generate the audio output correctly.

  • Input-Language: the input language (AUTO-DETECT option is available with OpenAI provider)

  • Default input language: select the default input language

  • Openchannel Account: an Openchannel Account must be indicated

  • Show original message: the agent will also be able to see the transcript in the original language, and not only the translation

  • Enable Provider Change For Agent: the agent can change the following transcription parameters: provider, input language and output language

  • Skip Agent confirmation on start: Enabling this option, the window with the choice of provider, input language(s) and output language will not be displayed.

Other options only available for Google and OpenAI:

  • Processed audio length:

    • between 2.0 and 8.0 seconds, recommended value 3.3

    • The recording is divided into smaller audio files. With this option you choose the duration in seconds of these audio files to be sent to the provider for transcription. Reducing the duration reduces the latency between the time you speak and the time the transcription takes place. However, this may reduce the quality of the transcription.

  • Consecutive processed audio overlap:

    • between 0.1 and 3.0, recommended value 0.7

    • This option affects the overlapping time of two consecutive audio files to be transcribed. Increasing this value will reduce the probability of words being lost between two consecutive audios, but will increase the probability of the final part of the first transcript being repeated in the first part of the next transcript.

Translate

In the Plugins → Live call translator → Translate → General tab, you need to select the default translation provider and define the target language, that is the language of the person being called.

Under Plugins → Live call translator → Translate → Configurations section you can choose the provider to be used for a specific language:

Example:

  • Input language: Italian

  • Output language: English

  • Provider: OpenAI

Note: It is not necessary to define all language pairs. If a requested translation pair is not explicitly configured, the system follows this logic:

  1. It looks for a provider configured for the specified output language.

  2. If none is found, it selects a provider for the first language in the same macro-region (e.g., if en-GB is requested, it will fall back to en-US if available).

  3. If this search also fails, the system uses the default translation provider.


TTS

In Plugins → Live Call Translator → TTS tab, the configuration is similar to the general translation settings. Here, you can:

  • Select a default TTS provider.

  • Choose a specific provider, voice type, and speech synthesis engine for each language, if you want to override the default settings.

Logs

Finally, the Logs tab allows you to view all plugin-related logs, providing insights into its activity, errors, and operational status.

Logs

The Plugins → Live Call Translator → Logs section allows you to check the proper functioning of the application, and to eventually check for errors.

You can find logs in folder /var/log/xcally/live-call-translator
As mentioned, the log files are organized by component and by type, and can be of four types:

  • <component_name>-combined.log, includes the list of all complete logs for that component;

  • <component_name>-combined.date.log (e.g. api-combined.2024-01-01.log) : these files gather the log details for the specified component, including both debug and error messages, and for the day specified in the date part of the name. The logs are rotated daily, meaning that you’ll have a new combined log file for each component for each day. The rotation limit is set to 30 days, so after 30 days the oldest logs will be deleted;

  • <component_name>-error.log, includes the list of all error logs for that component;

  • <component_name>-error.date.log: these files gather the log details related to errors only, and for the day specified in the date part of the name. The logs are rotated daily, meaning that you’ll have a new combined log file for each component for each day. The rotation limit is set to 30 days, meaning that after 30 days the oldest logs will be deleted.

  • TranslatorService_backend.log: includes the complete logs for the backend service.

  • TranslatorService_latency.log: includes information on provider latency times. Average, minimum and maximum latency are provided per call.

TranslatorService_backend.log details:

Error type

Text

Description

Solution

Error type

Text

Description

Solution

Configuration

error: 'Plugin license is not enabled for this server'

error: 'Plugin license is not enabled for this server'

Request machine token enabling. You can find the token by following: Settings → License

Configuration

ERROR provider initialization: error_specification

The error specified in the configuration of the indicated provider is present.

Verify that all information entered for that provider is correct

Configuration

ERROR - asr_streaming_[provider_name]_translate: [Errno 2] No such file or directory:

System cannot find the recording file

Check if registration is enabled for the queue or for the outbound route used, check if split recording is enabled. See documentation

Configuration

ERROR - Exception on /api/createPlayTTS [POST] […] Invalid Value, Bad language pair:

The language code used does not exist or is not supported by the provider

Set the chosen language for the desired provider that supports the language or check that the language code entered exists

Configuration

ERROR - asr_streaming_openai_translate: Error code: 400 - {'error': {'message': "Invalid language 'arb'. Language parameter must be specified in ISO-639-1 format.", 'type': 'invalid_request_error', 'param': 'language', 'code': 'invalid_language_format'}}

The language code used does not exist or is not supported by the provider

Set the chosen language for the desired provider that supports the language or check that the language code entered exists

Configuration

ERROR - api rpc status is: 401

Motion username or Motion Password wrong

Check Motion username and Motion Password in Live Call Translator General Settings

 

ERROR - asr_streaming_openai_translate: Request timed out.

The request to OpenAI went into timeout

If the error occurs constantly, check the connection status and check the status of the OpenAI API at the following link OpenAI Status

Error

ERROR - asr_streaming_openai_translate: Error code: 400 - {'error': {'message': 'Audio file is too short. Minimum audio length is 0.1 seconds.', 'type': 'invalid_request_error', 'param': 'file', 'code': 'audio_too_short'}}

If it occurs at the end of the conversation it's not a problem. If it occurs several times in the same call it could be an error

Please collect relevant logs and proceed to open a ticket

Error

ERROR - trim_silence: data length must be a multiple of '(sample_width * channels)'

If it only occurs at the beginning of the call it's not a problem. if it occurs repeatedly within the same call it could be an error

Please collect relevant logs and proceed to open a ticket

Error

ERROR - : incomplete initialization

System failed to initialise correctly

Please collect relevant logs and proceed to open a ticket

Error

ERROR - update_call_info call not found

System failed to update the call data

Please collect relevant logs and proceed to open a ticket

translateService-error.yyyy-mm-dd.log details:

Error type

Text

Description

Solution

Error type

Text

Description

Solution

Error


Error starting translate service connect ECONNREFUSED 127.0.0.1:5000

The frontend cannot communicate with the backend service

Check that the backend service is active by executing the command 'ps ax | grep python'. If no live call translator plugin-related python process is present, restart the plugin. If it is present, contact support.

Troubleshooting

If the plugin is not working, you can check whether it is active.

Navigate to Settings → System → Processes and verify in the list if the plugin is running (ONLINE label).

If not, go to App Zone → Plugins, open the plugin list, click the three-dots menu of the specific plugin, select Uninstall plugin, and then Install plugin again.

Please note that uninstalling only stops the processes; it does not remove the plugin itself, so all previously entered configurations are preserved.