I have a problem to Play audio message from aws lex code hook .is there any option can return audio file instead of text response on content .guys any ideas please share me.
Amazon LEX does not talk. If you want speaking functionality, look at using Amazon Polly, which is a service that turns text into lifelike speech.
Amazon Lex uses Polly to deliver audio responses.
You'll find the output voice setting under the general settings tab of your Lex bot in the Amazon Lex Console.
Programatically you need to invoke the PostContent method instead of PostText. The PostContent method accepts an audio stream and in turn returns an audio stream.
This page from the Developer Guide describes the main points to consider when sending and receiving voice streams to and from the Lex runtime API.
Amazon Lex Developer Guide | PostContent
Related
I am a new in AWS services and we want to build a simple demo that detect a special word and: [1] trigger an action [2] responses (as speech during the call).
For example, if the user say: "Help" I want to reply "OK" and make an operation (AWS lambda).
We're using Twilio, and Twilio should streaming the audio.
As I understand I have two options, Android Lex and Transcribe, when Lex is for bots and transcribe just translate the speech and can't get involved in conversation.
So the questions are:
What Services should I use to trigger an action when the special word is recognize AND involved in the conversation?
Can I streaming the call directly to AWS service via Twilio?
Edit
To be more clear: The communication will be with two persons in real time, and I want to make interject during their call when someone say "Help" I want to add a bot voice to the conversation and say "OK", for example"
[Person 1]: Hi, how are you
[Person 2]: HELP ...
[BOT]: OK (like a third person in a conference call..).
I am not fully clear on the interaction taking place with the user, before they interject with help. Are they listening to a bot, media file, TTS, or communicating with another person in real time?
For realtime analysis, you would need to use Twilio Media Streams, which streams the voice conversation to a service that could then convert the speech to text in near real time, looking for keywords, and then programmatically perform some action based on those keywords.
An example of using Twilio Media streams with Lex:
Use Amazon Lex as a conversational interface with Twilio Media Streams
I was wondering if anybody has ever experimented with this issue I'm having and could give me any input on the subject.
As it stands right now I'm trying to see if there is a way to grab a users input through the AWS Connect. I understand that there is already a "Get User Input" block in the GUI that is available for me to use, unfortunately it does not offer the fine grain control I am looking for with requests and responses from Lex.
Right now I am able to Post Content to Lex and get responses just fine, as well as output speech using Amazon Polly via my Lambda. This works great for things that do not require a user to have to give feedback for a question.
For example if a client asks
"What time is my appointment?"
and we give back
"Your appointment is for X at X time, would you like an email with
this confirmation?"
I want to be able to capture what the user says back within that same lambda.
So the interaction would go like so:
User asks a question.
Lambda POST's it to Lex and gets a response
Amazon Polly says the response - i.e: 'Would you like an email to confirm?'
Lambda then picks up if the user says yes or no - POST's info to Lex
Gets response and outputs voice through Polly.
If anybody has any information on this please let me know, thank you!
Why do you make so much complications to implement IVR system using Amazon Connect. I have done the complete IVR automated system to one of my biggest US banking client. Use the below procedure to achieve what you desire.
Build a complete interactive lex bot(So that you can avoid amazon poly & using lex post content api). It is advised to build each bot has only one intent in it.
In connect using "Get User Input" node map the lex bot which you have created earlier with the question to be asked "What time is my appointment?". Once this question has been played the complete control goes to lex and then you fulfilled your intent from lex side, you can come back to connect as like that.
Refer AWS contact center for the clear idea.
I’ve created a Lex bot that is integrated with an Amazon Connect work flow. The bot is invoked when the user calls the phone number specified in the Connect instance, and the bot itself invokes a Lambda function for initialisation & validation and fulfilment. The bot asks several questions that require the caller to provide simple responses. It all works OK, so far so good. I would like to add a final question that asks the caller for their comments. This could be any spoken text, including non-English words. I would like to be able to capture this Comment slot value as an audio stream or file, perhaps for storage in S3, with the goal of emailing a call centre administrator and providing the audio file as an MP3 or WAV attachment. Is there any way of doing this in Lex?
I’ve seen mention of ‘User utterance storage’ here: https://aws.amazon.com/blogs/contact-center/amazon-connect-with-amazon-lex-press-or-say-input/, but there’s no such setting visible in my Lex console.
I’m aware that Connect can be configured to store a recording in S3, but I need to be able to access the recording for the current phone call from within the Lambda function in order to attach it to an email. Any advice on how to achieve this, or suggestions for a workaround, would be much appreciated.
Thanks
Amazon Connect call recording can only record conversations once an agent accepts the call. Currently Connect cannot record voice in the Contact Flows. So in regards to getting the raw audio from Connect, that is not possible.
However, it looks like you can get it from lex if you developed an external application (could be lambda) that gets utterances: https://docs.aws.amazon.com/lex/latest/dg/API_GetUtterancesView.html
I also do not see the option to enable or disable user utterance storage in Lex, but this makes me think that by default, all are recorded: https://docs.aws.amazon.com/lex/latest/dg/API_DeleteUtterances.html
I'm using the Amazon Lex service. My input is always a text message, but sometimes I'd like a spoken response in addition to the text. I configured an output voice in the Lex settings.
I've tried adding a header amz-lex:accept-content-types=SSML to the request, but it returns with Invalid Bot Configuration: No usable messages given the current slot and sessionAttribute set. (Service: AmazonLexRuntime; Status Code: 400; Error Code: BadRequestException;. The same request works just fine when I ask for PlainText. And even if I ask for SSML,PlainText it'll respond with plain text only.
Do I need to configure something else inside Lex to allow it to do voice responses?
Lex cannot actually output voice by itself.
Lex will always output a JSON response and that response needs to be processed by the channel the user is accessing Lex with. So that channel is what outputs either text or voice based on how it processes the response message delivered from Lex.
Amazon Lex can handle speech-to-text.
Amazon Polly can do the reverse: text-to-speech.
If you go to the above Lex page, they have a few examples of using Lex for conversation logic and then Polly for text-to-speech and outputting voice to the user.
You can utilize SSML(Speech Synthesis Markup Language) for this, even to test voice in Lex Test Bot Console, using message content-type.
Using SSML tags, you can customize and control aspects of speech, such as pronunciation, volume, and speech rate.
SSML comes with a variety of directives with which you can customize pronunciation and create based on your requirement . Eg - say-as directive
`"message": {
"contentType": "SSML",
"content": "<speak> Hi " + data["User ID"].split('.')[0]+", Your Reference Number <say-as interpret-as="characters">" + "ABC"+event.currentIntent.slots.RefNo+ "</say-as> is ," + data["Status"] +"</speak>"
}`
Introduced in 2018 - https://aws.amazon.com/about-aws/whats-new/2018/02/announcing-responses-capability-in-amazon-lex-and-ssml-support-in-text-response/
I've created a lex sms chatbot and published it to a Twilio SMS Channel. Within my java lambda fulfillment handleRequest function I receive 2 parameters: an Input object and a Context. Input has some type of system generated userId, but I need the phone number of the incoming sms message from Twilio.
I've configured Twilio to call Lex via webhook per these instructions:
http://docs.aws.amazon.com/lex/latest/dg/twilio-bot-association.html#twilio-bot-assoc-create-assoc
I'm using the system created webhook callback url between Lex and Twilio and would like to avoid writing my own Lex-Twilio go between with Gateway API and lambda if possible.
I think I'm missing a configuration in Twilio to send this to Lex maybe? I haven't setup any TwilML app or anything, just linked the SMS number to my webhook callback url for lex. Everything works fine except getting the incoming sms number.
UPDATE:
I exhausted all possibilities with my input object and context in my java lambda function. I guess there's no way to get at the payload from twilio in Java. I tried switching to an input stream handler but still only had the input format defined in the documentation here: http://docs.aws.amazon.com/lex/latest/dg/lambda-input-response-format.html
I had to bite the bullet and build my own gateway API and Node handler. It's just a layer that sits before lex and translates twilio into lex input format and vis versa on the response. It updates the userId with the incoming phone number.
This tutorial was very useful in doing this:
https://aws.amazon.com/blogs/ai/integrate-your-amazon-lex-bot-with-any-messaging-service/
It was a lot of work just to get one additional field, but It does provide me the flexibility to switch SMS providers in the future easier.
I am not sure as what exactly stopped you from getting the user phone number in your lambda function. You can always get sender phone number from 'userId' field from the input event in your lambda function. You can refer to the documentation to get more information.