I want to expose a data streaming endpoint with 'Transfer-Encoding: chunked', a constantly open connection that streams data to a client.
My first though was to create something on Spring like here which works but we are using API Gateway and I found out that it does not support chunked transfer mode. Is there any other way to expose such an endpoint within the AWS ecosystem like Lambda or Kinesis or something else? Thanks in advance.
Related
This apparently is so simple question but I have been trying hard and came accross many questions and articles but none of them really answers this.
There are many ways to design the architecture after the log is ingested and transferred to AWS. We can use ES, Kinesis and other services.
My problem is how to transfer the application client logs to AWS securly, anywhere, could be S3, Lambda, Kindesis, ElasticSearch. We can surely have an endpoint exposed but that will be open. How to authenticate this to make sure malicious users don't spam our logs? what is the best way to ingest and transfer logs to AWS from mobile applicatios?
Creat an API that collect logs and ingest to kinesis expose it with API gateway use API keys aur authentication API to generate bearer token.
In application create functionality that make API call.
We did this in one of our application successfully
One way is to use Amplify SDK on the client application to log
securely.
Have a look at this link:
https://medium.com/swlh/add-monitoring-to-your-amplify-app-by-using-amplify-framework-d4c43b2bb84b
I deployed a ES cluster to AWS EKS and it can be access through application load balancer. I have created an index in the cluster. I am going to deploy a Kinesis firehose to use http_endpoint as the target to stream data to ES cluster.
But how can I configure what http verb firehose uses to put data to ES endpoint? For example, my ES endpoint is http://xxxx.com/myindex and I'd like the firehose to send POST request to this endpoint. How can I configure it to use POST rather than PUT?
Kinesis Firehose doesn't give you that much control over what the HTTP requests look like, unfortunately, since there are only a handful of configuration options for the HTTP Endpoint destination.
Kinesis sends HTTP requests according to their own specification, and expects responses to look in a specific way. You can find the specification here.
This specification is not compatible with Elasticsearch's Index API as you discovered. On a side note, you will probably want to use the Bulk API for performance reasons, since Kinesis will buffer events and deliver multiple ones in one request.
So, you will most likely need something in-between Kinesis and your Elasticsearch cluster to handle the differences in regards to the HTTP requests and responses. This could be a simply nginx proxy, for instance, or it could be something like an API Gateway which invokes a Lambda function. This Lambda function then sends a request to your index' Bulk API and constructs a response for Kinesis according to the HTTP specification.
I hope that helps.
I am looking to try and integrate AWS Connect to Nuance Speech.
Ideally, I would like to use a VXML interface from within Amazon Connect. But Amazon Connect will not support a native VXML interface.
The choices I have are to use either PSTN breakout from Connect to Nuance via a PBX that supports VXML or to pull out a text stream from Amazon connect Lex and pass to Nuance as a text string.
Anyone got any other experience how to do this without having to invest in a PBX to do the XML conversion.
Did you have a look at the Voximal or the Voxibot solutions?
You can use an AWS image to instance a VoiceXML interpreter in the Amazon Cloud.
May it could help.
This question already has answers here:
Call REST API for Amazon Kinesis with Setting up API Gateway
(2 answers)
Closed 5 years ago.
I want to add data to Kinesis Stream by Hitting an HTTP Endpoint, specifying some headers and data.
I know its possible via API Gateway -> AWS Service Proxy -> Kinesis Stream.
I dont want to use API Gateway.
Also I dont want to use the KPL Library.
The task is to provide an Endpoint to a Tibco Service and Tibco Service is going to ingest data into the stream.
Architecture Changes are welcome.!
The Kinesis API endpoints is what you're looking for.
However, handling the authentication and signing of the requests is going to be much work, which is already done for you in the AWS CLI and SDKs.
If possible, use those to submit your Kinesis data.
If this is not possible, you will need to create your own endpoint to receive the more "basic" request, and convert that into an AWS authenticated request. That's either done using the API gateway (which you state you do not want to do), or you host your own web server on an EC2 instance.
AWS has provided many tools to make things easy for you. Either use them, or roll your own.
I am building real-time dashboard using aws services, currently my application using MySQL database(RDS), which service and how would be designed real-time dashboard using Amazon web service, currently my approach is to use kinesis with redshift and connect my application via JDBC connector or use kinesis with s3 and use Athena to show real-time aggregators.
please help.
Thanks in advance.
Although the Amazon Kinesis Docs mention that Kinesis streams can be used to send updates to dashboards, Kinesis has no native mechanism to do this alone. For some very good security reasons it's unwise to allow clients (i.e. from a webpage you serve) to access backend services like Kinesis directly.
Instead, you'll want to set up your application layer (Java, in this case) to listen to the Kinesis streams and expose any relevant events to your client, storing any changes you need to keep track of in your database (RDS, in this case). For a real-time dashboard I'd recommend using something like WebSockets to send events in real time to your webpage from your server, as they're widely supported and easy to use. Heres a tutorial on how to implement WebSockets on GlassFish