We've configured WSO2 using the below ISO8583 endpoint to receive messages and communicate with a RESTful API.
https://docs.wso2.com/display/ESBCONNECTORS/Configuring+ISO8583+Inbound+Operations
The provider we're integrating with are expecting that we will initiate the connection with their server on the 5000 port, and only then will they begin sending transactions to us. I'm primarily experienced with RESTful APIs so I'm not familiar with WSO2 and opening and maintaining a socket connection. How do I need to configure WSO2 to create and maintain the connection with the host / port from which we expect all transaction to then originate?
I assume this will be via the ISO8583 connector however need guidance.
https://docs.wso2.com/display/ESBCONNECTORS/Configuring+ISO8583+Connector+Operation
Could you please elaborate more on your requirements. While reading the description, I do not think that you have a requirement to consume ISO8583 standard messages. Please use this connector only if you are consuming ISO8583 standard messages.
You can refer to the documentation on the inbound endpoint in document [1] to further clarification regarding the listening inbound endpoint, in which you can define a host and a port. Please elaborate more on the type of messages that need to be consumed how to interact with the backend to provide with a more infomative answer
[1]-https://docs.wso2.com/display/EI650/WSO2+EI+Inbound+Endpoints
Related
I've installed WSO2 Api Manager 4.0.0 on internal server and I have no idea how to tell it to use proxy server to connect to Choreo analytics.
I see following error in the log:
:Provided authentication endpoint https://analytics-event-auth.choreo.dev/auth/v1 is not reachable.
I've tried to set http_proxy, https_proxy, HTTP_PROXY and HTTPS_PROXY environment variables with flag java.net.useSystemProxies=true to api_manager.sh but they seem to not work and I don't see any traffic coming from this server through proxy.
Adding java flags http.proxyHost and http.proxyPort didn't helped too.
If I start it from server with internet access then it works just fine with Choreo.
Is there any way to set the proxy for APIM?
WSO2 API Manager's gateway component talks to an API in the Analytics cloud in order to fetch required credentials to publish events to the cloud. The failure that you have pointed out occurs at the point of talking to this API. It appears that proxy settings have not been configured for this particular HTTP client. Please see here.
Event publishing is the next step and uses AMQP protocol. Therefore I think it would not go through the HTTP/S proxy. However, if the gateway has no access to the internet, this step will fail again regardless of the API call is fixed to honour the proxy settings.
Currently, analytics does not have the support to publish events through a proxy. It seems that in order to honour proxy settings in event publishing, the protocol needs to be changed to Websocket.
Is it possible to use AWS Application Loadbalancer for RSocket?
An AWS Application Loadbalancer can also be used for WebSocket connections and my project uses RSocket with WebSocket as its transport. This made me wonder if it is possible to use this loadbalancer for RSocket aswell.
On one hand I would think it is possible to use this loadbalancer, as it only receives a connection and passes this to the target RSocket server.
On the other hand, if all RSocket frames go through the loadbalancer, it might not know how to handles these frames, which would make it not possible to use.
I couldn't find much about RSocket and loadbalancing online besides this post .But this is client side loadbalancing and I was looking for server side loadbalancing.
And this post .But this uses LoadBalanceSocketClient while I want to find out if an AWS Application Loadbalancer can be used.
Here follows a simple diagram of what I would like to have (if possible):
The RSocket client connects to the loadbalancer which passes the connection to a RSocket server (for example server A). Then the client and RSocket server A can communicate.
AWS will see this as a typical websocket service. So as long as it lets HTTP/1.1 connections through and lets them upgrade to WebSocket there shouldn't be a problem. This is very standard so it shouldn't be an issue. Ideally it won't see individual frames of the traffic, and you app will handle all frames on a single WebSocket connection. But it looks like the API Gateway support does deal with individual messages https://docs.aws.amazon.com/apigateway/latest/developerguide/apigateway-set-up-websocket-deployment.html. You should ignore the RSocket client load balancing, and focus on AWS WebSocket routing.
As an example, with GCP (instead of AWS) the complexity is that this bumps you up from AppEngine Standard to Flexible. The demo site https://demo.rsocket.io/ is deployed to GCP and exposes websockets.
The additional kink, is that you possibly want stateful routing if you want client resumption.
I am using google firestore as my DB with a spring web service. I am using com.google.firebase package to build data access layer and to query the database.
Can someone please tell me what protocol is used underneath for communication between application and database. Is it TCP or something else ?
Atul
The protocol is well documented. You can access it via a REST API or a gRPC API.
It is http over TCP. All Internet traffic runs on TCP or UDP but there are usually higher protocol on top of the low level transport layer.
I want to map an API Gateway endpoint with a Socket.io server endpoint, in order to authenticate users through Cognito and, if successful, redirect to the Socket.io server and establish a socket with optional namespace and rooms.
Is that makes sense? I didn't found any example, and API Gateway has only recently enabled a WebSocket API but without support for Socket.io
Your question has two parts:
First, the API Gateway using Cognito to authenticate your client;
Second, assuming you are using an EC2 running Node.JS with Socket.IO using API Gateway as an endpoint for your clients.
For the First part, you may use the following reference from AWS documentation.
There are several sub-parts when you talk about AWS Cognito, for example including AIM permissions Method Execution to enable API resource endpoint HTTP method.
For the second point, enable API Gateway to establish a synchronous connection with EC2 port running Socket.io you may read some references like this one.
You should configure your API Gateway:
Protocol WebSocket connection
Select your Route Selection expression ,e.g. \$default
Map the target backend for each $connect, $disconnect and $default
Use integration type AWS Service
Select EC2 and fill the rest of configs.
The answer by Rafael focuses more on using the Websocket API Gateway which in my opinion is still relatively new and there is some space to improvements. Plus I don't like having lambda integrations with database access because without RDS proxy they exceed the db connections really fast, and I don't think HTTP integration adds anything to the whole thing because you're performing HTTP request in the end but it's called through the Websocket API.
One thing I agree on with Rafael is that you need to have an EC2 instance running socket.io whether it's in Node.js or python (I used python with Flask in my case).
I managed to connect to my socket.io by using the HTTP API Gateway and setting allow_upgrades=False so http protocol won't be upgraded to ws protocol, because HTTP API Gateway doesn't support ws. My HTTP API Gateway is just forwarding socket.io requests to the load balancer, and good thing about that is that you can define access control on each route defined in the HTTP API Gateway.
The socket.io on my EC2 instance is defined like this:
socketio = SocketIO(async_handlers=True, allow_upgrades=False, cors_allowed_origins='*')
And my client connects to it by simply calling the route defined in the HTTP API Gateway which has proxy integration enabled.
https://xxxxxxxxx.execute-api.us-west-2.amazonaws.com/socket.io/{proxy}
Final result - client connected to socket
Before websocket technology, if you wanted real-time data in your browser, you needed a wasteful polling strategy. That's why websocket technology was introduced. However, it took some time before browsers supported it. On top of that, it wasn't that good at handling reconnects.
Socket-io gave us early-access to a reliable solution by combining multiple protocols, and adding several features to improve the stability and to recover from errors. With new releases, the protocol changed, and more flags and options were added.
That evolution made socket-io what it is today, which isn't exactly an "open standard". For that reason, it will probably never be decently supported on AWS.
Some possible solutions:
Having said that, browsers have evolved and most of them support websockets now. So, you could consider to migrate (back) from socket-io to plain old websockets. Nevertheless, you probably want to add a "heartbeat" that sends back and forth ping/pong messages to detect disconnects (which is one of those things that socket-io has built-in).
However, if you like GraphQL, then you should certainly consider AWS AppSync, which amongst others supports GraphQL subscriptions to push notifications to the client. Apollo client is extremely popular and reliable.
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 3 years ago.
Improve this question
Are there any serverless technologies that allow a serverless websocket server to be built?
I know the nature of long running connections is that they are stateful, but if the only state is the connection itself at the transport layer then it seems like there could be a serverless product that abstracts this away so you only deal with the application layer. Is there a cloud provider (AWS, Azure, etc) that allows this? I can't see a way for AWS Lambda or Azure Functions to achieve this.
Anyone got any ideas? Just checking.
Thanks
With the release of WebSocket support for AWS API Gateway, you can create a server-less WebSocket API.
A WebSocket API is composed of one or more routes. To determine which route a particular inbound request should use, you provide a route selection expression. The expression is evaluated against an inbound request to produce a value that corresponds to one of your route’s routeKey values. API gateway routes the request to corresponding lambda function.
From AWS blog example,
The application is composed of the WebSocket API in API Gateway that handles the connectivity between the client and servers (1). Two AWS Lambda functions react when clients connect (2) or disconnect (5) from the API. The sendMessage function (3) is invoked when the clients send messages to the server. The server sends the message to all connected clients (4) using the new API Gateway Management API. To track each of the connected clients, use a DynamoDB table to persist the connection identifiers (You can also use it to store other state information about the connection).
You can extend this to send messages to clients on any changes to data using DynamoDB Streams
Currently AWS Lambda and Azure Functions doesn't support this. If you plan to setup an scalable environment in AWS with websockets, you can use Application Load Balancer and in front of ECS cluster or EC2 instances with Websocket supported server like NodeJS.
Another solution is to go with fully managed services, like Google Firebase Service or Pubnub in your architecture to handle the real-time part.
if the only state is the connection itself at the transport layer
That's not really the case. Web socket connections exchange keep-alives as layer 7 payload. Others might argue that it's more accurately described as a sublayer somewhere between layers 6 and 7... but in any event, it is well-above the transport layer.
And many applications use web sockets in other ways that are also not stateless. Once connected, then authenticated, there's no need to continually re-authenticate, because the client on the socket now will be the same client 15 minutes from now, and this is overhead that would not be avoidable in a serverless environment -- every action on a websocket would need to be re-authenticated. For another example, with a constant data stream, the server might keep track of what has been sent or what specific subset of the stream the client is interested in.
If you aren't maintaining (or don't need) a persistent connection to a server, the question could be asked "why are you using a web socket?"
Perhaps also relevant: HAProxy, a commonly used load balancer with web socket support, maintains a persistent connection to a single back-end server for each current web socket connection. If that backend server goes offline, there's no provision in the balancer to choose another back-end for the existing connection. The client will need to reconnect.
AWS IoT provides MQTT endpoints and it supports MQTT + WebSocket on port 443. This might be the closest thing you can get as a hosted service on AWS.
Check this link: AWS IoT Protocols
You can define rules that trigger Lambdas on AWS IoT or pass them to Kinesis and process streams through Lambdas.
Fanout can do this. It works as a proxy that can translate WebSocket client activity into a series of HTTP requests. This allows FaaS backends like Lambda to manage raw WebSockets. The function is invoked only when there is activity to react to.
Docs: https://fanout.io/docs/devguide.html#custom-websocket-api
WebSocket-over-HTTP protocol: http://pushpin.org/docs/protocols/websocket-over-http/
Python helper library: https://github.com/fanout/python-faas-grip
AWS Lambda/Azure Functions/GCP Functions don't support web-socket protocol. So it's normal to use third-party services for this purpose.
In additional to comments above I can propose Ably: https://www.ably.io/ .
It provides api/libraries to handle web-socket without pain.