I have the following setup:
The ember frontend is connected to a websocket server
The backend pushes records (real-time data) via websocket to the clients, as stringified JSON
The client receives the data, and must now update the store with this new data received
The problem that I have is that I do not know to process the raw JSON data to make it compatible to what is in the store. I can of course parse the json (JSON.parse), but this is just a part of what the REST adapter is doing.
When doing a normal REST request, more or less what happens is that:
server generates reply -> REST adapter converts it -> it gets pushed to the store
But now, since I am not using the REST adapter to process this data (because this is not a request triggered in the client side, but a notification coming from the server side), I do not know how to trigger the normal processing that the REST adapter performs.
How can I trigger the REST adapter programmatically? Can I pass it the stringified JSON coming from the websockets server?
Is it possible to hook the REST adapter to a generic websockets callback, where the only thing I have is the stringified JSON coming from the websockets server?
This is the code that I have (inspired in web2py)
function connect_websocket() {
console.log('connect_websocket > connecting to server');
var callback = function(e) {
var data = JSON.parse(e.data);
console.log('Data received > data=%o', data);
// TODO;
// - process the data as the REST adapter would do, and push new / updated records to the store.
// - handle record deletes too (how?)
};
if(!$.web2py.web2py_websocket('ws://127.0.0.1:8888/realtime/mygroup', callback)) {
alert('html5 websocket not supported by your browser, try Google Chrome');
}
}
I have taken a look at EmberSockets, but as far as I understand that does not offer a generic method of updating records in the store, but just a very specialized way of updating properties in the controllers (which requires a lot of configuration too).
What I am looking for is a generic method of triggering the ember REST adapter from a websockets server. Is there such a thing?
Related
I am using graph-gophers library to build graphql application on the server side. I have a lambda works as resolvers and websocket apigateway which works as transport layer. The lambda will take the request from apigateway then call graph-gophers library schema methods to trigger resolver. It works well for query and mutation. But I am not sure how to make it work for subscription.
The library graph-gophers requires the subscription resolver to return a go channels which it listens to. but in case of lambda which is a short lived application, it can't keep alive for a long time. That means I can't use channel for publishing data.
What I am doing is to save the socket connection id on a database, when there is a need to publish data, I will grab the connection id from db to find all the subscribers. But I don't know how to trigger the publish in this case. Anyone has any idea about how to do that?
Is it possible to invoke a AWS Step function by API Gateway endpoint and listen for the response (Until the workflow completes and return the results from end step)?
Currently I was able to find from the documentation that step functions are asynchronous by nature and has a final callback at the end. I have the need for the API invocation response getting the end results from step function flow without polling.
I guess that's not possible.
It's async and also there's the API Gateway Timeout
You don't need get the results by polling, you can combine Lambda, Step Functions, SNS and Websockets to get your results real time.
If you want to push a notification to a client (web browser) and you don't want to manage your own infra structure (scaling socket servers and etc) you could use AWS IOT. This tutorial may help you to get started:
http://gettechtalent.com/blog/tutorial-real-time-frontend-updates-with-react-serverless-and-websockets-on-aws-iot.html
If you only need to send the result to a backend (a web service endpoint for example), SNS should be fine.
This will probably work: create an HTTP "gateway" server that dispatches requests to your Steps workflow, then holds onto the request object until it receives a notification that allows it to send a response.
The gateway server will need to add a correlation ID to the payload, and the step workflow will need to carry that through.
One plausible way to receive the notification is with SQS.
Some psuedocode that's vaguely Node/Express flavoured:
const cache = new Cache(); // pick your favourite cache library
const gatewayId = guid(); // this lets us scale horizontally
const subscription = subscribeToQueue({
filter: { gatewayId },
topic: topicName,
});
httpServer.post( (req, res) => {
const correlationId = guid();
cache.add(correlationId, res);
submitToStepWorkflow(gatewayId, correlationId, req);
});
subscription.onNewMessage( message => {
const req = cache.pop(message.attributes.correlationId);
req.send(extractResponse(message));
req.end();
});
(The hypothetical queue reading API here is completely unlike aws-sdk's SQS API, but you get the idea)
So at the end of your step workflow, you just need to publish a message to SQS (perhaps via SNS) ensuring that the correlationId and gatewayId are preserved.
To handle failure, and avoid the cache filling with orphaned request objects, you'd probably want to set an expiry time on the cache, and handle expiry events:
cache.onExpiry( (key, req) => {
req.status(502);
req.send(gatewayTimeoutMessage());
req.end();
}
This whole approach only makes sense for workflows that you expect to normally complete in the sort of times that fit in a browser and proxy timeouts, of course.
I have a function to give recommendations to users. This function need to make a lot of calcs to start, but after start it use the already calculed matrix on memory. After this, any other calc that is made, "fills" the object in memory to continuous learning.
My intention is to use this function to website users, but the response need to come from the same "object" in memory and need to be sequential by request because it is not thread safe.
How is the best way to get this working? My first idea was use signalr so the user dont need to wait to response and a queue to send the requests to objects. But how the signalr can receive the response for this specific request?
The entire flow is:
User enter on a page.
A javascript will call a service with the user ID and actual page.
The server will queue the ID an page.
The service will be calculating the results for each request on queue and sending responses.
The server will "receive" the response and send back to client.
The main problem is that I dont see a way to the service receive the response to send back to client until it is complete, without need to be looping in queues.
Thanks!
If you are going to use SignalR, I would suggest using a hub method to accept these potentially long running requests from the client. By doing so it should be obvious "how the signalr can receive the response for this specific request".
You should be able to queue your calculations from inside your hub method where you will have access to the caller's connection id (via the Context.ConnectionId property).
If you can await the results of your queued operation inside of the hub method you queue from, you can then simply return the result from your hub method and SignalR will flow the result back to the calling JavaScript. You can also use Clients.Caller.... to send the result back.
If you go this route I suggest you use async/await instead of blocking request threads waiting for your long-running calculations to complete.
http://www.asp.net/signalr/overview/signalr-20/hubs-api/hubs-api-guide-server
If you can't process your calculation results from the same method you queued the calculation from, you still have options. Just be sure to queue the caller's connection id and a request id along with the calculation to be processed.
Then, you can process the results of all your calculations from outside of your hub using GlobalHost.ConnectionManager.GetHubContext:
private IHubContext _context = GlobalHost.ConnectionManager.GetHubContext<MyHub>()
// Call ProcessResults whenever results are ready to send back to the client
public void ProcessResults(string connectionId, uint requestId, MyResult result)
{
// Presumably there's JS code mapping request id's to results
// if you can have multiple ongoing requests per client
_context.Clients.Client(connectionId).receiveResult(requestId, result);
}
http://www.asp.net/signalr/overview/signalr-20/hubs-api/hubs-api-guide-server#callfromoutsidehub
Is there a way to add records to a client model and indicate to Ember that these records are already on the server side?
So, for instance if my Person model has records for "Peter" and "Paul" but then later the server adds "Mary" and my client becomes aware of that in a non ember-data sort of way. I'd like to be able to have the client add "Mary" in a way that will not cause state problems with the subsequent interactions.
For those that must know the "use case" ... I'm trying to:
have the first request to findAll() for a given model to pull the full set of data back from the server
have subsequent requests in a session call a custom AJAX request that only returns differences to the resultset since that last request
I want to be able to push these differences into the client model without screwing up it's "state"
Sometimes we create and update EmberData records on the client with data from websockets. In this case, the changes are already on the server, so we just want to make the changes on the client without changing state, etc, exactly as you describe.
When we create new records on the client, we push them into the store, e.g.:
this.store.push('post', { id: 1, body: 'Ember!' });
And, when we update existing records on the client, we update them in the store, e.g.:
this.store.update('post', { id: 1, hidden: true });
I am using Microsoft Synch Service Framework 4.0 for synching Sql server Database tables with SqlLite Database on the Ipad side.
I want to customize the C# code. The client will be sending me some parameters e.g. SessionId with the synch services. Depending on the parameters, the code will decide whether to continue with the synching or not. Now, the below code is for initializing the synch service. Where can I put the custom code for processing the parameters. Basically,
If ( Session Id already exists)
// Continue with synching
else
{
Do not continue with synching
}
I believe this is possible using Interceptors: Request, Response OR Conflict Interceptors.
Could anyone provide an example so that I can write business logic before synching
i think you should be writing a Request Interceptor.
here's a quick/basic intro on how to write one.
Well, I faced some problem while using Interceptors when downloading changes from server.
Client is sending me "Server_Code" through header.
if (serverCode == "DEV")
{
foreach (IOfflineEntity entity in context.IncomingChanges)
{
context.RejectChange(entity, "Invalid Server");
}
}
When client is downloading changes from server, I get IncomingChanges value = 0. Thus I cannot reject changes.
How about using filters for sync. i.e. Sync will happen based on the parameters sent by the client.
If client sends SessionId, then sync will happen only for that particular SessionId.
Any ideas?