Redis to Socket.IO - django

I am making a real-time webapp using Django. Django does all the heavy lifting. To make the app real-time, when a change is made in Django, it is published to redis. I then set up a node.js app that marshals the data from redis to socket.io.
So right now things look like this.
Django -> Redis -> Node.js -> Socket.IO -> Browser
I'm OK with using redis and think it is great. I looked into gevent-socketio but it is not what I am looking for. Is there a way to do:
Django -> Redis -> Socket.IO -> Browser?

Redis and socket.io don't communicate directly.. you need server in between, especially if You use websockets, long-polling etc.

You can use django-socketio.
What you have done is similar to Trello:
http://blog.fogcreek.com/the-trello-tech-stack/

We started a project called bus.io on npm. It should help you. It simplifies connecting sockets to a pub sub and distributing messages to them.
On the server.
var express = require('express')
var app = express();
app.use(express.static(__dirname+'/public'));
var server = require('http').Server(app).listen(3000);
var bus = require('bus.io')(server);
On the client.
<script type="text/javascript src="/bus.io/bus.io.js">
var sock = io.connect();
sock.on('connect', function () {
sock.message().action('say').content('hello').deliver();
});
sock.on('say', function (msg) {
console.log(msg.content());
});
</script>
This example demonstrates building a message and delivering it to the server. By default the message will loop back to the sender. A "sender" is an actor and by default a sender will send a message to itself. You can customize the actor and target. So for instance you wanted to send a message to everyone.
On the server you could add this to force all your messages to go to everyone.
sock.in(function (msg, sock, next) {
msg.target('everyone').deliver();
});
Or on the client.
sock.message().target('everyone').action('say').content('hello').deliver();
There is a shorthand too.
sock.message({target:'everyone', action:'say', content:'hello'}).deliver();
You can also change how the actor is assigned, by default it is the socket.id.
bus.actor(function (sock, cb) {
//so if you had a session you could get the user name from the session
cb(null, sock.handshake.session.user);
});
Messages are automatically propagated from the client to other clients through the bus. The bus is built ontop of redis and socket.io.
If you have any questions or would like some help contact checkout our http://turbonetix.io or the project on github https://github.com/turbonetix/bus.io.git

Related

SignalR returns "Connection ID required" when ran in an AWS ECS container

I am trying to get a .Net Core 3.1 SignalR Web API with an Angular front end to work with websockets.
The code works perfectly fine when ran locally, either from within in the IDE or via docker run. However, once the code gets deployed to an ECS instance in AWS behind an API Gateway the web sockets refuse to connect.
I setup my mappings like so:
app.UsePathBase("/ui");
app.UseEndpoints(endpoints =>
{
endpoints.MapControllerRoute(
"default",
"{controller}/{action=Index}/{id?}")
.RequireCors(PolicyName);
endpoints.MapHub<SessionHub>("/ws/session");
endpoints.MapHub<SessionsHub>("/ws/sessions");
});
And on the client I connect to the hub like so:
this.sessionsHubConnection = new HubConnectionBuilder()
.withUrl(`${window.origin}/ws/sessions`, {
skipNegotiation: true,
transport: HttpTransportType.WebSockets,
accessTokenFactory: () => this.getAccessToken()
})
.withAutomaticReconnect()
.build();
The following Fiddler trace shows the initial HTTP request to initialize the websocket connection and the error being returned by kestrel.
I tweaked my web socket middleware for handling the access token to also Console.Write some additional debugging statements that I think might prove insightful:
public async Task Invoke(HttpContext httpContext)
{
var request = httpContext.Request;
Console.WriteLine($"Starting connection id: {httpContext.Connection.Id}");
// web sockets cannot pass headers so we must take the access token from query param and
// add it to the header before authentication middleware runs
if (request.Path.StartsWithSegments("/ws", StringComparison.OrdinalIgnoreCase)
&&
request.Query.TryGetValue("access_token", out var accessToken))
{
request.Headers.Add("Authorization", $"Bearer {accessToken}");
}
try
{
var sb = new StringBuilder();
sb.AppendLine($"Connection Id: {httpContext.Connection.Id}");
Console.WriteLine(sb.ToString());
await _next(httpContext);
sb = new StringBuilder();
sb.AppendLine($"Status code {httpContext.Response.StatusCode}"); <-- this line
sb.AppendLine($"Connection Id: {httpContext.Connection.Id}"); <-- and this line
Console.WriteLine(sb.ToString());
}
catch (Exception e)
{
Console.WriteLine(e.Message);
Console.WriteLine(e.StackTrace);
throw;
}
}
And in the AWS logs you can see that the connection Id is present but is being ignored by the EndpointMiddleware(?) for some reason.
Any idea what could be causing this? Two ideas I have yet to be able to rule out are:
The HTTP/S termination at our API gateway is confusing Kestrel since the browser client is building the socket request under HTTPS, but as far as Kestrel is concerned everything is communicating over HTTP?
app.UsePathBase("/ui"); is confusing Kestrel and all the web socket paths should actually be /ui/ws/session?

Host web UI with executable application?

This may be a very stupid question but I have spent nearly 5 hours doing research on the web and found nothing to fully clarify my doubts.
In few words I have been asked for a possible employer to develop certain executable application as part of a "Technical Test". Supposedly they're measuring my expertise working with WCF. I was given two days to develop such App and all the information about it is the following:
Deliverable:
                - An executable that
                                * When APP is ran, it should host a WCF service (SERVICE) as well as a
web UI (UI) accessible by web browsers.
                                * Through the UI, user should be able to add or delete messages stored in a
database (DB).
                                * The UI should also display the current list of messages stored in the DB.
                                * If changes are made to the DB, those changes should show up in the UI
without the need to reload the page.
                - All of the project source code.
               
Additional notes:
Use of existing libraries is allowed as long as they are clearly referenced
Now, I understand that you can host a WCF Web Service using a Console Application (among other options) and the Service will be alive as long as the application is running. I also know that any Web Application can access this service by just adding a Service Reference, creating a client of its type and calling its methods. My confusion begins when they ask me to put all together in one executable application:
When APP is ran, it should host a WCF service (SERVICE) as well as a web UI (UI) accessible by web browsers.
What is that supposed to mean?? How can I host a Web UI using an executable?? Am I supposed to develop something like IIS and at the same time somehow define the html and server side code on the APP?
I did some research and I found a class(HttpListener) that allows you to open an http port, listen and then send back some html thru it. A very simple class. If this is a solution I can't see how to implement it. Other than that I couldn't find anything else on the web.
I would appreciate any opinion on the matter, even if I'm not able to develop the solution in time I would like to know how to do it. And if I'm missing some important basic concept regarding WCF or Web Hosting please I would greatly appreciate some clarification. Thanks in advance.
You can use OWIN to "self host" web apps.
An overview and further information can be found here - http://codeopinion.com/self-host-asp-net-web-api/
I solved the problem by hosting the web UI in the service itself. A service operation can return anything, even a Stream of bytes with the html for the browser to render. Here's the code.
[WebGet(BodyStyle = WebMessageBodyStyle.WrappedRequest)]
public Stream GetUserInterface()
{
var appDirectoryName = Path.GetDirectoryName(System.Reflection.Assembly.GetExecutingAssembly().Location);
var htmlFilePath = appDirectoryName + "\\UI.html";
var buffer = File.ReadAllBytes(htmlFilePath);
if (WebOperationContext.Current != null)
WebOperationContext.Current.OutgoingResponse.ContentType = "text/html";
return new MemoryStream(buffer);
}
As you can see on the same directory than the executable I placed a UI.html file, this file contains all my UI html, javascript and css. Then I convert it to an array of bytes and return that to the browser.
So the only thing I have to do to access the UI is run the application and then browse to this operation. Eg: http://localhost:8080/MyService/GetUserInterface.
For the database part I used SQlite, in this way the application became a standalone that can be installed in a PC and run immediately without the need of Database or Web hosting. Exactly what the test requested.
Alternatively the class that I mentioned in my question (HttpListener) can also be used to host the Web UI instead of the service. This is another solution.
private static void HostUI()
{
while (true)
{
using (var listener = new HttpListener())
{
listener.Prefixes.Add("http://localhost:7070/");
listener.Start();
var context = listener.GetContext();
var response = context.Response;
//The .html file will be in the same folder where the .exe is
var appDirectoryName = Path.GetDirectoryName(System.Reflection.Assembly.GetExecutingAssembly().Location);
var htmlFilePath = appDirectoryName + "\\UI.html";
var buffer = File.ReadAllBytes(htmlFilePath);
response.ContentType = "text/html";
response.ContentLength64 = buffer.Length;
Console.WriteLine(buffer.Length);
var output = response.OutputStream;
output.Write(buffer, 0, buffer.Length);
output.Close();
listener.Close();
}
}
}
The reason why I used an infinite loop is because the HttpListener class implementation processes only one order by loop, so in order to able to request the UI multiple times you need to do this.
Then you can browse to http://localhost:7070/ and you'll see the UI too.
You can put this code in an independent thread to host the Web UI without affecting the main thread.

EmberJS and WebSocket | Best Approach?

I have a again which I can't answer for my self properly, maybe because of my lack in expierience with EmberJS.
I have to develop a management interface in EmberJS, using Symfony 2 for the backend, which should act and feel like a desktop application. So far so good, but since alot of people will work with the data inside this application, i would really like to use a WebSocket adapter implementation for EmberJS, since every connected client should always know about changes in entities immediately (or asap). I could write a WebSocket adapter for EmberJS but my problem here is that the WebSocket will do much more then RESTful operations, also the server will send messages without any EmberJS request (e.g. an entity changed and the server broadcasting this change to all clients). That means that i need a "command" structure on top of RESTful operations which, as far as my expierience goes, will not work with a pure DS Adapter.
For example:
Maybe i will trigger a controller method that will send a websocket message like this:
{command: "say", parameters: {message: "Hello guys!"} }
This command is not Entity (DS) related and will never go into the application store.
Another example would be like this:
{command: "load entity", parameters: {type: "Vendor\Bundle\Entity\Type", id: 43} }
Which would load an entity which should be stored in the application store.
Well, as i said, im not that familiar with EmberJS that I could figure out which the best approach could be. Should I bypass the DS Adapter completely and check for "isDirty" and just the push methods after loading entities? I'm happy about any idea you have!
As far as I understand your question, you want to push changes from your backend to your single page app?
You can push custom JSON into your application's store in Ember by using self.store.push('modelName', json). Have a look at the docs for a better undestanding.
So for example if your server sends you JSON via websocket that looks like this
{
- "message": {
"type": "fooModel",
"data": {
... // Model attributes here
}
}
}
you can push the data into your store. The following snippet would work with SocketIO for example:
App.ApplicationRoute = Ember.Route.extend({
activate: function() {
// connect to the websocket once we enter the application route
var socket = window.io.connect('http://localhost:8080');
var self = this;
socket.on('message', function(data){
self.store.push(data.type, data.item);
});
}
});
You can easily modify this snippet to fit your needs.

How to associate socket.io sockets in node.js app with logged user in web app?

I need associate logged users in my django web app with their socket.io sockets in node.js app, because for real-time part of web i want to use node.js. Session data will be stored in database. I think i can access cookies in browser, so i can send to node.js app cookie value, that acts like user identifier, but i think this isn't good idea. Is there any other way how to do this?
The cookie can be accessed inside the authorization event. So if you're already using a cookie to store session authentication data, it's pretty easy to integrate that into socket.io. There are some pre-built modules for doing this sort of thing, like SessionSockets.
This article gives some insights into how to do it yourself. The most important section is exerpted below:
var parseCookie = require('connect').utils.parseCookie;
sio.set('authorization', function (data, accept) {
// check if there's a cookie header
if (data.headers.cookie) {
// if there is, parse the cookie
data.cookie = parseCookie(data.headers.cookie);
// note that you will need to use the same key to grad the
// session id, as you specified in the Express setup.
data.sessionID = data.cookie['express.sid'];
} else {
// if there isn't, turn down the connection with a message
// and leave the function.
return accept('No cookie transmitted.', false);
}
// accept the incoming connection
accept(null, true);
});
sio.sockets.on('connection', function (socket) {
console.log('A socket with sessionID ' + socket.handshake.sessionID
+ ' connected!');
});

How do I capture (in Fiddler) the HTTP requests issued by Nancy.Testing.Browser

I have the following NancyFX unit test.
var browser = new Browser(new UnitTestBootstrapper());
var response = browser.Post("/login", with =>
{
with.FormValue("UserName", userName);
with.FormValue("Password", password);
});
response.ShouldHaveRedirectedTo("/home");
You can see that I use an instance of Nancy.Testing.Browser to POST some form values. I would like to capture this Http request in Fiddler but I am not sure how to set-up the Browser (a proxy perhaps?)
Thanks
You can't because they never hit the network; that's the whole point of the browser class - to give you end to end testing without the performance hit/configuration issues of having to use hosting/http/networking/browser rendering.
If you want to go via the networking stack then use something like Selenium, or spin up a self host and poke it with EasyHttp or manually with HttpClient.