A client talks to Django, and django uses node.js to do some javascript related work to give back the client a http response.
I wonder how I should set up the link(?) between the django and node.js.
Simply, I could use python's requests library and talk http, but is this best I can do?
If I were to build the communication link in c++, I would create non-block socket with Send/Recv Thread and use mutex(or similar) between the django view code and the send/recv thread.
I guess that's what is called asynchronous io in node.js world.
Is there a similar thing in python so that I could use on django side to talk to another server?
I heard many big companies use Thrift, would it fit here?
I also see gevent might be relevant keyword here, but not sure.
I am not sure if my answer is still relevant, but I'll give it a try.
IMHO, the best solution for you would be to have a RESTful API for your Django app. This has several advantages:
it provides a lot of decoupling between your Django app and your Node.js one, so in case you ever want to reuse any of them or to replace one of them, it will be easy
it allows you to design an API for each of them and to hide the rest of your implementation (your Django app should not care how Node.js does its job, but only the response that it provides)
there are many great frameworks out there that can help you quickly build your APIs (at leat for Django there is Django REST framework)
If you don't want to build a RESTful API, python's request library is probably the best and easiest way.
Good luck.
Related
I am making a website with django now and I want to implement a live notification feature like the one on facebook or SE.
I did some research and it seems although there's two options: ajax long polling and websockets, the latter is the way to go.
However, as you know the go to plugin for websocket 'socket.io' turns out to be a node.js plugin and the django port only seems to support python 2 and the project seems pretty much dead. I am using python 2.7 as my project interpreter but I want to future proof myself so that if I upgrade to python3 later, I don't find myself not being able to use this functionality.
So my question is this:
Is there a straight forward and future ready way to implement websocket which will be used to send live notifications and chats in django env?
Django itself is build in blocking manner, i.e. with synchronous approach. So, you cannot open persistent websocket with django app, as it will block entire django thread.
If you want to enable notification/chat within django project environment, i would recommend to use centrifuge. It is written in python, but async (non-blocking) framework is used: tornado.
But, you don't need to even know how it works, as it provides simple REST api to communicate with it.
Simplified workflow, check docs for more details:
Start centrifuge at same server, as your django project (or on another but with low latency between them)
Your front-end will open websocket with centrifuge, not with django project.
When you need to send notification, send it to centrifuge from django via REST api, and centrifuge will deliver it to needed clients!
I've already tried it and it works!
Django doesn't provide what you're looking for out of the box. You'll have to use a third party library. One that works across frameworks is Pusher.
I think you must go for Firebase it gives you awesome synchronization and any how you are going to use chat on frontend so its does not have to do anything with django environment so you can update you backend asynchron in callback with firbase. Also firebase with AngularJS provides you really really awesome three way binding.
I've a huge django project and have to use Instagram API and its subscriptions model to work. For the subscriptions, my server has to be very responsive and be ready to work asynchronously to set up a hook so as to receive notifications once the user posts. Or that's what the documentation suggests. Now will it be a good thing to use Tornado there? Just for that small part or can I do it using Django in an effective way? if so, how?
You can use the WSGI container on top of Tornado to host any WSGI application, including Django, however, when you do that the WSGI application is still running as a blocking application and will not magically be running as an asynchronous application. So, when Django is handling a request there is no ability to handle another request at the same time within Django. The solution at that point is not much different to running a single threaded WSGI server and you would need to have multiple Tornado instances to handle concurrent requests.
So all really depends on what you mean by asynchronous. You certainly can't make use of Tornado's direct asynchronous programming API in Django. Thus there isn't really any great benefit from using Tornado with Django via the WSGI interface.
As I understand you are talking about this paragraph in Instagram docs
You should build your system to accept multiple update objects per payload - though often there will be only one included. Also, you should acknowledge the POST within a 2 second timeout--if you need to do more processing of the received information, you can do so in an asynchronous task.
That's another type of "asynchronous" that Tornado provides.
I think Django + Celery will suite better for this.
Your application will work in this way:
You receive JSON-data from Instagram
Create a celery-task, e.g. instagram_process.delay(request.raw_post_data) or instagram_process.delay(request.body) according to your Django version
Response to Instagram with 200 status code
In instagram_process task you do all your procession - parse JSON, store it do database and anything else you need.
If you want to check X-Hub-Signature you can either do it between steps 1 and 2, or pass this header to the task and verify the signature at step 4.
You can use tornado.wsgi to integrate Tornado with other WSGI compliant frameworks. Check out this demo project for details:
https://github.com/bdarnell/django-tornado-demo
I have several questions about Tornado and other web frameworks.
1) Tornado claims to be a webserver (a non-blocking one, therefore much performant), so some people said it does not play the role of django --i.e., they say tornado is not a web framework.
However, it does provide a web framework I think (http://www.tornadoweb.org/documentation#main-modules) -- in this way, it seems to replace django as the web development framework.
Is my above understanding correct?
2) Normally, several Tornados are set up behind Nginx. Tomcat is also normally set up behind Apache web server. Can I say Tornado plays exactly same role of Tomcat does for Java web server? If the answer is yes, then Tornado IS a web framework.
3) I read some article saying using Tornado and Django together, such as http://www.jeremybowers.com/blog/post/3/on-deploying-tornado-web-server-framework/, but I read some article online claiming that "if you use Django, then you lose the asynchronous from Tornado", is this true or false?
A related question though, if Tornado is itself a web framework as I said in 1), why people bother using Django at all? (to result the plugin?)
Can someone give me a 101 introduction?
To answer the question,
NO, Tornado is not a replacement to Django. It's an alternative.
YES, they are complementary to each other but not in the same process (*)
I would use Django when it's a big team effort and/or needs to run on a SQL database.
I would use Tornado for everything else. Ie. personal projects, WebSocket-related apps, or apps that use a NoSQL backend.
(*) Don't bother running Django inside Tornado or the other way around unless you really have a strong requirement for that.
Tornado is a web server and a web framework by most definitions, but it's quite a minimal framework (compared to Rails or Django). Tornado modules are loosely coupled, so it's possible to use just the web server component (or even just the lower level IO loop). Compared to Django, there are a lot of areas where Tornado doesn't have as much functionality. For example, there isn't really a model abstraction in Tornado, you have to roll your own.
When you run a Tornado app behind Nginx, that's your app running – Nginx just proxies to it. I believe Tomcat is an application server, distinct from your application. I wouldn't call Tomcat a web framework.
[Update: Django added support for async views in 3.0 and some ORM queries in 4.0. This point is no longer accurate] Django is not asynchronous, so generally your app will block while using the Django components. This may not be a big deal, it depends what you're doing. The Tornado devs have stated (paraphrasing heavily) that for most applications, the biggest win comes from not blocking while waiting for the client, i.e. web browser. Blocking on the database, for example, is not a big deal if you keep your queries fast.
There are a lot of pros and cons for both Django and Tornado, and there are many alternatives to both - it's definitely not just a choice between the two. Here's a very quick list of why you might want to use Django though:
Pros for Django:
it's a fuller stack (admin pages for example are very easy to implement)
it's much more established (plugins, tutorials, etc.)
it's better documented
its ORM is very nice
I'm planning to write a comet application in Django, but as far as I can see articles on this topic are scarce, and the comet libraries that are available don't necessarily lend themselves to use with Django (especially the development environment).
So far the best options I've seen (with one article on how to get it started in Django each) are APE and Orbited. Orbited, however, doesn't seem to still be actively developed (easy_install doesn't even work, the domains have expired), and APE should work fine, but how does one get it to work with python manage.py runserver ?
So my question is - if you are writing a comet application in Django, what library is most commonly used, if any?
My solution for this would be to write all of the models and main site in Django, but use another server for the ajax which is more suited to long polling or persistent sockets.
For example: SocketTornad.IO
https://github.com/SocketTornadIO/SocketTornad.IO
Inside the tornado socket.io server, the idea was to import the required Django models to work with the data required by the comet / websocket views.
Doing that through manage.py or any plain django server seems very difficult, and it would much easier to let the tornado socket.io server do the specialized comet/websocket work for you.
But, I have not actually implemented it yet, since basic non comet ajax has met our needs fine. It is just a concept that was considered.
for one of my applications I'd like to provide a minimal web interface. This core application is written in C++ and uses Qt4 as a framework. Since I'm also using some libraries I wrote to calculate some things and do some complex data management, I'd like to use this existing code as a backend to the web interface.
Idea 1: Using an embedded web server
The first thing I tried (and which worked to some degree) was using an embedded web server (mongoose). As you can imagine, it is just a very thin library and you have to implement a lot of things yourself (like session management, cookies, etc.).
Idea 2: Using a normal web server and adding a fcgi/cgi/scgi backend to my application
The next thing that came to my head was using a mature but compact web server (for instance, lighttpd) and simple provide a fcgi/scgi/cgi backend to it. I could write the web application using a good framework, like Pylons, PHP, or RoR, (...) and simply have an URL prefix, like /a/... which allows me to directly talk to the backend.
I tried to implement the libfcgi into my application, but it looks messier than needed (for instance you'd have to implement your own TCP/IP sockets to pass on data between your app and the web server and tunnel it through the FCGI library, meh)
Idea 3: Creating a command line version of my application which does the most basic things and use a normal web server and framework to do the rest
This is the third idea that came to my head. It is basically about creating a web application using a traditional way (PHP, RoR, etc.) and using a command line version of my application to process data and return it when needed.
I've got some experience with creating web applications, but I never had to do something like this, so I'd like to hear some ideas or suggestions. I'd like to use JavaScript on the browsers (AJAX, that is) and pass some JSON constructs between web browser and server to make the user experience a bit smoother.
So what are your suggestions, ideas on this? I don't want to re-invent the wheel, honestly.
I would never expose a custom written application to the net as front-end, for that servers like apache or lighthttp are build. They give you some serious security out of the box.
As for interaction of your app with that webserver, it depends a bit on the load and what kind of experience you have with writing software in PHP, python or other languages supported by your web server (via interpreter of course).
A slight load, and a command line tool accessed from PHP might do perfectly well.
A more heavy load and you might wish to implement a simple (SOAP?) server with Qt and access that from a python (or php) script.
That way you don't need to do layout in you app, and you also don't need to implement security all that much.
I'm currently researching a similar situation (custom web app backend using Qt), and the least bad option is FastCGI. Found something you might be interested in. Not production ready without some serious testing, but this might be a good starting point for Qt - FastCGI interop: FastCGIQt
I've used the FastCGI Protocol Driver library for a similar project (also a Qt application), the download link is at the end of that page [Libfastcgi]. Integration with the application turned out actually comparatively easy. Lighttpd + mod_fastcgi was used as web server. Can't say anything about FastCGIQt, though.
You Wt works well to provide a web interface to Qt based applications. Both have a similar programming style, and there's an example that demonstrates the integration with Qt.
Here is example of embedded QML-server: https://github.com/ncp1402/ql-server