This question doesn't involve any code. I just want to know a way to run multiple instances of a django app and if it is really possible in the first place.
I make django apps and host them on Apache. With the apps, I noticed a conflict between multiple users accessing the web app.
Let us assume it is a web scraping app. If one user visits the app and run the scraper, the other user accessing the site from a different location doesn't seem to be able to visit the app or run the scraper unless the scraping that the first user started finishes.
Is it really possibe to make it independent for all different users accessing the app?
There are a few ways you could approach this. You might consider putting your app into a container (Google search: docker {your stack})
Then implement something like Docker Swarm or Kubernetes to allocate multiple instances of your app.
That being said, you might consider how to refactor your app to allow multiple users. It sounds like your scraping process locks things. But in reality, there's no reason your server should lock up during this.
It might be better to create your app so that when it received a request, like someone visiting the site, the server pays out the requested web page. When a user asks for a scrape/task to run, the server calls your scaper service or script asynchronously.
That way your app can still function while a scrape is in progress. This would be MUCH more resource efficient (and likely simpler) than spinning up tens or hundreds of instances of your entire app.
tl;Dr: containerization for multiple instances
Refactor app so a single user can't threadlock it.
Related
I am currently creating a website hosted by Django. I plan to use React as my frontend framework. I have done some research on putting them together but most say that I should go for the SPA model and have separate web servers for frontend and backend. The problem is that I wish to use apache as a prod server with django and avoid having 2 separate servers. I have read about the hybrid model and having django serve static files with react.
My Biggest concern is security as I have already setup apache for security and I aware that node.js is somewhat insecure.
What would the best approach be. The separate SPA model or the hybrid model.
I'd say it's okay to go for hybrid model if the project is small and you are the only one working on it and you only want to make things done. I think it's kinda messy to create apps like this unless they don't really worth the time.
But if it's a big project and more than one developer is working on it or will work on it then i highly recommend going with separate web servers one serving frontend app and one django app.
Also note that you don't really need 2 different servers. You can use one server for both and use 2 different which is still not necessary and you can use one web server to serve both.
And security not something that different models can cause to downgrade or upgrade. It's up to you to configure the server and write both frontend and backend apps secure enough to do the work for you.
There are more than one web servers that are as secure as they can be and they work with both django and react pretty well. I used nginx many times to host both django and react apps and i had no problem causing by nginx itself whatsoever.
And for last piece of advice if you will; Creating good quality apps requires a lot of time and energy, working with different technologies that do really good for what they are made for and if you are planning to be a really good developer you should come out of your comfort zone and adapt with new technologies that comes out and they are coming out pretty rapidly which requires you to learn constantly and do things in way you are not used to yet and making things work even if they doesn't seem to be good together at the first look.
I have an Django application at my work, only available on the internal network.
Currently we import data using Excel, but this is a terrible error prone process and I want to replace it.
I would like to provide a rich web application in Javascript which exposes some, but not all of the data from the main Django application (lookup values for menus). This would run on a server visible to the outside world.
So what is a good approach for this?
Management are concerned about security of making the main Django app available to the outside world, and I would prefer an intermediate tier as well - I think it would be easier to write a small server side app than to go through the current code and make sure it is secure enough to the outside world (I learned Django buildiong this app, so some of the older code is not done according to best practices, but does work as it needs to). I would also like it to hold the new data until someone has checked it looks OK before importing to the main database. (I am the only developer, so there are time considerations).
So two options I can think of just now.
1: Have a small Django app on an external facing server. This can communicate with the main app to get the values required for lookups, and store the input before it gets imported. The tables will essentially mirror the main app and need updated when the main app tables change.
2: Have something similar, but rather than use a database, use the external facing server to contact the REST interface on the internal server. Something like using Django non-relational to get data from the REST interface of the main app. Put an import table in the main database server to store the dats for approval.
Are either of these good / bad approaches?
Any other suggestions?
Are there any good resources for learning about n-tier apps?
If I understand you correctly you want a small Group of trusted users to be able to access an internal database. There is already an Internal Django App accessing that database.
Management is concerned about making this app or an extension of it available to the general Internet.
I think ther concerns are very valid. If you have only a limited set of users accessing the import functionality, push authentication out of the Django Web Application into the HTTP Server / Balancer / Frontend.
For example set up an apache external webserver forcing all access to your Django App beeing encrypted (HTTPS) and authenticated. Users can be authenticated via HTTP-Auth using static files on the server. Password changes / user additions have to be done by an admin logging into the server.
Only after completing this login the Django App with it's own authentication can be accessed. I would opt vor a smale seperate import App instead of extending the main app. This small app could run with reduced permissions on the main database for an defense in depth aproach.
This setup provides you with a litte additional interfaces / points of failures, while maintaining a small attack surface against random Internet users. You can hire a security consultant th audit your apache config and be assured that you locked out the greater Internet and only have to worry about HTTP-Authenticated users.
I have benn running such setups for 15 years by now. Users are annoyed by the double authentication and password saving in Internet Cafes is an issue whith HTTP-Auth but generally it is verry seamless if once set up.
I come from a Java background where the web application is always resident in the memory. This allows it to perform all initialization tasks at the startup itself and, unlike PHP, it does not have to do that again and again for every request.
I see a lot of options to run Django projects but not sure which one of them will allow me to achieve the above? Furthermore I already have a Nginx running at 80 so requests to Django needs to be routed via it.
Django is run by python, and has a process which stays loaded in the memory, much like java. Unlike php, Django will not reload all of its data per request, and it has an application scope.
This is the reason why there are so many options for php hosting, but not as many for Django.
There are a few ways to use Nginx with Django, just google "nginx django" and you get a lot of results which teach you how, for example: https://code.djangoproject.com/wiki/DjangoAndNginx
Firstly, some background - I have been mostly using VPS/DS and EC2 for the django apps I build for the past 3-4 years.
On webfaction, I have only been deploying "trivial" apps or "personal blog apps" using both django and wordpress.
My biggest hesitation about deploying anything serious (with large amount of sensitive personalised user data such as addresses and phone records or ecommerce related data or even accepting a Paypal related transaction) on webfaction is that it is a shared hosting environment.
I like webfaction a lot because they have made it really easy to deploy django apps without the need for me to spend some time doing system administration and installing my own packages (which I obviously need to do on a clean EC2 instance or a VPS/DS instance on linode for example).
The question is - is it very unsafe to use webfaction (since it is a shared hosting provider and I can actually list the other users' home directory by running "ls -la" after I ssh in) to run an ecommerce application or an application which has a lot of personalized, sensitive user data? I would appreciate specific technical examples which can help me better decide whether or not to run a "simple" (yes, I know the word simple is dangerously ambiguous/undefined) ecommerce app - which probably will use the paypal payment gateway on webfaction.
I can address the specific concerns you have listed:
While you may be able to see other user's files, it's because they haven't changed the permissions on their files. You can use standard linux file permissions to keep other users from reading or writing your files.
chmod 600 file
You will most likely be storing your information in a database hosted by webfaction. While it is a shared enironment, you do get your own database that is protected by a password and a username. In short, it has the same security in place that a database hosted on a VPS would have.
While you do have to take extra steps to ensure the other users on the system cannot gain access to your information, it is certainly possible to keep your site secure in webfaction's shared hosting environment.
At the moment I'm running a Django application that is the same for all of my clients.
Each client has its own subdomain e.g.:
http://client1.myapp.com/
http://client2.myapp.com/
client1 has a settings file, client2 has a settings file etc. The Django app is the same for everyone.
Nginx proxies the requests for each subdomain to a fcgi instance.
So, every client runs its own Django instance, consuming a lot of memory.
Is it possible to run one fcgi instance that switches to the right settings file based on the subdomain that is requested?
Thank you very, very much for your time.
Good question. There was an extensive discussion of this exact problem on the django developers mailing list a few weeks ago.
Basically there isn't a good official solution right now but lots of people are solving the problem themselves in various ways. If you read the whole thread you will probably have a better idea of how to proceed or who to talk to.