I'm very new to web server and I would like to propose to my users the capability to use Python through my website.
My main problem is that Python is not harmless, even if it is armless (sorry for this very bad play on words). So I need to use a kind of sandbox but for me this is more a concept than a technic that I can use it.
So what would be the best way to do that ?
Sandboxing
You will need support from the operating system to effectively sandbox an application.
On FreeBSD, you can use a jail. This has proven to be quite secure over the years. (While there have been vulnerabilities in this system, the concensus is that it is not possible for a program to break out of a jail without outside help. <1>)
On Linux you can use LXC.
On MS-Windows you could use sandboxie.
Have a look at the comparison of virtualization technologies before deciding what to do. Personally I'd suggest only using technologies that offer "root privilege isolation".
Another possibility would be to use a virtual machine. But that would probably have more overhead.
And no matter what you use, you still need the firewall on the host to redirect some traffic to the sandbox/virtual machine.
Python security
CPython itself
The source code for the standard CPython is regularly audited by coverity. In their 2012 scan they found 0.005 defects in 1000 lines of code. The average for open source projects is 0.69 defects per 1000 lines, and 1 defect/1000 lines is accepted as a good industry standard.
So CPython itself doesn't have many defects.
Web programming with Python
The OWASP Python security project has identified security concerns in the CPython source code, as well as security concerns in modules and functions.
The built-in eval() function deserves special mention here.
It executes the Python code given to it as a string without check or boundaries. So while it is sometimes very useful, this function should never ever be given untrusted input from the web!
For instance, don't be tempted to use it to give your web app a built in calculator.
Their top-10 list of web app vulnerabilities also makes for interesting reading.
<1> From the documentation;
Jails are a powerful tool, but they are not a security panacea. While it is not possible for a jailed process to break out on its own, there are several ways in which an unprivileged user outside the jail can cooperate with a privileged user inside the jail to obtain elevated privileges in the host environment.
Most of these attacks can be mitigated by ensuring that the jail root is not accessible to unprivileged users in the host environment. As a general rule, untrusted users with privileged access to a jail should not be given access to the host environment.
You can run a cgi enabled server on the local machine with a single command:
python -m http.server 8000 --cgi
It will serve whatever is in the working directory that you executed the command.
You can browse the pages served by pointing your browser to localhost:8000
If you are using python 3.4 or later, you can restrict this to the local interface so that no one from the outside can access the pages:
python -m http.server 8000 --cgi --bind 127.0.0.1
Related
Is there good alternatives to the django developement server (runserver) that are more performant,
especially in concurency and static serving, and that have the auto-reload function, without having to setup a full blown production environment ?
Im working on Windows so gunicorn cannot be used.
You can install and use the rungevent commant. It has auto-reload function and it's more performant than thread-based servers (it is greenlet-oriented). The only caveat is the static file serving: you must install a webserver or proxy like nginx for that.
Are you doing so high bulk tests in ur dev server so you suffer this -specially regarding static files-? If so, then you must emulate, as said, a productive environment (just have an nginx correctly configured pointing to the address:port you use for your rungevent command).
If static files is not your problem, install a rungevent command and try how it works.
No since dev sites are made to handle limited requests, runserver runs fine on a machine that can match the requirements of your app.
If you are dealing with a large scale dev project which your system cannot tolerate, then it's either time to reproduce a production environment or upgrade.
I find it difficult to believe that your application is that bad in terms of performance, again if you are trying to test the behavior of a full production site (in terms of DB entries etc) then its time to emulate the production environment.
If that is not the case, then I would start checking the underlying models / code of the project.
Well, if you don't want to use django dev server you will have to spend some time to setup anyway. But the good part is that you can do it only once. Sequential deploying will take very little time.
Not so much time ago I switched from fastcgi to uWSGI and it made my life much easier.
uWSGI is awesome! It has autoreload (which works both in daemon mode and when launched directly in terminal). When launched in terminal you can use debugger (e.g. pdb) during request just like you do in django dev-server. And of course you can debug with print in simple cases.
I'm using it with nginx which serves both static and uWSGI but it of course can be any server.
The most useful feature for me in this configuration is that you use the same thing both for dev and production. For simple projects after developing you just turn off autoreload and a few other options and it's ready.
First of all please let me be clear that I am a windows user and very new to the web world. For the past months I have been learning both python and django, and it has been a great experience for me. Now I have somehow created a small project that I would like to deploy in the production server. Since django has its built-in development server there was no problem for me. But now that I have to deploy it to a production server I googled around and found Nginx + uWSGI or Nginx + Gunicorn as the best option for it. And as uWSGI and Gunicord are incompatible with Windows, I think I should adapt Ubuntu or other Unix system.
So my questions are:
Just to be clear, as I will have to work with one of the above, please explain to me why do I need two servers?
If I have to adapt the Ubuntu environment, do I have to learn Ubuntu shell scripting, SSH and other stuff? Or the hosting provider will help me do that?
Please let me be aware of what else do I need for the above concerned.
Thank you so much for your time and please pardon if my question was a lame question. Hoping for positive response answers.
A typical configuration involves two server processes (which can be run together on the same actual hardware or virtual server) so that the proxy server in front can buffer slow clients. For instance: a slow client will connect to nginx with a request. Nginx will pass the request on to Gunicorn and Gunicorn will respond. Nginx will then consume the Gunicorn response immediately, freeing up the Gunicorn resources right away. At that point, the slow client can take as much time as it wants to consume the response from Nginx without tying up much in the way of server resources. Alternatives to the two-server-process model are to use async workers with Gunicorn and put Gunicorn itself in front, or to use an async-sync combo like Waitress. Nginx in front has the added benefit of doubling as a ready-to-use statics server, though.
Note that "slow clients" can describe: mobile phones that lose their connection and leave the TCP socket hanging until timeout mid-request; mobile phones that are just slow; unreliable connections of all types; hostile denial-of-service clients who are deliberately trying to use server resources; sometimes any old connection that has a hiccup or malfunction for any reason. So this is a problem that will affect nearly any site.
You won't need shell scripting per se but getting used to Ubuntu will take some time. There is a lot to learn even outside of scripting, like how to use the package manager, how to configure packages once they're installed in ways that won't confound future updates, etc. And you will definitely have to learn to use SSH; it is one of the most fundamental server administration tools in the *nix world.
An alternative to learning to use Ubuntu or another server platform is to use a Platform-as-a-Service option like Heroku, as PaaS hosting providers really will take care of all of that stuff for you. I recommend this approach. That having been said, even though I think PaaS is a good option for people who want to focus on development and not server admin regardless of their level of skill, it's also true that a little bit of experience with Linux server platforms goes a long way in helping you to understand the environment that your code runs in. So even if you go with PaaS, you would still benefit from tinkering with Ubuntu a little (or a lot).
Another benefit from a PaaS is that normally their infrastructure handles the Nginx part of the deal (buffering of slow requests via proxy). This is the case with Heroku, for instance. So you won't have to worry about that part of the infrastructure at all.
This part of the question is too broad to answer, but let me know in the comments if you need clarification.
I'm doing it almoast like in this tutorial: http://michal.karzynski.pl/blog/2013/06/09/django-nginx-gunicorn-virtualenv-supervisor/
Nginx is my proxy to django app running on gunicorn and its serving statics, virtualenv for my python enviroment, supervisor to watch my app's running.
It's possible you will run in some error's if not using Postgresql, ask then I will help (used MySQL in the past now it's Postgresql)
Firstly, there's no need to use Ubuntu if you're happier with Windows. I don't know if nginx works on Windows, but I'd be very surprised if it doesn't (in fact, here are the nginx docs for installing on Windows). Apache, meanwhile, definitely does work on Windows. The Django documentation has a full explanation of how to set up Apache/mod_wsgi to serve Django.
You don't need two servers. I'm not sure why you think you do: the usual reason for that is to have the static assets on a separate server, but you don't mention that as a reason. Since you're only talking about a small site, though, you don't even need to do that. One server configured to serve both Django and the static assets will do fine. Again, the docs explain exactly how to do that.
Does anyone have a good way to set up multiple CFML engines, and versions of them, together in a suitable environment for cross testing a CFML based application.
Ideally, I'd like this to be Ubuntu Server based as I'm using it with VirtualBox (under Windows 7). Plus it'd be helpful if it was possible to switch between, so my laptop can cope with one at a time rather than all running at once. I'm thinking of the following:
Adobe ColdFusion 9
Adobe ColdFusion 10
Railo 3.3.x
Railo 4.x
OpenBD 2.x
I'd also like to get them serving from the same shared directory, so I don't have to have a copy of the code for each engine. Cheers
You mentioned being able to "switch between, so my laptop can cope with one at a time rather than all running at once", I'm guessing that you are thinking that each one will run on a different VM, or that they might require a huge amount of memory. I don't think you need to worry about that. Unless you require that they be on different machines, I think you could do this all on one VM and with one instance of a servlet container (like Tomcat).
From a high-level view, here is how I would do it.
Install Tomcat
Create or download .wars for each of the engines.
Deploy said .wars to that one instance of Tomcat
Set up Tomcat to use each of those servlets from a different host name (server.xml)
Create a code directory outside of Tomcat for your one copy of the code
Set up a Symbolic link in each webapp to link the code folder into the servlet
You should then be able to hit the same source from each engine by visiting the different host names in the browser.
I may be missing something. It has been a long time since I set something like this up. You'll likely need to make a bunch of tweaks (JVM settings, switching to Sun/ORACLE JVM vs. OpenJDK, etc).
I don't think running this many engines will cause you great trouble. In my experiences, for development, I have had 3 instances of CF9 running on Tomcat using only 189mb of RAM. And each additional instance did not increase that number by 1/3. Far less. It would not surprise me if you could run all of those handily with less than 512md of RAM. Possibly even 256mb if you are really hurting on memory.
I hope this helps.
For ColdFusion 10, Railo and OpenBD you would be looking at deploying with standalone installations of Tomcat, Jetty or JBoss.
ColdFusion 9, probably the easiest solution is "Enterprise Multiserver configuration" setup.
With these kinds of installation they are pretty much platform agnostic.
The things to be aware of are the web server, proxy and jndi ports that are used by each installation, but only if you want to run more than one server at a time.
After that it's whether you are bothered about proxying from apache or Nginx to the server instances and the connector you want to use.
No idea if this helps...
Since you've mentioned the VirtualBox, I'll share my personal approach to this task. It includes few fairly simple steps:
Install Ubuntu Server as VirtualBox guest (host is also Ubuntu).
Set up only basic software like JVM and updates. Set up virtual
machine networking as bridged adapter to use my Wi-Fi connection.
Configure my Wi-Fi router DHCP to assign static IP for MAC address of the virtual machine.
Add entry to my (host) system hosts: ip_assigned_to_vm virtual.ubuntu
Set up guest additions and mount my ~/www directory inside the machine to access web applications.
Now, when I need another machine for experiments, or some other configuration of software (I've tested ACF 10 and Railo 4 this way) I do two things:
Clone existing clean machine.
Make sure it is using the same MAC address with bridged interface.
That's it.
It doesn't matter which of the machines I run, they all can be accessed as http://virtual.ubuntu (of course, it requires proper web-server configuration on the guest). Same time they are independent and it is completely safe to make anything I wish and test anything that runs on Ubuntu.
Obvious downsides are that I can run just one machine at a time, plus much more disk space is used. Not a problem to me.
I've tried approach with Tomcat and multiple WARs, but it has couple of issues: I can't use different JVM and Tomcat settings, also if I screw the setup -- all the Tomcat hosts are down.
Hope this helps.
I know that it's bad to use the Django web server in production. There's been at least one Stackoveflow question on this already.
But I'm wondering about where to draw the line between development and production? If I'm only allowing HTTP access to one (or a few) IP addresses, then I know I'm in development. What if I open it to all IP addresses, but only e-mail a couple friends to see what they think of what I've built?
As far as I can tell, the problems with using the Django server are:
It's single-threaded
Security
I don't think (1) is likely to be an issue if I'm only sharing it with a few people. For (2)--what's the worst-case scenario? Does it make a difference that I'm running on an Amazon EC2 server that I could very easily restart from a backup if something bad happened?
Well, the answer is very simple actually, you've left development when you have something you must protect: real user personal information, real data in your database that you'd be afraid to lose, etc.
Security isn't a concern until these things are present. The rule about not using the dev server in "production" is guidance, not mandatory. You can fire up the dev server in your production environment any time you want. However, you'd be silly to do so and then open up universal access to it, once your site is truly live and in use by the world.
Setting up mod_wsgi (or some other WSGI container) on a development machine takes all of 5 minutes, and can help you sort out deployment issues before you actually reach deployment. So really, why ever use the development server if you don't have to?
I am a total newbie in servers/hosting etc, although I have some experience in programming in C,Java,etc. So excuse me if the question is 'absurd'.
I recently bought service from a hosting site,namely this(hostmds). I have some code I've written in C++ and I want to run it in the hosting site. So my question is:
Is this possible, or will I have to rewrite everything in a new language?
What should my approach be?
Edit: I have a Shared-Hosting account.
You will have to get a "virtual private server" account from your host in order to do this. This will enable you to compile your program on your host machine and run it essentially as if it were a separate machine under your control.
This means you will also be responsible for maintaining your own HTTP server program (such as Apache, if running on a Linux/Unix host), and your own database servers and other support.
If you have a "shared hosting" account (the most common low cost option) with SSH support, you may be able to compile your program, and even run it, but you will be subject to the whims (capricious or otherwise) of the administrators of your system (that it, you may find that libraries you need are removed or moved around)
What type of hosting is this?
What kind of application is this, is it a daemon?
Depending on the amount of access rights you have, you can run the code in the cgi-bin folder or through the shell of the server.
Depending on the OS/compiler you've used to write your code in you might have to modify some things so that it'll work on the target OS. You should probably add some more details. :)
Many hosting services provide CGI/FastCGI/SCGI that can be used for running C++ webapps. However, it depends on your host whether you can actually do this, as it may be difficult to get binaries built on some other system to run on the web hosting service (if you even can upload them in the first place).
On shell services and virtual servers you can also run daemons (that directly listen to a port), but especially on shell services you cannot listen on low ports (0..1024), for security reasons.
Notice that the cheapest hosting packages generally only allow PHP at most, so you will need something more expensive for more access.
It is best to ask the hosting provider for further information, as these things wildly differ from host to another.