I'm trying to use server push with my Django app. Of all the different solutions I've seen, django-socketio seemed to be the easiest to implement, and I got it working when launching the server through manage.py. However, when it goes to production, I'd like it to be served through Apache. In the past, I've done this with wsgi, but django-socketio isn't playing nicely with the default wsgi script. Is there something simple I can just change in my django.wsgi that lets apache do the right thing? If not, what would be the suggested way to handle this?
EDIT: Here's the WSGI script I was normally using (without any kind of push server), plus a bit more explanation.
import os, sys
locale = os.path.realpath(__file__)
ROOT_DIR = locale[:locale.find('/server/apache/')]
sys.path.append(ROOT_DIR)
sys.path.append(ROOT_DIR+'/server')
os.environ['DJANGO_SETTINGS_MODULE'] = 'server.settings'
os.environ['PYTHON_EGG_CACHE'] = '/var/www/.python-eggs'
import django.core.handlers.wsgi
application = django.core.handlers.wsgi.WSGIHandler()
sys.stdout = sys.stderr
The sys.stdout = sys.stderr is just to redirect any test print statements to apache's error log. The problem is, when I use this, I get an error complaining that request.environ doesn't have the key "socketio" or "DJANGO_SOCKETIO_PORT". I can add DJANGO_SOCKETIO_PORT easily enough (os.environ['DJANGO_SOCKETIO_PORT']="9000"), but from what I can tell, request.environ['socketio'] is set to an instance of SocketIOProtocol somewhere in django-socketio's internals. Also, after looking at the command that django-socketio added to manage.py, I noticed that it creates an instance of SocketIOServer, and calls serve_forever on it, but I have no idea where to put that in my code. Hopefully this will make it easier to see what I'm trying to get done.
Related
Deploying my django website with S3 as storage which runs fine locally to pythonanywhere gives a strange error I can't google a solution for:
"TypeError: a bytes-like object is required, not 'str'"
What I'm doing wrong?
I've tried to put my environment variables out of settings.env (aws keys, secret_key, etc) ad set them directly in my settings.py app. + every suggestion I could find but it's still the same :(
here's my /var/www/username_pythonanywhere_com_wsgi.py:
# +++++++++++ DJANGO +++++++++++
# To use your own Django app use code like this:
import os
import sys
from dotenv import load_dotenv
project_folder = os.path.expanduser('~/portfolio_pa/WEB') # adjust as appropriate
load_dotenv(os.path.join(project_folder, 'settings.env'))
# assuming your Django settings file is at '/home/myusername/mysite/mysite/settings.py'
path = '/home/corebots/portfolio_pa'
if path not in sys.path:
sys.path.insert(0, path)
os.environ['DJANGO_SETTINGS_MODULE'] = 'WEB.settings'
## Uncomment the lines below depending on your Django version
###### then, for Django >=1.5:
from django.core.wsgi import get_wsgi_application
application = get_wsgi_application()
###### or, for older Django <=1.4
#import django.core.handlers.wsgi
#application = django.core.handlers.wsgi.WSGIHandler()
I'd expect the website to run fine just like it does locally.
Boto library doesn't have a good Python3 support. This particular issue is known in the boto bugtracker: https://github.com/boto/boto/issues/3837
The best way of fixing this is to use boto3 which has decent Python3 support and is a generally most supported AWS SDK for Python.
The reason why it works on your local machine and doesn't work on production is that pythonanywhere setup seems to be using proxy which triggers this incompatible boto code. See the actual calling code: https://github.com/boto/boto/blob/master/boto/connection.py#L747
Your error traceback confirms this.
Unfortunately, I'm not familliar with the django-photologue, but a brief look doesn't suggest that it strongly depends on boto3. Maybe I'm wrong.
I still think that the best way is to go with boto3. As a backup strat you can fork boto with a fix for this issue and install that instead of the official one from PyPI: https://github.com/boto/boto/pull/3699
I'm trying to get Falcon up and running on Webfaction. I'm not exactly a network guru, so I'm having a tough time wrapping my head around how these applications are served.
My Webfaction App is set up as mod_wsgi 4.5.3/Python 2.7
To my understanding, Falcon will run on any WSGI server. When I swet up my mod_wsgi server, is it automatically configured for something like Falcon to run on? Or do I still need to install something like Gunicorn?
When I set up my webfaction app, I received a directory structure like this:
app/htdocs/index.py
And inside the index.py file, I put the example found at Falcon Tutorial
import falcon
class ThingsResource(object):
def on_get(self, req, resp):
"""Handles GET requests"""
resp.status = falcon.HTTP_200
resp.body = 'Hello world!'
# falcon.API instances are callable WSGI apps
wsgi_app = api = falcon.API()
# Resources are represented by long-lived class instances
things = ThingsResource()
# things will handle all requests to the '/things' URL path
api.add_route('/things', things)
I understand there are also instructions for running WSGI, but that is where my confusion is at - is the webfaction server already running WSGI, or do I still need something like Gunicorn, and if so - what is the best way to configure? Do I need a cron to keep running Gunicorn?
Thanks!
Update:
I checked error logs and received a WSGI error about not having a variable named "application",
So I changed:
wsgi_app = api = falcon.API()
to:
application = falcon.API()
This cleared out the error, but now when I visit mydomain.com/things, I get an error 404 (Not found / Does not exist).
So, this brings me back to my original question of what the next steps are? It seems as if the url isn't being routed correctly, so it is most likely something to do with httpd.conf file or similar - again, this is my first go at getting something like this set up live.
Here is the answer (at least for the initial question, I'm willing to bet I'll mess up something else in the near future on the same project).
Essentially, I was able to put the tutorial code in the index.py file that Webfaction generates when setting up an app & mounting on a domain. So, my tutorial code looks something like this:
import falcon
class ThingsResource(object):
def on_get(self,req,resp):
resp.status = falcon.HTTP_200
resp.body = 'Hello World!'
api = application = falcon.API()
things = ThingsResource()
api.add_route('/things', things)
Since I couldn't find much info for launching a Falcon app on Webfaction, I looked at how similar applications run on Webfaction (Flask in this example). That being said, I found a snippet on the Flask docu showing how to get set up on webfaction. I'm not sure if this means my entire application will work, but I do know that the Falcon tutorial code at least works. Essentially, I just had to edit the httpd.conf file per the instructions found here: Flask Webfaction
WSGIPythonPath /home/yourusername/webapps/yourapp/htdocs/
#If you do not specify the following directive the app *will* work but you will
#see index.py in the path of all URLs
WSGIScriptAlias / /home/yourusername/webapps/yourapp/htdocs/index.py
<Directory /home/yourusername/webapps/yourapp/htdocs/>
AddHandler wsgi-script .py
RewriteEngine on
RewriteBase /
WSGIScriptReloading On
</Directory>
I hope this helps anybody with a similar issue.
I'm writing my first Flask application. The application itself runs fine. I just have a newbie question about logging in production mode.
The basic structure:
app/
app/templates/
app/static
config.py
flask/... <- virtual env with flask + extensions
run.py
The application is started by run.py script:
#!flask/bin/python
import os.path
import sys
appdir = os.path.dirname(os.path.abspath(__file__))
if appdir not in sys.path:
sys.path.insert(1, appdir)
from app import app as application
if __name__ == '__main__':
application.run(debug=True)
and is started either directly or from an Apache 2.4 web server. I have these lines in the apache config:
WSGIPythonHome /usr/local/opt/app1/flask
WSGIScriptAlias /app1 /usr/local/opt/app1/run.py
In the former case, the debug=True is all I need for the development.
I'd like to have some logging also for the latter case, i.e. when running under Apache on a production server. Following is a recommendation from the Flask docs:
if not app.debug:
import logging
from themodule import TheHandlerYouWant
file_handler = TheHandlerYouWant(...)
file_handler.setLevel(logging.WARNING)
app.logger.addHandler(file_handler)
It needs some customization, but that's what I want - instructions for the case when app.debug flag is not set. Similar recommendation was given also here:
How do I write Flask's excellent debug log message to a file in production?
Please help: where do I have to put this code?
UPDATE: based on the comments by davidism and the first answer I've got I think the app in the current simple form is not suitable for what I was asking for. I will modify it to use different sets of configuration data as recommended here: http://flask.pocoo.org/docs/0.10/config/#development-production . If my application were larger, I would follow the pech0rin's answer.
UPDATE2: I think the key here is that the environment variables should control how the application is to be configured.
I have had a lot of success with setting up my logging configurations inside a create_app function. This uses the application factory pattern. This allows you to pass in some arguments or a configuration class. The application is then specifically created using your parameters.
This allows you initialize the application, setup logging, and do whatever else you want to do, before the application is sent back to be run.
For example:
def create_app(dev=False):
app = Flask(__name__)
if dev:
app.config['DEBUG'] = True
else:
...
app.logger.addHandler(file_handler)
return app
This has worked very well for me in production environments. YMMV
I'm working on a system that has two django projects. A server and a client. The server is responsible for managing several client instances. This system relies on Sentry/Raven to process error logging.
My problem is that Sentry needs me to create and configure each client(sentry project) by hand. Since the number of client instances is large and I already have to do this by hand on my server project. I was trying to automatize the process, so that when I create a new client on the server, it creates a new Sentry project.
Much like in this question, I tried to access directly to the Sentry ORM on my project. But this revealed to be a dead end. So I wrote a python scrypt, to do this.
In said script, I import the DJANGO_SETTINGS_MODULE from sentry and work my way around with it until I have what I need.
sys.path.append("/sentry/")
os.environ.setdefault("DJANGO_SETTINGS_MODULE", 'sentry_configuration_file')
from sentry.models import *
#Do my thing here
If I run the script on my shell, it works perfectly.
However, when I use subprocess to call it inside of my Django project
from subprocess import call
call("/sentry/venv/bin/python /sentry/my_script.py", shell=True)
The script generates the following error on the "from sentry.models import * line:
ImportError("Could not import settings '%s' (Is it on sys.path?): %s" % (self.SETTINGS_MODULE, e))
ImportError: Could not import settings 'configurations.settings' (Is it on sys.path?): No module named configurations.settings
You may have noticed that sentry is installed inside a virtualenv. However, I don't need it activated when I call this script on my bash, as long as I provide the correct path to the virtualenv's python.
I'm lost here. I see no reason in particular for the script to fail using subprocess.call when it runs fine using the shell.
Any pointers would be greatly apreciated.
Thanks.
If anyone ever comes across with this question, I managed to solve the issue by replacing subprocess.call by subprocess.Popen
The cool thing about Popen is that you can specify the environment of the process with the argument "env"
So
my_env = os.environ
my_env["DJANGO_SETTINGS_MODULE"] = "sentry_configuration_file"
result = Popen(command, shell=True, env=my_env)
Worked like a charm.
I seem to have run into a bit of an issue.
I am busy creating an app, and over the last few weeks setup my server to use Git, mod_wsgi to host this app.
Since deploying it, everything seems to be running smoothly however, I had to go through all my files and insert the absolute url of the project to make sure it works fine.
on my local machine
from registration.models import UserRegistration
on server
from myapp.registration.models import UserRegistration
Am I doing something wrong?
And this has also caused an issue for me where I cannot access my django admin interface.
All i get is this: Caught an exception while rendering: No module named registration
Exception Value: Caught an exception while rendering: No module named registration
As far as I am concerned my app has all the relevant urls, but it does not seem to work.
Thank you in advance
The problem is occurring because somehow your local machine is adding the myapp directory to the PYTHONPATH, as well as its parent directory. The way to fix this is to modify your .wsgi script to add both these directories to sys.path:
import sys
sys.path.insert(0, '/path/to/parent')
sys.path.insert(0, '/path/to/parent/myapp')
Read and use improved WSGI script in:
http://blog.dscpl.com.au/2010/03/improved-wsgi-script-for-use-with.html
This will set up environment to better match Django built in development server and you shoud hopefully not see a difference between the two, especially in respect of how Python module search path is handled.