I'm working on a system that has two django projects. A server and a client. The server is responsible for managing several client instances. This system relies on Sentry/Raven to process error logging.
My problem is that Sentry needs me to create and configure each client(sentry project) by hand. Since the number of client instances is large and I already have to do this by hand on my server project. I was trying to automatize the process, so that when I create a new client on the server, it creates a new Sentry project.
Much like in this question, I tried to access directly to the Sentry ORM on my project. But this revealed to be a dead end. So I wrote a python scrypt, to do this.
In said script, I import the DJANGO_SETTINGS_MODULE from sentry and work my way around with it until I have what I need.
sys.path.append("/sentry/")
os.environ.setdefault("DJANGO_SETTINGS_MODULE", 'sentry_configuration_file')
from sentry.models import *
#Do my thing here
If I run the script on my shell, it works perfectly.
However, when I use subprocess to call it inside of my Django project
from subprocess import call
call("/sentry/venv/bin/python /sentry/my_script.py", shell=True)
The script generates the following error on the "from sentry.models import * line:
ImportError("Could not import settings '%s' (Is it on sys.path?): %s" % (self.SETTINGS_MODULE, e))
ImportError: Could not import settings 'configurations.settings' (Is it on sys.path?): No module named configurations.settings
You may have noticed that sentry is installed inside a virtualenv. However, I don't need it activated when I call this script on my bash, as long as I provide the correct path to the virtualenv's python.
I'm lost here. I see no reason in particular for the script to fail using subprocess.call when it runs fine using the shell.
Any pointers would be greatly apreciated.
Thanks.
If anyone ever comes across with this question, I managed to solve the issue by replacing subprocess.call by subprocess.Popen
The cool thing about Popen is that you can specify the environment of the process with the argument "env"
So
my_env = os.environ
my_env["DJANGO_SETTINGS_MODULE"] = "sentry_configuration_file"
result = Popen(command, shell=True, env=my_env)
Worked like a charm.
Related
Deploying my django website with S3 as storage which runs fine locally to pythonanywhere gives a strange error I can't google a solution for:
"TypeError: a bytes-like object is required, not 'str'"
What I'm doing wrong?
I've tried to put my environment variables out of settings.env (aws keys, secret_key, etc) ad set them directly in my settings.py app. + every suggestion I could find but it's still the same :(
here's my /var/www/username_pythonanywhere_com_wsgi.py:
# +++++++++++ DJANGO +++++++++++
# To use your own Django app use code like this:
import os
import sys
from dotenv import load_dotenv
project_folder = os.path.expanduser('~/portfolio_pa/WEB') # adjust as appropriate
load_dotenv(os.path.join(project_folder, 'settings.env'))
# assuming your Django settings file is at '/home/myusername/mysite/mysite/settings.py'
path = '/home/corebots/portfolio_pa'
if path not in sys.path:
sys.path.insert(0, path)
os.environ['DJANGO_SETTINGS_MODULE'] = 'WEB.settings'
## Uncomment the lines below depending on your Django version
###### then, for Django >=1.5:
from django.core.wsgi import get_wsgi_application
application = get_wsgi_application()
###### or, for older Django <=1.4
#import django.core.handlers.wsgi
#application = django.core.handlers.wsgi.WSGIHandler()
I'd expect the website to run fine just like it does locally.
Boto library doesn't have a good Python3 support. This particular issue is known in the boto bugtracker: https://github.com/boto/boto/issues/3837
The best way of fixing this is to use boto3 which has decent Python3 support and is a generally most supported AWS SDK for Python.
The reason why it works on your local machine and doesn't work on production is that pythonanywhere setup seems to be using proxy which triggers this incompatible boto code. See the actual calling code: https://github.com/boto/boto/blob/master/boto/connection.py#L747
Your error traceback confirms this.
Unfortunately, I'm not familliar with the django-photologue, but a brief look doesn't suggest that it strongly depends on boto3. Maybe I'm wrong.
I still think that the best way is to go with boto3. As a backup strat you can fork boto with a fix for this issue and install that instead of the official one from PyPI: https://github.com/boto/boto/pull/3699
I am trying to run a custom django management command from my views. I have the view ready to execute the command as shown below:
from django.core.management import call_command
import django
def send_queued_mails():
# Run Django Setup
django.setup()
call_command('send_all_queued_mails')
But, when the command is executed on my windows machine, I get the following error:
os.symlink(self.pid_filename, self.lock_filename)
OSError: symbolic link privilege not held
I can tackle this manually by running the terminal as Administrator but I want to run the command through my views and with escalated privileges.
Any ideas or suggestions are appreciated.
PS: I also tried using OS level command as shown below:
from subprocess import call
call(["python", "manage.py", "send_all_queued_mails"])
But I am getting the same error as above.
I found a solution which works well for me and might work for others as well. I am using django-post_office for sending emails. I traced back to the line where the error originated, it was as follows:
if hasattr(os, 'symlink'):
os.symlink(self.pid_filename, self.lock_filename)
else:
# Windows platforms doesn't support symlinks, at least not through the os API
self.lock_filename = self.pid_filename
The comments clearly stated that Windows does not support symlink, so, I modified the code a bit to avoid the error.
if hasattr(os, 'symlink') and platform.system() != 'Windows':
os.symlink(self.pid_filename, self.lock_filename)
else:
# Windows platforms doesn't support symlinks, at least not through the os API
self.lock_filename = self.pid_filename
This is not the exact solution as it does not solve the problem of escalated privileges while running that command. But, if you are facing a similar error, you can directly assign the file you want to create the symlink for to the desired file.
If anyone knows a better way please do answer.
I have been running into this problem for a short while now and simply can't find a solution anywhere. I am using Google App Engine to run a default Python 2.7 app with Django 1.5 (via GAE SDK) created through PyCharm. I can upload the app successfully, but upon visiting the actual page, I get a Server Error. Then, checking the logs in Google App Engine, I see this:
ImportError: <module 'main' from '/base/data/home/apps/s~eloquent-ratio-109701/1.388053784931450315/main.pyc'> has no attribute application
After searching the internet for a while, I was able to find a few posts which address this issue, but attempting them never seemed to solve my problem. For example: This problem was solved by replacing "application" with "app" in the following lines:
application = django.core.handlers.wsgi.WSGIHandler()
util.run_wsgi_app(application)
In fact, I had run into this same issue before and this solution provided a fix for me in the past, however at that time I was running a separate app and it was not through the GAE.
I checked the Django documentation for version 1.5 here, but the code and suggestions there don't seem to conflict with what I currently have in my project.
I read a bit more about this type of problem, and saw another post that suggested checking the app's wsgi.py file to ensure that it is named 'application' or 'app' respectively, so that one could then use that same name throughout the rest of the application. However, upon checking those settings I saw that 'application' was used there too:
from django.core.wsgi import get_wsgi_application
application = get_wsgi_application()
There's even a line in settings.py which uses the same nomenclature to declare the WSGI application:
WSGI_APPLICATION = 'Chimera.wsgi.application'
I'm really having trouble debugging this. I get the feeling it's really dumb and I just can't see it, but unfortunately I'm not particularly good at this kind of stuff -- I'm still a bit of a novice in this field.
Does anyone have any idea what I could try in an attempt to fix this issue?
UPDATE: I started making line by line changes and testing things, and eventually I found that the GAE log changes depending on the input for the "script" under app.yaml. So if I change the script under "handlers" between "main.app" and "main.application", it adjusts the log output to refer to "app" or "application" respectively. So that line in the app.yaml file tells the app what to look for, but I'm still not seeing why it can't be found. Not sure what else I could change to test it out. I wish I knew a bit more about the actual inner workings so that I could figure out why the app is confused about the attribute. Is it trying to run before it even gets instantiated or something?
Source code below:
main.py
import os, sys
os.environ['DJANGO_SETTINGS_MODULE'] = 'Chimera.settings'
from google.appengine.ext.webapp import util
from django.conf import settings
settings._target = None
import django.core.handlers.wsgi
import django.core.signals
import django.db
import django.dispatch.dispatcher
def main():
application = django.core.handlers.wsgi.WSGIHandler()
util.run_wsgi_app(application)
if __name__ == '__main__':
main()
app.yaml
application: eloquent-ratio-109701
version: 1
runtime: python27
api_version: 1
threadsafe: true
handlers:
- url: /.*
script: main.application
libraries:
- name: django
version: 1.5
wsgi.py
import os
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "Chimera.settings")
from django.core.wsgi import get_wsgi_application
application = get_wsgi_application()
Full log from GAE:
Traceback (most recent call last):
File "/base/data/home/runtimes/python27/python27_lib/versions/1/google/appengine/runtime/wsgi.py", line 240, in Handle
handler = _config_handle.add_wsgi_middleware(self._LoadHandler())
File "/base/data/home/runtimes/python27/python27_lib/versions/1/google/appengine/runtime/wsgi.py", line 302, in _LoadHandler
raise err
ImportError: <module 'main' from '/base/data/home/apps/s~eloquent-ratio-109701/1.388053784931450315/main.pyc'> has no attribute application
Thanks for helping me out.
In your main.py file (i.e. the main module) application is a variable inside the main() function, not an attribute of the main module. Basically you don't need a main() function.
GAE has some specific support for using Django, I'd strongly suggest going through the Django Support documentation and the Django App example.
Based on the comment made by #DanielRoseman I discovered that declaring the app inside of the main() function caused an issue because the app attribute was then only accessible at the main() function level, as it was a member variable of main() as opposed to a global variable. Although the default application files were structured this way by PyCharm, it seems that it was incorrect. I'm not sure if this is a compatibility issue, but regardless, moving the app declaration outside of the main() function adjusts the scope in a way which allows for other parts of the project to access it, solving my problem.
Thank you #DanielRoseman for the comment.
I'm trying to use server push with my Django app. Of all the different solutions I've seen, django-socketio seemed to be the easiest to implement, and I got it working when launching the server through manage.py. However, when it goes to production, I'd like it to be served through Apache. In the past, I've done this with wsgi, but django-socketio isn't playing nicely with the default wsgi script. Is there something simple I can just change in my django.wsgi that lets apache do the right thing? If not, what would be the suggested way to handle this?
EDIT: Here's the WSGI script I was normally using (without any kind of push server), plus a bit more explanation.
import os, sys
locale = os.path.realpath(__file__)
ROOT_DIR = locale[:locale.find('/server/apache/')]
sys.path.append(ROOT_DIR)
sys.path.append(ROOT_DIR+'/server')
os.environ['DJANGO_SETTINGS_MODULE'] = 'server.settings'
os.environ['PYTHON_EGG_CACHE'] = '/var/www/.python-eggs'
import django.core.handlers.wsgi
application = django.core.handlers.wsgi.WSGIHandler()
sys.stdout = sys.stderr
The sys.stdout = sys.stderr is just to redirect any test print statements to apache's error log. The problem is, when I use this, I get an error complaining that request.environ doesn't have the key "socketio" or "DJANGO_SOCKETIO_PORT". I can add DJANGO_SOCKETIO_PORT easily enough (os.environ['DJANGO_SOCKETIO_PORT']="9000"), but from what I can tell, request.environ['socketio'] is set to an instance of SocketIOProtocol somewhere in django-socketio's internals. Also, after looking at the command that django-socketio added to manage.py, I noticed that it creates an instance of SocketIOServer, and calls serve_forever on it, but I have no idea where to put that in my code. Hopefully this will make it easier to see what I'm trying to get done.
I seem to have run into a bit of an issue.
I am busy creating an app, and over the last few weeks setup my server to use Git, mod_wsgi to host this app.
Since deploying it, everything seems to be running smoothly however, I had to go through all my files and insert the absolute url of the project to make sure it works fine.
on my local machine
from registration.models import UserRegistration
on server
from myapp.registration.models import UserRegistration
Am I doing something wrong?
And this has also caused an issue for me where I cannot access my django admin interface.
All i get is this: Caught an exception while rendering: No module named registration
Exception Value: Caught an exception while rendering: No module named registration
As far as I am concerned my app has all the relevant urls, but it does not seem to work.
Thank you in advance
The problem is occurring because somehow your local machine is adding the myapp directory to the PYTHONPATH, as well as its parent directory. The way to fix this is to modify your .wsgi script to add both these directories to sys.path:
import sys
sys.path.insert(0, '/path/to/parent')
sys.path.insert(0, '/path/to/parent/myapp')
Read and use improved WSGI script in:
http://blog.dscpl.com.au/2010/03/improved-wsgi-script-for-use-with.html
This will set up environment to better match Django built in development server and you shoud hopefully not see a difference between the two, especially in respect of how Python module search path is handled.