I am trying to make logparser.py in django project which parses the data coming from different servers.
And on running the command on terminal :
$ python logparser.py
This error is coming :
Traceback (most recent call last):
File "logparser.py", line 13, in <module>
SMTP_CONF = settings.SMTP_CONF
File "/home/arya/.virtualenv/Devel/.virtualenvs/hu/local/lib/python2.7/site-packages/django/conf/__init__.py", line 53, in __getattr__
self._setup(name)
File "/home/arya/.virtualenv/Devel/.virtualenvs/hu/local/lib/python2.7/site-packages/django/conf/__init__.py", line 48, in _setup
self._wrapped = Settings(settings_module)
File "/home/arya/.virtualenv/Devel/.virtualenvs/hu/local/lib/python2.7/site-packages/django/conf/__init__.py", line 134, in __init__
raise ImportError("Could not import settings '%s' (Is it on sys.path?): %s" % (self.SETTINGS_MODULE, e))
ImportError: Could not import settings 'hma.settings' (Is it on sys.path?): No module named hma.settings
my logparser.py contains:
import re
import os
import fnmatch
import gzip
import bz2
from collections import defaultdict
from django.core.mail import send_mail
from django.core.mail.backends import smtp
import smtplib
from email.mime.text import MIMEText
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "hma.settings")
from django.conf import settings
SMTP_CONF = settings.SMTP_CONF
def send_email(self,fromaddress,toaddresses,content,subject):
smtp_server = SMTP_CONF["SERVER"]
smtp_username = SMTP_CONF["USERNAME"]
smtp_password = SMTP_CONF["PASSWORD"]
smtp_port = SMTP_CONF["PORT"]
msg = MIMEText(content, 'html', _charset='utf-8')
msg['Subject'] ='Alert message for bad and internal server error'
msg['From'] = fromaddress
msg['To'] = toaddresses
server = smtplib.SMTP(smtp_server,smtp_port)
server.starttls()
server.login(smtp_username,smtp_password)
server.send_mail(fromaddress,toaddresses,msg.as_string())
server.quit()
return True
I know I am doing wrong something with command [python manage.py], but i need to run like this. Any solution for this exporting django settings to separate python file??
Well, This is the exact Usecase why Django provided an ability to create custom commands. You can use all the features of django, in your script, Its like your script will be running inside a Django Container. Here is the Documentation https://docs.djangoproject.com/en/dev/howto/custom-management-commands/.
In case you don't want to use custom management commands though there is also a simple way to run your code within Django's context. Simply put the following at the beginning of your python file, you want to run:
from django.conf import settings
from django.core import management
management.setup_environ(settings)
Related
stucture of the app
flask_app
__init__.py
models.py
routes.py
run.py
__init__.py
from flask import Flask
from flask_bcrypt import Bcrypt
import os
from flask_app.models import db
app = Flask(__name__)
db.init_app(app)
bcrypt = Bcrypt(app)
from flask_app import routes
models.py
from flask_sqlalchemy import SQLAlchemy
from flask_app import app, bcrypt
db = SQLAlchemy()
run.py
from flask_app import app
if __name__ == '__main__':
app.run(debug=True)
This happens first
Traceback (most recent call last):
File
"c:\users\cristovao\documents\mqs_development\flaskexperiment\env\lib\site-packages\flask\cli.py", line 240, in locate_app
__import__(module_name)
File "C:\Users\cristovao\Documents\MQS_Development\FlaskExperiment\run.py", line 4, in <module>
from flask_app import app # importing from __init__.py within flask_app package
File "C:\Users\cristovao\Documents\MQS_Development\FlaskExperiment\flask_app\__init__.py", line 6, in <module>
from flask_app.models import db
File "C:\Users\cristovao\Documents\MQS_Development\FlaskExperiment\flask_app\models.py", line 2, in <module>
from flask_app import app, bcrypt
ImportError: cannot import name 'app' from 'flask_app' (C:\Users\cristovao\Documents\MQS_Development\FlaskExperiment\flask_app\__init__.py)
and then in models.py after this error I write from flask__app.__init__ import app bcrypt and I got another error
Traceback (most recent call last):
File "c:\users\cristovao\documents\mqs_development\flaskexperiment\env\lib\site-packages\flask\cli.py", line 240, in locate_app
__import__(module_name)
File "C:\Users\cristovao\Documents\MQS_Development\FlaskExperiment\run.py", line 4, in <module>
from flask_app import app # importing from __init__.py within flask_app package
File "C:\Users\cristovao\Documents\MQS_Development\FlaskExperiment\flask_app\__init__.py", line 6, in <module>
from flask_app.models import db
File "C:\Users\cristovao\Documents\MQS_Development\FlaskExperiment\flask_app\models.py", line 2, in <module>
from flask_app.__init__ import app, bcrypt
File "C:\Users\cristovao\Documents\MQS_Development\FlaskExperiment\flask_app\__init__.py", line 6, in <module>
from flask_app.models import db
ImportError: cannot import name 'db' from 'flask_app.models' (C:\Users\cristovao\Documents\MQS_Development\FlaskExperiment\flask_app\models.py)
when I run my app using flask run it gives me those import errors cannot import app and bcrypt from flask_app to models.py and can't import db from models.py to __init__py. I dont understand why since run.py is being separated from my package (flask__app).
When I use your code, I get an error because of a circular import.
ImportError: cannot import name 'app' from partially initialized module 'flask_app' (most likely due to a circular import) (/home/jugmac00/Tests/stackoverflow/flask_app/__init__.py)
I can get rid of that when I remove this line from models.py
from flask_app import app, bcrypt
The easiest way to get rid of circular imports is to use the app factory pattern.
https://flask.palletsprojects.com/en/1.1.x/patterns/appfactories/
This application factory pattern took me a while to understand, but it is really worthwile to try to understand and use it.
There is a mindblowing good video on that, from this year's Flask-Conference:
https://www.youtube.com/watch?v=xNo-eOfZH5Q
If this sounds to hard, then my advice would be... just put everything in one file - that is not too bad, unless the app grows really big.
I've searched SO and Google and can't find an answer to my problem.
I've launched my virtualenv and ran this command in the terminal:
python bin/process_messages.py
and this error occurs:
Stacktrace:
Traceback (most recent call last):
File "bin/process_messages.py", line 6, in <module>
from xyz.models import get_sku
File "/Users/myname/.environments/xyz_env/lib/python3.6/site-packages/xyz/models.py", line 19, in <module>
class Suppliers(models.Model):
File "/Users/myname/.environments/xyz_env/lib/python3.6/site-packages/django/db/models/base.py", line 110, in __new__
app_config = apps.get_containing_app_config(module)
File "/Users/myname/.environments/xyz_env/lib/python3.6/site-packages/django/apps/registry.py", line 247, in get_containing_app_config
self.check_apps_ready()
File "/Users/myname/.environments/xyz_env/lib/python3.6/site-packages/django/apps/registry.py", line 125, in check_apps_ready
raise AppRegistryNotReady("Apps aren't loaded yet.")
django.core.exceptions.AppRegistryNotReady: Apps aren't loaded yet.
I thought it wasn't running django.setup so I added that to the script. Here is my code:
#!/usr/bin/env python
import os
import django
import boto3
from xyz.settings import SQS_QUEUE_NAME
from xyz.models import get_sku
__author__ = 'me'
def check_django_environment(default_settings):
# Environment setup for Django project files:
os.sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
if not os.environ.get('DJANGO_SETTINGS_MODULE'):
# Don't override settings if it is specified.
os.environ['DJANGO_SETTINGS_MODULE'] = default_settings
from django.conf import settings
return getattr(settings, 'DEBUG', None)
check_django_environment('xyz.settings')
django.setup()
# Get the service resource
sqs = boto3.resource('sqs')
# Get the queue
queue = sqs.get_queue_by_name(QueueName=SQS_QUEUE_NAME)
for message in queue.receive_messages():
print(message)
if message.message_attributes is not None:
print(message.message_attributes)
#sku = message.message_attributes
db_sku = get_sku(sku)
print(db_sku)
break
My Installed apps:
INSTALLED_APPS = [
'django.contrib.admin',
'django.contrib.auth',
'django.contrib.contenttypes',
'django.contrib.sessions',
'django.contrib.messages',
'django.contrib.staticfiles',
'xyz',
]
By importing get_sku, you are importing your models before django.setup() has run. You need to move this import down so it happens after django.setup().
In a stand-alone script that uses Django, I generally have two groups of imports. The first contains the bare minimum to get Django setup, the second contains all the other imports, including models etc.:
#!/usr/bin/env python
import os
import django
__author__ = 'me'
def check_django_environment(default_settings):
# Environment setup for Django project files:
os.sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
if not os.environ.get('DJANGO_SETTINGS_MODULE'):
# Don't override settings if it is specified.
os.environ['DJANGO_SETTINGS_MODULE'] = default_settings
from django.conf import settings
return getattr(settings, 'DEBUG', None)
check_django_environment('xyz.settings')
django.setup()
import boto3
from xyz.settings import SQS_QUEUE_NAME
from xyz.models import get_sku
In addition to knbk's answer, you can also just use manage.py.
Here's an example script, let's say script.py:
from xyz.models import get_sku
if __name__ == '__main__':
print(get_sku)
And you run it like:
./manage.py shell < script.py
Maybe not what you're looking for, but worth knowing, nonetheless.
I am trying to use Pycharm Community Edition to improve on my code in my Django application, but I cannot run all of my Django code that I'd like. I keep getting this traceback...
Traceback (most recent call last):
File "C:/Users/Jaysp_000/firstSite/PROJECTone/blog_static/views.py", line 1, in <module>
from django.views.decorators.csrf import csrf_exempt
File "C:\Python34\lib\site-packages\django\views\decorators\csrf.py", line 3, in <module>
from django.middleware.csrf import CsrfViewMiddleware, get_token
File "C:\Python34\lib\site-packages\django\middleware\csrf.py", line 14, in <module>
from django.utils.cache import patch_vary_headers
File "C:\Python34\lib\site-packages\django\utils\cache.py", line 26, in <module>
from django.core.cache import caches
File "C:\Python34\lib\site-packages\django\core\cache\__init__.py", line 34, in <module>
if DEFAULT_CACHE_ALIAS not in settings.CACHES:
File "C:\Python34\lib\site-packages\django\conf\__init__.py", line 48, in __getattr__
self._setup(name)
File "C:\Python34\lib\site-packages\django\conf\__init__.py", line 42, in _setup
% (desc, ENVIRONMENT_VARIABLE))
django.core.exceptions.ImproperlyConfigured: Requested setting CACHES, but settings are not configured. You must either define the environment variable DJANGO_SETTINGS_MODULE or call settings.configure() before accessing settings.
This error seems to involve the django.views.decortors.csrf.csrf_exempt that I imported to my views.py file. I've tried other files, and they have given me no issues. There is something in particular about this import, but I don't know what.
from django.views.decortors.csrf import csrf_exempt
#csrf_exempt
def handle_hook(request):
from django.http import HttpResponse
from django.core.management import call_command
result = call_command('update_blog', verbosity = 0)
return HttpResponse(result)
The same kind of issue shows up when I am trying to run the code on the python shell (I use 3.4) and when I import django.http.request as request. I type in handle_hook(request), and I get the same kind of error.
Im being told that I must either define the environment variable DJANGO_SETTINGS_MODULE or call settings.configure() before accessing settings, but I haven't a clue on how to do that. I've looked around and I am not certain if those methods specifically speak to my issue. Any clues?
Go to the Run Menu, select Edit Configurations..., then select the run configuration for you tests.
Select the environment variables button. You'll see one existing variable, which is PYTHONUNBUFFERED
Under that add (for example) DJANGO_SETTINGS_MODULE=mysitename.settings
I have installed django 1.6 using pip on Windows 7. When I try to run django-admin.py, I get this error
C:\django-admin.py startproject test007
Traceback (most recent call last):
File "C:\Python27\Scripts\django-admin.py", line 2 in ?
from django.core import management
File "C:\Python27\Lib\site-packages\django\core\management\__init__.py", line 55
except ImportError as e:
^
SyntaxError: invalid syntax
in the \management__init__.py, the imports are
import collections
import os
import sys
from optparse import OptionParser, NO_DEFAULT
import imp
from django.core.exceptions import ImproperlyConfigured
from django.core.management.base import BaseCommand, CommandError, handle_default_options
from django.core.management.color import color_style
from django.utils.importlib import import_module
from django.utils import six
# For backwards compatibility: get_version() used to be in this module.
from django import get_version
Here's the relevant block from the same file
parts = app_name.split('.')
parts.append('management')
parts.reverse()
part = parts.pop()
path = None
# When using manage.py, the project module is added to the path,
# loaded, then removed from the path. This means that
# testproject.testapp.models can be loaded in future, even if
# testproject isn't in the path. When looking for the management
# module, we need look for the case where the project name is part
# of the app_name but the project directory itself isn't on the path.
try:
f, path, descr = imp.find_module(part, path)
except ImportError as e:
if os.path.basename(os.getcwd()) != part:
raise e
else:
if f:
f.close()
while parts:
part = parts.pop()
f, path, descr = imp.find_module(part, [path] if path else None)
if f:
f.close()
return path
in which the same line 55 from the traceback is in the try/except block towards the bottom. I've uninstalled and reinstalled but, to no avail.
It works when I give the/full/path/to/django-admin.py but it shouldn't be required.
I made a database ruter for myapp application
in file /myproject/myapp/routers.py
class ShardingRouter(object):
def db_for_read(self, model, **hints):
return 'default'
def db_for_write(self, model, **hints):
return 'default'
def allow_relation(self, obj1, obj2, **hints):
return None
def allow_syncdb(self, db, model):
return None
In settings.py I have:
from django.db import connections
DATABASE_ROUTERS = ['myproject.myapp.routers.ShardingRouter',]
This works well for normal application running through wsgi, but I have one CLI script /myproject/parser_jobs.py it is starting with cron or manually from CLI:
import os, sys
if __name__ == "__main__":
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "settings")
...
...
if __name__ == "__main__":
do_some_long_boring_work()
now, when i run this script, I'm getting import error:
Traceback (most recent call last):
File "/hosting/myproject/myproject/parser_jobs.py", line 20, in <module>
import settings
File "/hosting/myproject/myproject/settings.py", line 46, in <module>
from django.db import connections
File "/usr/local/lib/python2.7/dist-packages/django/db/__init__.py", line 11, in <module>
if DEFAULT_DB_ALIAS not in settings.DATABASES:
File "/usr/local/lib/python2.7/dist-packages/django/utils/functional.py", line 184, in inner
self._setup()
File "/usr/local/lib/python2.7/dist-packages/django/conf/__init__.py", line 42, in _setup
self._wrapped = Settings(settings_module)
File "/usr/local/lib/python2.7/dist-packages/django/conf/__init__.py", line 95, in __init__
raise ImportError("Could not import settings '%s' (Is it on sys.path?): %s" % (self.SETTINGS_MODULE, e))
ImportError: Could not import settings 'myproject.myproject' (Is it on sys.path?): cannot import name connections
Looks like it found settings.py, but while importing it fails on from django.db import connections. If i comment this string, it works, but without my db router:( I can add using() everywhere, but it's not cool.
So, website works good, but cli script fails. Please, help!
update: /hosting/myproject/myproject/parser_jobs.py worked good from cli, before I added DB router
pprint of sys.path in this script:
['/hosting/myproject/myproject',
'/hosting/myproject',
'/usr/lib/python2.7',
'/usr/lib/python2.7/plat-linux2',
'/usr/lib/python2.7/lib-tk',
'/usr/lib/python2.7/lib-old',
'/usr/lib/python2.7/lib-dynload',
'/usr/local/lib/python2.7/dist-packages',
'/usr/lib/python2.7/dist-packages',
'/usr/lib/python2.7/dist-packages/PIL',
'/usr/lib/python2.7/dist-packages/gst-0.10',
'/usr/lib/python2.7/dist-packages/gtk-2.0',
'/usr/lib/python2.7/dist-packages/ubuntu-sso-client',
'/usr/lib/pymodules/python2.7']
update: wsgi script, website works good with it:
import os
import sys
path = '/hosting/myproject'
if path not in sys.path:
sys.path.insert(0, path)
os.environ['DJANGO_SETTINGS_MODULE'] = 'myproject.settings'
import django.core.handlers.wsgi
application = django.core.handlers.wsgi.WSGIHandler()
my script was in the same folder with settings.py
when I moved it from /hosting/myproject/myproject/parser_jobs.py to /hosting/myproject/parser_jobs.py (where in django1.4 manage.py should be) it become working correct.
Also I changed os.environ.setdefault("DJANGO_SETTINGS_MODULE", "settings") to os.environ.setdefault("DJANGO_SETTINGS_MODULE", "myproject.settings")
And now it works!
In previous location parser_jobs.py works only without db router