I wonder where I should initialize my MongoDB connection in my Django projects.
Currently I am initializing the client and db before my view functions in views.py:
import pymongo
from django.conf import settings
client = pymongo.MongoClient(settings.MONGO_URI)
db = client.get_default_database()
def some_view(request):
pass
However I also need to use MongoDB in my models.py in conjunction with Django signals. What do you suggest?
Maybe settings.py? Or even root __init__.py? Then you can import client and db everywhere you need it.
I've decided to use project/mongodb.py (same folder as settings.py)
import pymongo
from django.conf import settings
client = pymongo.MongoClient(settings.MONGO_URI)
mongodb = client.get_default_database()
I am using two different settings files for local and production. Therefore, this approach makes it possible to use environment dependent settings, while enabling me to access mongodb variable from anywhere in the project.
Related
The figure shows my django structure. I don't have a django app according to django glossary.
When django starts, I want to load a system configuration (I know how to run a Linux command) and save the result to a place where view.py can access. How would I implement it?
I tried the following options:
First, I looked at Django settings, but it discourages altering values in settings.py at runtime. Second, from what I understand, Django Applications is saying I need to have a django app in order to use AppConfig.ready() which sets something global settings. Unless my folder structure is horribly wrong, I don't want to change it or switch to a django app.
I'm using django 3.1 on Linux.
You could alter django settings at runtime like this however it is discouraged.
from django.conf import settings
settings.CUSTOM_KEY = True
I want to use redis-py directly to work with flask, instead of other wrapper lib (e.g. flask-redis, flask-and-redis) of flask plugins. How to initialize redis-client in factory function create_app() of flask application ? Or how to properly just initialize redis.StrictRedis (maybe not compatible with other flask plugins of redis ??) ? Because there're some operations related to token persistence in router of other modules using this redis.StrictRedis object.
Any advice, please ?
Interesting question, I think using the wrappers may make things easier but if you just want to use redis as a cache you can very easily import redis and just create/instantiate alongside the flask app.
import redis
from flask import Flask
# make redis
redis_cache = redis.StrictRedis()
# make flask app
app = Flask(__name__)
# business logic
#app.route('/<string:item>')
def index(item):
# if cache hit then get from redis
if redis_cache.exists(item):
value = redis_cache.get(item)
# cache miss
else:
value = 'Not in cache'
return value
This is the simplest way.
You can also follow this link to create the setup function and add the redis instantiation in the create_app function.
Can I specific a different configuration for the test database? Or alternatives simply use a different user in production?(how to manage that as settings file needs to be updated as well?) The testing requirements for postgresql require in my opinion too many privs like create DB and in my case I also need to create an extension upon db creation which means superuser privileges.
This is very messy, but a valid workaround nonetheless. We use a snippet of code like this in settings.py to run tests using a local sqlite3 database.
import sys
if 'test' in sys.argv or 'test_coverage' in sys.argv:
DATABASES['default']['ENGINE'] = 'django.db.backends.sqlite3'
DATABASES['default']['NAME'] = os.path.join(BASE_DIR, 'db.sqlite3')
I am currently using (or hoping to use) both PostgreSQL and MongoDB for my Django project. When it comes to MongoDB I've been looking at PyMongo to make the connection, and add/edit info using Python. Most likely I will be calling PyMongo in my views.py to read/insert/edit data from MongoDB.
For the sake of this question, let's say I have a view called worksheet, which querys some data from MongoDB relevant to the current user. However I would like to maintain a persistent connection to the database so that I do not have to make a new connection every time a user visits the worksheet view.
Is there any way I could make a persistent connection so I am not wasting time/resources making new connections to MongoDB every time? Thanks.
I wanted the exact same thing and this is how I did it.
from pymongo import MongoClient
from threading import local
from django.conf import settings
_mongo_client = local()
def mongo_client():
client = getattr(_mongo_client, 'client', None)
if client is None:
client = MongoClient(settings.MONGODB_URI)
_mongo_client.client = client
return client
And then in your views you use it the client like this.
mongo_client().db.orders.find(...)
This is what I have been doing and it works, my only concern would be if the mongodb client correctly reestablishes lost connections... But I think it does because I have not been having any issues.
I have a model with an ImageField that is backed by django-storages' S3Boto. I have a test the exercises the "upload image" view, but the fact that it is uploading the image to S3 is slowing down my test suite.
In the interest of speeding up my tests, what is the best practice for dealing with this issue? Should I mock out S3Boto? Perhaps there is a memory backed storage backend that works well for testing (automatic cleanup would be nice!)?
I just had this problem too. I got much faster tests by using dj-inmemorystorage.
The quick way of setting this up is by creating a test_settings.py in the same folder as your settings:
from settings import *
DEFAULT_FILE_STORAGE = 'inmemorystorage.InMemoryStorage'
...and calling ./manage.py test --settings=project.test_settings to run the tests.
My preferred way is to set up a custom test runner:
In project/test_runner.py:
from django.conf import settings
from django.test.runner import DiscoverRunner
class FastTestRunner(DiscoverRunner):
def setup_test_environment(self):
super(FastTestRunner, self).setup_test_environment()
# Don't write files
settings.DEFAULT_FILE_STORAGE = 'inmemorystorage.InMemoryStorage'
# Bonus: Use a faster password hasher. This REALLY helps.
settings.PASSWORD_HASHERS = (
'django.contrib.auth.hashers.MD5PasswordHasher',
)
Note: This also sets the PASSWORD_HASHER, because it significantly improves User creation time. This should NOT be set in production.
In project/settings.py:
TEST_RUNNER = 'project.test_runner.FastTestRunner'
The requirements:
pip install dj-inmemorystorage
UPDATE: changed from django-inmemorystorage to dj-inmemorystorage.
UPDATE 2: Removed django-discover-runner, as it's now the default test runner in django, and fixed the link to the PASSWORD_HASHER related blog post.
I also use S3Boto but for testing, I prefer having custom settings which include using the file system storage. You can have your custom settings declared in a file which you can then import and use in your test cases. Even so, you can mock the file storage so that the files are not actually written to disk.
Here's a sample test_settings.py
# myproject/myproject/test_settings.py
from django.test import override_settings
common_settings = override_settings(
DEFAULT_FILE_STORAGE='django.core.files.storage.FileSystemStorage',
PASSWORD_HASHERS=(
'django.contrib.auth.hashers.UnsaltedMD5PasswordHasher',
),
)
Usage:
from django.test import TestCase
from myproject.test_settings import common_settings
#common_settings
class MyTestCase(TestCase):
"""Tests go here"""
On mocking the file system storage you can check out my answer here on SO.
Just ran into this as well so I thought I'd put my solution up. My solution uses Mock
import mock
from django.core.files.storage import FileSystemStorage
from django.test import TestCase
class ATestCase(TestCase):
def setUp(self):
# Stuff Happens
def tearDown(self):
# more Stuff
#mock.patch('storages.backends.s3boto.S3BotoStorage', FileSystemStorage)
def test_file_stuff(self):
self.assertMagicPonies(True)
Some gotchas - make sure you have a sane MEDIA_ROOT setup in the settings. as of django 1.4, you can't use the testing context manager to override MEDIA_ROOT, so you need a separate settings config for it (https://code.djangoproject.com/ticket/17787) This was fixed in 1.6. Also, make sure your upload_to works in normal filesystem, or you will get permission errors.
I would propose to use the standard Django Storage for testing, where you can a define custom path for storage and cleanup that path in your test suite once your done. Both the storage and the path can be set in the settings and overwritten for testing.