I am trying to mock out redis in my Django application. I have tried several different methods but none seem to work. What am I doing wrong?
My primary redis instance is called with:
redis_client = redis.from_url(os.environ.get("REDIS_URL"))
That instance is imported in other parts of the app in order to add and retrieve data.
In my tests I tried doing:
import fakeredis
from mock import patch
class TestViews(TestCase):
def setUp(self):
redis_patcher = patch('redis.Redis', fakeredis.FakeRedis)
self.redis = redis_patcher.start()
self.redis.set('UPDATE', 'Spring')
print(redis_client.get('UPDATE'))
def tearDown(self):
self.redis_patcher.stop
When running the tests I want the 'UPDATE' variable to be set. But instead every instance of redis_client fails saying the server is not available. How can I mock out redis and set values, so that they are available when testing my app?
You should mock an item where it is used, not where it came from.
So if redis_client is used in a view like this:
myapp/views.py
from somemodule import redis_client
def some_view_that_uses_redis(request):
result = redis_client(...)
Then in your TestViews you should patch redis_client like this:
class TestViews(TestCase):
def setUp(self):
redis_patcher = patch('myapp.views.redis_client', fakeredis.FakeRedis)
self.redis = redis_patcher.start()
Related
As you know django give you clear database in testing, but I have a ready() method that create some data for me and I need to query these data in my tests.
class YourAppConfig(AppConfig):
default_auto_field = 'django.db.models.AutoField'
name = 'Functions.MyAppsConfig'
def ready(self):
from django.contrib.auth.models import Permission
from django import apps
from django.contrib.contenttypes.models import ContentType
try:
Permission.objects.get_or_create(....)
MyOtherModel.objects.get_or_create(....)
except:
pass
class TestRules(APITestCase):
def test_my_model(self):
....
x = MyOtherModel.objects.filter(....).first()
# x = None # <=========== problem is here ========= I need to see the data that I created in the ready method
....
You can use the fixtures for that, in each Test case you can fixtures to it as stated documentation example is
class Test(TransactionTestCase):
fixtures = ['user-data.json']
def setUp():
…
Django will load the fixtures before under test case
I am developing a package for my testing purpose called dbtest. This package is because i am using MySQLdb for connecting databases and hence it is very tedious task to write sql queries while testing. So i created a new package and all queries can be accessed with separate functions. I avoided django ORM because my database table have multiple foreign keys and primary keys.
Below present is a part of the package.
package.py
from django.test import TestCase
dbcon='connector'
class testcase(TestCase):
flag_user=[]
#classmethod
def setUpClass(cls):
global dbcon
dbcon=MySQLdb.connect(host=dbHost,port=dbPort,user=dbUser,passwd=dbPasswd,db=dbname)
super(testcase, cls).setUpClass()
cursor = dbcon.cursor()
sql=open("empty.sql").read()
cursor.execute(sql)
cursor.close()
views.MySQLdb=Mockdb()
#classmethod
def tearDownClass(cls):
dbcon.close()
def user_table(self,username=username,email=email):
cache=[username]
self.flag_user.append(cache)
cmpdata=(username,email)
insert_table(tablename_user,cmpdata)
def delete(self,table):
last_entry=self.flag_user[-1]
query_user = 'delete from USER where USERNAME=%s'
cursor=dbcon.cursor()
query=eval('query_%s'%table)
cursor.execute(query,last_entry)
dbcon.commit()
del self.flag_user[-1]
tests.py
from package import testcase
class showfiles(testcase):
def setUp(self):
print "setup2"
self.user_table(username='vishnu',email='vishnu#clartrum.com')
def tearDown(self):
print "teardown2"
self.delete("user")
def test_1(self):
print "test dbtest link feature"
def test_2(self):
print "test health/errorfiles with valid device"
self.user_table(username='vishnu',email='vishnu#clartrum.com')
The insert_table in package execute insert operation in sql and delete method deletes the last entry from user. empty.sql creates tables for the database.
Actually when i run the tests, finally the flag_user should contain only [['vishnu']]. But i get [['vishnu'],['vishnu']] and this is because delete function in teardown doesn't updating the value.
I think this is due to class instances ? Am i right or not?
Here :
class testcase(TestCase):
flag_user=[]
you create flag_user as a class attribute (shared by all instances).
Then here:
def user_table(self,username=username,email=email):
cache=[username]
self.flag_user.append(cache)
You append to the (class level) flag_user attribute (it's accessed thru the instance but it's still the class attribute)
But here:
def delete(self,table):
delete_table(tablename)
self.flag_user=[]
you create a flag_user attribute on the instance itself, which is totally disconnected from the eponym class attribute.
The simplest solution is to use an instance attribute right from the start instead of using a class attribute:
# package.py
from django.test import TestCase
dbcon='connector'
class testcase(TestCase):
def setUp(self):
self.flag_user = []
and don't forget to call testcase.setUp in child classes:
# tests.py
from package import testcase
class showfiles(testcase):
def setUp(self):
super(showfile, self).setUp()
self.user_table(username='vishnu',email='vishnu#clartrum.com')
The alternative solution if you really want a class attribute (I can't imagine why you would but...) is to modify testcase.delete() so it really clears the flag_user class attribute instead of creating an instance attribute, which is done by explicitely asking python to rebind the attribute on the class itself (type(obj) returns obj.__class__ which is the class the instance belongs to):
def delete(self,table):
delete_table(tablename)
type(self).flag_user = []
I have a Django project that pulls data from legacy database (read only connection) into its own database, and when I run integration tests, it tries to read from test_account on legacy connection.
(1049, "Unknown database 'test_account'")
Is there a way to tell Django to leave the legacy connection alone for reading from the test database?
I actually wrote something that lets you create integration test in djenga (available on pypi) if you want to take a look at how to create a separate integration test framework.
Here is the test runner I use when using the django unit test framework:
from django.test.runner import DiscoverRunner
from django.apps import apps
import sys
class UnManagedModelTestRunner(DiscoverRunner):
"""
Test runner that uses a legacy database connection for the duration of the test run.
Many thanks to the Caktus Group: https://www.caktusgroup.com/blog/2013/10/02/skipping-test-db-creation/
"""
def __init__(self, *args, **kwargs):
super(UnManagedModelTestRunner, self).__init__(*args, **kwargs)
self.unmanaged_models = None
self.test_connection = None
self.live_connection = None
self.old_names = None
def setup_databases(self, **kwargs):
# override keepdb so that we don't accidentally overwrite our existing legacy database
self.keepdb = True
# set the Test DB name to the current DB name, which makes this more of an
# integration test, but HEY, at least it's a start
DATABASES['legacy']['TEST'] = { 'NAME': DATABASES['legacy']['NAME'] }
result = super(UnManagedModelTestRunner, self).setup_databases(**kwargs)
return result
# Set Django's test runner to the custom class defined above
TEST_RUNNER = 'config.settings.test_settings.UnManagedModelTestRunner'
TEST_NON_SERIALIZED_APPS = [ 'legacy_app' ]
from django.test import TestCase, override_settings
#override_settings(LOGIN_URL='/other/login/')
class LoginTestCase(TestCase):
def test_login(self):
response = self.client.get('/sekrit/')
self.assertRedirects(response, '/other/login/?next=/sekrit/')
https://docs.djangoproject.com/en/1.10/topics/testing/tools/
You should theoretically be able to use the override settings here and switch to a dif
I'm using django with neo4j as database and noemodel as OGM. How do I test it?
When I run python3 manage.py test all the changes, my tests make are left.
And also how do I make two databases, one for testing, another for working in production and specify which one to use how?
I assume the reason all of your changes are being retained is due to using the same neo4j database for testing as you are using in development. Since neomodel isn't integrated tightly with Django it doesn't act the same way Django's ORM does when testing. Django will do some helpful things when you run tests using its ORM, such as creating a test database that will be destroyed upon completion.
With neo4j and neomodel I'd recommend doing the following:
Create a Custom Test Runner
Django enables you to define a custom test runner by setting the TEST_RUNNER settings variable. An extremely simple version of this to get you going would be:
from time import sleep
from subprocess import call
from django.test.runner import DiscoverRunner
class MyTestRunner(DiscoverRunner):
def setup_databases(self, *args, **kwargs):
# Stop your development instance
call("sudo service neo4j-service stop", shell=True)
# Sleep to ensure the service has completely stopped
sleep(1)
# Start your test instance (see section below for more details)
success = call("/path/to/test/db/neo4j-community-2.2.2/bin/neo4j"
" start-no-wait", shell=True)
# Need to sleep to wait for the test instance to completely come up
sleep(10)
if success != 0:
return False
try:
# For neo4j 2.2.x you'll need to set a password or deactivate auth
# Nigel Small's py2neo gives us an easy way to accomplish this
call("source /path/to/virtualenv/bin/activate && "
"/path/to/virtualenv/bin/neoauth "
"neo4j neo4j my-p4ssword")
except OSError:
pass
# Don't import neomodel until we get here because we need to wait
# for the new db to be spawned
from neomodel import db
# Delete all previous entries in the db prior to running tests
query = "match (n)-[r]-() delete n,r"
db.cypher_query(query)
super(MyTestRunner, self).__init__(*args, **kwargs)
def teardown_databases(self, old_config, **kwargs):
from neomodel import db
# Delete all previous entries in the db after running tests
query = "match (n)-[r]-() delete n,r"
db.cypher_query(query)
sleep(1)
# Shut down test neo4j instance
success = call("/path/to/test/db/neo4j-community-2.2.2/bin/neo4j"
" stop", shell=True)
if success != 0:
return False
sleep(1)
# start back up development instance
call("sudo service neo4j-service start", shell=True)
Add a secondary neo4j database
This can be done in a couple ways but to follow along with the test runner above you can download a community distribution from neo4j's website. With this secondary instance you can now swap between which database you'd like to use utilizing the command line statements used in the calls within the test runner.
Wrap Up
This solution assume's you're on a linux box but should be portable to a different OS with minor modifications. Also I'd recommend checking out the Django's Test Runner Docs to expand upon what the test runner can do.
There currently isn't mechanism for working with test databases in neomodel as neo4j only has 1 schema per instance.
However you can override the environment variable NEO4J_REST_URL when running the tests like so
export NEO4J_REST_URL=http://localhost:7473/db/data python3 manage.py test
The way I went about this was to give in and use the existing database, but mark all test-related nodes and detach/delete them when finished. It's obviously not ideal; all your node classes must inherit from NodeBase or risk polluting the db with test data, and if you have unique constraints, those will still be enforced across both live/test data. But it works for my purposes, and I thought I'd share in case it helps someone else.
in myproject/base.py:
from neomodel.properties import Property, validator
from django.conf import settings
class TestModeProperty(Property):
"""
Boolean property that is only set during unit testing.
"""
#validator
def inflate(self, value):
return bool(value)
#validator
def deflate(self, value):
return bool(value)
def __init__(self, *args, **kwargs):
super().__init__(*args, **kwargs)
self.default = True
self.has_default = settings.UNIT_TESTING
class NodeBase(StructuredNode):
__abstract_node__ = True
test_mode = TestModeProperty()
in myproject/test_runner.py:
from django.test.runner import DiscoverRunner
from neomodel import db
class NeoDiscoverRunner(DiscoverRunner):
def teardown_databases(self, old_config, **kwargs):
db.cypher_query(
"""
MATCH (node {test_mode: true})
DETACH DELETE node
"""
)
return super().teardown_databases(old_config, **kwargs)
in settings.py:
UNIT_TESTING = sys.argv[1:2] == ["test"]
TEST_RUNNER = "myproject.test_runner.NeoDiscoverRunner"
I am trying to access access application configuration inside a blueprint authorisation.py which in a package api. I am initializing the blueprint in __init__.py which is used in authorisation.py.
__init__.py
from flask import Blueprint
api_blueprint = Blueprint("xxx.api", __name__, None)
from api import authorisation
authorisation.py
from flask import request, jsonify, current_app
from ..oauth_adapter import OauthAdapter
from api import api_blueprint as api
client_id = current_app.config.get('CLIENT_ID')
client_secret = current_app.config.get('CLIENT_SECRET')
scope = current_app.config.get('SCOPE')
callback = current_app.config.get('CALLBACK')
auth = OauthAdapter(client_id, client_secret, scope, callback)
#api.route('/authorisation_url')
def authorisation_url():
url = auth.get_authorisation_url()
return str(url)
I am getting RuntimeError: working outside of application context
I understand why that is but then what is the correct way of accessing those configuration settings?
----Update----
Temporarily, I have done this.
#api.route('/authorisation_url')
def authorisation_url():
client_id, client_secret, scope, callback = config_helper.get_config()
auth = OauthAdapter(client_id, client_secret, scope, callback)
url = auth.get_authorisation_url()
return str(url)
Use flask.current_app in place of app in the blueprint view.
from flask import current_app
#api.route("/info")
def get_account_num():
num = current_app.config["INFO"]
The current_app proxy is only available in the context of a request.
Overloading record method seems to be quite easy:
api_blueprint = Blueprint('xxx.api', __name__, None)
api_blueprint.config = {}
#api_blueprint.record
def record_params(setup_state):
app = setup_state.app
api_blueprint.config = dict([(key,value) for (key,value) in app.config.iteritems()])
To build on tbicr's answer, here's an example overriding the register method example:
from flask import Blueprint
auth = None
class RegisteringExampleBlueprint(Blueprint):
def register(self, app, options, first_registration=False):
global auth
config = app.config
client_id = config.get('CLIENT_ID')
client_secret = config.get('CLIENT_SECRET')
scope = config.get('SCOPE')
callback = config.get('CALLBACK')
auth = OauthAdapter(client_id, client_secret, scope, callback)
super(RegisteringExampleBlueprint,
self).register(app, options, first_registration)
the_blueprint = RegisteringExampleBlueprint('example', __name__)
And an example using the record decorator:
from flask import Blueprint
from api import api_blueprint as api
auth = None
# Note there's also a record_once decorator
#api.record
def record_auth(setup_state):
global auth
config = setup_state.app.config
client_id = config.get('CLIENT_ID')
client_secret = config.get('CLIENT_SECRET')
scope = config.get('SCOPE')
callback = config.get('CALLBACK')
auth = OauthAdapter(client_id, client_secret, scope, callback)
Blueprints have register method which called when you register blueprint. So you can override this method or use record decorator to describe logic which depends from app.
The current_app approach is fine but you must have some request context. If you don't have one (some pre-work like testing, e.g.) you'd better place
with app.test_request_context('/'):
before this current_app call.
You will have RuntimeError: working outside of application context , instead.
You either need to import the main app variable (or whatever you have called it) that is returned by Flask():
from someplace import app
app.config.get('CLIENT_ID')
Or do that from within a request:
#api.route('/authorisation_url')
def authorisation_url():
client_id = current_app.config.get('CLIENT_ID')
url = auth.get_authorisation_url()
return str(url)
You could also wrap the blueprint in a function and pass the app as an argument:
Blueprint:
def get_blueprint(app):
bp = Blueprint()
return bp
Main:
from . import my_blueprint
app.register_blueprint(my_blueprint.get_blueprint(app))
I know this is an old thread. But while writing a flask service, I used a method like this to do it. It's longer than the solutions above but it gives you the possibility to use customized class yourself. And frankly, I like to write services like this.
Step 1:
I added a struct in a different module file where we can make the class structs singleton. And I got this class structure from this thread already discussed. Creating a singleton in Python
class Singleton(type):
_instances = {}
def __call__(cls, *args, **kwargs):
if cls not in cls._instances:
cls._instances[cls] = super(Singleton, cls).__call__(*args, **kwargs)
else:
cls._instances[cls].__init__(*args, **kwargs)
return cls._instances[cls]
Step 2:
Then I created a Singleton EnvironmentService class from our Singleton class that we defined above, just for our purpose. Instead of recreating such classes, create them once and use them in other modules, routes, etc. import. We can access the class with the same reference.
from flask import Config
from src.core.metaclass.Singleton import Singleton
class EnvironmentService(metaclass=Singleton):
__env: Config = None
def initialize(self, env):
self.__env = env
return EnvironmentService()
def get_all(self):
return self.__env.copy()
def get_one(self, key):
return self.__env.get(key)
Step 3:
Now we include the service in the application in our project root directory. This process should be applied before the routes.
from flask import Flask
from src.services.EnvironmentService import EnvironmentService
app = Flask(__name__)
# Here is our service
env = EnvironmentService().initialize(app.config)
# Your routes...
Usage:
Yes, we can now access our service from other routes.
from src.services.EnvironmentService import EnvironmentService
key = EnvironmentService().get_one("YOUR_KEY")