What is the simplest way to avoid field collisions working with pymongo? I have a very simple structure for a Location class (name, slug, description and geolocation) and I would like to ensure that no duplicate names are allowed. I am using flask and pymongo?
I was trying this:
from flask import Flask
from flask.ext.pymongo import PyMongo
app = Flask(__name__)
mongo = PyMongo(app)
mongo.db.court.ensureIndex( { "name": 1, "slug": 1 } )
but it gives me an error: RuntimeError: working outside of application context.
use unique indexes and you'll have no two documents that have same values for a field. this doesn't have to be flask-specific, but it is rather mongodb-specific.
if you're lazy or indexes give you headache, just use _id field as the location name. in this case you have to make sure your documents don't get overwritten.
The best place to put your calls to ensure_index is someplace before you call run() on your flask app. You want to make sure your indexes are in place before you attempt to service any requests, because building the index while the site is live will make it pretty unresponsive. The error you are getting is because you need the application context. Try:
app = Flask(__name__)
mongo = PyMongo(app)
if __name__ == '__main__':
with app.app_context():
mongo.db.court.ensure_index( [("name", ASCENDING), ("slug", ASCENDING)], unique=True )
app.run()
As #thkang said, you should use a unique index to enforce that no two documents have the same value for a field or set of fields taken together. See more about this and pymongo's ensure_index syntax at the pymongo docs.
ensure_index has been deprecated. Use create_index instead.
from flask import Flask
from flask_pymongo import PyMongo
import pymongo
app = Flask(__name__)
mongo = PyMongo(app)
with app.app_context():
mongo.db.court.create_index([("name", pymongo.ASCENDING), ("slug", pymongo.ASCENDING)], unique=True)
Related
This is my first post on Stackoverflow and I'm new to Django, I hope you'll understand.
I want to use Django to provide a portal with authentication, which will have to consume an Keystone/Openstack API, to create/delete Projects, grant/remove rights.
Openstack provides a RestFul API, on which I have to authenticate (I provide credentials, and receive back a token).
I have 2 possibilities to access this API:
Using python client: python-keystoneclient
Using directly the restfulAPI
Nevermind the option 1 or 2, I'm able to login and interact with the API, I do this in the view.
My problem is, each time I change the page/view, I have to authenticate again. I don't know how to use/share the "session or client object" in other views.
>>> from keystoneauth1.identity import v3
>>> from keystoneauth1 import session
>>> from keystoneclient.v3 import client
>>> auth = v3.Password(auth_url='https://my.keystone.com:5000/v3',
... user_id='myuserid',
... password='mypassword',
... project_id='myprojectid')
>>> sess = session.Session(auth=auth)
>>> keystone = client.Client(session=sess, include_metadata=True)
I tried to pass the object as a session variable with request.session and request.session.get, but the object is not serializable. I serialized it, but I can't use it on the other view.
Maybe I shouldn't access the API in the view?
I'm sure I'm not the first in this usecase, regardless of the remote API. But I googled a lot without finding a proper way. Maybe I don't search with the right words
Thanks for your help.
I did it like this and it works well:
in apps.py:
from django.apps import AppConfig
from keystoneauth1.identity import v3
from keystoneauth1 import session
from keystoneclient.v3 import client
class MyAppConfig(AppConfig):
default_auto_field = 'django.db.models.BigAutoField'
name = 'mySharedVar'
auth = v3.Password(auth_url='https://my.keystone.com:5000/v3', user_id='myuserid', password='mypassword',project_id='myprojectid')
ses1 = session.Session(auth=auth)
in my views, I can now access the "share variable" with:
from keystoneauth1.identity import v3
from keystoneauth1 import session
from keystoneclient.v3 import client
#login_required
def list_project(request):
sharedVar=apps.get_app_config('mySharedVar')
keystone = client.Client(session=sharedVar.ses1, include_metadata=True)
pl = keystone.projects.list()
context = {
"title": "Project List",
"projects": pl.data
}
return render(request, "myapp/list_project.html",context=context)
I hope this can help someone.
I have a Flask app and I use SQLAlachemy(without Flask extension cause I need to create my own class based SQLAlchemy and so on).
My app has a connection to its database over engine and it works fine but now I need to make my engine dynamically and get db_name from Flask.g
Engine is declared in models.py
models.py
engine = create_engine(f"postgresql://postgres:postgres#localhost:5434/{g['tenant']}", convert_unicode=True)
db_session = scoped_session(sessionmaker(autocommit=False,
autoflush=False,
bind=engine))
Base = declarative_base()
Base.query = db_session.query_property()
For the start app I use wsgi.py:
from app import app
if __name__ == "__main__":
app.run(port=5002)
when I type python wsgi.py I receive an error.
RuntimeError: Working outside of application context.
This typically means that you attempted to use functionality that needed
to interface with the current application object in some way. To solve
this, set up an application context with app.app_context(). See the
documentation for more information.
In general I understand that I use an engine which is outside the context. The issue is - I cannot figure out how to pass my engine variable to context.
I try to make create app func:
def create_app():
app = Flask(__name__)
with app.app_context():
engine = create_engine(f"postgresql://postgres:postgres#localhost:5434/{g['tenant']}", convert_unicode=True)
return app
Also I tried app.app_context().push(engine)
But it doesn't work. How I can solve this issue?
The problem is that the flask object g only ever exists when a request is currently in progress on flask. (Without a request, there is no flask g, because g is specifically a global for individual requests)
what you'd have to do is to create that engine after the request starts, which slows down the route a bit. The #app.before_request decorator might help you here:
#app.before_request
def create_engine_for_request():
engine = create_engine(f"postgresql://postgres:postgres#localhost:5434/{g['tenant']}", convert_unicode=True)
("before_request" is already "during a request" - its just the first thing that flask does "when a request starts")
I want to create a new user's login and random/same password using CSV file using the ImportExport module Django. I've tried many examples but for no use. Can you please suggest anything? I've even tried to make password field in the import csv file. Let me know any tutorial/link that you can share to achieve this.
If you're looking to populate the database with dummy data, there is a module called faker in python.
Here is a small code snippet showing how you can use faker:
import os
# Configure settings for project
# Need to run this before calling models from application!
os.environ.setdefault('DJANGO_SETTINGS_MODULE','(YOUR DJANGO PROJECT).settings')
import django
# Import settings
django.setup()
import random
from (YOUR DJANGO APP).models import User
from faker import Faker
fakegen = Faker()
def populate(N=5):
for entry in range(N):
# Create Fake Data for entry
fake_name = fakegen.name().split()
fake_first_name = fake_name[0]
fake_last_name = fake_name[1]
fake_email = fakegen.email()
# Create new User Entry
user = User.objects.get_or_create(first_name=fake_first_name,
last_name=fake_last_name,
email=fake_email)[0]
if __name__ == '__main__':
print("Populating the databases...")
populate(20)
print('Populating Complete')
django-import-export contains several hooks which you can use to manipulate data prior to insertion in the db. This seems like the ideal way to generate a random password for each user.
You can combine this with Django's make_random_password() function. It doesn't seem like a good idea to have passwords in the import spreadsheet, but maybe you need to do that for some reason.
You will need to create your own Resource subclass, and override the before_save_instance(), for example:
class UserResource(resources.ModelResource):
def before_save_instance(self, instance, using_transactions, dry_run):
pw = User.objects.make_random_password()
instance.set_password(pw)
class Meta:
model = User
fields = ('id', 'username', 'password')
import_id_fields = ['id']
I'm trying to build an API server using Django. I have a few tables and I need to run queries based on the parameters passed in by url:
http://server.com/api/request/p1=123&p2=321...
and the server would extract p1 and p2 from url and run queries using them, and then return the result in json or xml.
I have tried Tastypie, and it's very easy to set up for retrieving data from one model. But my case is a bit complicated than that, and sometimes I need to run spatial queries. So if I could run the query and return result as json/xml, it would be great!
Very new to the backend technology, any help for a start point is appreciated!
Thanks!
[Edit]
Just want to make my case clearer. Say I ran a raw query using qs = cursor.execute(sql), etc., and I want to return that result as json/xml to a api call. Can I do that with either Tastypie or Rest Framework? Or can I do it without any help from 3rd party framework?
Here's a view I use return json, you should be able to adapt it pretty easy:
import json
from django.http import HttpResponse
from django.template.defaultfilters import slugify
from .models import *
def response_times(request):
response_data = {} #Create an empty dictionary
websites = Website.objects.all() #Query your models
for site in websites:
key = slugify(site.name)
response_data[key] = {}
history = History.objects.filter(website=site)[:60]
response_data[key]['response_times'] = []
for response in history:
response_data[key]['response_times'].append({'time': str(response.time), 'timestamp': response.added.strftime('%s')})
return HttpResponse(json.dumps(response_data), content_type="application/json")
It seems you have in the result QuerySet of some Model instances. So you could specify serializer to work with your model and use it to serialize from model instances to native python data using Rest Framework. Look here for details.
I would like to read some session variables from a test (Django TestCase)
How to do that in a clean way ?
def test_add_docs(self):
"""
Test add docs
"""
# I would access to the session here:
self.request.session['documents_to_share_ids'] = [1]
response = self.client.get(reverse(self.document_add_view_id, args=[1]), follow=True)
self.assertEquals(response.status_code, 200)
As of Django 1.7+ this is much easier. Make sure you set the session as a variable instead of referencing it directly.
def test_something(self):
session = self.client.session
session['somekey'] = 'test'
session.save()
andreaspelme's workaround is only needed in older versions of django. See docs
Unfortunately, this is not a easy as you would hope for at the moment. As you might have noticed, just using self.client.session directly will not work if you have not called other views that has set up the sessions with appropriate session cookies for you. The session store/cookie must then be set up manually, or via other views.
There is an open ticket to make it easier to mock sessions with the test client: https://code.djangoproject.com/ticket/10899
In addition to the workaround in the ticket, there is a trick that can be used if you are using django.contrib.auth. The test clients login() method sets up a session store/cookie that can be used later in the test.
If you have any other views that sets sessions, requesting them will do the trick too (you probably have another view that sets sessions, otherwise your view that reads the sessions will be pretty unusable).
from django.test import TestCase
from django.contrib.auth.models import User
class YourTest(TestCase):
def test_add_docs(self):
# If you already have another user, you might want to use it instead
User.objects.create_superuser('admin', 'foo#foo.com', 'admin')
# self.client.login sets up self.client.session to be usable
self.client.login(username='admin', password='admin')
session = self.client.session
session['documents_to_share_ids'] = [1]
session.save()
response = self.client.get('/') # request.session['documents_to_share_ids'] will be available
If you need to initialize a session for the request in tests to manipulate it directly:
from django.contrib.sessions.middleware import SessionMiddleware
from django.http import HttpRequest
request = HttpRequest()
middleware = SessionMiddleware()
middleware.process_request(request)
request.session.save()
If testing my django project with pytest, I can not see any modifications to the session that are made in the view. (That is because the Sessions middleware doesn't get called.)
I found the following approach to be useful:
from unittest.mock import patch
from django.test import Client
from django.contrib.sessions.backends import db
def test_client_with_session():
client = Client()
session = {} # the session object that will persist throughout the test
with patch.object(db, "SessionStore", return_value=session):
client.post('/url-that-sets-session-key/')
assert session['some_key_set_by_the_view']
client.get('/url-that-reads-session-key/')
This approach has the benefit of not requiring database access.
You should be able to access a Client's session variales through its session property, so I guess self.client.session['documents_to_share_ids'] = [1] should be what you are looking for!