I've written application in Django and tested it using SQLite. But now in production I want to use PostgreSQL and there's problem, because calling:
empty_list = []
Foo.objects.exclude(pk__in=empty_list).delete()
raises ProgrammingError (bad query - postgres doesn't accept in queries empty IN () )
I don't want to rewrite all code and wrap all excludes and filters in method that checks for empty list. I also don't want to write thousand of ifs or modify Django code. Is there any elegant solution to solve this?
OK, I found this: http://www.tryolabs.com/Blog/2013/07/05/run-time-method-patching-python/
It's simple, just write sth like this:
from django.db.models import QuerySet
def new_exclude(self, *args, **kwargs):
kw = dict()
for key, value in kwargs.items():
if is_not_empty_list(value) and is_not_empty_queryset(value): #other checks?
kw[key] = value
if len(kw):
return old_exclude(self, *args, **kw)
else:
return self
old_exclude = QuerySet.exclude
QuerySet.exclude = new_exclude
But personally I don't like this solution, so I won't accept it and I'm waiting for better one.
Related
Let's say that we have a database with existing data, the data is updated from a bash script and there is no related model on Django for that. Which is the best way to create an endpoint on Django to be able to perform a GET request so to retrieve the data?
What I mean is, that if there was a model we could use something like:
class ModelList(generics.ListCreateAPIView):
queryset = Model.objects.first()
serializer_class = ModelSerializer
The workaround that I tried was to create an APIView and inside that APIView to do something like this:
class RetrieveData(APIView):
def get(self, request):
conn = None
try:
conn = psycopg2.connect(host=..., database=..., user=..., password=..., port=...)
cur = conn.cursor()
cur.execute(f'Select * from ....')
fetched_data = cur.fetchone()
cur.close()
res_list = [x for x in fetched_data]
json_res_data = {"id": res_list[0],
"date": res_list[1],
"data": res_list[2]}
return Response({"data": json_res_data)
except Exception as e:
return Response({"error": 'Error'})
finally:
if conn is not None:
conn.close()
Although I do not believe that this is a good solution, also is a bit slow ~ 2 sec per request. Apart from that, if for example, many Get requests are made at the same time isn't that gonna create a problem on the DB instance, e.g lock table etc?
So I was wondering which is a better / best solution for this kind of problems.
Appreciate your time!
Django 1.10.1
On a page I have prepared a lot of controls. Some of them are organized as dynamically changing formsets. So, I don't even know how many of them are present.
I need a chained filters with AND, OR and NOT logical operations.
For example, I need something like this:
Entry.objects.filter(headline__startswith='What').exclude(pub_date__gte=datetime.date.today()).filter(pub_date__gte=datetime(2005, 1, 30)).filter(Q(pub_date=date(2005, 5, 2)) | Q(pub_date=date(2005, 5, 6))
Once more the number of filters is changing.
I was planning to ac like that: loop through request.POST and depending on a dozen of conditions prepare a string. The same string:
"Entry.objects.filter(headline__startswith='What').exclude(pub_date__gte=datetime.date.today()).filter(pub_date__gte=datetime(2005, 1, 30)).filter(Q(pub_date=date(2005, 5, 2)) | Q(pub_date=date(2005, 5, 6))"
So, the string is correct. But I can't make it work with exec().
I asked here: why it is not working. And the answer was: it will not work, run the Python code directly.
I can construct something like that:
entries = Entry.objects.filter( **kwargs )
But this is just one filter. I can't imagine how to chain such filters.
Could you help me here&
You can chain a queryset on multiple lines - adding additional queries depending on view logic. These querysets are not executed as they are created, only when they are called.
I am not sure if it is the "best" or most pythonic way, but something like this works very well for me. This is using Django Rest Framework helper methods but it should be applicable without:
def get(self, request, format=None, *args, **kwargs):
name = request.GET.get('name', None)
location = request.GET.get('location', None)
id = request.GET.get('id', None)
results = Model.objects.all().order_by('-created') # this would be the default with no filters on it
results = results.filter(name__iexact=name) if name else results
results = results.filter(location__icontains=location) if location else results
results = results.filter(id__exact=id) if id else results
# process/serialize/send response
Not knowing which parameters you will get can be tricky, but an idea may be to loop over your kwargs and follow the same pattern in the loop - here's a stab (note I haven't tested it):
def get(self, request, format=None, *args, **kwargs):
results = Model.objects.all().order_by('-created') # this would be the default
for param, value in kwargs.items():
results = results.filter(***need to convert param to orm field***= value)
New to django. I'm doing my best to implement CRUD using Django, mongodb, and mongoengine. I'm able to query the database and render my page with the correct information from the database. I'm also able to change some document fields using javascript and do an Ajax POST back to the original Django View class with the correct csrf token.
The data payload I'm sending back and forth is a list of each Document Model (VirtualPageModel) serialized to json (each element contains ObjectId string along with the other specific fields from the Model.)
This is where it starts getting murky. In order to update the original document in my View Class post function I do an additional query using the object id and loop through the dictionary items, setting the respective fields each time. I then call save and any new data is pushed to the Mongo collection correctly.
I'm not sure if what I'm doing to update existing documents is correct or in the spirit of django's abstracted database operations. The deeper I get the more I feel like I'm not using some fundamental facility earlier on (provided by either django or mongoengine) and because of this I'm having to make things up further downstream.
The way my code is now I would not be able to create a new document (although that's easy enough to fix). However what I'm really curious about is how I would know when to delete a document which existed in the initial query, but was removed by the user/javascript code? Am I overthinking things and the contents of my POST should contain a list of ObjectIds to delete (sounds like a security risk although this would be an internal tool.)
I was assuming that my View Class might maintain either the original document objects (or simply ObjectIds) it queried and I could do my comparisions off of that set, but I can't seem to get that information to persist (as a class variable in VolumeSplitterView) from its inception to when I received the POST at the end.
I would appreciate if anyone could take a look at my code. It really seems like the "ease of use" facilities of Django start to break when paired with Mongo and/or a sufficiently complex Model schema which needs to be directly available to javascript as opposed to simple Forms.
I was going to use this dev work to become django battle-hardened in order to tackle a future app which will be much more complicated and important. I can hack on this thing all day and make it functional, but what I'm really interested in is anyone's experience in using Django + MongoDB + MongoEngine to implement CRUD on a Database Schema which is not vary Form-centric (think more nested metadata).
Thanks.
model.py: uses mongoengine Field types.
class MongoEncoder(JSONEncoder):
def default(self, o):
if isinstance(o, VirtualPageModel):
data_dict = (o.to_mongo()).to_dict()
if isinstance(data_dict.get('_id'), ObjectId):
data_dict.update({'_id': str(data_dict.get('_id'))})
return data_dict
else:
return JSONEncoder.default(self, o)
class SubTypeModel(EmbeddedDocument):
filename = StringField(max_length=200, required=True)
page_num = IntField(required=True)
class VirtualPageModel(Document):
volume = StringField(max_length=200, required=True)
start_physical_page_num = IntField()
physical_pages = ListField(EmbeddedDocumentField(SubTypeModel),
default=list)
error_msg = ListField(StringField(),
default=list)
def save(self, *args, **kwargs):
print('In save: {}'.format(kwargs))
for k, v in kwargs.items():
if k == 'physical_pages':
self.physical_pages = []
for a_page in v:
tmp_pp = SubTypeModel()
for p_k, p_v in a_page.items():
setattr(tmp_pp, p_k, p_v)
self.physical_pages.append(tmp_pp)
else:
setattr(self, k, v)
return super(VirtualPageModel, self).save(*args, **kwargs)
views.py: My attempt at a view
class VolumeSplitterView(View):
#initial = {'key': 'value'}
template_name = 'click_model/index.html'
vol = None
start = 0
end = 20
def get(self, request, *args, **kwargs):
self.vol = self.kwargs.get('vol', None)
records = self.get_records()
records = records[self.start:self.end]
vp_json_list = []
img_filepaths = []
for vp in records:
vp_json = json.dumps(vp, cls=MongoEncoder)
vp_json_list.append(vp_json)
for pp in vp.physical_pages:
filepath = get_file_path(vp, pp.filename)
img_filepaths.append(filepath)
data_dict = {
'img_filepaths': img_filepaths,
'vp_json_list': vp_json_list
}
return render_to_response(self.template_name,
{'data_dict': data_dict},
RequestContext(request))
def get_records(self):
return VirtualPageModel.objects(volume=self.vol)
def post(self, request, *args, **kwargs):
if request.is_ajax:
vp_dict_list = json.loads(request.POST.get('data', []))
for vp_dict in vp_dict_list:
o_id = vp_dict.pop('_id')
original_doc = VirtualPageModel.objects.get(id=o_id)
try:
original_doc.save(**vp_dict)
except Exception:
print(traceback.format_exc())
I've got a line in python:
streams = Stream.objects.filter(info__isnull = False)\
.order_by('-score', 'info__title')
I feed that to a RequestContext
c = RequestContext(request, {
'online_streams': streams.filter(online = True),
'offline_streams': streams.filter(online = False)
})
It sorts a table of ~50 rows by their score and title. This is good, but there is one row I would like to always be top. Simply fetching it beforehand and filtering it out of the line above, then give it to the RequestContext separately won't work, since I'd have to modify the template. Also I might change which row should be on top later, or perhaps make it multiple rows instead.
Is there a filter argument I can insert that says something in the lines of "Order all rows by score then title, except row ID 8, which should always be top"? Or could I perhaps change the order of the QuerySet manually after the filtering?
Thanks for any suggestions
You can override Model manager method:
class MyManager(models.Manager):
def filter(self, *args, **kwargs):
return super(MyManager, self).filter(*args, **kwargs).exclude(id=8).order_by(...)
then in model
class Stream(models.Model):
...
objects = MyManager()
EDIT
to make sure it is included you can query:
from django.db.models import Q
return super(MyManager, self).filter(Q(id=8) | Q(*args, **kwargs)).order_by(...)
I couldn't find a way to use query sets to do this, so I ended up converting the query set to a list and sorting it using the magic compare function. It ended up looking something like this:
The magic sorter, checking for names that should be stuck "on top":
# Comparison magic
def __cmp__(self, other):
# On top
if self.name in stream_ontop:
return 1
if other.name in stream_ontop:
return -1
# Score
if self.score > other.score:
return 1
if self.score < other.score:
return -1
# Title alphabetical
if self.info.title > other.info.title:
return -1
if self.info.title < other.info.title:
return 1
return 0
Then I fetched and sorted them like this from my model class
# Custom sorting
#staticmethod
def getAllSorted(online):
streams = list(Stream.objects.filter(online=online, info__isnull=False))
streams.sort(key=None, reverse=True)
return streams
I am writing an application in Django, which uses [year]/[month]/[title-text] in the url to identitfy news items. To manage the items I have defined a number of urls, each starting with the above prefix.
urlpatterns = patterns('msite.views',
(r'^(?P<year>[\d]{4})/(?P<month>[\d]{1,2})/(?P<slug>[\w]+)/edit/$', 'edit'),
(r'^(?P<year>[\d]{4})/(?P<month>[\d]{1,2})/(?P<slug>[\w]+)/$', 'show'),
(r'^(?P<year>[\d]{4})/(?P<month>[\d]{1,2})/(?P<slug>[\w]+)/save$', 'save'),
)
I was wondering, if there is a mechanism in Django, which allows me to preprocess a given request to the views edit, show and save. It could parse the parameters e.g. year=2010, month=11, slug='this-is-a-title' and extract a model object out of them.
The benefit would be, that I could define my views as
def show(news_item):
'''does some stuff with the news item, doesn't have to care
about how to extract the item from request data'''
...
instead of
def show(year, month, slug):
'''extract the model instance manually inside this method'''
...
What is the Django way of solving this?
Or in a more generic way, is there some mechanism to implement request filters / preprocessors such as in JavaEE and Ruby on Rails?
You need date based generic views and create/update/delete generic views maybe?
One way of doing this is to write a custom decorator. I tested this in one of my projects and it worked.
First, a custom decorator. This one will have to accept other arguments beside the function, so we declare another decorator to make it so.
decorator_with_arguments = lambda decorator: lambda * args, **kwargs: lambda func: decorator(func, *args, **kwargs)
Now the actual decorator:
#decorator_with_arguments
def parse_args_and_create_instance(function, klass, attr_names):
def _function(request, *args, **kwargs):
model_attributes_and_values = dict()
for name in attr_names:
value = kwargs.get(name, None)
if value: model_attributes_and_values[name] = value
model_instance = klass.objects.get(**model_attributes_and_values)
return function(model_instance)
return _function
This decorator expects two additional arguments besides the function it is decorating. These are respectively the model class for which the instance is to be prepared and injected and the names of the attributes to be used to prepare the instance. In this case the decorator uses the attributes to get the instance from the database.
And now, a "generic" view making use of a show function.
def show(model_instance):
return HttpResponse(model_instance.some_attribute)
show_order = parse_args_and_create_instance(Order, ['order_id'])(show)
And another:
show_customer = parse_args_and_create_instance(Customer, ['id'])(show)
In order for this to work the URL configuration parameters must contain the same key words as the attributes. Of course you can customize this by tweaking the decorator.
# urls.py
...
url(r'^order/(?P<order_id>\d+)/$', 'show_order', {}, name = 'show_order'),
url(r'^customer/(?P<id>\d+)/$', 'show_customer', {}, name = 'show_customer'),
...
Update
As #rebus correctly pointed out you also need to investigate Django's generic views.
Django is python after all, so you can easily do this:
def get_item(*args, **kwargs):
year = kwargs['year']
month = kwargs['month']
slug = kwargs['slug']
# return item based on year, month, slug...
def show(request, *args, **kwargs):
item = get_item(request, *args, **kwargs)
# rest of your logic using item
# return HttpResponse...