Django - get_or_create triggers RunTimeError: maximum recursion depth exceeded - django

This is my first post on stackoverflow, so any criticisms about how I ask the question are welcome.
In my code, I get this error:
RuntimeError: maximum recursion depth exceeded
Here is the code (the content is irrelevant, I just recreated the error in the simplest way possible). Basically I am trying to override __init__. I want to do something if the object is in the database and something else if it is not.
class Question(models.Model):
text = models.CharField(max_length=140)
asked = models.BooleanField(default=False)
def __init__(self, text, *args):
#called the __init__ of the superclass.
super(Question, self).__init__()
self, c = Question.objects.get_or_create(text=text)
if c:
print 'This question will be asked!'
self.asked = True
self.save()
else:
print 'This question was already asked'
assert self.asked == True
The error occurs when calling the constructor:
Question('how are you?')
I understand that the problem comes from the get_or_create method. Looking at the error message,
---> 12 self, c = Question.objects.get_or_create(text=text)
...
---> 146 return self.get_query_set().get_or_create(**kwargs)
...
---> 464 obj = self.model(**params)
get_or_create calls the constructor of the object at some point. Which then calls get_or_create again etc...
EDIT: What I would like to achieve is basically being able to write:
Question('How are you?')
and having the object returned if it's in the database, or the newly created (and saved) one if it's not. Instead of something like:
> try:
> q = Question.objects.get(text='How are you?')
> except Question.DoesNotExist:
> q = Question(text='How are you?')
> q.save()
So I guess that the only way to achieve this is by overriding the __init__. Isn't it possible or is it conceptually wrong (or both)? Thanks!

You shouldn't really try and to this in the __init__. (In fact, it's best to leave the __init__ of Django models well alone.) It should go in the form, or the view.
In any case, you can't overwrite an instance inside itself by assigning to self - that's just a local variable like any other and will go out of scope at the end of the method.
Also note that you can use the defaults parameter to get_or_create to pass default values to be set on the new instance if an existing one is not found:
question, created = Question.objects.get_or_create(text=text, defaults={'asked': True})
Edit after question update Your edit makes it even clearer that __init__ is really not the place to be doing this. Don't forget, even evaluating a normal queryset will instantiate model objects, which means calling __init__ - so just getting an instance from the database will run into problems. Don't do this.
Instead, if you really need this to be provided by the model - even though it's just one line of code, as shown above - you could define a classmethod:
class Question(models.Model):
...
#classmethod
def query(cls, text):
question, _ = cls.objects.get_or_create(text=text, defaults={'asked': True})
return question
Then you could do Question.query('How are you') and it would return either the new or the existing item.

The logic within your __init__ needs to be moved somewhere else, like to a view. get_or_create returns two values: 1) the object and 2) if the object had to be created or not. See the docs for more details.
def some_view(request):
c, created = Question.objects.get_or_create(text=text)
if not created:
print 'This question was already asked'
else:
print 'This question will be asked!'
c.asked = True
c.save()

Like #Scott Woodall said, you need to move you init logic.
that is What's happening:
When you call Question('how are you?'), you go to __init__ method.
The __init__ calls Question.objects.get_or_create(text=text), who does't find a Question with that text, so try to create a new Question, calling __init__ (again).
You are re-entering in the method call forever.
Question('how are you?') # <-- You call this method
|
+-- def __init__(self, text, *args):
|
+-- Question.objects.get_or_create(text=text) # This try to create a model, calling __init__ method!
|
+-- def __init__(self, text, *args):
|
+-- Question.objects.get_or_create(text=text) # This try to create a model, calling __init__ method!
|
+ # So on...
I think you should add a unique=True to text Question field.
See the #Scott answer

Related

Django object "lock" and context rendering

I have a simple(I think) question, about Django context rendering.
I'll step right into it -
Basically what I need is, some temp table, which in my case, I called Locked. And when a user presses a button, which Is a form, that object goes straight to the table Locked(just a simple insert). And inside that table there is a field called is_locked, and if its True, that object needs to go gray, or to have some lock icon inside the html table.
Just some kind of a viewable sign, that an object is inside the table Locked, and that another user can't access it.
But, my problem is, since in my views.py, my lock function is not returning exact html where I want to render that locker icon, instead, it returns another html.
Is there any way, to render same context, on 2 html pages? Thank's.
This is my code :
views.py
def lock(request, pk):
# Linking by pk.
opp = get_object_or_404(OpportunityList, pk=pk)
opp_locked = get_object_or_404(Locked, pk=pk)
# Taking two parametters for 2 fields.
eluid = Elementiur.objects.get(eluid=pk)
user = User.objects.get(username=request.user)
# Dont bother with this one! Just pulling one field value.
field_name = 'optika_korisnik'
obj = OpportunityList.objects.get(pk=pk)
field_object = OpportunityList._meta.get_field(field_name)
field_value = getattr(obj, field_object.attname)
# This is the main part! This is where i'm inserting data into Locked table.
if opp_locked.DoesNotExist:
opp_locked.id = int(eluid.eluid)
opp_locked.locked_eluid = eluid
opp_locked.locked_comment = field_value
opp_locked.locked_user = user
opp_locked.locked_name = 'Zaključao korisnik - ' + request.user.username
opp_locked.is_locked = True
opp_locked.save()
# This is what has to be returned, but i need context on the other page.
return render(request, 'opportunity/detalji/poziv.html',
{'opp': opp, 'locked': opp_locked})
else:
# This return has the context that i need(from the first opp_locked variable)
return render(request, 'opportunity/opp_optika.html', {'locked_test': opp_locked})
I can provide more code, but i think that it's not important for this type of question, because all of the logic is happening inside the lock finction, and last two returns.
I just had a quick overview of your snippet sorry if this not help you but you need to review it a little bit.
You call DoesNotExist on an instance of a Locked model
if opp_locked.DoesNotExist: [...]
that's not how you should use this exception.
You have a method .exists() that is available but only for Querysets.
Also if your instance does not exists you are alredy returning an Http404 response when you use get_object_or_404() method.
And perhaps you should avoid sharing primary keys between instances and replace them with models.OneToOneField (OneToOnefield)
Since i got no answers, i added a new field, is_locked, into my Locked model and that solved it.

In Django, how can I prevent a "Save with update_fields did not affect any rows." error?

I'm using Django and Python 3.7. I have this code
article = get_article(id)
...
article.label = label
article.save(update_fields=["label"])
Sometimes I get the following error on my "save" line ...
raise DatabaseError("Save with update_fields did not affect any rows.")
django.db.utils.DatabaseError: Save with update_fields did not affect any rows.
Evidently, in the "..." another thread may be deleting my article. Is there another way to rewrite my "article.save(...)" statement such that if the object no longer exists I can ignore any error being thrown?
A comment by gachdavit suggested using select_for_update. You could modify your get_article function to call select_for_update prior to fetching the article. By doing this, the database row holding the article will be locked as long as the current transaction does not commit or roll back. If another thread tries to delete the article at the same time, that thread will block until the lock is released. Effectively, the article won't be deleted until after you have called the save function.
Unless you have special requirements, this is the approach I'd take.
I'm not aware of any special way to handle it other than to check to see if the values have changed.
article = update_model(article, {'label': label})
def update_model(instance, updates):
update_fields = {
field: value
for field, value in updates.items()
if getattr(instance, field) != value
}
if update_fields:
for field, value in update_fields.items():
setattr(instance, field, value)
instance.save(update_fields=update_fields.keys())
return instance
Edit:
Another alternative would be to catch and handle the exception.
This is hacky, but you could override _do_update in your model and simply return True. Django itself does something kind of hacky on line 893 of _do_update to suppress the same exception when update_fields contains column names that do not appear in the model.
The return value from _do_update triggers the exception you are seeing from this block
I tested the override below and it seemed to work. I feel somewhat dirty for overriding a private-ish method, but I think I will get over it.
def _do_update(self, base_qs, using, pk_val, values, update_fields, forced_update):
updated = super(Article, self)._do_update(base_qs, using, pk_val, values, update_fields, forced_update)
if not updated and Article.objects.filter(id=pk_val).count() == 0:
return True
return updated
This solution could be genericized and moved to a mixin base class if you need to handle this for more than one model.
I used this django management command to test
from django.core.management.base import BaseCommand
from foo.models import Article
class Command(BaseCommand):
def handle(self, *args, **kwargs):
Article.objects.update_or_create(id=1, defaults=dict(label='zulu'))
print('Testing _do_update hack')
article1 = Article.objects.get(id=1)
article1.label = 'yankee'
article2 = Article.objects.get(id=1)
article2.delete()
article1.save(update_fields=['label'])
print('Done. No exception raised')

Create REST API with Neo4J and Django

I am trying to create a REST API with Neo4j and Django in the backend.
The problem is that even when I have Django models using Neo4Django , I can't use frameworks like Tastypie or Piston that normally serialize models into JSON (or XML).
Sorry if my question is confusing or not clear, I am newbie to webservices.
Thanks for you help
EDIT: So I started with Tastypie and followed the tutorial on this page http://django-tastypie.readthedocs.org/en/latest/tutorial.html. I am looking for displaying the Neo4j JSON response in the browser, but when I try to access to http://127.0.0.1:8000/api/node/?format=json I get this error instead:
{"error_message": "'NoneType' object is not callable", "traceback": "Traceback (most recent call last):\n\n File \"/usr/local/lib/python2.6/dist-packages/tastypie/resources.py\", line 217, in wrapper\n response = callback(request, *args, **kwargs)\n\n File \"/usr/local/lib/python2.6/dist-packages/tastypie/resources.py\", line 459, in dispatch_list\n return self.dispatch('list', request, **kwargs)\n\n File \"/usr/local/lib/python2.6/dist-packages/tastypie/resources.py\", line 491, in dispatch\n response = method(request, **kwargs)\n\n File \"/usr/local/lib/python2.6/dist-packages/tastypie/resources.py\", line 1298, in get_list\n base_bundle = self.build_bundle(request=request)\n\n File \"/usr/local/lib/python2.6/dist-packages/tastypie/resources.py\", line 718, in build_bundle\n obj = self._meta.object_class()\n\nTypeError: 'NoneType' object is not callable\n"}
Here is my code :
api.py file:
class NodeResource (ModelResource): #it doesn't work with Resource neither
class meta:
queryset= Node.objects.all()
resource_name = 'node'
urls.py file:
node_resource= NodeResource()
urlpatterns = patterns('',
url(r'^api/', include(node_resource.urls)),
models.py file :
class Node(models.NodeModel):
p1 = models.StringProperty()
p2 = models.StringProperty()
I would advise steering away from passing Neo4j REST API responses directly through your application. Not only would you not be in control of the structure of these data formats as they evolve and deprecate (which they do) but you would be exposing unnecessary internals of your database layer.
Besides Neo4Django, you have a couple of other options you might want to consider. Neomodel is another model layer designed for Django and intended to act like the built-in ORM; you also have the option of the raw OGM layer provided by py2neo which may help but isn't Django-specific.
It's worth remembering that Django and its plug-ins have been designed around a traditional RDBMS, not a graph database, so none of these solutions will be perfect. Whatever you choose, you're likely to have to carry out a fair amount of transformation work to create your application's API.
Django-Tastypie allows to create REST APIs with NoSQL databases as well as mentioned in http://django-tastypie.readthedocs.org/en/latest/non_orm_data_sources.html.
The principle is to use tastypie.resources.Resource and not tastypie.resources.ModelResource which is SPECIFIC to RDBMS, then main functions must be redefined in order to provide a JSON with the desired parameters.
So I took the example given in the link, modified it and used Neo4j REST Client for Python to get an instance of the db and perform requests, and it worked like a charm.
Thanks for all your responses :)
Thanks to recent contributions, Neo4django now supports Tastypie out of the box! I'd love to know what you think if you try it out.
EDIT:
I've just run through the tastypie tutorial, and posted a gist with the resulting example. I noticed nested resources are a little funny, but otherwise it works great. I'm pretty sure the gents who contributed the patches enabling this support also know how to take care of nested resources- I'll ask them to speak up.
EDIT:
As long as relationships are specified in the ModelResource, they work great. If anyone would like to see examples, let me know.
Well my answer was a bit vague so I'm gonna post how a solved the problem with some code:
Assume that I want to create an airport resource with some attributes. I will structure this in 3 different files (for readability reasons).
First : airport.py
This file will contain all the resource attributes and a constructor too :
from models import *
class Airport(object):
def __init__ (self, iata, icao, name, asciiName, geonamesId, wikipedia, id, latitude, longitude):
self.icao = icao
self.iata = iata
self.name = name
self.geonamesId = geonamesId
self.wikipedia = wikipedia
self.id = id
self.latitude = latitude
self.longitude = longitude
self.asciiName = asciiName
This file will be used in order to create resources.
Then the second file : AirportResource.py:
This file will contain the resource attributes and some basic methods depending on which request we want our resource to handle.
class AirportResource(Resource):
iata = fields.CharField(attribute='iata')
icao = fields.CharField(attribute='icao')
name = fields.CharField(attribute='name')
asciiName = fields.CharField(attribute='asciiName')
latitude = fields.FloatField(attribute='latitude')
longitude = fields.FloatField(attribute='longitude')
wikipedia= fields.CharField(attribute='wikipedia')
geonamesId= fields.IntegerField(attribute='geonamesId')
class Meta:
resource_name = 'airport'
object_class = Airport
allowed_methods=['get', 'put']
collection_name = 'airports'
detail_uri_name = 'id'
def detail_uri_kwargs(self, bundle_or_obj):
kwargs = {}
if isinstance(bundle_or_obj, Bundle):
kwargs['id'] = bundle_or_obj.obj.id
else:
kwargs['id'] = bundle_or_obj.id
return kwargs
As mentioned in the docs, if we want to create an API that handle CREATE, GET, PUT, POST and DELETE requests, we must override/implement the following methods :
def obj_get_list(self, bundle, **kwargs) : to GET a list of objects
def obj_get(self, bundle, **kwargs) : to GET an individual object
def obj_create(self, bundle, **kwargs) to create an object (CREATE method)
def obj_update(self, bundle, **kwargs) to update an object (PUT method)
def obj_delete(self, bundle, **kwargs) to delete an object (DELETE method)
(see http://django-tastypie.readthedocs.org/en/latest/non_orm_data_sources.html)
Normally, in ModelResource all those methods are defined and implemented, so they can be used directly without any difficulty. But in this case, they should be customized according to what we want to do.
Let's see an example of implementing obj_get_list and obj_get :
For obj_get_list:
In ModelResource, the data is FIRSTLY fetched from the database, then it could be FILTERED according to the filter declared in META class ( see http://django-tastypie.readthedocs.org/en/latest/interacting.html). But I didn't wish to implement such behavior (get everything then filter), so I made a query to Neo4j given the query string parameters:
def obj_get_list(self,bundle, **kwargs):
data=[]
params= []
for key in bundle.request.GET.iterkeys():
params.append(key)
if "search" in params :
query= bundle.request.GET['search']
try:
results = manager.searchAirport(query)
data = createAirportResources(results)
except Exception as e:
raise NotFound(e)
else:
raise BadRequest("Non valid URL")
return data
and for obj_get:
def obj_get(self, bundle, **kwargs):
id= kwargs['id']
try :
airportNode = manager.getAirportNode(id)
airport = createAirportResources([airportNode])
return airport[0]
except Exception as e :
raise NotFound(e)
and finally a generic function that takes as parameter a list of nodes and returns a list of Airport objects:
def createAirportResources(nodes):
data= []
for node in nodes:
iata = node.properties['iata']
icao = node.properties['icao']
name = node.properties['name']
asciiName = node.properties['asciiName']
geonamesId = node.properties['geonamesId']
wikipedia = node.properties['wikipedia']
id = node.id
latitude = node.properties['latitude']
longitude = node.properties['longitude']
airport = Airport(iata, icao, name, asciiName, geonamesId, wikipedia, id, latitude, longitude)
data.append(airport)
return data
Now the third manager.py : which is in charge of making queries to the database and returning results :
First of all, I get an instance of the database using neo4j rest client framework :
from neo4jrestclient.client import *
gdb= GraphDatabase("http://localhost:7474/db/data/")
then the function which gets an airport node :
def getAirportNode(id):
if(getNodeType(id) == type):
n= gdb.nodes.get(id)
return n
else:
raise Exception("This airport doesn't exist in the database")
and the one to perform search (I am using a server plugin, see Neo4j docs for more details):
def searchAirport(query):
airports= gdb.extensions.Search.search(query=query.strip(), searchType='airports', max=6)
if len(airports) == 0:
raise Exception('No airports match your query')
else:
return results
Hope this will help :)

django multiwidget subclass not calling decompress()

I am trying to implement a MultiValueField for IP Adress/Domain Name entries. It works as expected for entering data.
My Problem is that if I want to display the form bound to specific data, the IP Address/Domain Name field stays empty. All other fields are filled with the desired data. If I use a normal CharField, I get the data that I would expect. But it does not work with my custom field.
I have tracked it down to the fact that my custom MultiWidget does not call its decompress method.
Here is my Field:
class accessIPField(forms.MultiValueField):
"""
custom Field for access IP
"""
def __init__(self, *args, **kwargs):
self.fields=(
forms.IPAddressField(label='IP Adress'),
forms.CharField(max_length=50,label='Domain Name')
)
self.widget=accessIPWidget()
super(accessIPField,self).__init__(self.fields,self.widget, *args, **kwargs)
def compress(self,data_list):
if data_list:
return " ".join(data_list)
And here is my widget:
class accessIPWidget(forms.MultiWidget):
"""
Widget to display IP Adress / Domain name pairs
"""
def __init__(self,*args,**kwargs):
self.widgets=(forms.TextInput(),forms.TextInput())
super(accessIPWidget,self).__init__(self.widgets,*args,**kwargs)
def decompress(self,value):
print 'decompress called'
if value:
return value.rsplit()
return [None,None]
def format_output(self, rendered_widgets):
return u'\n'.join(rendered_widgets)
The whole thing is called (in a larger context) as
self.fields['access_IPs'] = accessIPField()
Now as you can see, I put a print statement in my compress method, and I never get to see that statement. Also, if I rename compress to something like foobar, I would expect (according to the django code for MultiWidget) to get the NotImplementedError, which is not the case. Any suggestions?
I am using python 2.6.5, django 1.1 on ubuntu server 10.04.
It turns out that the problem was with the value_from_datadict() method as implemented by MultiWidget. First of all, it allready returned a list, so that is why decompress() was not called in the first place. Secondly, it allways returen a [None,None] list, so that is why the bound form stayed empty.
I needed to implement my own (within my accessIPWidget class):
def value_from_datadict(self, data, files, name):
try:
return data.get(name,None).rsplit()
except AttributeError:
return [widget.value_from_datadict(data, files, name + '_%s' % i) for i, widget in enumerate(self.widgets)]
Now the last line is what the original method did. In order to get the data into the bound form, I needed to add data.get(name,None).rsplit().
As far as I understand, the original value_from_datadict method only works for unbound fields. Because it changes the name of the original field to name + '_%s', which is what you get when pressing the submit button. In order to fill in a bound method, the datadict needs to be queried for 'name' only.
Hm, not shure if there is a way around this, but it seems to me that this behaviour should at least be documented somewhere.
Maybe I misunderstood something?

Django: Overriding the save() method: how do I call the delete() method of a child class

The setup =
I have this class, Transcript:
class Transcript(models.Model):
body = models.TextField('Body')
doPagination = models.BooleanField('Paginate')
numPages = models.PositiveIntegerField('Number of Pages')
and this class, TranscriptPages(models.Model):
class TranscriptPages(models.Model):
transcript = models.ForeignKey(Transcript)
order = models.PositiveIntegerField('Order')
content = models.TextField('Page Content', null=True, blank=True)
The Admin behavior I’m trying to create is to let a user populate Transcript.body with the entire contents of a long document and, if they set Transcript.doPagination = True and save the Transcript admin, I will automatically split the body into n Transcript pages.
In the admin, TranscriptPages is a StackedInline of the Transcript Admin.
To do this I’m overridding Transcript’s save method:
def save(self):
if self.doPagination:
#do stuff
super(Transcript, self).save()
else:
super(Transcript, self).save()
The problem =
When Transcript.doPagination is True, I want to manually delete all of the TranscriptPages that reference this Transcript so I can then create them again from scratch.
So, I thought this would work:
#do stuff
TranscriptPages.objects.filter(transcript__id=self.id).delete()
super(Transcript, self).save()
but when I try I get this error:
Exception Type: ValidationError
Exception Value: [u'Select a valid
choice. That choice is not one of the
available choices.']
... and this is the last thing in the stack trace before the exception is raised:
.../django/forms/models.py in save_existing_objects
pk_value = form.fields[pk_name].clean(raw_pk_value)
Other attempts to fix:
t =
self.transcriptpages_set.all().delete()
(where self = Transcript from the
save() method)
looping over t (above) and deleting each item individually
making a post_save signal on TranscriptPages that calls the delete method
Any ideas? How does the Admin do it?
UPDATE: Every once in a while as I'm playing around with the code I can get a different error (below), but then it just goes away and I can't replicate it again... until the next random time.
Exception Type:
MultiValueDictKeyError Exception
Value: "Key 'transcriptpages_set-0-id'
not found in "
Exception Location:
.../django/utils/datastructures.py in
getitem, line 203
and the last lines from the trace:
.../django/forms/models.py in _construct_form
form = super(BaseInlineFormSet, self)._construct_form(i, **kwargs)
.../django/utils/datastructures.py in getitem
pk = self.data[pk_key]
In the end it was a matter of timing. When deleting only a single child object out of many, there was no problem. If I was deleting too many child objects at once, the error could happen, because the delete action was attempting to reference ids that were not around. This is why nothing worked, not signals, not [object]_set. I fixed it by using jquery to set a hidden variable in the edit form for the child object, which caused the object to first process the update (slowing down it's processing) and THEN the delete.
You are probably trying to access the Transaction.id before is has been created. Additionally, you can try to access the TransactionPage objects through the Transaction object via transactionpage_set (see query docs about FOO_set notation).
def save(self):
super(Transcript, self).save()
if self.doPagination:
self.transaction_set.all().delete() # .all() may be optional
pages = ... # whatever you do to split the content
self.numPages = len(pages)
self.save() # yes, a second save if you store numPages in Transaction
for page_number in range(self.numPages):
TransactionPage.objects.create(content=pages[page_number],
order=page_number, transaction=self)
You could also switch to not storing numPages in the Transaction and access it via a property instead.
class Transaction(models.Model):
# ...
# replace numPages with
#property
def page_count(self):
return self.transactionpage_set.count() or 1
And then if you took it one step further you could always use the TransactionPage objects for display purposes. This would allow you to get rid of the extra self.save() call in the above save() method. It will also let you simplify your templates by always displaying TransactionPage.content instead of conditionally displaying Transaction.body if you are paginating and TransactionPage.content otherwise.
class Transaction(models.Model):
body = models.TextField()
paginate = models.BooleanField()
#property
def page_count(self):
return self.transactionpage_set.count() # no more "or 1" cruft!
def save(self):
super(Transcript, self).save()
self.transaction_set.delete() # might need an .all() before .delete()
if self.paginate:
# Do whatever you do to split the body into pages.
pages = ...
else:
# The only page is the entire body.
pages = [self.body]
for (page_number, page_content) in enumerate(pages):
TransactionPage.objects.create(content=page_content,
order=page_number, transaction=self)
Another possible solution might be to override save_formset() in the ModelAdmin to prevent the update completely:
def save_formset(self, request, form, formset, change):
if (form.cleaned_data['do_pagination'] and
formset.model == TranscriptPages):
formset.changed_objects = []
formset.new_objects = []
formset.deleted_objects = []
else:
formset.save()
Then, you can do whatever you like in Modal.save(). Note that there's probably a more elegant/breakproof way to stop the formset from processing if one wanted to dig into the internals a bit more.