Haystack + Whoosh throwing root:Error updating <appname> using default - django

Just started on Whoosh + Haystack.
I tested a test model which indexed fine and gave results. But indexing fails for some unknown reason on the second model (the important one)
Here is the model
class Sale(models.Model):
brand = models.ForeignKey('Brand', related_name='sales_p')
outlets = models.ManyToManyField('Outlet', null=True, blank=True, related_name='sales')
child_brands = models.ManyToManyField('Brand', null=True, blank=True, related_name='sales_c')
child_categories = models.ManyToManyField('Category', null=True, blank=True, related_name='sales_c')
#collecting subcategories for subcategory sale
child_subcategories = models.ManyToManyField('SubCategory', null=True, blank=True,related_name='subcategory')
The search index for this is
class SaleIndex(indexes.SearchIndex, indexes.Indexable):
text = indexes.CharField(document=True, use_template=True)
outlets = indexes.CharField(model_attr='outlets')
child_brands = indexes.CharField(model_attr = 'child_brands')
child_categories = indexes.CharField(model_attr = 'child_categories')
child_subcategories = indexes.CharField(model_attr = 'child_subcategories')
content_auto = indexes.CharField(model_attr='brand')
def get_model(self):
return Sale
def index_queryset(self, using=None):
return Sale.objects.all()
def prep_childbrands(self, object):
return [brand.name for brand in self.child_brands.all()]
No one else in the world seems to have faced this ever. Am new to programming in general. Is there something I am missing that is obvious?
Traceback if that helps:
Indexing 84 Sales
ERROR:root:Error updating website using default
Traceback (most recent call last):
File "/usr/local/lib/python3.4/dist-packages/haystack/management/commands/update_index.py", line 189, in handle_label
self.update_backend(label, using)
File "/usr/local/lib/python3.4/dist-packages/haystack/management/commands/update_index.py", line 234, in update_backend
do_update(backend, index, qs, start, end, total, self.verbosity)
File "/usr/local/lib/python3.4/dist-packages/haystack/management/commands/update_index.py", line 89, in do_update
backend.update(index, current_qs)
File "/usr/local/lib/python3.4/dist-packages/haystack/backends/whoosh_backend.py", line 191, in update
doc = index.full_prepare(obj)
File "/usr/local/lib/python3.4/dist-packages/haystack/indexes.py", line 207, in full_prepare
self.prepared_data = self.prepare(obj)
File "/usr/local/lib/python3.4/dist-packages/haystack/indexes.py", line 198, in prepare
self.prepared_data[field.index_fieldname] = field.prepare(obj)
File "/usr/local/lib/python3.4/dist-packages/haystack/fields.py", line 159, in prepare
return self.convert(super(CharField, self).prepare(obj))
File "/usr/local/lib/python3.4/dist-packages/haystack/fields.py", line 106, in prepare
return current_object()
File "/usr/local/lib/python3.4/dist-packages/django/db/models/fields/related.py", line 843, in __call__
manager = getattr(self.model, kwargs.pop('manager'))
KeyError: 'manager'

There seems to be a bug in Haystack + Django 1.7 that is causing this error. This is also referenced in this other question which I had not noticed because of the Xapian mention (we are using Whoosh)
Django-haystack with xapian engine: can't execute update_index if model has ManyToManyField

Related

Django : How to annotate, group by user and annotate again using the first annotation?

I a Django (1.11) project with a ReadingSession model as follows:
class ReadingSession(models.Model):
user = models.ForeignKey(User)
started_at = models.DateTimeField()
ended_at = models.DateTimeField()
And I want to display the sum of the reading sessions durations for each of some given users, in one database request.
Among other attempts, I've tried :
usernames = ['user1', 'user2',]
q = ReadingSession.objects.filter(user__username__in=usernames)\
.annotate(duration=Func(F('ended_at'), F('started_at'), function='age'))\
.order_by().values('user') \
.annotate(total_duration=Sum('duration'))\
.values('user', 'total_duration')
print q
but it fails with :
Traceback (most recent call last):
File ".../misc_queries.py", line 74, in <module>
test_query()
File ".../misc_queries.py", line 51, in test_query
.annotate(total_duration=Sum('duration'))
File ".../lib/python2.7/site-packages/django/db/models/query.py", line 945, in annotate
clone.query.add_annotation(annotation, alias, is_summary=False)
File "/.../lib/python2.7/site-packages/django/db/models/sql/query.py", line 973, in add_annotation
summarize=is_summary)
File ".../lib/python2.7/site-packages/django/db/models/aggregates.py", line 19, in resolve_expression
c = super(Aggregate, self).resolve_expression(query, allow_joins, reuse, summarize)
File ".../lib/python2.7/site-packages/django/db/models/expressions.py", line 548, in resolve_expression
c.source_expressions[pos] = arg.resolve_expression(query, allow_joins, reuse, summarize, for_save)
File "/.../lib/python2.7/site-packages/django/db/models/expressions.py", line 471, in resolve_expression
return query.resolve_ref(self.name, allow_joins, reuse, summarize)
File "..../local/lib/python2.7/site-packages/django/db/models/sql/query.py", line 1472, in resolve_ref
return self.annotation_select[name]
KeyError: 'duration'
Which makes sense, as the "values('user')" statement filtered the fields returned by the 'filter()' statement.
But on the other side, if I add 'duration' in the first 'values()' statement, the results are not groupped by user anymore.
Do you see what I'm doing wrong here or have you another way to achieve this ?
Thanks in advance !
I'm sure you can do it as a complex db query, but how about returning the data and iterating over it in the view?
queryset = ReadingSession.objects.all()
results = {}
for record in queryset:
if not record.user in results:
results[record.user] = timedelta(microseconds=0)
time_diff = record.ended_at - record.started_at
results[record.user] += time_diff
return results
That will return a dictionary of results[username] = [sum of session times]
You can probably use Window functions:
https://docs.djangoproject.com/en/3.1/ref/models/expressions/#window-functions
from django.db import models
usernames = ('user1', 'user2',)
q = ReadingSession.objects.filter(
user__username__in=usernames
).annotate(
duration=models.Func(models.F('ended_at'), models.F('started_at'), function='age')
).annotate(
total_duration=models.Window(
expression=models.Sum('duration'),
partition_by=[models.F('user')],
)
).values('user', 'total_duration')
print q

Django 1.9.5: FieldError for reverse lookup fields occasionally

I'm getting a frustrating intermittent error in Django, when attempting to run the objects.all() for a Queryset that is prefetching. There is an issue where occasionally the model._meta seems to be missing fields, between instantiation of a queryset and running an iteration through it. It's almost as if the queryset's prefetch doesn't actually get run in time for the iteration of the list of objects.
In this example, data.service_log is simply a queryset with some prefetched items called servicelog. When I run the queryset in the shell, I can look at all the fields in the self.names_to_path(lookup_splitted, self.get_meta()) method on the queryset. They are all there, specifically the "servicelog."
Notice this Traceback for this error it says that "servicelog" is not an available field, yet it lists it in the list of fields to choose from. This seems to be a Django bug, but I can't be sure because I can't explain or isolate the behavior. I can't possibly be the only person getting this error. It seems to be in django/db/models/sql/query.py in the names_to_paths() method. Here's the code that is failing to resolve:
query.py names_to_paths():
field = None
try:
field = opts.get_field(name)
except FieldDoesNotExist:
if name in self.annotation_select:
field = self.annotation_select[name].output_field
if field is not None:
# Fields that contain one-to-many relations with a generic
# model (like a GenericForeignKey) cannot generate reverse
# relations and therefore cannot be used for reverse querying.
if field.is_relation and not field.related_model:
raise FieldError(
"Field %r does not generate an automatic reverse "
"relation and therefore cannot be used for reverse "
"querying. If it is a GenericForeignKey, consider "
"adding a GenericRelation." % name
)
try:
model = field.model._meta.concrete_model
except AttributeError:
model = None
else:
# We didn't find the current field, so move position back
# one step.
pos -= 1
if pos == -1 or fail_on_missing:
field_names = list(get_field_names_from_opts(opts))
available = sorted(field_names + list(self.annotation_select))
raise FieldError("Cannot resolve keyword %r into field. "
"Choices are: %s" % (name, ", .join(available)))
break
field does not get set in the first try, then the condition statementif field is not Nonefails so we enter theelseblock. There theposgets reduced by one, but since this field is 'servicelog' it is already at 0. However, when I try this in the shell, it always finds thefieldwithopts.get_field('servicelog')`. Only when run from WSGI and Apache2 does this failure occur. Again, it isn't all the time, which makes it extremely difficult to test. I'm perplexed by this, and am not sure where to look for clues. PLEASE if anybody has ANY ideas of what to explore, I'd be so appreciative.
Traceback (most recent call last):
File "/var/www/fast/services/views/edit.py", line 12897, in service_log
for service in data.service_log:
File "/usr/local/lib/python2.7/dist-packages/django/db/models/query.py", line 258, in __iter__
self._fetch_all()
File "/usr/local/lib/python2.7/dist-packages/django/db/models/query.py", line 1076, in _fetch_all
self._prefetch_related_objects()
File "/usr/local/lib/python2.7/dist-packages/django/db/models/query.py", line 656, in _prefetch_related_objects
prefetch_related_objects(self._result_cache, self._prefetch_related_lookups)
File "/usr/local/lib/python2.7/dist-packages/django/db/models/query.py", line 1457, in prefetch_related_objects
obj_list, additional_lookups = prefetch_one_level(obj_list, prefetcher, lookup, level)
File "/usr/local/lib/python2.7/dist-packages/django/db/models/query.py", line 1556, in prefetch_one_level
prefetcher.get_prefetch_queryset(instances, lookup.get_current_queryset(level)))
File "/usr/local/lib/python2.7/dist-packages/django/db/models/fields/related_descriptors.py", line 802, in get_prefetch_queryset
queryset = queryset._next_is_sticky().filter(**query)
File "/usr/local/lib/python2.7/dist-packages/django/db/models/query.py", line 790, in filter
return self._filter_or_exclude(False, *args, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/django/db/models/query.py", line 808, in _filter_or_exclude
clone.query.add_q(Q(*args, **kwargs))
File "/usr/local/lib/python2.7/dist-packages/django/db/models/sql/query.py", line 1243, in add_q
clause, _ = self._add_q(q_object, self.used_aliases)
File "/usr/local/lib/python2.7/dist-packages/django/db/models/sql/query.py", line 1269, in _add_q
allow_joins=allow_joins, split_subq=split_subq,
File "/usr/local/lib/python2.7/dist-packages/django/db/models/sql/query.py", line 1149, in build_filter
lookups, parts, reffed_expression = self.solve_lookup_type(arg)
File "/usr/local/lib/python2.7/dist-packages/django/db/models/sql/query.py", line 1035, in solve_lookup_type
_, field, _, lookup_parts = self.names_to_path(lookup_splitted, self.get_meta())
File "/usr/local/lib/python2.7/dist-packages/django/db/models/sql/query.py", line 1330, in names_to_path
"Choices are: %s" % (name, ", ".join(available)))
FieldError: Cannot resolve keyword u'servicelog' into field. Choices are: additional_county_worker_notes, adoption_disrupted, adoption_first_name, adoption_last_name, adoption_middle_name, adoption_placement_date, adoption_placement_reason, adoption_placement_reason_id, adoption_placement_type, adoption_risk_level, adoption_risk_level_id, adoption_termination_date, adoption_termination_destination, adoption_termination_reason, adoption_termination_reason_id, adoptive_placement, agency, agency_id, all_items_outstanding, all_items_past_due, appeal_process_date, attached_file, attends_college, attorney_email_address, attorney_email_address_id, attorney_fax, attorney_fax_id, attorney_investigator_email_address, attorney_investigator_email_address_id, attorney_investigator_fax, attorney_investigator_fax_id, attorney_investigator_name, attorney_investigator_phone, attorney_investigator_phone_id, attorney_name, attorney_phone, attorney_phone_id, blood_related_to_applicants, blood_relationship, casa_email_address, casa_email_address_id, casa_fax, casa_fax_id, casa_name, casa_phone, casa_phone_id, certification_items_outstanding, certification_items_past_due, classification, classification_id, client, client_id, county_adoption_worker, county_adoption_worker_cell, county_adoption_worker_cell_id, county_adoption_worker_email_address, county_adoption_worker_email_address_id, county_adoption_worker_fax, county_adoption_worker_fax_id, county_adoption_worker_office, county_adoption_worker_office_id, county_adoption_worker_phone, county_adoption_worker_phone_id, county_adoption_worker_title, county_case_number, county_worker, county_worker_cell, county_worker_cell_id, county_worker_email_address, county_worker_email_address_id, county_worker_fax, county_worker_fax_id, county_worker_office, county_worker_office_id, county_worker_phone, county_worker_phone_id, county_worker_title, court, court_case_name, court_case_number, court_department, court_id, created, current_grade, date_identified_adoptive, date_placed_with_home, deleted, discharge_summary, eligibility_worker, eligibility_worker_email_address, eligibility_worker_email_address_id, eligibility_worker_phone, eligibility_worker_phone_id, emergency_placement, employed_80_hours, enable_discharge_summary, expected_type, expected_type_id, extended_family_contact_allowed, final_payment_amount, finalization_date, foreign_placement, hearing_36626_date, homestudy, id, incident_placement_1, incident_placement_2, incident_placement_3, incident_placement_4, individualized_plan_review, inhousemove, interpretive_summary, item_due, items_approved, items_pending, items_rejected, items_update_requested, la_county_id, medi_cal, medi_cal_eligibility_phone, medi_cal_eligibility_phone_id, medi_cal_eligibility_worker, medi_cal_id, modified, monthly_monitored_visit_hours, mother_child, move_in_type, move_out_type, new_protective_custody_petition, non_minor_dependent, non_truant, notes, number_of_files_required, other_school_contact, other_school_contact_first_name, other_school_contact_last_name, parent_payment_override_annually, parent_payment_override_daily, parent_payment_override_monthly, parental_contact_allowed, parental_group, parental_group_id, payment_amount, percent_certified, percent_items_complete, person_number, placement, placement_date, placement_id, placement_payment_override_annually, placement_payment_override_daily, placement_payment_override_monthly, placement_reason, placement_reason_details, placement_reason_id, placement_self, placer_shelter_bed, prior_placement, progress_summary, projected_adoption_36626_date, projected_adoption_finalization_date, projected_adoption_placement_date, recordreview, requires_educational_support, requires_mental_health_services, respite, school, school_different, school_id, school_liaison_email, school_liaison_first_name, school_liaison_last_name, school_liaison_phone, school_liaison_phone_extension, school_notes, serial_number, servicecontact_onbehalf, servicedeliverylog, servicelog, social_worker_at_termination, social_worker_at_termination_id, special_health_care_needs, state_case_number, teachers, termination_date, termination_destination, termination_reason, termination_reason_details, termination_reason_id, therapist, therapy_code, therapy_supervision_requirements, treatment_abilities, treatment_needs, treatment_preferences, treatment_strengths, treatmentneed, update_requested, update_requested_by, update_requested_by_id, update_requested_date, update_requested_note, updateable, use_number_required, uses_psychotropic_medication, visit_frequency_override, visit_frequency_override_id, visitation_restrictions, who_can_pickup_at_home, who_can_pickup_at_school, who_can_visit
UPDATE - adding models/offending view code
models.py
class ParentalGroup(models.Model):
many fields...
class Placement(models.Model):
parental_group = models.ForeignKey(ParentalGroup, null=True, blank=True)
many more fields...
class ServiceLog(models.Model):
parental_group = models.ForeignKey(ParentalGroup, null=True, blank=True)
placement = models.ManyToManyField(Placement, blank=True)
many more fields...
views.py:
data.service_log = ServiceLog.objects.filter(
parental_group=data.pg,
).prefetch_related(
Prefetch(
'placement',
queryset=Placement.objects.all(),
to_attr='placements'
),
)
for service in data.service_log:
some code to generate data to pass to template...
return render_to_response(...)
I had a similar problem using Django 1.8.6 on Gunicorn/Django runserver. I haven't managed to reproduce the error in shell/notebook environment either.
I solved the randomly occurring FieldError by adding related_name to the ManyToManyField. I used a through model in the ManyToManyField though.
In your case:
class ServiceLog(models.Model):
parental_group = models.ForeignKey(ParentalGroup, null=True, blank=True)
placement = models.ManyToManyField(Placement, blank=True, related_name='servicelog')
Jan

Please help me decipher a traceback for "ValueError: invalid literal for int()" in my models

I am trying to create a list of ProductPart objects where ProductPart.part matches the value of an instance of Variety.variety_name. Here is a demonstration of my problem.
> v = Variety.objects.get(id=2)
> p = ProductPart.objects.get(id=1)
> v.variety_name
'Non Pareil (Inshell)'
> p.part
<Variety: Non Pareil (Inshell)>
> a = ProductPart.objects.values().filter(part=v.variety_name)
...
ValueError: invalid literal for int() with base 10: 'Non Pareil (Inshell)'
This pattern seems to work elsewhere in my models, but for some reason it isn't working here. Does anyone know what I am doing wrong? Can someone help me understand why it is reaching for an int() object?
Traceback:
Traceback (most recent call last):
File "<input>", line 1, in <module>
File "C:\Projects\AlmondKing\AlmondKing\InventoryLogs\models.py", line 49, in _total_sales_of_variety
sold = ProductPart.objects.values().filter(part=self.variety_name)
File "C:\Users\Adam\Envs\AlmondKing\lib\site-packages\django\db\models\query.py", line 679, in filter
return self._filter_or_exclude(False, *args, **kwargs)
File "C:\Users\Adam\Envs\AlmondKing\lib\site-packages\django\db\models\query.py", line 697, in _filter_or_exclude
clone.query.add_q(Q(*args, **kwargs))
File "C:\Users\Adam\Envs\AlmondKing\lib\site-packages\django\db\models\sql\query.py", line 1304, in add_q
clause, require_inner = self._add_q(where_part, self.used_aliases)
File "C:\Users\Adam\Envs\AlmondKing\lib\site-packages\django\db\models\sql\query.py", line 1332, in _add_q
allow_joins=allow_joins, split_subq=split_subq,
File "C:\Users\Adam\Envs\AlmondKing\lib\site-packages\django\db\models\sql\query.py", line 1194, in build_filter
lookups, value)
File "C:\Users\Adam\Envs\AlmondKing\lib\site-packages\django\db\models\fields\related.py", line 1740, in get_lookup_constraint
ookup_class(target.get_col(alias, source), val), AND)
File "C:\Users\Adam\Envs\AlmondKing\lib\site-packages\django\db\models\lookups.py", line 96, in __init__
self.rhs = self.get_prep_lookup()
File "C:\Users\Adam\Envs\AlmondKing\lib\site-packages\django\db\models\lookups.py", line 134, in get_prep_lookup
return self.lhs.output_field.get_prep_lookup(self.lookup_name, self.rhs)
File "C:\Users\Adam\Envs\AlmondKing\lib\site-packages\django\db\models\fields\__init__.py", line 727, in get_prep_lookup
return self.get_prep_value(value)
File "C:\Users\Adam\Envs\AlmondKing\lib\site-packages\django\db\models\fields\__init__.py", line 985, in get_prep_value
return int(value)
ValueError: invalid literal for int() with base 10: 'Non Pareil (Inshell)'
Models:
class Variety(models.Model):
product_group = models.ForeignKey(ProductGroup)
variety_name = models.CharField(max_length=140)
husked = models.BooleanField()
finished = models.BooleanField() #defunct
description = models.CharField(max_length=500, blank=True)
class ProductPart(models.Model):
product = models.ForeignKey(Product)
part = models.ForeignKey(Variety)
qty = models.DecimalField(max_digits=28, decimal_places=2)
You are trying to match a foreign key to a string, you need to provide the model
ProductPart.objects.values().filter(part=v)
Note: the values isn't necessary either
ProductPart.objects.filter(part=v)
From the documentation for Values
a QuerySet subclass that returns dictionaries when used as an iterable, rather than model-instance objects.
In other words, you are losing some control over your objects with your current approach
According to Django ORM you should address model objects directly in a filter (it makes code easy readable)
v = Variety.objects.get(id=2)
a = ProductPart.objects.filter(part=v)
That's an error you get when you try to parse a string to an int and is not a valid integer: int('hello')
You are trying to do something weird in this query:
a = ProductPart.objects.values().filter(part=v.variety_name)
Basically, part is a foreign key to Variety (a foreign key in this case is an integer) and you are trying to compare it with a string v.variety_name.
I think the correct way to do it is like this:
a = ProductPart.objects.filter(part=v)
You should use:
a = ProductPart.objects.values().filter(part__variety_name=v.variety_name)
Check this Django Doc
Lookups that span relationships

ManyToManyField error on production server, fine on dev server

My Django app is raising a puzzling error when using a production served version (via Apache, and Nginx for static), that is not evident for the dev server version on localhost.
I have the models :
class AdaptationLibrary(models.Model):
description = models.CharField( u'Description of adaptation',max_length=400,blank=True,null=True)
name = models.CharField(u'Adaptation Name',max_length=60)
applies_to = models.ManyToManyField(Archetype,related_name =u'archetype_adaptations',null=True,blank=True)
adaptations = jsonfield.JSONField(u'adaptation library items', null=True, blank=True)
def __unicode__(self):
return self.name
and ..
class Archetype(models.Model):
archetype = models.CharField(max_length=20, choices=ARCHETYPE_CHOICES,unique=True)
archetype_family = models.CharField(max_length=60,choices=ARCHETYPE_FAMILY_CHOICES,null=True)
replacement_cost_default = models.FloatField("Default complete asset replacement cost - ($)",null=True, blank=True)
lifespan_default = models.FloatField("Default asset life (yrs)", null=True, blank=True)
age_default = models.FloatField("Default age - (yrs)", null=True, blank=True)
storage_time_default = models.FloatField("Default storage/retention time (hrs)", null=True, blank=True)
def __unicode__(self):
return self.archetype
when I attempt to retrieve related Archetype objects via :
library_archetypes = library_item.applies_to.all()
I get the following error :
FieldError: Cannot resolve keyword u'archetype_adaptations' into field.
Choices are: age_default, archetype, archetype_family, asset, cemlo2, id,
lifespan_default, new_dependency, replacement_cost_default, storage_time_default
Dev and local versions are using the same database, and apart from the call to the AdaptationLibrary ManyToManyField, the rest of the app functions fine.
Can anybody shed some light on this issue?
Cheers
Edit:
As per Rohan's suggestion that it's a migration problem - I've gone the whole box and dice of resetting and re-converting to south. Dev server is still happy - Apache served version throws same errors. Both versions are using the same DB.
(full traceback error) :
ERROR Traceback (most recent call last):
File "C:\Python27\Lib\site-packages\dajaxice\core\DajaxiceRequest.py", line 181, in process
response = '%s' % thefunction(self.request, **argv)
File "C:/Python27/sites/Adaptwater\proto4\ajax.py", line 2636, in populate_adaptation_library
initial_data = dict_to_library_form(library_item_id = library_to_load)
File "C:/Python27/sites/Adaptwater\proto4\ajax.py", line 2556, in dict_to_library_form
library_archetypes = library_item.applies_to.all()
File "C:\Python27\Lib\site-packages\django\db\models\manager.py", line 116, in all
return self.get_query_set()
File "C:\Python27\Lib\site-packages\django\db\models\fields\related.py", line 543, in get_query_set
return super(ManyRelatedManager, self).get_query_set().using(db)._next_is_sticky().filter(**self.core_filters)
File "C:\Python27\Lib\site-packages\django\db\models\query.py", line 621, in filter
return self._filter_or_exclude(False, *args, **kwargs)
File "C:\Python27\Lib\site-packages\django\db\models\query.py", line 639, in _filter_or_exclude
clone.query.add_q(Q(*args, **kwargs))
File "C:\Python27\Lib\site-packages\django\db\models\sql\query.py", line 1250, in add_q
can_reuse=used_aliases, force_having=force_having)
File "C:\Python27\Lib\site-packages\django\db\models\sql\query.py", line 1122, in add_filter
process_extras=process_extras)
File "C:\Python27\Lib\site-packages\django\db\models\sql\query.py", line 1316, in setup_joins
"Choices are: %s" % (name, ", ".join(names)))
FieldError: Cannot resolve keyword u'archetype_adaptations' into field. Choices are: age_default, archetype, archetype_family, asset, cemlo2, id, lifespan_default, new_dependency, replacement_cost_default, storage_time_default
Ok - sorry for the self answer, but I fixed the problem, even if I'm still mostly in the dark as to what caused it. After some additional serious googling, I found discussion about the problem in terms of ordering of imports, and model definitions both. For example :
http://chase-seibert.github.com/blog/2010/04/30/django-manytomany-error-cannot-resolve-keyword-xxx-into-a-field.html
After placing the AdaptationLibrary model ahead of Archetype in models.py (and quoting "Archetype" for m2m setup) it appears to be happy. Unless I'm missing something blindingly obvious, this feels like a voodoo fix. Up until now I'd been fastidiously putting referred to models before their referring buddies. But it's a fix nonetheless - so now back to work.
Cheers & Thanks.
Try to use String instead of unicode...sometimes, I have had this problem where Apache is not able to understand UTF-8.

Django model field with default value set violates not null constraint when saved

My field:
signup_date = models.DateTimeField(blank=True,default=datetime.now)
My error when saving:
IntegrityError: null value in column "signup_date" violates not-null constraint
I'm trying to make a simple unit test where I create a bound instance of a ModelForm from a dict and save it.
Thanks.
Traceback (most recent call last):
File "<console>", line 1, in <module>
File "/home/django/signupform/signup/insert_test_data.py", line 27, in <module>
SDF.save()
File "/usr/lib/python2.5/site-packages/Django-1.2.1-py2.5.egg/django/forms/models.py", line 371, in save
fail_message, commit, construct=False)
File "/usr/lib/python2.5/site-packages/Django-1.2.1-py2.5.egg/django/forms/models.py", line 86, in save_instance
instance.save()
File "/usr/lib/python2.5/site-packages/Django-1.2.1-py2.5.egg/django/db/models/base.py", line 435, in save
self.save_base(using=using, force_insert=force_insert, force_update=force_update)
File "/usr/lib/python2.5/site-packages/Django-1.2.1-py2.5.egg/django/db/models/base.py", line 528, in save_base
result = manager._insert(values, return_id=update_pk, using=using)
File "/usr/lib/python2.5/site-packages/Django-1.2.1-py2.5.egg/django/db/models/manager.py", line 195, in _insert
return insert_query(self.model, values, **kwargs)
File "/usr/lib/python2.5/site-packages/Django-1.2.1-py2.5.egg/django/db/models/query.py", line 1479, in insert_query
return query.get_compiler(using=using).execute_sql(return_id)
File "/usr/lib/python2.5/site-packages/Django-1.2.1-py2.5.egg/django/db/models/sql/compiler.py", line 783, in execute_sql
cursor = super(SQLInsertCompiler, self).execute_sql(None)
File "/usr/lib/python2.5/site-packages/Django-1.2.1-py2.5.egg/django/db/models/sql/compiler.py", line 727, in execute_sql
cursor.execute(sql, params)
File "/usr/lib/python2.5/site-packages/Django-1.2.1-py2.5.egg/django/db/backends/util.py", line 15, in execute
return self.cursor.execute(sql, params)
File "/usr/lib/python2.5/site-packages/Django-1.2.1-py2.5.egg/django/db/backends/postgresql_psycopg2/base.py", line 44, in execute
return self.cursor.execute(query, args)
IntegrityError: null value in column "signup_date" violates not-null constraint
from django.db import models
from django.contrib.localflavor.us.models import PhoneNumberField
from datetime import datetime
from models_dropdowns import *
class SignupData(models.Model):
first_name = models.CharField(max_length=128,verbose_name='First Name')
last_name = models.CharField(max_length=128,verbose_name='Last Name')
street1 = models.CharField(max_length=128,verbose_name='Street Address 1')
street2 = models.CharField(max_length=128,verbose_name='Street Address 2')
city = models.CharField(max_length=128)
state = models.CharField(max_length=2)
zip = models.IntegerField(verbose_name='Zip Code')
phone_number = PhoneNumberField(verbose_name='Phone Number XXX-XXX-XXXX')
email = models.EmailField(verbose_name='Email Address')
contact_method = models.ForeignKey('ContactMethodChoice',blank=False,default=-1,verbose_name='Preferred contact method')
birth_date = models.DateField(verbose_name='Birth Date')
policy_number = models.CharField(max_length=128,verbose_name='American Family Auto Insurance Policy Number')
vin = models.CharField(max_length=128,verbose_name='Vehicle Identification Number (VIN)')
vehicle_make = models.ForeignKey('VehicleMakeChoice',verbose_name='VehicleMake')
vehicle_model = models.CharField(max_length=128,verbose_name='Vehicle Model')
vehicle_year = models.ForeignKey('VehicleYearChoice',verbose_name='Vehicle Year')
vehicle_ownership = models.ForeignKey('VehicleOwnershipChoice',blank=False,default=-1,verbose_name='Vehicle Ownership')
vehicle_use = models.ForeignKey('VehicleUseChoice',blank=False,default=-1,verbose_name='Use of Vehicle')
terms_and_conditions = models.BooleanField(verbose_name='I Agree to the terms and conditions. (add link)')
form_user_role = models.ForeignKey('FormUserRoleChoice',blank=False,default=-1,verbose_name='Your Role')
participate_in_feedback = models.BooleanField(verbose_name='<b>Opportunity to provide feedback.</b>...<br><br>',help_text='Please check the box if you would like to participate.')
signup_date = models.DateTimeField(blank=True,default=datetime.now,verbose_name='')
I use the following code to create a bound instance of the form and save it. I run this by importing it from ./manage.py shell.
from signupform.signup.forms import SignupDataForm
keys = ('first_name','last_name','street1','street2','city','state','zip','phone_number','email','contact_method','birth_date','policy_number','vin','vehicle_make','vehicle_model','vehicle_year','vehicle_ownership','vehicle_use','terms_and_conditions','form_user_role','participate_in_feedback')
data = [
('firstname1','lastname1','test1','test1','test1','XX',55555,'555-555-5555','test#asdf.com',1,'01/01/01','####-####-##-##-AAAA-AA','123456789abcdefgh',1,'model',1996,1,1,True,1,True),
('firstname2','lastname2','test2','test2','test2','XX',55555,'555-555-5555','test#asdf.com',1,'01/01/01','####-####-##-##-AAAA-AA','123456789abcdefgh',1,'model',1996,1,1,True,1,True),
('firstname3','lastname3','test3','test3','test3','XX',55555,'555-555-5555','test#asdf.com',1,'01/01/01','####-####-##-##-AAAA-AA','123456789abcdefgh',1,'model',1996,1,1,True,1,True),
('firstname4','lastname4','test4','test4','test4','XX',55555,'555-555-5555','test#asdf.com',1,'01/01/01','####-####-##-##-AAAA-AA','123456789abcdefgh',1,'model',1996,1,1,True,1,True),
]
for d in data:
tmpDict = {}
for i in range(0,len(keys)):
tmpDict[keys[i]] = d[i]
SDF = SignupDataForm(tmpDict)
if not SDF.is_valid():
print SDF.errors
else:
SDF.save()
On problem in you code for me is that you should not write
default=datetime.now()
but instead
default = datetime.now
passing the function as default and not the result of the function at the time the code is parsed. Currently, all your SignupData will have the same signup_date
That said, I am not sure that this is the cause of your bug.
Make sure you're actually using Django to create your objects. Otherwise you're by-passing all the Django magic for setting up default values.
So use something like this:
SignupData.objects.create(blah)
To use your dictionary, use dictionary unpacking:
SignupData.objects.create(**my_dictionary)
BTW, you probably want the default parameter to be a callable, i.e. omit the parens after the now function. This way the function will be called every time a new object is created. With the parens it will only be called once and the same value will be used for all newly created objects.