I recently upgraded to Django 1.4 from 1.1. I have been running WadofStuff Django Serializers 1.0.0. After upgrade, I noticed that dates from my django model get serialized with a 'T' character inserted:
{"pk": 7, "model": "ao.message", "fields": {"content_file": "bar.wav", "date": "2012-07-04T10:58:46", "summary_file": "foo.wav"}}
What's up with that 'T'? Can/should it be removed? Is there a way to specify my desired output date format to the serializer (say, if I didn't want it to return with a 'T')?
Thanks
A single point in time can be represented by concatenating: a complete date expression, the letter T as a delimiter, and a valid time expression. For example "2007-04-05T14:30" (Wikipedia Link )
For further details regarding this T insertion in datetime format as far as python is concerned, you may go to this link: "Python Datetime Representations". The first example specifically illustrates your problem and suggests its solutions too.
Related
Model file includes :
class Foo(models.Model):
bar = models.DateTimeField()
Fixture file.json includes:
{
"model": "app.Foo",
"pk": "1",
"fields": {
"bar": "2018/4/20",
}
},
when I'm trying to add fixtures with "python manage.py loaddata" result is:
django.core.serializers.base.DeserializationError: Problem installing fixture
'C:\Projects\TestProject\app\fixtures\file.json': ['“2018/4/20” value has an
invalid format. It must be in Y
YYY-MM-DD HH:MM[:ss[.uuuuuu]][TZ] format.']: (app.Foo:pk=1) field_value was '2018/4/20'
so my question is how can I add date format with "YYYY/MM/DD" to my Model files ?
your using DateTimeField() and your getting data is only date that's why it's getting the error instead of DateTimeField() use DateField()
for example:class Foo(models.Model):
bar = models.DateField()
The Django models.DateField and models.DateTimeField expect input either as python datetime objects or in ISO format (with "-" separators, not "/" separators). This is not something you can easily change because it is also what is used to save the values in the database. So what you want isn't possible.
The only option I see is that you write your own management command that first parses your json fixture, changing the dates format to ISO and saving to a new file, then calling the loaddata command from within your own handle() method.
Note: The corresponding form fields, forms.DateField, are flexible and accept various input formats and allow you to specify which input format you want to use. In fact, if you enable i18n then they will accept international input formats like 2020/03/12. But that doesn't really help you with loaddata which doesn't use a form at all.
I have a SearchIndex whose results I want to be sorted based on a DatetimeField. However when I try to manage.py rebuild_index, I get a ValueError complaining about the datetime being a... datetime.
In case it matters, I use timezones and pytz, but for the sorting I want, timezones do not really matter, I just want the newest first kind of thing.
The Index
I have removed some irrelevant fields.
class ArticleIndex(indexes.SearchIndex, indexes.Indexable):
text = indexes.CharField(document=True, use_template=True)
publish_date = indexes.DateTimeField(model_attr='publish_date')
The view/url pair
posts_sqs = SearchQuerySet().order_by('-publish_date')
urlpatterns += patterns(
'haystack.views',
url(r'^search/$', search_view_factory(
view_class=PostsSearchView,
template='pages/search.html',
searchqueryset=posts_sqs,
form_class=ThreeCharMinSearchForm), # a custom form
name='pages.search'),
)
The rebuild_index Exception
ValueError: datetime.datetime(2015, 1, 23, 16, 31, 28, tzinfo=<UTC>) is not unicode or sequence
I have tried to implement prepare_publish_date methods that return strftime representations ('%Y %m %d %H %M' and '%d %m %Y %H %M') with both naive and aware datetimes, timetuples, "epoch times" with a CharField instead of a DateTimeField and I can't remember what else and all failed, except for the "epoch time" version, which was terribly slow though.
As a last note, I use Python 2.7.8, Django 1.6.10 and before I tried to do this sorting, the index was working nicely (even better that what was expected), so I am pretty sure the rest of the implementation is correct.
I understand that it is Whoosh that's expecting unicode, but I don't know what to do exactly. Any thoughts?
Thanks for the feedback everyone, because when noone answers your question on SO, the feedback basically is "Dude, what you 're saying doesn't make sense, question your assumptions and check everything.".
So, in my case, there was a customized WhooshSearchBackend which was stripped down and did not account for DateTimeFields. For anyone stumbling upon this, Whoosh and Haystack can handle datetimes just fine. If they don't, check your setup.
After much hardship, I have managed to convert my django project, that previously ran with sqlite, to run with MongoDB.
This is great, beside from the fact that my old version had a massive initial_data.json file, which now fails to work with the new db when running django's syncdb command.
EDIT:
this is an example of the initial_data.json file :
[{"pk":1,
"model": "vcb.dishtype",
"fields": {
"photo": "images/dishes/breakfast.jpg",
"name": "Breakfast"
}
},
{"pk":2,
"model": "vcb.dishtype",
"fields": {
"photo": "images/dishes/bread_and_pastry.jpg",
"name": "Bread and pastry"
}
}]
and after running the syncdb I get:
DeserializationError: Problem installing fixture
'C:\Users..\initial_data.json' : u'pk'
It seems to be a problem with the MongoDB objectId and how I defined the initial_data file.
I tried to remove all the pks fields from the file, but still the same error.
EDIT
I tried putting just two fixtures, if I don't set the pk, I get the same error as above. If I do set it, I get :
"DatabaseErroe: Problem installing fixture 'C:..\initial_data.json':
could not load vcb.dishtype(pk=1): AutoField (default primary key)
values must be strings representing an ObjectId on MongoDB (got u'1'
instead)".
which is a similar problem I had with the django Site, that was solved with the help of this thread: Django MongoDB Engine error when running tellsiteid
This raises my suspicion that there's a different way to set the fixtures in this infrastructure. Maybe syncdb isn't the way to go, and there should be a sort of dump maybe?
I've searched google, and nothing seems to tackle this problem. I'm obviously asking the wrong questions.
what should I do, to create fixtures in my altered project?
thanks a lot,
Nitzan
From your error message, I assume you are using Django MongoDB Engine?
Your pk values must be valid ObjectIds, try using values like:
'000000000000000000000001'
'000000000000000000000002'
etc
You can get ObjectIds or check that you have correct values:
>>> from bson.objectid import ObjectId
>>> ObjectId()
> ObjectId('52af59bac38f8420734d064d')
>>> ObjectId('000000000000000000000001')
> ObjectId('000000000000000000000001')
>>> ObjectId('bad')
*error*
I have a json field in my database which is like
jsonfield = {'username':'chingo','reputation':'5'}
how can i write a query so that i can find if a user name exists. something like
username = 'chingo'
query = User.objects.get(jsonfield['username']=username)
I know the above query is a wrong but I wanted to know if there is a way to access it?
If you are using the django-jsonfield package, then this is simple. Say you have a model like this:
from jsonfield import JSONField
class User(models.Model):
jsonfield = JSONField()
Then to search for records with a specific username, you can just do this:
User.objects.get(jsonfield__contains={'username':username})
Since Django 1.9, you have been able to use PostgreSQL's native JSONField. This makes search JSON very simple. In your example, this query would work:
User.objects.get(jsonfield__username='chingo')
If you have an older version of Django, or you are using the Django JSONField library for compatibility with MySQL or something similar, you can still perform your query.
In the latter situation, jsonfield will be stored as a text field and mapped to a dict when brought into Django. In the database, your data will be stored like this
{"username":"chingo","reputation":"5"}
Therefore, you can simply search the text. Your query in this siutation would be:
User.objects.get(jsonfield__contains='"username":"chingo"')
2019: As #freethebees points out it's now as simple as:
User.objects.get(jsonfield__username='chingo')
But as the doc examples mention you can query deeply, and if the json is an array you can use an integer to index it:
https://docs.djangoproject.com/en/2.2/ref/contrib/postgres/fields/#querying-jsonfield
>>> Dog.objects.create(name='Rufus', data={
... 'breed': 'labrador',
... 'owner': {
... 'name': 'Bob',
... 'other_pets': [{
... 'name': 'Fishy',
... }],
... },
... })
>>> Dog.objects.create(name='Meg', data={'breed': 'collie', 'owner': None})
>>> Dog.objects.filter(data__breed='collie')
<QuerySet [<Dog: Meg>]>
>>> Dog.objects.filter(data__owner__name='Bob')
<QuerySet [<Dog: Rufus>]>
>>> Dog.objects.filter(data__owner__other_pets__0__name='Fishy')
<QuerySet [<Dog: Rufus>]>
Although this is for postgres, I believe it works the same in other DBs like MySQL
Postgres: https://docs.djangoproject.com/en/2.2/ref/contrib/postgres/fields/#querying-jsonfield
MySQL: https://django-mysql.readthedocs.io/en/latest/model_fields/json_field.html#querying-jsonfield
This usage is somewhat anti-pattern. Also, its implementation is not going to have regular performance, and perhaps is error-prone.
Normally don't use jsonfield when you need to look up through fields. Use the way the RDBMS provides or MongoDB(which internally operates on faster BSON), as Daniel pointed out.
Due to the deterministic of JSON format,
you could achieve it by using contains (regex has issue when dealing w/ multiple '\' and even slower), I don't think it's good to use username in this way, so use name instead:
def make_cond(name, value):
from django.utils import simplejson
cond = simplejson.dumps({name:value})[1:-1] # remove '{' and '}'
return ' ' + cond # avoid '\"'
User.objects.get(jsonfield__contains=make_cond(name, value))
It works as long as
the jsonfield using the same dump utility (the simplejson here)
name and value are not too special (I don't know any egde-case so far, maybe someone could point it out)
your jsonfield data is not corrupt (unlikely though)
Actually I'm working on a editable jsonfield and thinking about whether to support such operations. The negative proof is as said above, it feels like some black-magic, well.
If you use PostgreSQL you can use raw sql to solve problem.
username = 'chingo'
SQL_QUERY = "SELECT true FROM you_table WHERE jsonfield::json->>'username' = '%s'"
User.objects.extra(where=[SQL_EXCLUDE % username]).get()
where you_table is name of table in your database.
Any methods when you work with JSON like with plain text - looking like very bad way.
So, also I think that you need a better schema of database.
Here is the way I have found out that will solve your problem:
search_filter = '"username":{0}'.format(username)
query = User.objects.get(jsonfield__contains=search_filter)
Hope this helps.
You can't do that. Use normal database fields for structured data, not JSON blobs.
If you need to search on JSON data, consider using a noSQL database like MongoDB.
When using localized list of "choices" for a model field, the admin doesn't show the translated values in the list view.
Short example:
from django.utils.translation import ugettext_lazy as _
class OrderStates:
STATES = (
(STATE_NEW, _("New")),
(STATE_CANCELLED, _("Cancelled")), )
class Order(models.Model):
state = models.IntegerField(choices=OrderStates.STATES)
# ..
class OrderAdmin(admin.ModelAdmin):
list_display = [ 'id', 'state', 'address', 'user']
# ..
admin.site.register(Order, OrderAdmin)
The localized versions of "New" and "Cancelled" show up correctly in the front-end and in the admin form when editing an order. But in the admin list view I get blank fields - regardless of the language I switch to, including English. Column names are fine.
This only happens with Python 2.3 (talk about niche questions). The choices display correctly everywhere with Python 2.5. I don't get any errors or warnings in neither.
Tried using ugettext instead of ugettext_lazy for the options, which didn't work. ugettext_noop sort of works - it at least shows the original english versions instead of blank fields.
Am I doing something wrong or is this a bug?
This is probably a bug somewhere in Django, not calling force_unicode on the item correctly. The original code you pasted is correct. You don't mention what Django version you're using, so I'd reccomend trying the latest 1.0.3 or 1.1 release to see if that happens to fix it, else check the ticket tracker to see if it's already been reported (note that if it hasn't been fixed yet it probably won't be at all, since 1.1 is the last version to support 2.3).
try using:
import gettext as _
Though, that may break if some of your translations use non-ascii values. Actually, this should have been fixed some time ago, see Ticket #5287.
Hope this helps.