I transfered my project to another computer and get an error while running a view.
I'm getting some informations of a model and want to save them to XML by using XMLGenerator.
On the one computer it works fine, type() of the model.charField() returns "unicode"
On the new computer it did not work, type() of the model.charField() returns "str"
The working computer has Python 2.7.2
The not working computer has Python 2.5.2
So on the not working computer I did not get unicode which can be handled by XMLGenerator. I tried to work around the problem by running .decode("utf-8") on the string which is served by the model and it worked.
But how can I know what encoding the string is? I guessed now that it has the same encoding as in the database but am I right?
regards Martin
could you please check the mysql collation settings? if those are also the same?
from django doc:
"In many cases, this default will not be a problem. However, if you really want case-sensitive comparisons on a particular column or table, you would change the column or table to use the utf8_bin collation. The main thing to be aware of in this case is that if you are using MySQLdb 1.2.2, the database backend in Django will then return bytestrings (instead of unicode strings) for any character fields it receive from the database."
see django doc collation settings
Let's say we have this:
a = unicode('a')
b = str('b')
A fast check is to do:
print type(a)
print type(b)
If you want to validate them you can do:
if isinstance(a, str):
if isinstance(a, unicode):
A way would to to typecast the content:
c = str(a)
d = unicode(b)
Related
I'm running the latest version of Django on postgres. I'm trying to store emojis in my postgres DB in a way that a React Native app can properly render it. Below I have the initial emojis variables setup that'll go into the table. I've copy and pasted the emojis from here. How do I store emojis in my postgres DB so that a React Native app can render it properly?
I tried following this blog, which suggests adding ’OPTIONS’: {’charset’: ’utf8mb4’} to DATABASES under settings.py, but I get this error django.db.utils.ProgrammingError: invalid dsn: invalid connection option "charset". Seems like this only works for MySQL DBs. How can I store emojis in a Django postgres DB?
Like in the comments suggested, you need to put quotes around the emojis since they're just chars. Though, something like flags is actually two chars. So that's something to be careful about. All your computer is doing is converting unicode to a rendered emoji that's platform dependent.
The emojis that you're using should be unicode supported. On your computer, they're definitely supported. For the most part, additional unicode support for new emojis is very quickly implemented once published on client machines. There should be no problem with emojis in strings. This is a nice video kinda explaining emojis by Tom Scott who keeps getting interviews about emojis: https://www.youtube.com/watch?v=sTzp76JXsoY
I'm not an expert so please correct me if I'm wrong.
In your models you need to use a CharField or a TextField to store emojis, that need to be passed as characters (for example "😄" and not directly 😄). Your database must use utf8 to support emojis, connect to your database with a SQL shell, to check the current encoding run:
SHOW CLIENT_ENCODING;
If the output is not UTF8 run:
SET CLIENT_ENCODING='UTF8';
Now remove ’OPTIONS’: {’charset’: ’utf8mb4’} from your Django settings.
My database contains a dictionary. When I read the dictionary from the database and try to do something with it it fails because the dictionary has been automatically converted into a string. Any way to avoid Django turning the dict into a string?
you can also use simplejson.loads() and simplejson.dumps() to deserialize and serialize the dictionary. It is a bit more work, but it ensures that you are not dependent on database.
There are options for MySQL and Postgres but I don't think there's an equivalent for sqlite.
For MySQL JSONField: https://django-mysql.readthedocs.io/en/latest/model_fields/json_field.html
Similarly for Postgres:
https://docs.djangoproject.com/en/3.0/ref/contrib/postgres/fields/#jsonfield
There's built in support to query the contents of the fields which is pretty neat. The docs show examples.
Solved the issue with help from the responses I got here.
When capturing and saving the JSON from the webhook (as that's where the JSON is coming from in my project), I had to do the strange step of serialising and deserialising the JSON before saving it to my database. This process got rid of all the \r and \t charactors which are passed by request.body but make the JSON invalid:
t = Transaction(data=json.dumps(json.loads(request.body)))
t.save()
To load the JSON from database into a python dictionary that I can then use in my code I used json.loads:
data = json.loads(t.data)
I'm completely baffled by a problem I found today: I have a PostgreSQL database with tables which are not managed by Django, and completely normal queries via QuerySet on these tables. However, I've started getting Unicode exceptions and when I went digging, I found that my QuerySets are returning non-Unicode strings!
Example code:
d = Document.objects.get(id=45787)
print repr(d.title), type(d.title)
The output of the above statement is a normal string (without the u prefix), followed by a <str> type identifier. What's more, this normal string contains UTF-8 data as expected, in raw byte form! If I call d.title.decode('utf-8'), I get valid Unicode strings!
Even more puzzling, some of the fields work correctly. This same table / model contains another field, html_filename of the same type (TextField) which is returned correctly, as a Unicode string!
I have no special options, the database data is correctly encoded, and I don't even know where to begin searching for a solution. This is Django 1.6.2.
Update:
Database Server encoding is UTF8, as usual, and the data is correctly encoded. This is on PostgreSQL 9.1 on Ubuntu.
Update 2:
I think I may have found the cause, but I don't know why it behaves this way: I thought the database fields were defined with the text type, as usual, but instead they are defined as citext (http://www.postgresql.org/docs/9.1/static/citext.html). Since the Django model is unmanaged, it looks like Django doesn't interpret the field type as being worthy of converting to Unicode. Any ideas how to force Django to do this?
Apparently, Django will not treat fields of type citext as textual and return them as Unicode strings.
I am following the Django tutorial for version 1.6 using PostgreSQL on Arch Linux. I'm on part 1 up to this point:
"Give the Poll a couple of Choices. The create call constructs a new
Choice object, does the INSERT statement, adds the choice to the set
of available choices and returns the new Choice object. Django creates
a set to hold the "other side" of a ForeignKey relation
(e.g. a poll's choices) which can be accessed via the API.
>>> p = Poll.objects.get(pk=1)
# Display any choices from the related object set -- none so far.
>>> p.choice_set.all()
[]
When I run p.choice_set.all(), instead of an empty list, I get an error in my shell:
ProgrammingError: column "poll_id" of relation "polls_choice" does not exist
This is first that I've seen "choice_set" in the tutorial, and I'm not sure what it does. I tried the whole exercise several times and everything worked right until this point -- I can't figure out what I'm doing wrong.
Drop Poll and Choice tables and run python manage.py syncdb to recreate them according to your current models.py
I am not sure why I am getting this error:
Exception Type: UnicodeEncodeError
Unicode error hint
The string that could not be encoded/decoded was: he Théâtre d
The full traceback is here: http://dpaste.com/686751/ (I put it in a dpaste due to it's length)
I am really confused about this because it works flawlessly on our staging and has been for a year or so now, it's finally on the live server, I copied over the database to the live server and now if I edit anything or add a new page with any sort of french accents I received the above error. I've been googling for hours with not much luck.
In my research I have found some issues with DB collation but I have tried to recreate the database as utf8_general_ci, and converted the tables respectively and still no luck. Any idea?
I should also note that the apps listed in the installed apps are one's we've developed and use for about 13 other live and large web sites on the same server and with the same types of characters.
baffled
Jeff
in model add u''
def __unicode__(self):
return u"%s" % self.your_field
maybe the servers have different library versions?
afaik, the way to fix those errors is using the smart_unicode function in the unicode method in models, as mentioned here:
django unicode encode/decode errors