My site needs to be able to serve data in different languages. I set it so it uses utf-8 and the db settings are set to that as well. I've been getting different different unicode errors over the admin.
For example:
In the admin list, when a field from the list contains a non ascii char. (i get UnicodeDecodeError)
When adding a new entry, a UnicodeEncodeError if the unicode method for the model returns an utf-8 decode (which fixes #1).
When using a filter_horizontal in the admin, if data from the used model contains non ascii chars, then the filter disappears from the form.
If I set the unicode method for the model to return for example:
return u'%s' % unicode(self.tag)
That seems to fix #1 and #2, but then that's when I get #3.
I have been looking a lot for a solution, but can't find something that fixes all different errors. What's the best way to deal with those?
from django.utils.encoding import smart_unicode
...
def __unicode__(self):
return smart_unicode(self.tag)
It is noteworthy that you can bypass unicode by simply encoding your data in hexadecimal before storing it in your database.
Something like this is sufficient
MyModel(name=name.encode('hex'), password=password).save()
You can then execute name.decode('hex') to return the data back to its former representation.
Related
I have a Flask-SQLAlchemy site which uses the SQLAlchemy enum type for certain columns, to restrict possible input values.
For example, I have a "payment type" enum that has a few payment type options, like so:
class PaymentType(enum.Enum):
fixed = "fixed"
variable = "fixed_delivery"
quantity_discount = "quantity_discount"
When I use this in a dropdown/select I can specify the options like so:
prd_payment_type = SelectField(_('prd_payment_type'), choices=SelectOptions.PaymentTypes.All(0, _('PleaseSelect')))
The All() function I'm calling returns the different enum values as options, and also adds a custom "Please select..." option to the dropdown list. It looks like this:
class PaymentTypes(object):
#staticmethod
def All(blank_value=None, blank_text=None):
ret = [(i.value, _(i.name)) for i in PaymentType]
SelectOption.add_blank_item(ret, blank_value, blank_text)
return ret
So far, so good, I get a nice dropdown, with the correct options. the problem arises when I want to display the form using an already existing SQLAlchemy object from the database.
When I fetch the object from SQLAlchemy, the prd_payment_type property contains the enum PaymentType.quantity_discount, not just a string value 'quantity_discount':
Because WTForms (presumably) matches the pre-selected option in the dropdown to a string value "quantity_discount", it doesn't match the enum in the SQLAlchemy model and the option is not selected in the dropdown.
The SelectField of WTForms accepts a list of tuples containing strings as choices. I can't feed it my enum.
On the other hand, the SQLAlchemy model returns an enum type as the property for prd_payment_type. I could maybe override this property or something, but that feels like a lot of work to support SQLAlchemy's enums in WTForms.
Is there a standard solution for working with SQLAlchemy's enums in a WTForms SelectField or should I abandon using enums altogether and just store strings, maybe from a list of constants to keep them in check?
I have found a solution myself. It seems that when the enum has a
def __str__(self):
return str(self.value)
That's enough for WTForms to match the database value to the SelectField value.
I have made a baseclass that derives from enum.Enum and added the code above to return the enum value as a string representation.
This solution was based on these similar problems I found later on:
https://github.com/flask-admin/flask-admin/issues/1315
Python Flask WTForm SelectField with Enum values 'Not a valid choice' upon validation
I'm reading an e-mail content through IMAP in my Django app.
When I try to assign some of the parsed content to the object and do .save() it returns:
ProgrammingError: You must not use 8-bit bytestrings unless you use a text_factory that can interpret 8-bit bytestrings (like text_factory = str). It is highly recommended that you instead just switch your application to Unicode strings.
When I print the variable type: . Field in the DB is defined as CharField. I tried TextField as well, but the result is the same.
How I can solve that?
if your mail text is in mail_text, do this:
mail_text = unicode(mail_text)
I have weird problem with django translations that i need help figuring out. Problem is that instead of getting translated string every time i seem to get randomly either translated string or default string.
I have created a class for putting "action" buttons on several pages. Many of those buttons are reusable so why should i put
blabla
into several templates when i can create simple toolbar and use
toolbar.add(tool)
in view and then just use templatetag for rendering all the tools.... anyway.
This is example of one tool:
class NewUserGroupButton(MyTool):
tool_id = 'new-user-group'
button_label = ugettext(u'Create new user group')
tool_href = 'new/'
js = ()
def render(self):
s = '<a id="%(tool_id)s" href="%(tool_href)s" class="button blue">%(button_label)s</a>'
return s % {'tool_id': self.tool_id, 'tool_href': self.tool_href, 'button_label':self.button_label}
The tools are rendered like this:
class ToolbarTools:
def __init__(self):
self.tools = []
def add(self, tool):
self.tools.append(tool)
def render(self):
# Organize tools by weight
self.tools.sort(key=lambda x: x.weight)
# Render tools
output = '<ul id="toolbar" class="clearfix">'
for tool in self.tools:
output += '<li id="%s">' % ('tool-'+ tool.tool_id)
output += tool.render() if tool.renderable else ''
output += '</li>'
output += '</ul>'
# Reset tools container
self.tools = []
return mark_safe(output)
im using ugettext to translate the string. using (u)gettext=lambda s:s or ugettext_lazy gives me either no translations or proxy object corresponding to function choices.
And like i said - while rest of the page is in correct language the toolbar buttons seem to be randomly either translated or not. The faulty behaviour remains consistent if i move between different pages with different "tools" or even refresh the same page several times.
Could someone please help me to figure out what is causing this problem and how to fix it?
Problem solved.
The issue itself (random translating) was perhaps caused by using ugettext. But at the same time i should have used ugettext_lazy instead.
And thus the problem really came from ugettext_lazy returning proxy object not translated string... and that is caused by this:
[Quote]
Working with lazy translation objects
The result of a ugettext_lazy() call can be used wherever you would use a unicode string (an object with type unicode) in Python. If you try to use it where a bytestring (a str object) is expected, things will not work as expected, since a ugettext_lazy() object doesn't know how to convert itself to a bytestring. You can't use a unicode string inside a bytestring, either, so this is consistent with normal Python behavior. For example:
This is fine: putting a unicode proxy into a unicode string.
u"Hello %s" % ugettext_lazy("people")
This will not work, since you cannot insert a unicode object
into a bytestring (nor can you insert our unicode proxy there)
"Hello %s" % ugettext_lazy("people")
[/Quote]
taken from here:
https://docs.djangoproject.com/en/dev/topics/i18n/translation/#working-with-lazy-translation-objects
Alan
I had the same issue just now. The problem was caused by incorrect Middleware order - I set LocaleMiddleware after CommonMiddleware. After I placed it between SessionMiddleware and CommonMiddleware, it seems to work correctly.
See more here: https://lokalise.com/blog/advanced-django-internationalization/
I know, the problem was solved a long ago, but it can be helpful for someone.
I have search results showing with facet options for drilling down the data. When a facet is selected it then changes the facet results. So if I am originally showing "places (10)", "images (5)", "people (3)", and "All (18)" as faceting options and I click on images I would still like to see all the others too even though the search results changed. Is there a good way to do this?
I just did this and it's actually quite achievable without rerunning the original search query. you just need to use session to store the original facets.
Here's my actual working code:
from haystack.views import FacetedSearchView
class StickyFacetedSearchView (FacetedSearchView):
def top_level_facets(self):
"""
When selecting a facet to drill down the results,
we need to keep the top level facet counts
"""
stored_query = self.request.session.get('query', None)
if stored_query != self.query:
self.request.session['query'] = self.query
self.request.session['facet_counts'] = self.results.facet_counts()
return self.request.session['facet_counts'] # Fail loudly
def extra_context(self):
""" add base_facets to extra_context"""
extra = super(StickyFacetedSearchView, self).extra_context()
extra['base_facets'] = self.top_level_facets()
return extra
Stick the above view in an app called 'mysearch' or similar, then use mysearch.StickyFacetedSearchView in urls.py instead of FacetedSearchView.
To access them use base_facets in the template, rather than facets.
It works perfectly.
I've run into problems with this too. The only answer we could find was to re-run the query each time without the drill-down to get the top-level facet results.
I have a field that has a max_length set. When I save a model instance, and the field's value is greater than max_length, Django enforces that max_length at the database level. (See Django docs on models: http://docs.djangoproject.com/en/dev/ref/models/fields/#django.db.models.CharField.max_length)
However, since I am using Postgres, I receive a DatabaseError exception like this:
DatabaseError: value too long for type character varying(1000)
I would prefer to instead auto-truncate the value (so I don't have an exception). Now, I can do this manually, but what I would really want is to have all of my models auto-truncate the value. (Not necessarily intelligently. Just cutting it off at the 999th character is fine.)
Should I just write a custom class that imports from models.Model and override the save() method, looping through each _meta.field, checking for the max_length, and then truncating? That seems inelegant and there must be a better way.
You could create a custom field that auto-truncates the field (I think this code should work, but double-check it):
class TruncatingCharField(models.CharField):
def get_prep_value(self, value):
value = super(TruncatingCharField,self).get_prep_value(value)
if value:
return value[:self.max_length]
return value
Then, instead of using models.CharField in your models.py file, you'd just use TruncatingCharField instead.
get_prep_value prepares the value for a field for insertion in the database, so it's the ideal place to truncate.
Why don't you use a TextField? From the manual:
For large amounts of text, use
TextField.
Why don't you use ModelForm. ModelForm enforces a validation, setting its default max_length to model field's max_length property, and raising proper validation error when form.is_valid() is called. That way you don't have to save the form, until form is validated.
Or, if you want to silently pass the validation and truncate suits best to you, write a simple django form, and write a clean method that truncates input string to the max_length and return stripped data. Take data from form.cleaned_data after form is validated and save the object.
All considering the fact, Forms are designed to validate data before going to DB.
That seems inelegant and there must be a better way.
The only reason the truncate behavior ever happens in the first place is because of MySQL's failure to adhere to the SQL Standard. The throwing of an exception is the correct response when attempting to INSERT a string into a VARCHAR field that is not wide enough to hold it. MySQL truncates and inserts instead.
If you want to silently corrupt your data, then you'll have to do it manually somehow--PostgreSQL will never do it for you.