Translating %% with gettext and jinja2 and pyramid - python-2.7

Doing i18n work with Python using Jinja2 and Pyramid. Seems to have a problem knowing how it should translate %%. I'm beginning to suspect the bug is in Jinja2.
So I've done some more investigation and it appears the problem is more with gettext than with jinja2 as illustrated with the repl
>>>gettext.gettext("98%% off %s sale") % ('holiday')
'98% off holiday sale'
>>>gettext.gettext("98%% off sale")
'98%% off sale'
>>>gettext.gettext("98% off %s sale") % ('holiday')
Traceback (most recent call last):
Python Shell, prompt 13, line 1
TypeError: %o format: a number is required, not str
It seems to be a chicken/egg problem.
If gettext translates %% -> % then the formatter blows up on it during parameter substitution.
If gettext doesn't translate %% -> % then when the formatter is not called (no params to insert) then the %% leaks out.
All this means the translators (most of whom are not computer programmers) have to be very careful in how they do the translation and everyone needs to be very careful with translations that include %.
Seems like we are doing this wrong (somehow) and there should be a more straightforward and uniform format for doing this. Right now we are coping by simply injecting a % as a format parameter.
Is there a better way to do this, or is this as good as it gets?
There is a .po file at the bottom
Unit test pretty much says it all, why is the last assertion failing? Is this a bug with Jinja2, or do I need to be dealing with this differently.
class Jinja2Tests(TestCase):
def test_percent_percent(self):
""" i18n(gettext) expresses 98% as 98%% only in some versions of jinja2 that has not
worked as expected. This is to make sure that it is working. """
env = Environment(extensions=['jinja2.ext.i18n'])
lang = gettext.translation('messages', path.abspath(path.join(path.dirname(__file__), 'data')))
env.install_gettext_translations(lang)
template = env.from_string(source="{{ _('98%% off %(name)s sale') | format(name='holiday') }}")
result = template.render()
self.assertEqual('98% off holiday sale(translated)', result)
template = env.from_string(source="{{ _('98%% off sale') }}")
result = template.render()
# THE LINE BELOW FAILS WITH:
# AssertionError: '98% off sale(translated)' != u'98%% off sale(translated)'
self.assertEqual('98% off sale(translated)', result)
And the MO file you have to compile to a PO file to run the above code.
# This file is distributed under the same license as the Uniregistrar project.
# FIRST AUTHOR <EMAIL#ADDRESS>, 2016.
#
msgid ""
msgstr ""
"Project-Id-Version: Uniregistrar 1.0\n"
"Report-Msgid-Bugs-To: mark#uniregistry.com\n"
"POT-Creation-Date: 2016-12-22 15:22-0500\n"
"PO-Revision-Date: 2016-11-14 16:42-0500\n"
"Last-Translator: FULL NAME <EMAIL#ADDRESS>\n"
"Language: en\n"
"Language-Team: en <LL#li.org>\n"
"Plural-Forms: nplurals=2; plural=(n != 1)\n"
"MIME-Version: 1.0\n"
"Content-Type: text/plain; charset=utf-8\n"
"Content-Transfer-Encoding: 8bit\n"
"Generated-By: Babel 2.3.3\n"
#: uniregistrar/constants.py:90
msgid "98%% off sale"
msgstr "98%% off sale(translated)"
#: uniregistrar/constants.py:90
msgid "98%% off %(name)s sale"
msgstr "98%% off %(name)s sale(translated)"

Related

How to translate .po file

I have a flask app and i wanted to add multiple languages to it. So i have using this demo on Flask_babel to do this.
Flask Label Demo
config.py
DEBUG = True
LANGUAGES = ['en', 'de', 'fr']
This is the app.py
from flask import Flask, request, g
from flask_babel import Babel
from config import Config
# set up application
app = Flask(__name__)
app.config.from_object(Config)
# set up babel
babel = Babel(app)
#babel.localeselector
def get_locale():
if not g.get('lang_code', None):
g.lang_code = request.accept_languages.best_match(app.config['LANGUAGES'])
return g.lang_code
babel.cfg
[python: app/**.py]
[jinja2: **/templates/**.html]
extensions=jinja2.ext.autoescape, jinja2.ext.with_
To start the process i first used this command
pybabel extract -F babel.cfg -o messages.pot .
and then this
pybabel init -i messages.pot -d app/translations -l de
It generates messages.po file
# German translations for PROJECT.
# Copyright (C) 2021 ORGANIZATION
# This file is distributed under the same license as the PROJECT project.
# FIRST AUTHOR <EMAIL#ADDRESS>, 2021.
#
msgid ""
msgstr ""
"Project-Id-Version: PROJECT VERSION\n"
"Report-Msgid-Bugs-To: EMAIL#ADDRESS\n"
"POT-Creation-Date: 2021-05-28 19:07+0530\n"
"PO-Revision-Date: 2021-05-28 19:08+0530\n"
"Last-Translator: FULL NAME <EMAIL#ADDRESS>\n"
"Language: de\n"
"Language-Team: de <LL#li.org>\n"
"Plural-Forms: nplurals=2; plural=(n != 1)\n"
"MIME-Version: 1.0\n"
"Content-Type: text/plain; charset=utf-8\n"
"Content-Transfer-Encoding: 8bit\n"
"Generated-By: Babel 2.9.1\n"
#: app/blueprints/multilingual/routes.py:38
msgid "Beautiful day in Portland!"
msgstr ""
#: app/blueprints/multilingual/routes.py:42
msgid "The Avengers movie was so cool!"
msgstr ""
#: app/blueprints/multilingual/routes.py:45 app/templates/base.html:15
msgid "Home"
msgstr ""
#: app/blueprints/multilingual/routes.py:51
msgid "The Cake is a Lie"
msgstr ""
#: app/blueprints/multilingual/templates/multilingual/cake.html:4
msgid "Hi, User!"
msgstr ""
#: app/blueprints/multilingual/templates/multilingual/cake.html:6
msgid "I promise you that there will be "
msgstr ""
#: app/blueprints/multilingual/templates/multilingual/cake.html:6
msgid "cake"
msgstr ""
#: app/blueprints/multilingual/templates/multilingual/cake.html:6
msgid " at the end of this article, so keep on reading!"
msgstr ""
#: app/blueprints/multilingual/templates/multilingual/index.html:4
msgid "Hi, "
msgstr ""
#: app/blueprints/multilingual/templates/multilingual/index.html:6
msgid " says: "
msgstr ""
#: app/templates/base.html:6
msgid "Welcome to Microblog"
msgstr ""
#: app/templates/base.html:16
msgid "Cake"
msgstr ""
How do i translate each and every msgid. The demo says to do this manually, but this is a demo and it only contains 10 lines but my project would contain hundreds of translations in 10 languages, so what do i use for that?
How do i convert the untranslated .po file to translated .po file by a library or program?
You can set up your own automated translation via deepl.com or any other translation api or you need to do it all by yourself with a help of an po editor.
Someone made this tool online https://www.ajexperience.com/po-translator/
It allows you to paste .po-files contents and choose the source language and target language. Tested, working.

French translation raises "ValueError('invalid token in plural form: %s' % value)"

I want to handle a french version of my website.
I use Django 2.2 with i18n and I already set locale variables in settings.py.
# Internationalization
# https://docs.djangoproject.com/en/2.2/topics/i18n/
LANGUAGES = (
('en', _('English')),
('fr', _('French')),
('it', _('Italian')),
('es', _('Spanish')),
)
LOCALE_PATHS = (
os.path.join(BASE_DIR, 'locale'),
)
TIME_ZONE = 'Europe/Paris'
USE_I18N = True
USE_L10N = True
USE_TZ = True
When I use ./manage.py makemessages -l fr, I correctly have a django.po french file but after ./manage.py compilemessages -l fr the server crashes with the following error (trimed) :
File "/usr/lib/python3.7/gettext.py", line 93, in _tokenize
raise ValueError('invalid token in plural form: %s' % value)
ValueError: invalid token in plural form: EXPRESSION
English, Italian and Spanish translations work well
EDIT : Well, the issue has been resolved, but I'm not really sure how. I deleted my venv, recreated it and french translation suddenly worked. Upgrading from Django 2.2.1 to 2.2.2 may be what caused the resolution.
For other language facing this error:
There exists a line that tells Django evaluating this expression, decide which form of the word it should use, and for some languages, this expression is not written, e.g. Farsi.
For these languages, a default line is written in your main .po file(not the specific ones):
"Plural-Forms: nplurals=INTEGER; plural=EXPRESSION;\n"
Here the EXPRESSION part should be changed to your language.
HERE you can read the exact format of EXPRESSION, but for short if your language has just two forms for singular and plural form change the line to this:
"Plural-Forms: nplurals=2; plural=(n != 1);\n"
And recompile your messages.
I had the same issue.
The reason was that i accidentally wrote a word in messages.po file in configuration lines (that are on top of .po file, containing "Project-Id-Version", "Plural-Forms", ...)
So i checked what changes i made with my VSC (git) and it was fixed.
Don't forget to recompile your .mo files

Django translations not showing from app's po

I've been developing an application which works in English or French (Canadian). Django settings are;
LANGUAGE_CODE = 'en'
LANGUAGES = [
('en', gettext('English')),
('fr-ca', gettext('Canadian French')),
]
LOCALE_PATHS = (
...
os.path.join(PROJECT_ROOT, 'console', 'locale'),
...
)
The app's locale path is console/locale/fr-CA/LC_MESSAGES/
The app however has recently stopped rendering the vast majority of the translations, while django debug toolbar, and other apps are displaying French without issue.
For example, I've got a form with 'First name', 'Last name', 'Email'. Yesterday this was correctly using the po file;
#: console/forms/participants.py:631
msgid "First Name"
msgstr "Prénom"
#: console/forms/participants.py:635
msgid "Last Name"
msgstr "Nom"
#: console/forms/participants.py:140 console/forms/participants.py:469
#: console/forms/participants.py:639 console/models/mixins.py:70
msgid "Email"
msgstr "Courriel"
But today, only the Email string is appearing in French. I assume ugettext is getting that from another application because I've tested it in the shell;
>>> from django.utils.translation import ugettext, activate
>>> activate('fr-ca')
>>> ugettext('Sunday')
u'dimanche'
>>> ugettext('Event')
u'Event'
>>> ugettext('Yes')
u'Oui'
>>> ugettext('Gender')
u'Gender'
>>> ugettext('enquiry')
u'enquiry'
>>> ugettext('Enquiry')
u'Enquiry'
>>> ugettext('Receive notifications about other events.')
u'Receive notifications about other events.'
These are all taken from the app's po file;
#: console/models/events.py:35 console/models/events.py:206
#: console/models/participants.py:81 console/models/vouchers.py:14
msgid "Event"
msgstr "Événement"
#: console/models/participants.py:113
msgid "Gender"
msgstr "Sexe"
#: console/models/participants.py:160
msgid "Receive notifications about other events."
msgstr "Recevez des notifications pour un événement."
It goes without saying I've ran the translation management commands (and can see the locale paths being output);
manage.py makemessages -l fr-CA
manage.py compilemessages -l fr-CA
You should remember that ugettext_lazy is a lazy evaluation and should be used in models and forms (as they load only once in Django) for views you should use gettext
Try delete (backup) translation files and recreate them again
Check templates for existence of translation blocks

Django localization: how to create language file with msgstr = msgid

When I create language file with django-admin makemessages -l I need them to be their msgstr equals not empty string but its msgid by default. Is it possible?

django i18n not working

A python newbie here.
I wanna my website support English and Chinese. So I just follow django book, chapter 19 internationalization. But it seems doesn't work for me, string I hope to be displayed as chinese, still english there. My code and settin is as following.
[settings.py]
LANGUAGE_CODE = 'zh-cn'
USE_I18N = True
USE_L10N = True
LANGUAGES = (
('en', 'English'),
('zh-cn', 'Chinese')
)
TEMPLATE_CONTEXT_PROCESSORS = {
'django.core.context_processors.i18n',
}
MIDDLEWARE_CLASSES = (
'django.middleware.locale.LocaleMiddleware',
'django.middleware.common.CommonMiddleware',
'django.contrib.sessions.middleware.SessionMiddleware',
'django.middleware.csrf.CsrfViewMiddleware',
'django.contrib.auth.middleware.AuthenticationMiddleware',
'django.contrib.messages.middleware.MessageMiddleware',
)
In my app views.py, I forcely set language code as 'zh-cn' in index
def index( request ):
response= render_to_response( 'index.htm' )
response.set_cookie('django_language','zh-cn')
return response
then I'd hope annother page that will be loaded after index.htm, will display a chinese string.
Annother page is renderred by upload.html
{% load i18n %}
<html>
<head>
{% block head %}
{% endblock %}
</head>
<body>
{% block body %}
<h1>{% trans 'Upload Demo' %}</h1>
{% endblock %}
</body>
</html>
After then, I do
django-admin.py makemessages -l zh-cn -e htm
in my django project folder, and I got django.po at
locale/zh-cn/LC_MESSAGES/django.po
which content is like
#, fuzzy
msgid ""
msgstr ""
"Project-Id-Version: PACKAGE VERSION\n"
"Report-Msgid-Bugs-To: \n"
"POT-Creation-Date: 2012-05-10 18:33+0800\n"
"PO-Revision-Date: YEAR-MO-DA HO:MI+ZONE\n"
"Last-Translator: FULL NAME <EMAIL#ADDRESS>\n"
"Language-Team: LANGUAGE <LL#li.org>\n"
"Language: \n"
"MIME-Version: 1.0\n"
"Content-Type: text/plain; charset=UTF-8\n"
"Content-Transfer-Encoding: 8bit\n"
#: uploader/template/base.htm:10
msgid "Upload Demo"
msgstr "上传文件"
Thereafter, I call following command to compile message
django-admin.py compilemessages
I got django.mo file at some folder with django.po
Fistly I access the index page, then I access another page, which has 'Upload Demo' string id. Actually I still see english string there.
And tried debug via printing language code, find that language has been set correctly.
context = RequestContext(request)
print context
translation.activate('zh-cn')
Lastly, I use
gettext locale/zh-cn/LC_MESSAGES/django.mo "Upload Demo"
really got 'Upload Demo'. So I think problem is here.
But why this happen? I really confused. Can any body help me.
Deeply appreciated any comment or help.
gettext locale/zh-cn/LC_MESSAGES/django.mo "Upload Demo"
I think I made a mistake. Above command return a string that is same as string you typed as string ID rather than translated string. In above command, it is "Upload Demo", That is if your change "Upload Demo" in above command as "bla bla", you will "bla bla".
Maybe it's too late, but I bet your problem probably was due to missing LOCALE_PATHS tuple in your project's settings.
After an hour of pulling my hair out, I solved the same problem by simply adding this to the settings.py file in my project:
LOCALE_PATHS = (
'/home/myuser/Projects/Some_project_root/My_django_project_root/locale',
)
And this is how that directory looks:
locale/
├── de
│   └── LC_MESSAGES
│   ├── django.mo
│   └── django.po
├── en
│   └── LC_MESSAGES
│   ├── django.mo
│   └── django.po
└── es
└── LC_MESSAGES
├── django.mo
└── django.po
Your codeblocks are a bit messy so it is quite hard to read it all. But you might want to start with your .mo file. It contains a #, fuzzy annotation. Fuzzy means that the build script was not sure about the translation and therefore requires attention of the translator (=you). Start by checking all #, fuzzy marked translations. If the translation is correct (or after you have corrected the wrong translation) remove the #, fuzzy annotation. Next run compile messages again. This could fix your problem.
#, fuzzy
msgid ""
msgstr ""
"Project-Id-Version: PACKAGE VERSION\n"
"Report-Msgid-Bugs-To: \n"
"POT-Creation-Date: 2012-05-10 18:33+0800\n"
"PO-Revision-Date: YEAR-MO-DA HO:MI+ZONE\n"
"Last-Translator: FULL NAME \n"
"Language-Team: LANGUAGE \n"
"Language: \n"
"MIME-Version: 1.0\n"
"Content-Type: text/plain; charset=UTF-8\n"
"Content-Transfer-Encoding: 8bit\n"
See also: django fuzzy string translation not showing up
Friendly regards,
Wout
You can see LANGUAGE_CODE is not the same as django_language. I think get_language in utils.py does not handle underscore LANGUAGES correctly. get_language will return zh-cn but it should not be cut into cn. Instead, zh-cn should be converted to zh_CN.
So you should use below code in setting.py
LANGUAGES = (
('en', _('English')),
('zh-cn', _('Simplified Chinese')),
)
and run the following command from terminal
django-admin.py makemessages -l zh_CN
django-admin.py compilemessages
This is working perfectly for me
I believe the issue lies within the makemessages and compilemessages arguments that you are passing. #Vishnu's answer above is correct.
TLDR
Don't use zh-cn for the locale. Use zh_CN.
django-admin.py makemessages -l zh_CN
django-admin.py compilemessages -l zh_CN
Explanation
The "locale name" as defined in the Django Docs is:
A locale name, either a language specification of the form ll or a combined language and country specification of the form ll_CC. Examples: it, de_AT, es, pt_BR. The language part is always in lower case and the country part in upper case. The separator is an underscore.
In the makemessages example (found at the bottom of the makemessages documentation), you will notice the use of the locale name convention:
django-admin makemessages --locale=pt_BR
django-admin makemessages --locale=pt_BR --locale=fr
django-admin makemessages -l pt_BR
django-admin makemessages -l pt_BR -l fr
django-admin makemessages --exclude=pt_BR
django-admin makemessages --exclude=pt_BR --exclude=fr
django-admin makemessages -x pt_BR
django-admin makemessages -x pt_BR -x fr
You will notice the same in the compilemessages example:
django-admin compilemessages --locale=pt_BR
django-admin compilemessages --locale=pt_BR --locale=fr -f
django-admin compilemessages -l pt_BR
django-admin compilemessages -l pt_BR -l fr --use-fuzzy
django-admin compilemessages --exclude=pt_BR
django-admin compilemessages --exclude=pt_BR --exclude=fr
django-admin compilemessages -x pt_BR
django-admin compilemessages -x pt_BR -x fr
I have experienced a similar problem: compilemessagesseems to be working only on locale in the current directory. This was a remedy:
find . -name "locale" | while read VAR1; do CURR="``pwd``";cd $VAR1/..; echo "in $VAR1";django-admin.py compilemessages ; cd $CURR; done`
There is an outstanding problem in this code which is as follow:
In settings.py Make sure django.middleware.locale.LocaleMiddleware in the MIDDLEWARE setting comes after SessionMiddleware and CacheMiddleware and before CommonMiddleware if those other middlewares are used.
# settings.py
MIDDLEWARE = [
'django.contrib.sessions.middleware.SessionMiddleware',
'django.middleware.cache',
# ...
'django.middleware.locale.LocaleMiddleware',
# ...
'django.middleware.common.CommonMiddleware',
]
for more information see: 1 , 2