id	summary	reporter	owner	description	type	status	component	version	severity	resolution	keywords	cc	stage	has_patch	needs_docs	needs_tests	needs_better_patch	easy	ui_ux
3063	[patch] i18n broken for msgids with extended characters	Antti Kaihola	hugo	"Message ids with non-us-ascii characters in a UTF-8 encoded .po file are not recognized. Django returns the original string even though a translation is provided.

The reason is that Python's {{{GNUTranslations.gettext()}}} expects the message id as a unicode string, but {{{gettext()}}} in {{{trans_real.py}}} calls it with UTF-8 encoded message ids.

Even if this is fixed by decoding the message id before calling {{{t.gettext()}}}, there is another problem with missing messages: Python's {{{GNUTranslations.gettext()}}} returns UTF-8 encoded strings, but when a message is not found, it just returns the original unicode message id, and the template engine chokes when trying to join mixed unicode and normal strings.

The Python {{{gettext}}} documentation recommends to use {{{ugettext()}}} instead of {{{gettext()}}} so everything is handled in unicode.

I know it's exceptional to use non-us-ascii message ids, but my strong opinion is that since we're beginning to live in a unicode world, it shouldn't be a problem anymore. And I'm sure many people have a situation where an existing single-language non-English site is i18n'ed and it's simply most practical just to wrap existing non-English messages in {{{{%trans%}}}} tags and {{{_()}}} calls. A multi-lingual site might even use only languages with non-us-ascii character sets.

I'll post a patch which fixes both problems described above."	defect	closed	Internationalization		normal	fixed	i18n		Design decision needed	1	0	0	0	0	0