Opened 14 years ago
Last modified 8 months ago
#14974 new New feature
Add support for translation backends other than gettext
Reported by: | Marinho Brandão | Owned by: | nobody |
---|---|---|---|
Component: | Internationalization | Version: | dev |
Severity: | Normal | Keywords: | |
Cc: | ethan.jucovy@…, aurelio@…, ojiidotch@…, kitsunde@…, stas@…, rtnpro, Andi Albrecht, Ülgen Sarıkavak | Triage Stage: | Accepted |
Has patch: | yes | Needs documentation: | yes |
Needs tests: | no | Patch needs improvement: | yes |
Easy pickings: | no | UI/UX: | no |
Description
Using gettext has been a problem on many projects. It's weird to translate and is limited to (static) potfiles (nothing convince me that Pootle Server and Rosetta are great ideas, actually they are just quick fixes for a bad implemented tool).
So, I'd like to suggest you to make a contrib app to have all i18n functions.
Not only the existing functions but provide backend system for alternative ways to make translations, to replace gettext for database stored message strings, or dynamic/smart translations.
Would be helpful also if the same application has a way to translate database objects, like django-multilingual (1) and polyglot (2) do.
I can work on it if the idea get approved.
Change History (21)
comment:1 by , 14 years ago
Component: | Uncategorized → Internationalization |
---|---|
Triage Stage: | Unreviewed → Design decision needed |
follow-up: 3 comment:2 by , 14 years ago
Of course... so, because it is not for dynamic translations, it's a problem to do it if the only way that the framework offers to do is using gettext.
This is why I believe a backend system would help a lot, because who want to work with non-static translations (and at the level of model objects) can avoid gettext and use database or cache to do it.
I know that gettext is mature and fast, yet a limited sollution.
follow-up: 4 comment:3 by , 14 years ago
Replying to marinho:
Of course... so, because it is not for dynamic translations, it's a problem to do it if the only way that the framework offers to do is using gettext.
You're basically trying to justify a new feature by saying the existing feature doesn't cover your use case, which is a moot point in my opinion.
This is why I believe a backend system would help a lot, because who want to work with non-static translations (and at the level of model objects) can avoid gettext and use database or cache to do it.
Right, I don't quite understand what you mean by backend system though. An explanation (on the mailing list) might help to clarify what you mean. The big issue I see here (at least from what I understand) is that content translation and translation of UI elements are completely different stories because the way those content types differ in structure and storage. Adding a backend system implies having a common API, but I don't see any. Can you elaborate?
I know that gettext is mature and fast, yet a limited sollution.
It's not limited but really just not supposed to be used like you use.
comment:4 by , 14 years ago
Replying to jezdez:
I didn't revise the text below, please understand I'm doing it as faster as I can (today I'm working).
Ok, I tried to be clear on the mail list, but I'm going to right a fast story with more details below:
You know, most of i18n functions are in:
- utils/translation
- core/management/commands/compilemessages.py
- core/management/commands/makemessages.py
- views/i18n.py
- conf/locale
What I'm saying is about to change the points 1, 2, 3 and 4.
Keep in my the following structure:
- contrib/i18n
- contrib/i18n/init.py
- contrib/i18n/models.py <--- here we have the model classes for DB storage
- contrib/i18n/views.py <--- here we have the current "views/i18n.py"
- contrib/i18n/backends/init.py
- contrib/i18n/backends/base.py <--- here we have the backend base class
- contrib/i18n/backends/potfiles.py <--- here we have the backend class for gettext (with most of current "utils/translation" code)
- contrib/i18n/backends/db.py <--- here we have the backend class for database (using the model class for DB storage, etc)
- contrib/i18n/management/init.py
- contrib/i18n/management/commands/init.py
- contrib/i18n/management/commands/compilemessages.py <--- we move the current to here
- contrib/i18n/management/commands/makemessages.py <--- we move the current to here
Now a prototype of "contrib/i18n/models.py":
class MessageString(models.Model): class Meta: unique_together = ( ('hash','language'), ) hash = models.CharField() language = models.CharField() message_id = models.TextField() # if necessary message_str = models.TextField()
Now a prototype of "contrib/i18n/backends/base.py":
class BaseBackend(object): def gettext(self, msg): raise NotImplemented def ugettext(self, msg): raise NotImplemented def gettext_noop(self, msg): raise NotImplemented def get_language(self): return self._language # Of course, different def makemessages(self): raise NotImplemented def compilemessages(self): raise NotImplemented # etc...
Now a prototype of "contrib/i18n/backends/potfiles.py":
class GettextBackend(BaseBackend): potfile = 'django.po' # etc... def gettext(self, msg): return do_translate(message, 'gettext') # Like it does nowadays def ugettext(self, msg): return do_translate(message, 'ugettext') # Like it does nowadays def gettext_noop(self, msg): return message def makemessages(self): # do the same that makemessages.py actually does def compilemessages(self): # do the same that compilemessages.py actually does # etc...
Now a prototype of "contrib/i18n/backends/db.py":
class DatabaseBackend(BaseBackend): def get_hash(self, msg): return hashlib.sha1(msg).hexdigest() def gettext(self, msg): try: obj = MessageString.objects.get( language = self.get_language(), hash = self.get_hash(msg), ) return obj.message_str except MessageString.DoesNotExist: return msg def ugettext(self, msg): return do_translate(message, 'ugettext') # Like it does nowadays def gettext_noop(self, msg): return message def makemessages(self): # do the same that makemessages.py actually does, but stores using # MessageString instead of using the gettext tools def compilemessages(self): # do nothing, probably # etc...
Now the setting to set the current backend (settings.py):
I18N_BACKEND = 'django.contrib.i18n.backends.DatabaseBackend'
So, the template tags and ugettext functions must be changed to use the current
backend instead of just use gettext.
comment:5 by , 14 years ago
Just fixing: ignore the method "ugettext" from "DatabaseBackend". It should be like the method "gettext" from the same class.
comment:6 by , 14 years ago
Severity: | → Normal |
---|---|
Type: | → New feature |
comment:7 by , 13 years ago
Easy pickings: | unset |
---|---|
Triage Stage: | Design decision needed → Accepted |
UI/UX: | unset |
The original report proposes three main changes:
- 1) a django.contrib.i18n application — which is basically a large refactoring
- 2) support for translation backends (other file formats / string databases)
- 3) support for dynamic translations (translation of database content)
Refactorings are complicated, because they introduce backwards incompatibilities that only disappear after three releases — that's the length of our deprecation path. They're also very hard to review. We tend to accept them only when they're a pre-requisite to add interesting features. Anyway, the amount of effort needed for the refactoring is dwarfed by the the amount needed for the features, so it doesn't really matter at this step.
Features (2) and (3) are rather large — as a wild guess, defining, developing, testing and documenting them for Django would take a few hundred hours.
Reading the thread you started on this topic, I understand that your proposal is mostly about (2). As you say in your last message, what we need now is a more detailed proposal, explaining the consequences of the change:
- which backends would you include initially?
- how much would this affect performance?
- how would this affect distribution of translations for Django itself, and for pluggable third-party applications?
- etc.
A proof of concept in a fork on GitHub would be interesting too.
To sum up:
- I agree that pluggable translation backends and dynamic translations are interesting ideas. They're separate ideas, so let's limit this ticket to translation backends. I'm accepting the ticket with this scope.
- Please don't focus too much on refactoring — it adds more noise than value. Deal with the hard part (APIs, proof of concept) first. Try to be as much backwards compatible as possible. Then, if necessary, we can decide to refactor things.
comment:9 by , 13 years ago
Summary: | Contrib application for i18n functions → Add support for translation backends other than gettext |
---|
comment:10 by , 13 years ago
Cc: | added |
---|
comment:11 by , 12 years ago
Cc: | added |
---|
comment:12 by , 12 years ago
Cc: | added |
---|
I'm not sure point 3 in comment:7 should really be part of this ticket. Dynamic translations are a completely different thing than static translations. Wouldn't it make more sense to move the discussion etc about dynamic translations in Django to another ticket and keep this one purely for supporting additional backends for static translations?
comment:14 by , 12 years ago
#19480 is a proposal related to this ticket and has a pull request at https://github.com/django/django/pull/590
comment:15 by , 12 years ago
Cc: | added |
---|
comment:16 by , 11 years ago
Cc: | added |
---|
comment:18 by , 11 years ago
Cc: | added |
---|
comment:19 by , 10 years ago
Has patch: | set |
---|---|
Needs documentation: | set |
Patch needs improvement: | set |
The PR above doesn't merge cleanly and lacks documentation.
comment:20 by , 7 years ago
Cc: | added |
---|
comment:21 by , 8 months ago
Cc: | added |
---|
With all due respect, calling gettext problematic for non-static translation seems a bit off since it clearly wasn't made for dynamic translations. In any case, this requires thorough discussion on the mailing list before any approval.