Changes between Version 2 and Version 3 of ModelInterNationalization


Ignore:
Timestamp:
May 28, 2010, 8:34:15 AM (14 years ago)
Author:
Stefan Wehrmeyer
Comment:

major rewrite, API proposal

Legend:

Unmodified
Added
Removed
Modified

  • ModelInterNationalization

    v2 v3  
     1== Introduction ==
     2Translation of Content stored in Models is a concern for many. A variant of 3rd party solutions have been developed. However, these solutions have many drawbacks. This page will summarize current use cases and propose an API for possibility (4). Related resources, discussions and tickets are listed at the end.
     3
    14== Use Cases ==
    25
    3 === Multilingual application with no relations ===
    4 Some multilangual applications don't have relations between different language versions of objects (eg. blog, each entry in one language). Here it is only interesting which language the content has, which can be solved by programmer easyly. So here no infrastructural changes are needed, but after changes this version shouldn't become slower.
     6=== Simple language-aware model (1) ===
     7Some multilangual applications don't have relations between different language versions of objects (eg. blog, each entry in one language). Here it is only interesting which language the content has, which can be solved by programmer easily. This case works right now without any changes to Django.
     8
     9Example:
     10
     11{{{
     12class BlogEntry(models.Model):
     13    language = models.CharField(max_length=20, db_index=true)
     14    title = models.CharField(max_length=255)
     15    # other fields
     16}}}
    517
    618Specialities:
    … …  
    1022
    1123Model implementation:
    12  * object per language
    13 
    14 === Multilingual application with weak relations ===
    15 Some multilingual applications have weak relationships, so an object just needs to know which language it has and have a reference to objects with the other languages. (eg. wiki, see entry in other language link)
    16 
    17 Specialities:
    18  * each object is available in at least one language
    19  * there is not necessary a representation in an other language
    20  * if there is no representation in a language, it is not a automatic fallback
    21  * if there is a representation in an other language, a relation should be possible
    22 
    23 Model implementation:
    24  * object per language
    25  * link to group object, or grouping by groupid
    26 
    27 === Multilingual application with strong relations ===
    28 Some multilingual applications share the logic in the object but have different localizations. (eg. customizable workflows, browsergame)
    29 
    30 Specialities:
    31  * each object is available exactly once
    32  * each object has at least one but can have more than one representations in different languages
    33  * there is not necessary a representation in all selectable languages
    34  * if there is no representation in default language, there should be an automatic fallback
    35  * it should be possible to display the object in multiple languages on same page
    36 
    37 Model implementation:
    38  * one object serves all languages (no logic duplication)
    39 
    40 == Approach overview ==
    41 A brif overview of existing approaches with implementing projects.
    42 
    43  * propertycolumn is holding all translations (in a map)
    44   * [http://code.google.com/p/transdb/ transdb]
    45 
    46  * collumn per language for every translated proerty
    47   * [http://code.google.com/p/django-modeltranslation/ django-modeltranslation]
    48   * [http://code.google.com/p/django-transmeta/ django-transmeta]
    49 
    50  * big table holding all translations
    51   * [http://code.google.com/p/django-utils/wiki/TranslationService django-utils]
    52 
    53  * table per property holding all languages of this property
    54 
    55  * table for not internationalized properties and an extratable for the localized version
    56   * [http://code.google.com/p/django-multilingual/ django-multilingual]
    57    * language is saved as integer, which doesn't work for multiple site projects where language ids in setting can differ
    58   * [http://code.google.com/p/django-multilingual-model/ django-multilingual-model]
    59   * [http://github.com/foxbunny/django-i18n-model/ django-i18n-model]
    60 
    61  * object per language, the association between objects is provided by a goup-id (an update of the not internationalized fields would overwrite this fields in all other language objects)
    62   * [http://blog.rafaljonca.org/2009/03/third-style-of-django-multilingual-data.html a code example]
    63  * normal model with one language, additional table with all i18n fields in other languages
    64   * [http://code.google.com/p/django-pluggable-model-i18n/ django-pluggable-model-i18n] (status: experimental)
    65 
     24 * one object per language
     25
     26=== Multilingual model with multiple languages in one object  ===
     27The translations for all languages are stored on the same object.
     28
     29(2) One way could be:
     30
     31{{{
     32class BlogEntry(models.Model):
     33    title = models.CharField(max_length=255)
     34    title_de = models.CharField(max_length=255)
     35    title_ru = models.CharField(max_length=255)
     36    # etc.
     37}}}
     38
     39Advantages:
     40 * a conceptual object is represented by one real object
     41 
     42Disadvantages:
     43 * Schema changes necessary for additional languages
     44 * FKs cannot directly reference a specific language and also need a language attribute
     45
     46(3) Another way could be:
     47Translations are stored for each model in another model and are JOINed on demand.
     48
     49Advantages:
     50 * No schema changes
     51
     52Disadvantages:
     53 * JOINs can be considered evil
     54
     55=== Multilingual model with one object per language (4) ===
     56The Model is language-aware but also has a indirect reference to the same object in different languages via a common key:
     57
     58Example:
     59
     60{{{
     61class BlogEntry(models.Model):
     62    language = models.CharField(max_length=20, db_index=true)
     63    group_id = models.CharField(max_length=36) # e.g. UUID
     64    # All database objects that represent the same logical object in different languages
     65    # have the same group_id (possibly a UUID, may also be an integer).
     66   
     67    title = models.CharField(max_length=255)
     68    # other fields
     69   
     70}}}
     71
     72Advantages:
     73 * flexible in respect to additional languages
     74 * querying stays easy, since basic model capabilities hold
     75
     76Disadvantages:
     77 * multiple model objects for one conceptual object, does not really hurt
     78 * duplication of non-translatable fields (ie fields that stay the same across translations)
     79
     80
     81API for Model Translation in respect to (4) - a proposal/discussion:
     82
     83{{{
     84class BlogEntry(TranslatableModel):
     85    title = models.CharField(max_length=255)
     86    author = models.ForeignKey(User)
     87    # To mark fields as translatable/untranslatable:
     88    #  * title = models.CharField(max_length=255, translatable=True)
     89    #  * author = models.CharField(max_length=255, untranslatable=True)
     90    last_modified = models.DateTimeField(auto_now=True)
     91
     92    # or introduce a field in a Meta class
     93    class Meta:
     94        untranslatable_fields = ('last_modified','author',)
     95        #  or translatable_fields = ('title',)
     96       
     97    def __unicode__(self):
     98        return _(u"Blog entry '%s'") % self.title # works of course
     99
     100# Relations:
     101# A foreign key may conceptually be the following:
     102#  1. it either references an object in a specific language
     103#  2. or it references a conceptual object in all of its languages
     104
     105# 1.
     106class Image(models.Model):
     107    blog_entry_1 = models.ForeignKey(BlogEntry)
     108    # this ForeignKey can reference the PK of the BlogEntry object, no problem
     109    blog_entry_2 = models.ForeignKey(BlogEntry)
     110    # this ForeignKey may reference language and group_id as a key, more complicated
     111
     112# 2.
     113class Image(models.Model):
     114    blog_entry_1 = models.ForeignKey(BlogEntry)
     115    # this ForeignKey can reference the PK of an BlogEntry object and get all other languages via group_id
     116    blog_entry_2 = models.MultiLanguageForeignKey(BlogEntry)
     117    # this ForeignKey can reference the group_id of a BlogEntry object and get objects in all languages
     118   
     119
     120blog_entry = BlogEntry(title="Some title", author=some_user)
     121blog_entry.save() # saves in current language as default, new group
     122
     123# explicitly save for language, creates new group, new group_id is created
     124blog_entry.save(language="en")
     125# explicit language save, same group_id as other_blog_entry:
     126blog_entry.save(language="en",connect_to=other_blog_entry)
     127
     128# Manager API that does implicit current language retrieval:
     129blog_entry = BlogEntry.objects.get(group_id=UUID)
     130# returns blog entry in current language,
     131# ie behind the scenes: BlogEntry.objects.get(group_id=UUID, language=CURRENT_LANGUAGE)
     132
     133# Updates
     134blog_entry.author = some_other_user
     135blog_entry.title = "Some other title"
     136blog_entry.save()
     137# author is marked as untranslatable, therefore all instances must be updated
     138#   this can be a single query: UPDATE blog_blogentry SET author_id=<some_other_user_id> WHERE group_id=<UUID>
     139#   but needs #4102 to be fixed
     140# title is marked translatable, therefore only this instance is updated.
     141# these are two queries, but they may be reduced to one when changes on fields are tracked after instance creation
     142# and only one kind of fields (translatable xor translatable) are changed
     143
     144# Deletes
     145blog_entry.delete() # should only delete this language
     146blog_entry.delete(all_languages=True) # deletes all translated objects all languages
     147
     148# Fallbacks
     149# When a language is requested for which an object does not exist
     150try:
     151    blog_entry = BlogEntry.objects.get(group_id=UUID) # object does not exist in current language
     152except BlogEntry.DoesNotExistInLanguage as e: # a subclass of BlogEntry.DoesNotExist
     153    e.available_languages # list of available languages or may even contain the respective objects
     154    # List can be shown to user as an option to view object in other language
     155
     156}}}
     157
     158==== Things to note ====
     159 * any time 'current language' is used it means django.utils.translation.get_language() which uses thread locals
     160
     161==== Important Tickets ====
     162 * [http://code.djangoproject.com/ticket/4102 #4102]
     163 * many more
    66164
    67165== Aspects and design Questions ==
    68166 * setting default locale in Managers?
    69167 * setting default locale in related objects?
     168 * locale in URL?
     169 * Mark model fields as internationalized through attribute or through meta class (first approach is bad for subclassing)
     170 * getting current language easy, having easy access to other languages
     171 * Models are often Subclassed, do the subclasses lose flexibility?
    70172 * is {{{ __unicode__(self):}}} locals aware?
    71  * locale in URL
    72  * what if somebody needs to use internationalized objects with something complexyty adding like django-reversion, is it still possible, which limitations result form the solution?
    73  * getting current language easy, having easy access to other languages
    74  * ltr and rtl languages mixin issues if there are multiple languages used in a page as result of fallback
    75  * Models are often Subclassed, do the subclasses lose flexibility?
    76  * Mark modelfields as internationalized through attribute or through meta class (first approach is bad for subclassing)
    77  * fallback, you allways want/need a fallback (flexible fallback strategy? who decides how the fallback is working)
    78 == Inirect Issues, related problems ==
    79  * Language in the URI, namespaces aware of language, reverse should be language-aware
     173  * yes, by the design (4)
     174 * what if somebody needs to use internationalized objects with something complexity adding like django-reversion, is it still possible, which limitations result from a solution?
     175  * must work anyway if they only used the documented API
     176 * Language Fallbacks
     177  * you allways want/need a fallback (flexible fallback strategy? who decides how the fallback is working)
     178  * ltr and rtl languages mixin issues if there are multiple languages used in a page as result of fallback
     179  * Solution to all of that: problem of someone else (application developer)
     180
     181== Indirect Issues, related problems ==
     182 * Language in the URI, URL namespaces aware of language, reverse should be language-aware
    80183 * UI Multiforms for translations
    81  * oly languages officially supported by Django work or can be selected in settings.py (not sure if this is describing it exactly but there is a limitation)
    82  * indexable CharField could be very usefull for selection of different translations
     184 * only languages officially supported by Django work or can be selected in settings.py (not sure if this is describing it exactly but there is a limitation)
    83185 * language selection
    84186  * in URL as .../<locale>/...
    85187  * in subdomain
    86   * as GET request (is sometimes requestet by customers)
    87   * as POST (is bad for searchengines)
    88   * language databasetable similar to contenttypes databasetable
    89 
     188  * as query string ?lang=en
     189  * as cookie (generally bad)
     190  * language database table similar to ContentTypes database table?
     191
     192
     193== Approach overview ==
     194A brief overview of existing approaches with implementing projects.
     195
     196 * property column is holding all translations (in a map)
     197 * [http://code.google.com/p/transdb/ transdb]
     198 
     199 * column per language for every translated property
     200 * [http://code.google.com/p/django-modeltranslation/ django-modeltranslation]
     201 * [http://code.google.com/p/django-transmeta/ django-transmeta]
     202 
     203 * big table holding all translations
     204 * [http://code.google.com/p/django-utils/wiki/TranslationService django-utils]
     205 
     206 * table per property holding all languages of this property
     207 
     208 * table for not internationalized properties and an extratable for the localized version
     209 * [http://code.google.com/p/django-multilingual/ django-multilingual]
     210  * language is saved as integer, which doesn't work for multiple site projects where language ids in setting can differ
     211 * [http://code.google.com/p/django-multilingual-model/ django-multilingual-model]
     212 * [http://github.com/foxbunny/django-i18n-model/ django-i18n-model]
     213 
     214 * object per language, the association between objects is provided by a goup-id (an update of the not internationalized fields would overwrite this fields in all other language objects)
     215 * [http://blog.rafaljonca.org/2009/03/third-style-of-django-multilingual-data.html a code example]
     216 * normal model with one language, additional table with all i18n fields in other languages
     217 * [http://code.google.com/p/django-pluggable-model-i18n/ django-pluggable-model-i18n] (status: experimental)
     218 * [http://github.com/ojii/django-multilingual-ng]
    90219
    91220== Other resources ==
    … …  
    96225  * [http://groups.google.com/group/django-users/browse_frm/thread/d6e7eab4cc81c7b8  An other thread on django-users ]
    97226 * Blogposts:
    98   * [http://and-other-things.blogspot.com/2009/04/list-of-django-multilingual-model.html]
    99   * [http://vaig.be/2008/07/django-i18n-status.html Marc Garcia django-i18n-status]
     227  * [http://vaig.be/2008/07/django-i18n-status.html Marc Garcia django-i18n-status (July 2008)]
     228  * [http://and-other-things.blogspot.com/2009/04/list-of-django-multilingual-model.html| List of Django Multilingual Model solutions (April 2009)]
     229  * [http://www.muhuk.com/2010/01/dynamic-translation-apps-for-django/| Dynamic Translation Apps for Django (January 2010)]
    100230
    101231== bucket ==
Back to Top