I18N needs improvements

Initial patch (partial)

V1 patch includes all features except form i18n. Some parts are ugly and need to be coded better, it's just a starting point to see what needs to be changed.

I set milestone to 1.0 beta, because I think it should be included (if finished when 1.0-beta) on 1.0, so it's essential for djangonauts from many countries (most except the US, UK and Australia ;)

Would it be possible to handle this as individual smaller entities, instead of one monolithic patch.

Yes, we have to do this, but this isn't the right approach, since there's too much manual work. Also, there are tickets already open for most of these things, from memory.

What I want to do here is auto-generate all of this information from the Local Data Markup Language files from the Unicode group. It should be pretty easy to do and then we just drop stuff in the appropriate locale directories. Django can ship a script to parse the file for a new language and we can make that script rely on one download (since it's only needed for maintainers, Django doesn't have to ship it).

I'll leave this open until I hunt out all the dupes later on tonight. I agree we should do this pre-1.0, Marc, but lots more translation stuff like this is probably a burden we can avoid.

First draft of a script for importing data from CLDR to django

Attached import_cldr.py isn't finished, and isn't working ok. I just uploaded it for people that is working on improve django's i18n.

This will have to wait until after 1.0. There's still too much work required to get it in before the beta and it requires enough internal changes that it will be potentially destabilising.

I still don't like the first approach in the patch because it adds a lot of settings and makes assumptions such as i18n being enabled. So even it would need some work (and not be backwards compatible with a more general solution). So, sorry, Marc, but this just came a bit late in the process. It's very important, but not worth the risk at the moment.

We'll definitely get it in by 1.1, though.

I absolutely agree.

Sadly I realized on that when improving and merging the existing patches.

The big news are that it'll be another big fucking party for Django 1.1 if it includes it. ;)

Imminent duplicated effort alert: http://babel.edgewall.org/ . Can't we just depend on Babel? Also, when things get refactored -- contrib.localflavour should really not be part of Django trunk.

Please do not post random unrelated comments on tickets (localflavour has nothing to do with this ticket). We aren't duplicating all of Babel, obviously, since we're well aware of that project, and we aren't about to have Django depending on a third-party package the size of Babel. We are incorporating some necessary minimal stuff so that Django's continues to have batteries included.

#6974 was another duplicate.

Milestone post-1.0 deleted

Fixed in branches/soc2009/i18n-improvements.

diff of all work developed on the [soc2009/i18n]

diff of all work developed on the [soc2009/i18n] with .po and formats files removed

Not sure what the right place to report this is.

I'm given the soc2009/i18n-improvements branch a try, but have been running into the problem that my German locale is not being used in form submissions. I'm using de_DE as my LANGUAGE_CODE setting, and as far as I was able to tell this is because in django/utils/formats.py, it attempts to import the format file based on django.utils.translation.get_language(), which in my case would return "de-de", when in fact the module to import from is "django.conf.locale.de".

Loading the .mo files doesn't have the same problem, since the gettext loading mechanism seems to auto-fallback to the available "de" locale.

Another bug in that branch: floatformat doesn't correctly print the sign of negative values. Quick-fix here:

http://github.com/miracle2k/django/commit/9259cb9bd1f40167a67714fc037d504c1247d979

Replying to miracle2k:

Another bug in that branch: floatformat doesn't correctly print the sign of negative values. Quick-fix here: http://github.com/miracle2k/django/commit/9259cb9bd1f40167a67714fc037d504c1247d979

Thanks, I've fixed this in my current working branch: http://github.com/jezdez/django/tree/i18n

Replying to miracle2k:

Not sure what the right place to report this is. I'm given the soc2009/i18n-improvements branch a try, but have been running into the problem that my German locale is not being used in form submissions. I'm using de_DE as my LANGUAGE_CODE setting, and as far as I was able to tell this is because in django/utils/formats.py, it attempts to import the format file based on django.utils.translation.get_language(), which in my case would return "de-de", when in fact the module to import from is "django.conf.locale.de". Loading the .mo files doesn't have the same problem, since the gettext loading mechanism seems to auto-fallback to the available "de" locale.

Good catch, I'll work on a fix for this. It probably involves a simple <locale>[:2] check in case the long format is not found.

Updated patch with relevant parts of i18n GSoC

Fixed issue with LANGUAGE_CODE of sublanguages

Updated format discovery to also look in main locales if not found in sublocales formats

Added Serbian and Serbian latin format strings from #11637

More deprecation warnings and Renamed USE_FORMAT_I18N setting to USE_L10N

Mainly updates to docs

(In [11964]) Fixed #7980 - Improved i18n framework to support locale aware formatting (dates and numbers) and form processing.

Thanks to Marc Garcia for working on this during his Google Summer of Code 2009!

Additionally fixes #1061, #2203, #3940, #5526, #6449, #6231, #6693, #6783, #9366 and #10891.

I just created ticket #12550 with a patch for a bug caused by the changes comitted for this one. Since my patch only deals with DecimalField? and this is a large submission I am not sure if there might be other types affected. I think it might be good for someone with more knowledge of the django.contrib.admin codebase to review this.

This ticket was closed because the original fature has been implemented.

If you opened another ticket, there is no need to reopen this one. Doing so isn't the right way to bring attention to the issue.

As Ramiro said, we only need one ticket to track the follow-on bug, and the right ticket for that is the new one, not this one. Concerns about there possibly being other field types affected should be noted there. Tests covering all field types would be a way to find out if the problem affects other field types. A test needs to be developed anyway for the DecimalField case; it should not be hard to expand that test to also test other field types. That would be useful both to find out whether the problem affects other fields, and to put in the test suite in either case to ensure this kind of problem, for any field type, does not recur with any future code changes.

Sorry about that, my aim was not really to try and draw attention to that ticket, I already added tests and fixed it so that it does not happen for other field types.

I added notes here more to notify whoever was involved in this one that there might be other places (not related to admin list display necessarily) that was affected by this change and for anybody in the know to review the patch for this again.