Opened 7 years ago

Closed 5 years ago

Last modified 4 years ago

#7980 closed (fixed)

I18N needs improvements

Reported by: garcia_marc Owned by: jezdez
Component: Internationalization Version: master
Severity: Keywords: i18n-fixed
Cc: marc.garcia@…, nreilly@…, alexkon@…, marcoberi@…, miracle2k, jacques.beaurain@… Triage Stage: Fixed on a branch
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: yes
Easy pickings: UI/UX:

Description

As described in http://vaig.be/2008/07/26/django-i18n-status/ Django's i18n has many features missing.

This ticket is a proposal to:

  • Customize first day of week in admin's calendar (based on current locale).
  • Customize date/datetime formats when a date/datetime is displayed (based on current locale).
  • Customize number format when a decimal number is displayed (based on current locale).
  • Customize input format for date/datetime widgets (based on current locale). It also includes displaying existing data in the same format, when shown on the widget.
  • Customize input format for decimal number widgets (based on current locale). It also includes displaying existing data in the same format, when shown on the widget.

Some other i18n features should be added later, but those are basic for Web site i18n.

Attachments (10)

7980_v1.diff (23.2 KB) - added by garcia_marc 7 years ago.
Initial patch (partial)
import_cldr.py (2.8 KB) - added by garcia_marc 7 years ago.
First draft of a script for importing data from CLDR to django
i18n.diff (248.6 KB) - added by garcia_marc 6 years ago.
diff of all work developed on the [soc2009/i18n]
i18n_simple.diff (79.4 KB) - added by garcia_marc 6 years ago.
diff of all work developed on the [soc2009/i18n] with .po and formats files removed
i18n-formatting.1.diff (141.9 KB) - added by jezdez 5 years ago.
Updated patch with relevant parts of i18n GSoC
i18n-formatting.2.diff (142.2 KB) - added by jezdez 5 years ago.
Fixed issue with LANGUAGE_CODE of sublanguages
i18n-formatting.3.diff (143.8 KB) - added by jezdez 5 years ago.
Updated format discovery to also look in main locales if not found in sublocales formats
i18n-formatting.4.diff (146.3 KB) - added by jezdez 5 years ago.
Added Serbian and Serbian latin format strings from #11637
i18n-formatting.5.diff (146.9 KB) - added by jezdez 5 years ago.
More deprecation warnings and Renamed USE_FORMAT_I18N setting to USE_L10N
i18n-formatting.6.diff (150.3 KB) - added by jezdez 5 years ago.
Mainly updates to docs

Download all attachments as: .zip

Change History (36)

Changed 7 years ago by garcia_marc

Initial patch (partial)

comment:1 Changed 7 years ago by garcia_marc

  • Needs documentation set
  • Needs tests set
  • Patch needs improvement set

V1 patch includes all features except form i18n. Some parts are ugly and need to be coded better, it's just a starting point to see what needs to be changed.

I set milestone to 1.0 beta, because I think it should be included (if finished when 1.0-beta) on 1.0, so it's essential for djangonauts from many countries (most except the US, UK and Australia ;)

comment:2 Changed 7 years ago by Alex

Would it be possible to handle this as individual smaller entities, instead of one monolithic patch.

comment:3 Changed 7 years ago by mtredinnick

  • Triage Stage changed from Unreviewed to Accepted

Yes, we have to do this, but this isn't the right approach, since there's too much manual work. Also, there are tickets already open for most of these things, from memory.

What I want to do here is auto-generate all of this information from the Local Data Markup Language files from the Unicode group. It should be pretty easy to do and then we just drop stuff in the appropriate locale directories. Django can ship a script to parse the file for a new language and we can make that script rely on one download (since it's only needed for maintainers, Django doesn't have to ship it).

I'll leave this open until I hunt out all the dupes later on tonight. I agree we should do this pre-1.0, Marc, but lots more translation stuff like this is probably a burden we can avoid.

Changed 7 years ago by garcia_marc

First draft of a script for importing data from CLDR to django

comment:4 Changed 7 years ago by garcia_marc

Attached import_cldr.py isn't finished, and isn't working ok. I just uploaded it for people that is working on improve django's i18n.

comment:5 Changed 7 years ago by mtredinnick

  • milestone changed from 1.0 beta to post-1.0

This will have to wait until after 1.0. There's still too much work required to get it in before the beta and it requires enough internal changes that it will be potentially destabilising.

I still don't like the first approach in the patch because it adds a lot of settings and makes assumptions such as i18n being enabled. So even it would need some work (and not be backwards compatible with a more general solution). So, sorry, Marc, but this just came a bit late in the process. It's very important, but not worth the risk at the moment.

We'll definitely get it in by 1.1, though.

comment:6 Changed 7 years ago by garcia_marc

I absolutely agree.

Sadly I realized on that when improving and merging the existing patches.

The big news are that it'll be another big fucking party for Django 1.1 if it includes it. ;)

comment:7 Changed 7 years ago by mrts

Imminent duplicated effort alert: http://babel.edgewall.org/ . Can't we just depend on Babel? Also, when things get refactored -- contrib.localflavour should really not be part of Django trunk.

comment:8 Changed 7 years ago by mtredinnick

Please do not post random unrelated comments on tickets (localflavour has nothing to do with this ticket). We aren't duplicating all of Babel, obviously, since we're well aware of that project, and we aren't about to have Django depending on a third-party package the size of Babel. We are incorporating some necessary minimal stuff so that Django's continues to have batteries included.

comment:9 Changed 7 years ago by jacob

#6974 was another duplicate.

comment:10 Changed 7 years ago by anonymous

  • Cc nreilly@… added

comment:11 Changed 6 years ago by alexkon

  • Cc alexkon@… added

comment:12 Changed 6 years ago by anonymous

  • milestone post-1.0 deleted

Milestone post-1.0 deleted

comment:13 Changed 6 years ago by garcia_marc

  • Keywords i18n-fixed added; date decimal calendar i18n-rf removed

Fixed in branches/soc2009/i18n-improvements.

comment:14 Changed 6 years ago by garcia_marc

  • Triage Stage changed from Accepted to Fixed on a branch

comment:15 Changed 6 years ago by marcob

  • Cc marcoberi@… added

Changed 6 years ago by garcia_marc

diff of all work developed on the [soc2009/i18n]

Changed 6 years ago by garcia_marc

diff of all work developed on the [soc2009/i18n] with .po and formats files removed

comment:16 follow-up: Changed 6 years ago by miracle2k

  • Cc miracle2k added

Not sure what the right place to report this is.

I'm given the soc2009/i18n-improvements branch a try, but have been running into the problem that my German locale is not being used in form submissions. I'm using de_DE as my LANGUAGE_CODE setting, and as far as I was able to tell this is because in django/utils/formats.py, it attempts to import the format file based on django.utils.translation.get_language(), which in my case would return "de-de", when in fact the module to import from is "django.conf.locale.de".

Loading the .mo files doesn't have the same problem, since the gettext loading mechanism seems to auto-fallback to the available "de" locale.

comment:17 Changed 5 years ago by jezdez

  • Owner changed from garcia_marc to jezdez
  • Status changed from new to assigned

comment:18 follow-up: Changed 5 years ago by miracle2k

Another bug in that branch: floatformat doesn't correctly print the sign of negative values. Quick-fix here:

http://github.com/miracle2k/django/commit/9259cb9bd1f40167a67714fc037d504c1247d979

comment:19 in reply to: ↑ 18 Changed 5 years ago by jezdez

  • milestone set to 1.2
  • Needs documentation unset
  • Needs tests unset

Replying to miracle2k:

Another bug in that branch: floatformat doesn't correctly print the sign of negative values. Quick-fix here:

http://github.com/miracle2k/django/commit/9259cb9bd1f40167a67714fc037d504c1247d979

Thanks, I've fixed this in my current working branch: http://github.com/jezdez/django/tree/i18n

comment:20 in reply to: ↑ 16 Changed 5 years ago by jezdez

Replying to miracle2k:

Not sure what the right place to report this is.

I'm given the soc2009/i18n-improvements branch a try, but have been running into the problem that my German locale is not being used in form submissions. I'm using de_DE as my LANGUAGE_CODE setting, and as far as I was able to tell this is because in django/utils/formats.py, it attempts to import the format file based on django.utils.translation.get_language(), which in my case would return "de-de", when in fact the module to import from is "django.conf.locale.de".

Loading the .mo files doesn't have the same problem, since the gettext loading mechanism seems to auto-fallback to the available "de" locale.

Good catch, I'll work on a fix for this. It probably involves a simple <locale>[:2] check in case the long format is not found.

Changed 5 years ago by jezdez

Updated patch with relevant parts of i18n GSoC

Changed 5 years ago by jezdez

Fixed issue with LANGUAGE_CODE of sublanguages

Changed 5 years ago by jezdez

Updated format discovery to also look in main locales if not found in sublocales formats

Changed 5 years ago by jezdez

Added Serbian and Serbian latin format strings from #11637

Changed 5 years ago by jezdez

More deprecation warnings and Renamed USE_FORMAT_I18N setting to USE_L10N

Changed 5 years ago by jezdez

Mainly updates to docs

comment:21 Changed 5 years ago by jezdez

  • Resolution set to fixed
  • Status changed from assigned to closed

(In [11964]) Fixed #7980 - Improved i18n framework to support locale aware formatting (dates and numbers) and form processing.

Thanks to Marc Garcia for working on this during his Google Summer of Code 2009!

Additionally fixes #1061, #2203, #3940, #5526, #6449, #6231, #6693, #6783, #9366 and #10891.

comment:22 Changed 5 years ago by Jacques Beaurain <jacques.beaurain@…>

  • Cc jacques.beaurain@… added
  • Resolution fixed deleted
  • Status changed from closed to reopened

I just created ticket #12550 with a patch for a bug caused by the changes comitted for this one. Since my patch only deals with DecimalField and this is a large submission I am not sure if there might be other types affected. I think it might be good for someone with more knowledge of the django.contrib.admin codebase to review this.

comment:23 Changed 5 years ago by ramiro

  • Resolution set to fixed
  • Status changed from reopened to closed

This ticket was closed because the original fature has been implemented.

If you opened another ticket, there is no need to reopen this one. Doing so isn't the right way to bring attention to the issue.

comment:24 Changed 5 years ago by kmtracey

As Ramiro said, we only need one ticket to track the follow-on bug, and the right ticket for that is the new one, not this one. Concerns about there possibly being other field types affected should be noted there. Tests covering all field types would be a way to find out if the problem affects other field types. A test needs to be developed anyway for the DecimalField case; it should not be hard to expand that test to also test other field types. That would be useful both to find out whether the problem affects other fields, and to put in the test suite in either case to ensure this kind of problem, for any field type, does not recur with any future code changes.

comment:25 Changed 5 years ago by Jacques Beaurain <jacques.beaurain@…>

Sorry about that, my aim was not really to try and draw attention to that ticket, I already added tests and fixed it so that it does not happen for other field types.

I added notes here more to notify whoever was involved in this one that there might be other places (not related to admin list display necessarily) that was affected by this change and for anybody in the know to review the patch for this again.

comment:26 Changed 4 years ago by jacob

  • milestone 1.2 deleted

Milestone 1.2 deleted

Note: See TracTickets for help on using tickets.
Back to Top