Upgrading Django from 2.1 to 2.2 - strings are not translated

[אורי]

I tried to upgrade Django from 2.1 to 2.2 in ​Speedy Net, and I found out that strings are not translated to Hebrew any more. Not all the strings, some of the strings. Especially strings translated with ngettext_lazy.

Tests which fail with strings not translated - ​here.

In the Hebrew website I get error messages in English instead of Hebrew. For example "This password is too long. It must contain at most 120 characters." instead of "סיסמה זו ארוכה מדי. היא יכולה להכיל 120 תווים לכל היותר." (and other error messages, for example in the registration form).

Not all the strings translated with ngettext_lazy are not translated, but only some of them. For example, the error message "This password is too short. It must contain at least 8 characters." is translated to Hebrew in the Hebrew website and tests.

[אורי]

I'm not sure, but it might be related to this ​commit.

[אורי]

It's a regression, we need to bisect. Everything works in Django versions up to 2.1.13.

[אורי]

$ git bisect run ../speedy-net/speedy/net/tests_manage.py test speedy.core.accounts.tests.test_models.UserHebrewTestCase

98f526e67c2899b9d44ba7cdbab5d70b1baa0060 is the first bad commit
commit 98f526e67c2899b9d44ba7cdbab5d70b1baa0060
Author: Claude Paroz <claude@…>
Date: Sat Mar 30 11:38:12 2019 +0100

[2.2.x] Updated core translations from Transifex

:040000 040000 d4b445730ca9670d5a9521e1b9140a365ec35ea1 35eeac3fa09f541c819f21d3dfa3b0d985081fed M django
bisect run success

[Claude Paroz]

I guess this is due to the fact Transifex uses now a different Plural equation for Hebrew.
Used to be the simple Plural-Forms: nplurals=2; plural=(n != 1);, now it is:
Plural-Forms: nplurals=4; plural=(n == 1 && n % 1 == 0) ? 0 : (n == 2 && n % 1 == 0) ? 1: (n % 10 == 0 && n % 1 == 0 && n > 10) ? 2 : 3;"

I'm sorry for that change, but we unfortunately have no way AFAIK to change that ourselves on Transifex.

As Django is currently using the core translations plural equations for all catalogs, you have to ensure that all catalogs are using 4 plural variants (which I recognize might be not easy to do for third-party packages).

Duplicate of #30439, which have discussions on possible fixes in Django.

[אורי]

I commented on the other ticket as well. I don't understand why you need such a long formula to decide if a number is singular or plural. If it's 1 it's singular, otherwise it's plural. What's wrong about that?

[Claude Paroz]

To be clear, *I* did not decide anything. We are using the Transifex translation platform which currently imposes Plural strings to hosted projects. I don't know why they changed that, maybe you should try to talk to them?

[אורי]

Replying to Claude Paroz:

To be clear, *I* did not decide anything. We are using the Transifex translation platform which currently imposes Plural strings to hosted projects. I don't know why they changed that, maybe you should try to talk to them?

Is it possible to revert the same code that worked in all Django versions up to 2.1? I think it you (the Django developers) revert the code, it can solve this problem.

[Claude Paroz]

Do you understand the implications of what you ask? Do you suggest we stop updating translations?

[אורי]

I would like to clarify a few things:

1. Some of the strings used by "ngettext" or "ngettext_lazy" are unique to Speedy Net, some are not. Anyway I added translations for all of them in the Speedy Net django.po files, for the case that if Django will change the strings, they will still be translated.

2. When running ./make_all_messages.sh in Django 2.2, which runs make_messages for the locales en and he, nothing changes in the django.po files (except the date) - a third and fourth line are not added.

3. I think Django should handle django.po files when there are only 2 forms - singular and plural - and use the same plural forms for all forms of plurals, whatever that means. And in many cases the plural forms are identical, as I found out for example in ​https://github.com/django/django/blob/master/django/contrib/auth/locale/he/LC_MESSAGES/django.po#L223-L233. Actually from my knowledge of Hebrew, in most cases the plural form is the same for all numbers. In my case for example, Speedy Net, I only use plural although the singular is defined. I just copied the definition from Django validators with "ngettext_lazy" included. So I think there should be a fallback, so if only 2 forms are defined - singular and plural - then Django should use the same plural form for all plurals. This will also provide a backward compatibility to previous versions of Django.

4. Everything worked for me with Django versions up to 2.1. I think if such a big change is introduced to Django, which is not backward compatible, then new functions should be introduced by leaving "ngettext" or "ngettext_lazy" as they are. Or at least there should be some setting to allow backward compatibility. Especially since updating the django.po files might cause problems when downgrading to previous versions of Django, which don't support it.

5. If running make_messages will introduce 2 more plural forms in the django.po files, I expect the new forms to default to the plural form currently defined, to allow backward compatibility. And also, these functions should work when downgrading to a previous version of Django.

6. I never used Transifex directly. I only use Django. I never even heard about Transifex until today. Django developers can do whatever they want with third party dependencies. What matters to me is how Django works.

[אורי]

I checked and I didn't find anything related to this issue documented on the Django 2.2 release notes. Is it documented?

I created ticket https://code.djangoproject.com/ticket/30945