Opened 11 years ago

Closed 11 years ago

Last modified 11 years ago

#19919 closed Bug (fixed)

get_language_from_request() disregards "en-us" and "en" languages when matching Accept-Language header

Reported by: lanzz@… Owned by: Łukasz Langa
Component: Internationalization Version: 1.5-rc-1
Severity: Normal Keywords: localization
Cc: Triage Stage: Accepted
Has patch: no Needs documentation: no
Needs tests: yes Patch needs improvement: no
Easy pickings: no UI/UX: no

Description

Scenario:
Django settings with languages:

LANGUAGES = (
    ('bg', gettext('Bulgarian')),
    ('en-us', gettext('English')),
)

A request comes with header:

Accept-Language: en-US,en;q=0.8,bg;q=0.6,ru;q=0.4

Without a previously set language in the session or a cookie, get_language_from_request() should inspect the header and pick the "en-us" language, as it is of the highest priority. Now, I see that get_language_from_request() actually disregards the weight, but still it should pick the "en-us" language, as it is listed first in the header, and is an explicitly enabled language in the settings. This is hot what happens, however.

get_language_from_request() first normalizes each accepted language to its locale name, as mapped by Python's locale.locale_alias. This turns both "en-US" and "en" into the canonical locale name "en_US". Then it tests if either "en_US" or its primary component "en" is a supported language. It scans over all_locale_paths() and tests if any of them contains an en_US/LC_MESSAGES/django.mo file, which none does, because Django does not provide an "en_US" translation, thus the "en_US" locale is rejected. Then it sees that "en" is not listed in LANGUAGES, and rejects that as well. It ends up returning "bg" as detected language, even though it is of a lower priority AND listed later than both "en-US" and "en" in the header.

Since the "en-us" language is the default for the LANGUAGE_CODE setting, and there are cases where Django forces the language to "en-us", it seems very unexpected to me that it is not possible to use that language code in LANGUAGES without it being completely rejected during Accept-Language matching. I am forced into using "en" in my LANGUAGES, even though my management commands, for instance, will run under a forced "en-us" language (which would be technically unsupported as per the LANGUAGES setting).

It seems to me the problem in this case is line 443 in django/utils/translation/trans_real.py. At that point it would check if "en" is in LANGUAGES, even though we're actually processing the "en-US" entry from the Accept-Language header. It makes no sense to reject "en-US" because "en" is not in LANGUAGES, even though "en-US" itself IS listed there.

(Note that this has been tested with 1.5c2, though it is not an option in the Version select box here)

Change History (4)

comment:1 by Łukasz Langa, 11 years ago

Needs tests: set
Owner: changed from nobody to Łukasz Langa
Status: newassigned
Triage Stage: UnreviewedAccepted

comment:2 by Aymeric Augustin, 11 years ago

Component: UncategorizedInternationalization

comment:3 by Łukasz Langa <lukasz@…>, 11 years ago

Resolution: fixed
Status: assignedclosed

In 92ebb29c5376c1811e8607e96bddc90d24e44e2a:

Fixes #19919: get_language_from_request() disregards "en-us" and "en" languages
when matching Accept-Language

comment:4 by Aymeric Augustin <aymeric.augustin@…>, 11 years ago

In cac7b44571b497bfecad2b786ae7c6172aed842e:

Merge pull request #1100 from ambv/issue19919

Fixed #19919: get_language_from_request() disregards en-us and en

Note: See TracTickets for help on using tickets.
Back to Top