Django detects HTTP Accept-Language header in case-sensitive manner

[wayneye]

This issue was originally discussed in django-developers: ​https://groups.google.com/forum/#!topic/django-developers/1Y9LZSAOSnE

Per ​w3c, ​rfc2616 and ​bcp47, Language tags should be parsed in case-insensitive, however, I noticed that Django detects HTTP Accept-Language headers in case-sensitive manner.

For example, the following headers:

Chrome:   Accept-Language:  zh-TW,zh;q=0.8,en-US;q=0.6,en;q=0.4
Firefox:  Accept-Language:  zh-tw,zh;q=0.8,en-us;q=0.5,en;q=0.3

Django will correctly display Traditional Chinese for Chrome, but won't for Firefox because of lower-cased TW.

The fix contains two parts:

1. Fix potential case-sensitive places in code to follow case-insensitive (for example parse_accept_lang_header())
2. Fix ​documentation, correct the sentence "Browsers send the names of the languages they accept in the Accept-Language HTTP header using this format. Examples: it, de-at, es, pt-br. Both the language and the country parts are in lower case. ", which obviously incorrect, Chrome uses tags like zh-TW, pt-BR.

[Claude Paroz]

Case sensitivity should have been resolved by 2bab9d6d9ea095c4bcaeede2df576708afd46291
I have done some local tests and couldn't reproduce your issue. Having a failing test case would help.

[Carl Meyer]

Closing as needsinfo.

[Daniel Samuels]

I've just hit this same problem today, here's some example code:

views.py:

class ExampleView(TemplateView):
    template_name = 'example.html'

    def get_context_data(self, **kwargs: Any) -> Dict[str, Any]:
        context = super().get_context_data(**kwargs)

        context['language_code'] = translation.get_language()
        #        ^-- should be pt-BR, but is pt
        return context

test_views.py

def test_example_view(db, client):
    language_code = 'pt-BR'
    resp = client.get(reverse('example'), HTTP_ACCEPT_LANGUAGE=language_code)
    assert resp.context_data['language_code'] == language_code
    # ^-- AssertionError: pt-BR != pt

The code path that's going wrong is:

• LocaleMiddleware.process_request calls translation.get_language_from_request
• get_language_from_request calls parse_accept_lang_header which turns pt-BR ito pt-br
• get_language_from_request then calls get_supported_language_variant, passing pt-br as the lang_code
• get_supported_language_variant then runs if code in supported_lang_codes, which is False (note that 'pt-BR' in supported_lang_codes == True)
• get_supported_language_variant then returns the fallback lang_code pt

[Daniel Samuels]

I've created a simple reproduction using the above example code as a basis. You can find it here: ​https://github.com/danielsamuels/django_23689

[Claude Paroz]

Thanks for the test project!

[Zainab Amir]

Created a fix for it on my branch: ​https://github.com/django/django/compare/main...zainab-amir:ticket_23689
I will create a PR on django if no one has any suggestions or comments.

1. The documentation already mentions that the header should be case sensitive, so I fixed that
2. Also noticed that running django server with these settings throw an error as mentioned below. This is also fixed.

LANGUAGES = (
    ('EN-US', 'English (US)'),
    ('De', 'Deutsche'),
    ('ar', 'عربى'),
)

LANGUAGE_CODE = 'en-us'

ERROR:

ERRORS:
?: (translation.E004) You have provided a value for the LANGUAGE_CODE setting that is not in the LANGUAGES setting.

[Mariusz Felisiak]

I will create a PR on django if no one has any suggestions or comments.

You don't need to wait for an approval. PR is the right place for suggestions and comments.

Also noticed that running django server with these settings throw an error as mentioned below. This is also fixed.

I don't see anything to fix here 🤔

[Zainab Amir]

Replying to Mariusz Felisiak:

Also noticed that running django server with these settings throw an error as mentioned below. This is also fixed.

I don't see anything to fix here 🤔

This is the ripple effect of fixing the header parsing to be case insensitive.

[Zainab Amir]

​PR

[Mariusz Felisiak <felisiak.mariusz@…>]

In 901a1691:

Fixed #23689 -- Made parsing HTTP Accept-Language header case-insensitive.

Thank you Daniel Samuels for test project.

[Mariusz Felisiak <felisiak.mariusz@…>]

In f741dd5:

[4.1.x] Fixed #23689 -- Made parsing HTTP Accept-Language header case-insensitive.

Thank you Daniel Samuels for test project.

Backport of 901a1691982cab76349d33e51b72c40120ec927a from main