Test suite failing under PostgresSQL < 10 due to missing support for ICU collations

[Hannes Ljungberg]

When running the test suite on PostgresSQL versions below 10 it fails with the following error message when creating the test database:

Creating test database for alias 'default'...
Traceback (most recent call last):
  File "/tests/django/django/db/backends/utils.py", line 82, in _execute
    return self.cursor.execute(sql)
psycopg2.errors.UndefinedObject: collation "sv-x-icu" for encoding "UTF8" does not exist
LINE 1: ...al NOT NULL PRIMARY KEY, "char_field" varchar(10) COLLATE "s...
                                                             ^


The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "tests/runtests.py", line 589, in <module>
    options.timing,
  File "tests/runtests.py", line 320, in django_tests
    extra_tests=extra_tests,
  File "/tests/django/django/test/runner.py", line 717, in run_tests
    old_config = self.setup_databases(aliases=databases)
  File "/tests/django/django/test/runner.py", line 637, in setup_databases
    debug_sql=self.debug_sql, parallel=self.parallel, **kwargs
  File "/tests/django/django/test/utils.py", line 183, in setup_databases
    serialize=connection.settings_dict['TEST'].get('SERIALIZE', True),
  File "/tests/django/django/db/backends/base/creation.py", line 77, in create_test_db
    run_syncdb=True,
  File "/tests/django/django/core/management/__init__.py", line 172, in call_command
    return command.execute(*args, **defaults)
  File "/tests/django/django/core/management/base.py", line 394, in execute
    output = self.handle(*args, **options)
  File "/tests/django/django/core/management/base.py", line 89, in wrapped
    res = handle_func(*args, **kwargs)
  File "/tests/django/django/core/management/commands/migrate.py", line 214, in handle
    self.sync_apps(connection, executor.loader.unmigrated_apps)
  File "/tests/django/django/core/management/commands/migrate.py", line 349, in sync_apps
    editor.create_model(model)
  File "/tests/django/django/db/backends/base/schema.py", line 329, in create_model
    self.execute(sql, params or None)
  File "/tests/django/django/db/backends/base/schema.py", line 143, in execute
    cursor.execute(sql, params)
  File "/tests/django/django/db/backends/utils.py", line 66, in execute
    return self._execute_with_wrappers(sql, params, many=False, executor=self._execute)
  File "/tests/django/django/db/backends/utils.py", line 75, in _execute_with_wrappers
    return executor(sql, params, many, context)
  File "/tests/django/django/db/backends/utils.py", line 84, in _execute
    return self.cursor.execute(sql, params)
  File "/tests/django/django/db/utils.py", line 90, in __exit__
    raise dj_exc_value.with_traceback(traceback) from exc_value
  File "/tests/django/django/db/backends/utils.py", line 82, in _execute
    return self.cursor.execute(sql)
django.db.utils.ProgrammingError: collation "sv-x-icu" for encoding "UTF8" does not exist
LINE 1: ...al NOT NULL PRIMARY KEY, "char_field" varchar(10) COLLATE "s...

It feels like something that could've been introduced by ​https://github.com/django/django/pull/13207 and more specifically ​https://github.com/django/django/pull/13207/files#diff-9385ef62a9e10556c6768c66e8f6e2ebR62. Awesome stuff BTW!

Support for ICU collations was introduced in PostgreSQL 10 ​https://wiki.postgresql.org/wiki/New_in_postgres_10#ICU_Collation_Support

Not really sure how to approach this. I see three different solutions:

• Use non ICU collations for PostgreSQL but I'm pretty sure that it won't be possible to test case-insensitive collations in that case.
• Fallback to non ICU collations on versions below 10 and skip tests that explicitly test features provided by ICU collations
• Limit usage of collations on fields to version 10 and above with supports_collation_on_*.

I created an issue on the django/django-docker-box repo before I came to the conclusion that it wasn't an configuration error: ​https://github.com/django/django-docker-box/issues/21

[Mariusz Felisiak]

Thanks for the report.

Use non ICU collations for PostgreSQL but I'm pretty sure that it won't be possible to test case-insensitive collations in that case.
Fallback to non ICU collations on versions below 10 and skip tests that explicitly test features provided by ICU collations

We cannot control which locales are installed locally, so non ICU collations are not an option. I think we can skip these tests on PostgreSQL < 10.

[Mariusz Felisiak]

​PR

[GitHub <noreply@…>]

In 999cddd5:

Fixed #32073 -- Skipped collation tests on PostgreSQL < 10.

PostgreSQL < 10 doesn't support ICU collations.

Thanks Hannes Ljungberg for the report.