UnicodeDecodeError during loaddata when database uses non-default locale and error occurs in database layer

[Emanuel Koczwara]

loaddata raises UnicodeDecodeError when error message from database is not in ascii (database has non-default locale).

Here is the error I get:

(fulnartapp)emanuel@emanuel-desktop /work/fulnart/fulnartapp/fulnartapp $ python manage.py loaddata courses.json 
Traceback (most recent call last):
  File "manage.py", line 10, in <module>
    execute_from_command_line(sys.argv)
  File "/work/fulnart/fulnartapp/lib/python2.7/site-packages/django/core/management/__init__.py", line 385, in execute_from_command_line
    utility.execute()
  File "/work/fulnart/fulnartapp/lib/python2.7/site-packages/django/core/management/__init__.py", line 377, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/work/fulnart/fulnartapp/lib/python2.7/site-packages/django/core/management/base.py", line 288, in run_from_argv
    self.execute(*args, **options.__dict__)
  File "/work/fulnart/fulnartapp/lib/python2.7/site-packages/django/core/management/base.py", line 338, in execute
    output = self.handle(*args, **options)
  File "/work/fulnart/fulnartapp/lib/python2.7/site-packages/django/core/management/commands/loaddata.py", line 61, in handle
    self.loaddata(fixture_labels)
  File "/work/fulnart/fulnartapp/lib/python2.7/site-packages/django/core/management/commands/loaddata.py", line 99, in loaddata
    e.args = ("Problem installing fixtures: %s" % e,)
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc5 in position 1: ordinal not in range(128)

Here is patch for that:

diff --git a/django/core/management/commands/loaddata.py b/django/core/management/commands/loaddata.py
index af108e0..c45f8c1 100644
--- a/django/core/management/commands/loaddata.py
+++ b/django/core/management/commands/loaddata.py
@@ -95,7 +95,7 @@ class Command(BaseCommand):
         try:
             connection.check_constraints(table_names=table_names)
         except Exception as e:
-            e.args = ("Problem installing fixtures: %s" % e,)
+            e.args = ("Problem installing fixtures: %s" % force_text(e),)
             raise
 
         # If we found even one object in a fixture, we need to reset the

And now I can see database error:

(fulnartapp)emanuel@emanuel-desktop /work/fulnart/fulnartapp/fulnartapp $ python manage.py loaddata data.json 
Traceback (most recent call last):
  File "manage.py", line 10, in <module>
    execute_from_command_line(sys.argv)
  File "/work/fulnart/fulnartapp/lib/python2.7/site-packages/django/core/management/__init__.py", line 385, in execute_from_command_line
    utility.execute()
  File "/work/fulnart/fulnartapp/lib/python2.7/site-packages/django/core/management/__init__.py", line 377, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/work/fulnart/fulnartapp/lib/python2.7/site-packages/django/core/management/base.py", line 288, in run_from_argv
    self.execute(*args, **options.__dict__)
  File "/work/fulnart/fulnartapp/lib/python2.7/site-packages/django/core/management/base.py", line 338, in execute
    output = self.handle(*args, **options)
  File "/work/fulnart/fulnartapp/lib/python2.7/site-packages/django/core/management/commands/loaddata.py", line 61, in handle
    self.loaddata(fixture_labels)
  File "/work/fulnart/fulnartapp/lib/python2.7/site-packages/django/core/management/commands/loaddata.py", line 96, in loaddata
    connection.check_constraints(table_names=table_names)
  File "/work/fulnart/fulnartapp/lib/python2.7/site-packages/django/db/backends/postgresql_psycopg2/base.py", line 205, in check_constraints
    self.cursor().execute('SET CONSTRAINTS ALL IMMEDIATE')
  File "/work/fulnart/fulnartapp/lib/python2.7/site-packages/django/db/backends/utils.py", line 81, in execute
    return super(CursorDebugWrapper, self).execute(sql, params)
  File "/work/fulnart/fulnartapp/lib/python2.7/site-packages/django/db/backends/utils.py", line 65, in execute
    return self.cursor.execute(sql, params)
  File "/work/fulnart/fulnartapp/lib/python2.7/site-packages/django/db/utils.py", line 94, in __exit__
    six.reraise(dj_exc_type, dj_exc_value, traceback)
  File "/work/fulnart/fulnartapp/lib/python2.7/site-packages/django/db/backends/utils.py", line 63, in execute
    return self.cursor.execute(sql)
django.db.utils.IntegrityError: BŁĄD:  wstawianie lub modyfikacja na tabeli "invoices_extraitem" narusza klucz obcy "invoices_ext_invoice_id_3e907289ee08ea24_fk_invoices_invoice_id"
DETAIL:  Klucz (invoice_id)=(2) nie występuje w tabeli "invoices_invoice".

[Claude Paroz]

Duplicate of #20572