Django

Code

Ticket #4664 (closed: fixed)

Opened 1 year ago

Last modified 1 year ago

[unicode] UnicodeDecodeError with on postgresql 7.4 using psycopg1 backend on python 2.3

Reported by: Øyvind Saltvik <oyvind@saltvik.no> Assigned to: mtredinnick
Milestone: Component: Database wrapper
Version: unicode Keywords:
Cc: Triage Stage: Accepted
Has patch: 0 Needs documentation: 0
Needs tests: 0 Patch needs improvement: 0

Description 

Traceback (most recent call last):
File "/var/www/vhosts/3kanten.it/httpdocs/trunk/django/core/handlers/base.py" in get_response
  77. response = callback(request, *callback_args, **callback_kwargs)
File "/var/www/vhosts/3kanten.it/httpdocs/trekanten/kontakt/views.py" in contact
  50. return object_list(request, **kwargs)
File "/var/www/vhosts/3kanten.it/httpdocs/trunk/django/views/generic/list_detail.py" in object_list
  75. }, context_processors)
File "/var/www/vhosts/3kanten.it/httpdocs/trunk/django/template/context.py" in __init__
  100. self.update(processor(request))
File "/var/www/vhosts/3kanten.it/httpdocs/trekanten/meny/context_processors.py" in meny
  9. menyliste = list(Side.objects.filter(i_meny=True)) + list(Element.objects.all())
File "/var/www/vhosts/3kanten.it/httpdocs/trunk/django/db/models/query.py" in __iter__
  109. return iter(self._get_data())
File "/var/www/vhosts/3kanten.it/httpdocs/trunk/django/db/models/query.py" in _get_data
  471. self._result_cache = list(self.iterator())
File "/var/www/vhosts/3kanten.it/httpdocs/trunk/django/db/models/query.py" in iterator
  188. rows = cursor.fetchmany(GET_ITERATOR_CHUNK_SIZE)
File "/var/www/vhosts/3kanten.it/httpdocs/trunk/django/db/backends/postgresql/base.py" in typecast_string
  273. return smart_unicode(s, client_encoding)
File "/var/www/vhosts/3kanten.it/httpdocs/trunk/django/utils/encoding.py" in smart_unicode
  25. return force_unicode(s, encoding, strings_only, errors)
File "/var/www/vhosts/3kanten.it/httpdocs/trunk/django/utils/encoding.py" in force_unicode
  42. s = unicode(s, encoding, errors)

  UnicodeDecodeError at /kontakt/
  'ascii' codec can't decode byte 0xc3 in position 134: ordinal not in range(128)

the variable encoding is ascii on line 42

Attachments

Change History

06/22/07 17:19:50 changed by Øyvind Saltvik <oyvind@saltvik.no>

  • needs_better_patch changed.
  • version changed from SVN to unicode.
  • needs_tests changed.
  • needs_docs changed.

06/22/07 17:34:59 changed by Øyvind Saltvik <oyvind@saltvik.no>

Switching to psycopg2 solved the problem.

06/22/07 21:23:49 changed by mtredinnick

  • owner changed from adrian to mtredinnick.
  • summary changed from unicode: UnicodeDecodeError with on postgresql 7.4 using psycopg1 backend on python 2.3 to [unicode] UnicodeDecodeError with on postgresql 7.4 using psycopg1 backend on python 2.3.
  • stage changed from Unreviewed to Accepted.

So for some reason your default client encoding is ASCII (possibly configured to be non-international friendly in postgresql.conf). You, or any other user, may not be in control of that, so we have to work around. It's going to be annoying to have to try and change this value, though. We don't know what is available on the server-side and things will go badly wrong (Django will crash, most likely) if we get it wrong.

I don't have PostgreSQL 7.4 on any machine at the moment. Can you add a line just before line 97 (at the same indentation level that as line 97) that says 

cursor.execute("SET CLIENT_ENCODING to 'UNICODE'")

So you will have a "set client encoding" followed by "show client encoding". I want to see if the "set" takes effect. If so, we can try to set the encoding to UTF-8 if the client reports ASCII.

(I may have the "UNICODE" string wrong. It isn't listed as an explicit option in table 20.2, here, but it looks like it should be supported. Try a few alternative strings if that doesn't work.)

06/24/07 20:14:45 changed by Øyvind Saltvik <oyvind@saltvik.no>

that solved it

06/25/07 09:40:08 changed by mtredinnick

  • status changed from new to closed.
  • resolution set to fixed.

(In [5535]) Fixed #4664 -- Forced the client character set encoding to UTF-8 for PostgreSQL (via the psycopg backend). The previous version was causing problems on some setups, particularly PostgreSQL 7.x. Current code should work with 7.x and 8.x, no matter what the default client encoding is.

Add/Change #4664 ([unicode] UnicodeDecodeError with on postgresql 7.4 using psycopg1 backend on python 2.3)




Change Properties
Action