Django

Code

Ticket #3115 (closed: fixed)

Opened 2 years ago

Last modified 2 years ago

Postgresql backend should convert Unicode input to bytestrings

Reported by: Manuel Saelices <msaelices@yaco.es> Assigned to: adrian
Milestone: Component: Database layer (models, ORM)
Version: SVN Keywords:
Cc: Triage Stage: Unreviewed
Has patch: 0 Needs documentation: 0
Needs tests: 0 Patch needs improvement: 0

Description

ORM fails in PostgreSQL when you filter with unicode strings. For example:

>>> from django.contrib.auth.models import User
>>> User.objects.filter(username=u'admin')
...
ProgrammingError: ERROR:  column "admin" does not exist

SELECT "auth_user"."id","auth_user"."username","auth_user"."first_name","auth_user"."last_name","auth_user"."email","auth_user"."password","auth_user"."is_staff","auth_user"."is_active","auth_user"."is_superuser","auth_user"."last_login","auth_user"."date_joined" FROM "auth_user" WHERE ("auth_user"."username" = admin) ORDER BY "auth_user"."username" ASC

The problem is on psycopg library, here are an example:

>>> from django.db import connection
>>> cursor = connection.cursor()
>>> cursor.execute('SELECT * from auth_user WHERE username = %s', [u'admin'])
...
ProgrammingError: ERROR:  column "admin" does not exist

SELECT * from auth_user WHERE username = admin

Ok, it's true... it's not a django error, but for example, in newforms all is unicode... ¿what we do? ¿Use psycopg2? ¿convert all unicodes to ascii in lookups? ¿convert all params on postgresql?. Call is made on django/db/models/query.py, on this sentences:

     cursor = connection.cursor()
     select, sql, params = self._get_sql_clause()
     cursor.execute("SELECT " + (self._distinct and "DISTINCT " or "") + ",".join(select) + sql, params)

Attachments

Change History

12/08/06 12:02:47 changed by adrian

Yes, we ought to change the postgresql Django database backend to check whether each string is Unicode before adding it to the statement.

This also ties into another suggestion people have had, which is to have a DATABASE_CHARSET setting. This would describe which character set the database uses.

12/08/06 12:03:11 changed by adrian

  • summary changed from ORM problems with unicode in postgresql backend to Postgresql backend should convert Unicode input to bytestrings.

12/15/06 16:33:53 changed by adrian

See also #952.

12/26/06 23:14:02 changed by adrian

  • status changed from new to closed.
  • resolution set to fixed.

(In [4244]) Fixed #3115 -- Changed postgresql backend to convert all Unicode strings to bytestrings according to DEFAULT_CHARSET. This is necessary because psycopg1 does not apply database quoting to Unicode strings

01/17/07 16:12:17 changed by

  • milestone deleted.

Milestone Version 1.0 deleted


Add/Change #3115 (Postgresql backend should convert Unicode input to bytestrings)




Change Properties
Action