Code

Opened 4 years ago

Last modified 21 months ago

#13965 new Bug

psycopg2 throws an "can't adapt" error on ugettext_lazy translated strings

Reported by: mitar Owned by: nobody
Component: Internationalization Version: 1.2
Severity: Normal Keywords:
Cc: mmitar@…, phartig@…, poswald Triage Stage: Accepted
Has patch: no Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description

I have a custom model field (based on a CharField) for which I am setting a default value to a string I want for it to be translated for each user based on her language settings.

So in definition of my custom field I do something like:

class CustomField(fields.CharField):
  def __init__(self, *args, **kwargs):
    ...
    kwargs.setdefault('default', ugettext_lazy('Default value'))
    ...

    super(fields.CustomField, self).__init__(*args, **kwargs)

  def get_internal_type(self):
    return "CharField"

Sadly this fails when creating a default (empty) object with get_or_create. psycopg2 raises "can't adapt" error where it is visible that proxy object was passed as an argument.

Attachments (0)

Change History (15)

comment:1 Changed 4 years ago by mitar

  • Needs documentation unset
  • Needs tests unset
  • Patch needs improvement unset

The workaround is to use small lambda wrapper (default can be also a callable) with non-lazy ugettext. Or to use a lambda with force_unicode if you are getting lazily translated string from some other code somewhere else (this above is just a simple example).

comment:2 follow-up: Changed 4 years ago by zerok

Could you please provide some more details regarding version numbers and such? I tried to reproduce it using psycopg2 '2.2.2 (dt dec ext pq3)' on PostgreSQL 8.4.4 with following field::

header = CustomField(max_length=255,

verbose_name=_('Header'),
help_text=_("An optional header for this content"))

and it worked for me.

comment:3 in reply to: ↑ 2 Changed 4 years ago by zerok

... using Django 1.2.1 (sorry, hit the wrong button).

comment:4 Changed 4 years ago by mitar

I use Django 1.2.1, psycopg2 2.2.1, Debian 5.0.5. PostgreSQL 8.3.11.

A example session in Python shell:

>>> p = models.UserProfile.objects.get(pk=7)
>>> p.city = 'foobar'
>>> p.save()
>>> from django.utils.translation import ugettext_lazy 
>>> p.city = ugettext_lazy('foobar')
>>> p.save()
Traceback (most recent call last):
  File "<console>", line 1, in <module>
  File "/usr/lib/pymodules/python2.5/django/db/models/base.py", line 435, in save
    self.save_base(using=using, force_insert=force_insert, force_update=force_update)
  File "/usr/lib/pymodules/python2.5/django/db/models/base.py", line 501, in save_base
    rows = manager.using(using).filter(pk=pk_val)._update(values)
  File "/usr/lib/pymodules/python2.5/django/db/models/query.py", line 491, in _update
    return query.get_compiler(self.db).execute_sql(None)
  File "/usr/lib/pymodules/python2.5/django/db/models/sql/compiler.py", line 861, in execute_sql
    cursor = super(SQLUpdateCompiler, self).execute_sql(result_type)
  File "/usr/lib/pymodules/python2.5/django/db/models/sql/compiler.py", line 727, in execute_sql
    cursor.execute(sql, params)
  File "/usr/lib/pymodules/python2.5/django/db/backends/util.py", line 15, in execute
    return self.cursor.execute(sql, params)
  File "/usr/lib/pymodules/python2.5/django/db/backends/postgresql_psycopg2/base.py", line 44, in execute
    return self.cursor.execute(query, args)
DatabaseError: can't adapt type '__proxy__'

The same happens when using a translated string for default value for a field.

comment:5 Changed 4 years ago by mitar

I attached fields I am using to #5446. city field above is CityField.

comment:6 Changed 4 years ago by ch0wn

  • Cc phartig@… added

Same for lxml.etree._ElementStringResult even though it's coercable to bytestrings or unicode. Wrapping the element in unicode() fixes it.
Important to note is, that this works without manual casting with the mysql driver.

comment:7 follow-up: Changed 4 years ago by mk

I'm not sure whether lazy strings should be automatically converted to unicode on save(). I'd say this constitutes silent loss of information.

Btw, this does not work on sqlite either: It fails with an InterfaceError: InterfaceError: Error binding parameter 5 - probably unsupported type.

I'd close this ticket as "wontfix".

comment:8 in reply to: ↑ 7 Changed 4 years ago by mitar

Replying to mk:

I'm not sure whether lazy strings should be automatically converted to unicode on save(). I'd say this constitutes silent loss of information.

Ha? Lazy strings are just that - lazy. They should behave exactly the same as strict strings, only their evaluation is postponed. This behavior should be transparent, semantics should be the same. This should only have influence on performance. So you can change strict into lazy and vice versa without worrying that something will explode.

comment:9 Changed 4 years ago by mk

  • Triage Stage changed from Unreviewed to Design decision needed

No, semantics aren't the same. A lazy i18n string depends on the context and this is lost when the string is stored in a DB. I'll rather be warned (with an explosion, if it has to be) when I try doing something stupid. "Silent data loss" is a strong description for what's happening if a lazy string is translated upon entering storage, but I'd still argue that's what is happening here. And silent data loss is bad(tm).

I'll mark this design decision needed, hopefully someone from core will weigh in on this issue.

comment:10 Changed 4 years ago by mitar

Hm, but in this case this is not a silent data loss because it is stored for concrete user in his context. It is only fixation of his settings at the time of storage. Please read initial description. I have a custom model field with default value which should be populated based on context and that value should be fixed and stored in the database. It is the same thing as I would display to the user values in his language context and then store that value. Would we say that there was a loss of data?

comment:11 Changed 3 years ago by poswald

  • Cc poswald added
  • Severity set to Normal
  • Triage Stage changed from Design decision needed to Accepted
  • Type set to Bug

I expect lazy evaluation to mean: in the context of the user doing the saving. I would expect the postgres backend to evaluate and translate at the time of saving rather than throw an error in an attempt to save me from myself. I believe this is the way the other database back-ends do it, no? It's also not like this is an intentional exception being raised... I think it is because it is trying to write the function name into the SQL which is definitely not right.

The docs say this:

Always use lazy translations in Django models. Field names and table names should be marked for translation (otherwise, they won't be translated in the admin interface).

While they are talking about things like help text and labels here (not values that get stored in the db) it's the reason that people hit this issue: they define _() in their models.py to be lazy evaluation.

The point of lazy evaluation is to delay the string translation until the correct locale is in effect rather then when the model is first loaded up. In the case of saving data like field defaults, I think "lazy" means when the data is saved into the database. You're calling it silent data loss but I think the common expectation would be for lazy to mean "when being serialized" wether that be to html, json, or a database table. For example, I have a multi-lingual user signup where I create defaults when the user signs up. I want those defaults to be saved in the user's language when I save the model. The only time I would want translation to happen to a string being serialized to the db would be on a choices field type and in that case it wouldn't be marked as a translation string.

In short, I don't think anybody expects a lazy string to be translated *after* the data is retrieved from the database again (there's a template tag for that if you wanted it), and I don't think anyone expects it to blow up which is what it is doing. I think this issue is valid. It would be good to write a test case for it. I wonder if it worked in the past versions and is a regression.

comment:12 Changed 2 years ago by aaugustin

  • UI/UX unset

Change UI/UX from NULL to False.

comment:13 Changed 2 years ago by aaugustin

  • Easy pickings unset

Change Easy pickings from NULL to False.

comment:14 Changed 21 months ago by ericholscher

Just ran into this on updating permissions with:

class Meta:

permissions = (

('view_version', _('View Version')),

)

comment:15 Changed 21 months ago by ericholscher

Workaround was to remove the translatable string for View Version, and then run syncdb, and then add it back :/

Add Comment

Modify Ticket

Change Properties
<Author field>
Action
as new
The owner will be changed from nobody to anonymous. Next status will be 'assigned'
as The resolution will be set. Next status will be 'closed'
Author


E-mail address and user name can be saved in the Preferences.

 
Note: See TracTickets for help on using tickets.