select_related "row, fields misalignment" in SQLCompiler.fill_related_selections()

[anonymous]

Given this models for example:

class Place(models.Model):
    name = models.CharField(max_length=50)
    address = models.CharField(max_length=80)

class Restaurant(Place):
    serves_hot_dogs = models.BooleanField()
    serves_pizza = models.BooleanField()

When using select related to prefetch the child, the child fields are not initialized properly:

>>> Restaurant.objects.create(name='Bobs Cafe', address="Somewhere", serves_pizza=True, serves_hot_dogs=True)
<Restaurant: Restaurant object>

>>> p = Place.objects.all().select_related('restaurant')[0]

>>> vars(p.restaurant)
{'_place_ptr_cache': <Place: Place object>,
 '_state': <django.db.models.base.ModelState at 0x2e791d0>,
 'address': u'Somewhere',
 'id': 1L,
 'name': u'Bobs Cafe',
 'place_ptr_id': 1L,
 'serves_hot_dogs': 1,
 'serves_pizza': 1}

serves_pizza and serves_hot_dogs are set to 1 instead of True!
The cause of this bug is kind of tricky. It is actually a general defect because of a "row, fields misalignment" in SQLCompiler.fill_related_selections().
​https://github.com/django/django/blob/master/django/db/models/sql/compiler.py#L650:
self.query.related_select_cols returns:

[SelectInfo(col=(u't13781_restaurant', u'place_ptr_id'), field=<django.db.models.fields.AutoField: id>), 
SelectInfo(col=(u't13781_restaurant', 'serves_hot_dogs'), field=<django.db.models.fields.CharField: name>), 
SelectInfo(col=(u't13781_restaurant', 'serves_pizza'), field=<django.db.models.fields.CharField: address>)]

But it only turns up when using SQL backends that use the misaligned fields for converting the row values depending on the field type (mainly when resolve_columns() is implemented). So for example in MySQL this only happens when boolean fields on the child model are involved, because in this special case the field information is interpreted.
​https://github.com/django/django/blob/master/django/db/backends/mysql/compiler.py#L10:

if (field and field.get_internal_type() in ("BooleanField", "NullBooleanField") and
  value in (0, 1)):
  value = bool(value)

It won't appear using SQLite but i guess it will appear using Oracle (only tested SQLite and MySQL).

It might be somehow related to #21203 and #21126.

[slide21@…]

patch for ticket #21413

[Shai Berger]

I haven't reviewed the patch, just gave the bug a quick look. Replaced explicit links to tickets in the description with ticket references.