Potential data corruption issue with Oracle and Mysql due to SQLCompiler.resolve_columns row, fields misalignment when using defer and select_related

[Michael Manfre]

SQLCompiler.resolve_columns fields, values are misaligned with defer and select_related. Same data corruption issue as reported/fixed by issue #21126.

Given:

class Parent(models.Model):
	parentid = models.AutoField(primary_key=True)
	parent_bool = models.BooleanField(default=True)
	created = models.DateTimeField(auto_now=True)

class Child(models.Model):
	childid = models.AutoField(primary_key=True)
	parent = models.ForeignKey(Parent)

The following query

p = Parent.objects.create(parent_bool=True)
c = Child.objects.create(parent=p)

qs = Child.objects.all().select_related('parent').defer('parent__parent_bool')

will result in this field misalignment

        field=<django.db.models.fields.AutoField: childid>      value=1
        field=<django.db.models.fields.related.ForeignKey: parent>      value=1
        field=<django.db.models.fields.AutoField: parentid>     value=1
        field=<django.db.models.fields.BooleanField: parent_bool>       value=datetime.datetime(2013, 9, 30, 19, 59, 41, 490000, tzinfo=<UTC>)
        field=<django.db.models.fields.DateTimeField: created>  value=None

parent_bool should have been removed from fields, but it was not because results_iter doesn't properly compare the only_load fields to the correct db_table¹.

¹ ​https://github.com/django/django/blob/master/django/db/models/sql/compiler.py#L716

[Anssi Kääriäinen]

Pull request ​https://github.com/django/django/pull/1698 should solve this ticket.

The basic problem was that the only_load dictionary wasn't used correctly.

Unfortunately the only_load dictionary also contains wrong data in some complex situations, but that is not this ticket's problem. I will create another ticket for that issue.

[Michael Manfre]

The pull request fixes the simple example I included above and the issue as experienced in my full project.

Here's a pull request for the backpatch to 1.5.x :)
​​https://github.com/django/django/pull/1701

(ignore previous edit, venv was corrupted and including an unpatched django from a different venv)

[Anssi Kääriäinen <akaariai@…>]

In 9972a101e60d1c4bf1e0b6f3c7946c8716022711:

[1.6.x] Fixed #21203 -- resolve_columns fields misalignment

In queries using .defer() together with .select_related() the values
and fields arguments didn't align properly for resolve_columns().

Backpatch of bf13c75c0d94d606b8a077ff73bbd0440f05b396 from master.

[Anssi Kääriäinen <akaariai@…>]

In 7ebd10019d5c6c173825a41c5d435c80bee7b82a:

[1.5.x] Fixed #21203 -- resolve_columns fields misalignment

In queries using .defer() together with .select_related() the values
and fields arguments didn't align properly for resolve_columns().

Backpatch of 8c27247397cf16b17d0153ae059593c5a468de01 from master.