order_by interaction with "brew" distribution of Python 2.7.5 introduces edge-case regression

[Preston Holmes]

This commit:

​https://github.com/django/django/commit/69597e5bcc89aadafd1b76abf7efab30ee0b8b1a

introduced a change that interacts with the OS X specific "brew" installation of Python 2.7.5 to result in a regression in one test.

FAIL: test_tickets_2076_7256 (regressiontests.queries.tests.Queries1Tests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/admin/Desktop/django/tests/regressiontests/queries/tests.py", line 437, in test_tickets_2076_7256
    ['<Item: one>', '<Item: two>', '<Item: one>', '<Item: two>', '<Item: four>']
  File "/Users/admin/Desktop/django/django/test/testcases.py", line 776, in assertQuerysetEqual
    return self.assertEqual(list(items), values)
AssertionError: Lists differ: [] != [u'<Item: one>', u'<Item: two>...
 
Second list contains 5 additional elements.
First extra element 0:
<Item: one>
 
- []
+ [u'<Item: one>',
+  u'<Item: two>',
+  u'<Item: one>',
+  u'<Item: two>',
+  u'<Item: four>']
 
----------------------------------------------------------------------
Ran 1 test in 0.033s
 
FAILED (failures=1)

This is limited only to this patched distribution of Python. Unfortunately this is a rather common version to be running on developer machines.

This gist shows this error on a clean machine with the brew version of 2.7.5, and the python.org binary install version of 2.7.5

​https://gist.github.com/ptone/0764601b524fb87a690d

The crux of the issue can be seen in this PDB session started from the failing test:

(Pdb) Item.objects.filter(tags__isnull=False)
[<Item: four>, <Item: one>, <Item: one>, <Item: two>, <Item: two>]
(Pdb) Item.objects.filter(tags__isnull=False).order_by('tags')
[]

[Preston Holmes]

I'm inclined to not consider this a blocker, as it shouldn't effect production systems, and we can alert the homebrew project to fix their version.

It would be nice to track down what part of their patch is causing this, but that is going to be outside my abilities.

But this could lead to some unfortunate head scratching by developers, and may warrant a warning in the release notes?

[Preston Holmes]

ref #20964, another brew related issue

[loic84]

For the record, I think there might be similarities between the query from this test and the scenario described by @akaariai in https://code.djangoproject.com/ticket/20842#comment:3.

[Aymeric Augustin]

FWIW my inclination is to close this kind platform-specific edge-cases as wontfix. Homebrew is especially unreliable; its users should be aware that that they're living on the edge.

[Anssi Kääriäinen]

It would be interesting to see what is actually happening here. Getting empty list seems very strange. It doesn't look to be the same thing as discussed in #20842. What does the query look like, that is what does str(qs.query) return for the failing case?

Unfortunately I don't have access to the brew version of Python, so I can't test this. It seems likely that this is brew version doing something weird, and in that case I don't think it is Django's responsibility to fix this.

[loic84]

Model: tests.queries.models.Item.

>>> print(Item.objects.filter(tags__isnull=False).order_by('tags').query)
SELECT "queries_item"."id", "queries_item"."name", "queries_item"."created", "queries_item"."modified", "queries_item"."creator_id", "queries_item"."note_id" FROM "queries_item" INNER JOIN "queries_item_tags" ON ( "queries_item"."id" = "queries_item_tags"."item_id" ) INNER JOIN "queries_tag" ON ( "queries_item_tags"."tag_id" = "queries_tag"."id" ) WHERE "queries_item_tags"."tag_id" IS NOT NULL ORDER BY "queries_tag"."name" ASC
>>> print len(Item.objects.filter(tags__isnull=False).order_by('tags'))
0
>>> print Item.objects.filter(tags__isnull=False).order_by('tags').count()
5

>>> print Item.objects.filter(tags__isnull=False).query
SELECT "queries_item"."id", "queries_item"."name", "queries_item"."created", "queries_item"."modified", "queries_item"."creator_id", "queries_item"."note_id" FROM "queries_item" INNER JOIN "queries_item_tags" ON ( "queries_item"."id" = "queries_item_tags"."item_id" ) INNER JOIN "queries_note" ON ( "queries_item"."note_id" = "queries_note"."id" ) WHERE "queries_item_tags"."tag_id" IS NOT NULL ORDER BY "queries_note"."note" DESC, "queries_item"."name" ASC
>>> print len(Item.objects.filter(tags__isnull=False))
5
>>> print Item.objects.filter(tags__isnull=False).count()
5

[Anssi Kääriäinen]

loic84 verified that the issue is Brew's SQLite. It doesn't return correct results for the .order_by('tags') query. The second INNER JOIN is removing rows that shouldn't be removed. The same test passes on PostgreSQL and MySQL.

I don't think it is a good idea to have release notes about bugs in Brew's SQLite. So, closing as invalid.

[Preston Holmes]

I don't disagree with closing the ticket - hopefully this won't bite anyone too badly on their dev setup - as it will be a pain to track down the issue. Hopefully at least they will find this ticket which documents the issue.

[Preston Holmes]

For cross reference, opened this issue on the HomeBrew project

​https://github.com/mxcl/homebrew/issues/22134

[github@…]

Homebrew is especially unreliable; its users should be aware that that they're living on the edge.

Seems solved by going even closer to the edge: SQLite 3.8.0 seems to fix this (you need to brew update, brew upgrade sqlite and brew reinstall python.