Opened 18 months ago

Closed 18 months ago

Last modified 18 months ago

#20963 closed Bug (invalid)

order_by interaction with "brew" distribution of Python 2.7.5 introduces edge-case regression

Reported by: ptone Owned by: nobody
Component: Database layer (models, ORM) Version: 1.6-beta-1
Severity: Normal Keywords:
Cc: Triage Stage: Accepted
Has patch: no Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description (last modified by ptone)

This commit:

https://github.com/django/django/commit/69597e5bcc89aadafd1b76abf7efab30ee0b8b1a

introduced a change that interacts with the OS X specific "brew" installation of Python 2.7.5 to result in a regression in one test.

FAIL: test_tickets_2076_7256 (regressiontests.queries.tests.Queries1Tests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/admin/Desktop/django/tests/regressiontests/queries/tests.py", line 437, in test_tickets_2076_7256
    ['<Item: one>', '<Item: two>', '<Item: one>', '<Item: two>', '<Item: four>']
  File "/Users/admin/Desktop/django/django/test/testcases.py", line 776, in assertQuerysetEqual
    return self.assertEqual(list(items), values)
AssertionError: Lists differ: [] != [u'<Item: one>', u'<Item: two>...
 
Second list contains 5 additional elements.
First extra element 0:
<Item: one>
 
- []
+ [u'<Item: one>',
+  u'<Item: two>',
+  u'<Item: one>',
+  u'<Item: two>',
+  u'<Item: four>']
 
----------------------------------------------------------------------
Ran 1 test in 0.033s
 
FAILED (failures=1)

This is limited only to this patched distribution of Python. Unfortunately this is a rather common version to be running on developer machines.

This gist shows this error on a clean machine with the brew version of 2.7.5, and the python.org binary install version of 2.7.5

https://gist.github.com/ptone/0764601b524fb87a690d

The crux of the issue can be seen in this PDB session started from the failing test:

(Pdb) Item.objects.filter(tags__isnull=False)
[<Item: four>, <Item: one>, <Item: one>, <Item: two>, <Item: two>]
(Pdb) Item.objects.filter(tags__isnull=False).order_by('tags')
[]

Change History (12)

comment:1 Changed 18 months ago by ptone

  • Description modified (diff)

comment:2 Changed 18 months ago by ptone

I'm inclined to not consider this a blocker, as it shouldn't effect production systems, and we can alert the homebrew project to fix their version.

It would be nice to track down what part of their patch is causing this, but that is going to be outside my abilities.

But this could lead to some unfortunate head scratching by developers, and may warrant a warning in the release notes?

comment:3 Changed 18 months ago by ptone

ref #20964, another brew related issue

comment:4 Changed 18 months ago by loic84

For the record, I think there might be similarities between the query from this test and the scenario described by @akaariai in https://code.djangoproject.com/ticket/20842#comment:3.

comment:5 Changed 18 months ago by timo

  • Triage Stage changed from Unreviewed to Accepted

comment:6 Changed 18 months ago by aaugustin

FWIW my inclination is to close this kind platform-specific edge-cases as wontfix. Homebrew is especially unreliable; its users should be aware that that they're living on the edge.

comment:7 Changed 18 months ago by akaariai

It would be interesting to see what is actually happening here. Getting empty list seems very strange. It doesn't look to be the same thing as discussed in #20842. What does the query look like, that is what does str(qs.query) return for the failing case?

Unfortunately I don't have access to the brew version of Python, so I can't test this. It seems likely that this is brew version doing something weird, and in that case I don't think it is Django's responsibility to fix this.

comment:8 Changed 18 months ago by loic84

Model: tests.queries.models.Item

-- Item.objects.filter(tags__isnull=False).order_by('tags').query

SELECT "queries_item"."id", "queries_item"."name", "queries_item"."created", "queries_item"."modified", "queries_item"."creator_id", "queries_item"."note_id" FROM "queries_item" INNER JOIN "queries_item_tags" ON ( "queries_item"."id" = "queries_item_tags"."item_id" ) INNER JOIN "queries_tag" ON ( "queries_item_tags"."tag_id" = "queries_tag"."id" ) WHERE "queries_item_tags"."tag_id" IS NOT NULL ORDER BY "queries_tag"."name" ASC

-- Item.objects.filter(tags__isnull=False).query

SELECT "queries_item"."id", "queries_item"."name", "queries_item"."created", "queries_item"."modified", "queries_item"."creator_id", "queries_item"."note_id" FROM "queries_item" INNER JOIN "queries_item_tags" ON ( "queries_item"."id" = "queries_item_tags"."item_id" ) INNER JOIN "queries_note" ON ( "queries_item"."note_id" = "queries_note"."id" ) WHERE "queries_item_tags"."tag_id" IS NOT NULL ORDER BY "queries_note"."note" DESC, "queries_item"."name" ASC
Version 0, edited 18 months ago by loic84 (next)

comment:9 Changed 18 months ago by akaariai

  • Resolution set to invalid
  • Status changed from new to closed

loic84 verified that the issue is Brew's SQLite. It doesn't return correct results for the .order_by('tags') query. The second INNER JOIN is removing rows that shouldn't be removed. The same test passes on PostgreSQL and MySQL.

I don't think it is a good idea to have release notes about bugs in Brew's SQLite. So, closing as invalid.

comment:10 Changed 18 months ago by ptone

I don't disagree with closing the ticket - hopefully this won't bite anyone too badly on their dev setup - as it will be a pain to track down the issue. Hopefully at least they will find this ticket which documents the issue.

comment:11 Changed 18 months ago by ptone

For cross reference, opened this issue on the HomeBrew project

https://github.com/mxcl/homebrew/issues/22134

comment:12 Changed 18 months ago by github@…

Homebrew is especially unreliable; its users should be aware that that they're living on the edge.

Seems solved by going even closer to the edge: SQLite 3.8.0 seems to fix this (you need to brew update, brew upgrade sqlite and brew reinstall python.

Note: See TracTickets for help on using tickets.
Back to Top