Code

Opened 6 years ago

Closed 6 years ago

Last modified 3 years ago

#7786 closed (fixed)

queries test failure on Python 2.3

Reported by: Karen Tracey <kmtracey@…> Owned by:
Component: Database layer (models, ORM) Version: master
Severity: Keywords:
Cc: Triage Stage: Accepted
Has patch: no Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: UI/UX:

Description

Running the test suite on Python 2.3, I get a test failure in the queries regression test:

D:\u\kmt\django\trunk\tests>d:\bin\Python2.3.5\python.exe runtests.py --settings=testsettings queries
======================================================================
FAIL: Doctest: regressiontests.queries.models.__test__.API_TESTS
----------------------------------------------------------------------
Traceback (most recent call last):
  File "d:\u\kmt\django\trunk\django\test\_doctest.py", line 2180, in runTest
    raise self.failureException(self.format_failure(new.getvalue()))
AssertionError: Failed doctest test for regressiontests.queries.models.__test__.API_TESTS
  File "D:\u\kmt\django\trunk\tests\regressiontests\queries\models.py", line unknown line number, in API_TESTS

----------------------------------------------------------------------
File "D:\u\kmt\django\trunk\tests\regressiontests\queries\models.py", line ?, in regressiontests.queries.models.__test__.API_TESTS
Failed example:
    LoopX.objects.all()
Expected:
    Traceback (most recent call last):
    ...
    FieldError: Infinite loop caused by ordering.
Got:
    []
----------------------------------------------------------------------
File "D:\u\kmt\django\trunk\tests\regressiontests\queries\models.py", line ?, in regressiontests.queries.models.__test__.API_TESTS
Failed example:
    LoopZ.objects.all()
Expected:
    Traceback (most recent call last):
    ...
    FieldError: Infinite loop caused by ordering.
Got:
    []


----------------------------------------------------------------------
Ran 1 test in 0.591s

FAILED (failures=1)

I believe it's caused by this Python bug: http://bugs.python.org/issue1242657 which caused exceptions raised by __len__ to be silently swallowed. A traceback from when the exception is successfully raised:

>>> LoopX.objects.all()
Traceback (most recent call last):
  File "<console>", line 1, in <module>
  File "d:\u\kmt\django\trunk\django\db\models\query.py", line 143, in __repr__
    return repr(list(self))
  File "d:\u\kmt\django\trunk\django\db\models\query.py", line 155, in __len__
    self._result_cache.extend(list(self._iter))
  File "d:\u\kmt\django\trunk\django\db\models\query.py", line 268, in iterator
    for row in self.query.results_iter():
  File "d:\u\kmt\django\trunk\django\db\models\sql\query.py", line 204, in results_iter
    for rows in self.execute_sql(MULTI):
  File "d:\u\kmt\django\trunk\django\db\models\sql\query.py", line 1598, in execute_sql
    sql, params = self.as_sql()
  File "d:\u\kmt\django\trunk\django\db\models\sql\query.py", line 251, in as_sql
    ordering = self.get_ordering()
  File "d:\u\kmt\django\trunk\django\db\models\sql\query.py", line 610, in get_ordering
    self.model._meta, default_order=asc):
  File "d:\u\kmt\django\trunk\django\db\models\sql\query.py", line 663, in find_ordering_name
    order, already_seen))
  File "d:\u\kmt\django\trunk\django\db\models\sql\query.py", line 663, in find_ordering_name
    order, already_seen))
  File "d:\u\kmt\django\trunk\django\db\models\sql\query.py", line 657, in find_ordering_name
    raise FieldError('Infinite loop caused by ordering.')
FieldError: Infinite loop caused by ordering.

shows how __len__ swallowing exceptions would cause this test failure.

Don't know that it is worth fixing (apparently you'll just get empty lists instead of exceptions in cases where this is hit) but figured it was at least worth reporting in case anyone else runs across it. (I didn't find any likely match in my tracker search.)

Attachments (0)

Change History (14)

comment:1 follow-up: Changed 6 years ago by mtredinnick

  • Needs documentation unset
  • Needs tests unset
  • Owner changed from nobody to mtredinnick
  • Patch needs improvement unset
  • Triage Stage changed from Unreviewed to Accepted

Argh! Another __len__-related problem. I'm beginning to really hate its error handling (although, to be fair, if you did something bad in __len__ in, say, Python 1.5, the wheels really fell off the wagon, so the current safe behaviour is probably preferable).

I'll try to fix this. It's actually important.

comment:2 in reply to: ↑ 1 ; follow-up: Changed 6 years ago by Karen Tracey <kmtracey@…>

Replying to mtredinnick:

I'll try to fix this. It's actually important.

Python 2.3 is still that important, then? I know the answer was a resounding 'yes' when someone asked on one of the lists about 8 months ago...but that was 8 months ago. The problem only occurs on 2.3, not 2.4 or 2.5.

comment:3 Changed 6 years ago by mtredinnick

Yep. Python 2.3 is fully supported in Django 1.0.

comment:4 Changed 6 years ago by mtredinnick

Looks like there's not really a lot we can do about this. Arranging to not do anything in __len__ that might cause an exception is something I tried to do months ago for other reasons and the performance effects were terrible. So penalising python 2.4 and 2.5 for what is primarily a diagnostic message in this case isn't really worth it. Django works correctly if you don't make an error in Python 2.3 and I can live with some erroneous cases perform differently than they do in 2.4 and 2.5.

I've adjusted the test suite to not fail on those tests in 2.3, but that's as far as I'm going to go here. Not entirely satisfying, but we're fighting Python bugs and the way list() and friends are implemented internally in Python.

comment:5 Changed 6 years ago by mtredinnick

  • Resolution set to fixed
  • Status changed from new to closed

(In [7939]) Fixed #7786 -- Removed some tests from running when using Python 2.3.

The problem being "hidden" here is not serious. It won't affect correct code
and only gives a different failure mode for incorrect code. The moral is: don't
write incorrect code.

comment:6 in reply to: ↑ 2 ; follow-up: Changed 6 years ago by russellm

Replying to Karen Tracey <kmtracey@gmail.com>:

Replying to mtredinnick:

I'll try to fix this. It's actually important.

Python 2.3 is still that important, then? I know the answer was a resounding 'yes' when someone asked on one of the lists about 8 months ago...but that was 8 months ago. The problem only occurs on 2.3, not 2.4 or 2.5.

Elaborating on Malcolm's comments: Yes, Django 1.0 will support Python 2.3 - for two reasons:

  • RHEL4, Debian Sarge and OS X Tiger all provide Python 2.3. These are not entirely uncommon platform amongst conservative hosting providers (or older desktop installs)
  • Supporting Python implementations other than CPython (i.e., Jython, PyPy) is on the list for v1.0. Keeping the code compatible with CPython 2.3 is the easiest way to maintain support for these platforms.

comment:7 in reply to: ↑ 6 Changed 6 years ago by Karen Tracey <kmtracey@…>

Replying to russellm:

Thanks for the clarification. I asked mainly to make sure I wasn't wasting people's time reporting 2.3 problems; I thought it possible things had changed since the issue was raised on one of the lists more than six months ago, but I guess not. So, I'll keep testing on it and report problems as I find them (this and the other I opened recently are the only ones I hit on the current codebase).

comment:8 follow-up: Changed 6 years ago by Karen Tracey <kmtracey@…>

FYI this Python behavior of swallowing exceptions raised by __len__ has returned in the current (Beta 2) 2.6 release. I haven't tracked down yet whether that is intentional or an oversight on Python's part.

comment:9 in reply to: ↑ 8 ; follow-up: Changed 6 years ago by jcrocholl

  • Resolution fixed deleted
  • Status changed from closed to reopened

Replying to Karen Tracey <kmtracey@gmail.com>:

FYI this Python behavior of swallowing exceptions raised by __len__ has returned in the current (Beta 2) 2.6 release. I haven't tracked down yet whether that is intentional or an oversight on Python's part.

The problem also exists in Python 2.6beta3.

comment:10 in reply to: ↑ 9 Changed 6 years ago by kmtracey

Replying to jcrocholl:

The problem also exists in Python 2.6beta3.

Yes, the Python bug cited in the description has been re-opened for 2.6/3.1 (not sure why that's not 3.0, but the bug tracker has 3.1) but not fixed as of yet. Not sure at this point if Django should skip this test when running on 2.6 (as it does for 2.3) or leave things as-is in the hope that Python will re-fix bug #1242657 before 2.6 is actually released.

comment:11 Changed 6 years ago by mtredinnick

This behaviour isn't showstopper-critical. If that feature of Django stopped working, you wouldn't receive a particular error when you made a mistake. That means only people doing bad things will be harmed and they shouldn't be doing bad things in the first place. So I'm happy to just not run that test on Python 2.6, at least for now.

comment:12 Changed 6 years ago by mtredinnick

  • milestone set to 1.0
  • Owner mtredinnick deleted
  • Status changed from reopened to new

comment:13 Changed 6 years ago by mtredinnick

  • Resolution set to fixed
  • Status changed from new to closed

(In [8570]) Hid a few QuerySet regression tests from Python 2.6 due to a bug in the the
Python beta releases. Failures there mean that incorrect code won't raise an
error, but it's otherwise harmless (correct code still runs correctly).

Fixed #7786.

comment:14 Changed 3 years ago by jacob

  • milestone 1.0 deleted

Milestone 1.0 deleted

Add Comment

Modify Ticket

Change Properties
<Author field>
Action
as closed
as The resolution will be set. Next status will be 'closed'
The resolution will be deleted. Next status will be 'new'
Author


E-mail address and user name can be saved in the Preferences.

 
Note: See TracTickets for help on using tickets.