Opened 15 years ago

Closed 14 years ago

Last modified 14 years ago

#11615 closed (fixed)

./manage.py test returns incorrect status code if more than 255 bugs fail

Reported by: Chris Lamb Owned by: Gary Wilson
Component: Testing framework Version:
Severity: Keywords:
Cc: Triage Stage: Ready for checkin
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description

The Django test runner currently calls "sys.exit(number_of_failed_tests)" when finished. However, as exit codes under UNIX (etc.) systems are unsigned chars, if n % 256 == 0 bugs fail (where n > 0) the exit code for the program is 0 - ie. incorrectly denoting a "successful" build run.

A more general problem exists where, if n > 256 bugs fail, the exit code is actually modulus 256 the number of failed tests.

A patch is attached which normalises the values to either 0 and 1 on success and failure respectfully - we cannot return the number in this manner, so let's not pretend to.

Attachments (1)

test-runner-exit-status.diff (448 bytes ) - added by Chris Lamb 15 years ago.

Download all attachments as: .zip

Change History (8)

by Chris Lamb, 15 years ago

comment:1 by Alex Gaynor, 15 years ago

Triage Stage: UnreviewedAccepted

comment:2 by Chris Beaven, 15 years ago

Triage Stage: AcceptedReady for checkin

comment:3 by Gary Wilson, 14 years ago

Owner: changed from nobody to Gary Wilson
Status: newassigned

Looks like we need to correct this on the tests/runtests.py runner as well, fix to follow.

comment:4 by Gary Wilson, 14 years ago

Resolution: fixed
Status: assignedclosed

(In [12068]) Fixed #11615 -- Changed test runners to use an exit status code of 1 for any number of failed tests. The previous behavior of using an exit status code equal to the number of failed tests produced incorrect exit status codes when the number of test failures was 256 or greater. Thanks to lamby for the report and patch.

comment:5 by Gary Wilson, 14 years ago

(In [12069]) [1.1.X] Fixed #11615 -- Changed test runners to use an exit status code of 1 for any number of failed tests. The previous behavior of using an exit status code equal to the number of failed tests produced incorrect exit status codes when the number of test failures was 256 or greater. Thanks to lamby for the report and patch.

Backport of r12068 from trunk.

comment:6 by Gary Wilson, 14 years ago

(In [12070]) Added backwards incompatibility note to 1.1.2 release notes for the change in test runner exit status codes (refs #11615).

comment:7 by Gary Wilson, 14 years ago

(In [12071]) [1.1.X] Added backwards incompatibility note to 1.1.2 release notes for the change in test runner exit status codes (refs #11615).

Backport of r12070 from trunk.

Note: See TracTickets for help on using tickets.
Back to Top