Opened 7 years ago

Closed 7 years ago

Last modified 7 years ago

#11615 closed (fixed)

./manage.py test returns incorrect status code if more than 255 bugs fail

Reported by: Chris Lamb Owned by: Gary Wilson
Component: Testing framework Version:
Severity: Keywords:
Cc: Triage Stage: Ready for checkin
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: UI/UX:

Description

The Django test runner currently calls "sys.exit(number_of_failed_tests)" when finished. However, as exit codes under UNIX (etc.) systems are unsigned chars, if n % 256 == 0 bugs fail (where n > 0) the exit code for the program is 0 - ie. incorrectly denoting a "successful" build run.

A more general problem exists where, if n > 256 bugs fail, the exit code is actually modulus 256 the number of failed tests.

A patch is attached which normalises the values to either 0 and 1 on success and failure respectfully - we cannot return the number in this manner, so let's not pretend to.

Attachments (1)

test-runner-exit-status.diff (448 bytes) - added by Chris Lamb 7 years ago.

Download all attachments as: .zip

Change History (8)

Changed 7 years ago by Chris Lamb

comment:1 Changed 7 years ago by Alex Gaynor

Needs documentation: unset
Needs tests: unset
Patch needs improvement: unset
Triage Stage: UnreviewedAccepted

comment:2 Changed 7 years ago by Chris Beaven

Triage Stage: AcceptedReady for checkin

comment:3 Changed 7 years ago by Gary Wilson

Owner: changed from nobody to Gary Wilson
Status: newassigned

Looks like we need to correct this on the tests/runtests.py runner as well, fix to follow.

comment:4 Changed 7 years ago by Gary Wilson

Resolution: fixed
Status: assignedclosed

(In [12068]) Fixed #11615 -- Changed test runners to use an exit status code of 1 for any number of failed tests. The previous behavior of using an exit status code equal to the number of failed tests produced incorrect exit status codes when the number of test failures was 256 or greater. Thanks to lamby for the report and patch.

comment:5 Changed 7 years ago by Gary Wilson

(In [12069]) [1.1.X] Fixed #11615 -- Changed test runners to use an exit status code of 1 for any number of failed tests. The previous behavior of using an exit status code equal to the number of failed tests produced incorrect exit status codes when the number of test failures was 256 or greater. Thanks to lamby for the report and patch.

Backport of r12068 from trunk.

comment:6 Changed 7 years ago by Gary Wilson

(In [12070]) Added backwards incompatibility note to 1.1.2 release notes for the change in test runner exit status codes (refs #11615).

comment:7 Changed 7 years ago by Gary Wilson

(In [12071]) [1.1.X] Added backwards incompatibility note to 1.1.2 release notes for the change in test runner exit status codes (refs #11615).

Backport of r12070 from trunk.

Note: See TracTickets for help on using tickets.
Back to Top