Opened 6 years ago

Closed 6 years ago

Last modified 6 years ago

#11615 closed (fixed)

./manage.py test returns incorrect status code if more than 255 bugs fail

Reported by: lamby Owned by: gwilson
Component: Testing framework Version:
Severity: Keywords:
Cc: Triage Stage: Ready for checkin
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: UI/UX:

Description

The Django test runner currently calls "sys.exit(number_of_failed_tests)" when finished. However, as exit codes under UNIX (etc.) systems are unsigned chars, if n % 256 == 0 bugs fail (where n > 0) the exit code for the program is 0 - ie. incorrectly denoting a "successful" build run.

A more general problem exists where, if n > 256 bugs fail, the exit code is actually modulus 256 the number of failed tests.

A patch is attached which normalises the values to either 0 and 1 on success and failure respectfully - we cannot return the number in this manner, so let's not pretend to.

Attachments (1)

test-runner-exit-status.diff (448 bytes) - added by lamby 6 years ago.

Download all attachments as: .zip

Change History (8)

Changed 6 years ago by lamby

comment:1 Changed 6 years ago by Alex

  • Needs documentation unset
  • Needs tests unset
  • Patch needs improvement unset
  • Triage Stage changed from Unreviewed to Accepted

comment:2 Changed 6 years ago by SmileyChris

  • Triage Stage changed from Accepted to Ready for checkin

comment:3 Changed 6 years ago by gwilson

  • Owner changed from nobody to gwilson
  • Status changed from new to assigned

Looks like we need to correct this on the tests/runtests.py runner as well, fix to follow.

comment:4 Changed 6 years ago by gwilson

  • Resolution set to fixed
  • Status changed from assigned to closed

(In [12068]) Fixed #11615 -- Changed test runners to use an exit status code of 1 for any number of failed tests. The previous behavior of using an exit status code equal to the number of failed tests produced incorrect exit status codes when the number of test failures was 256 or greater. Thanks to lamby for the report and patch.

comment:5 Changed 6 years ago by gwilson

(In [12069]) [1.1.X] Fixed #11615 -- Changed test runners to use an exit status code of 1 for any number of failed tests. The previous behavior of using an exit status code equal to the number of failed tests produced incorrect exit status codes when the number of test failures was 256 or greater. Thanks to lamby for the report and patch.

Backport of r12068 from trunk.

comment:6 Changed 6 years ago by gwilson

(In [12070]) Added backwards incompatibility note to 1.1.2 release notes for the change in test runner exit status codes (refs #11615).

comment:7 Changed 6 years ago by gwilson

(In [12071]) [1.1.X] Added backwards incompatibility note to 1.1.2 release notes for the change in test runner exit status codes (refs #11615).

Backport of r12070 from trunk.

Note: See TracTickets for help on using tickets.
Back to Top