#14427 closed (fixed)
Add options to runtests.py to discover test pairings that fail
| Reported by: | Russell Keith-Magee | Owned by: | nobody |
|---|---|---|---|
| Component: | Testing framework | Version: | 1.2 |
| Severity: | Keywords: | ||
| Cc: | Triage Stage: | Design decision needed | |
| Has patch: | no | Needs documentation: | no |
| Needs tests: | no | Patch needs improvement: | no |
| Easy pickings: | no | UI/UX: | no |
Description
Although unittests and doctests are supposed to be self contained, sometimes they have leaky side effects. For example, if a test case activates a language setting, but doesn't deactivate it, subsequent tests will be run in the wrong locale, which can cause tests to fail that pass when run in isolation.
Test failures like this are painful to try and track down, so ./runtests.py should have some tools to help narrow down the cause.
Attachments (1)
Change History (4)
by , 14 years ago
| Attachment: | bisect_and_pair.diff added |
|---|
comment:1 by , 14 years ago
| Triage Stage: | Unreviewed → Design decision needed |
|---|
Attached patch adds two flags to runtests.py:
- --bisect=test_client
- --pair=test_client
--bisect does a bisection of the test suite looking for a test that causes the test_client test to fail
--pair runs the test_client test with every other test until a test failure is discovered.
comment:2 by , 14 years ago
| Resolution: | → fixed |
|---|---|
| Status: | new → closed |
comment:3 by , 14 years ago
Note:
See TracTickets
for help on using tickets.
Patch implementing bisection and pairing for runtests.py