Opened 3 weeks ago

Closed 13 days ago

Last modified 13 days ago

#36678 closed Bug (fixed)

Infinite retries in parallel test runner if _init_worker fails

Reported by: Jacob Walls Owned by: Jacob Walls
Component: Testing framework Version: dev
Severity: Release blocker Keywords:
Cc: Triage Stage: Ready for checkin
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description

Recent GitHub actions were timing out after 6 hours because of an infinite retry in the parallel test runner.

To reproduce, throw an error at the top of _init_worker.

We will fix the specific error source in #36677, but we should also fix the parallel test runner to have a bound on retries.

Change History (9)

comment:1 by Jacob Walls, 3 weeks ago

Summary: Parallel test runner retries indefinitiely if _init_worker failsInfinite retries in parallel test runner if _init_worker fails

comment:2 by Jacob Walls, 3 weeks ago

Has patch: set

comment:3 by Simon Charette, 3 weeks ago

Triage Stage: UnreviewedAccepted

comment:4 by Jacob Walls, 3 weeks ago

Patch needs improvement: set

Need to tack back toward an earlier approach that tracked the origin of the failure in _init_worker to avoid introducing an arbitrary time limit on tests.

comment:5 by Jacob Walls, 3 weeks ago

Patch needs improvement: unset

comment:6 by Jacob Walls, 3 weeks ago

Severity: NormalRelease blocker

I think we need to handle this before 6.0 final since while this was only a theoretical failure point before, #36083 is going to make this a realistic vector for errors.

comment:7 by Natalia Bidart, 13 days ago

Triage Stage: AcceptedReady for checkin

comment:8 by nessita <124304+nessita@…>, 13 days ago

Resolution: fixed
Status: assignedclosed

In 1aa69a7:

Fixed #36678 -- Limited retries in ParallelTestRunner.

Thanks Natalia Bidart for the review.

comment:9 by Natalia <124304+nessita@…>, 13 days ago

In 6938675:

[6.0.x] Fixed #36678 -- Limited retries in ParallelTestRunner.

Thanks Natalia Bidart for the review.

Backport of 1aa69a7491ce7f7f1f164a26a3dfaaa1aeeab217 from main.

Note: See TracTickets for help on using tickets.
Back to Top