Opened 3 weeks ago

Closed 3 days ago

#37255 closed Bug (fixed)

Tests rely on unspecified row order of SELECT without ORDER BY

Reported by: Yaroslav K Owned by: Yaroslav K
Component: Core (Other) Version: dev
Severity: Normal Keywords: test
Cc: Yaroslav K Triage Stage: Ready for checkin
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description

Many (~70) tests assert a specific row order for querysets whose SQL has no
ORDER BY. SQL standard does not define that order.

On PostgreSQL the rows usually arrive in physical storage order, which
on a freshly loaded table matches insertion order -- so these assertions
pass by coincidence rather than by contract. That breaks when rows are
updated or deleted and the space is reused, or when the planner picks a
different access path such as an index scan. A test like this can start
failing with no change to Django or to the test itself.

Proposed solution:

  1. add order_by() where the asserted sequence is meaningful
  2. use assertCountEqual() otherwise

Found by running the suite under
pg_disorder.

Change History (4)

comment:1 by Tim Graham, 3 weeks ago

Component: Testing frameworkCore (Other)
Triage Stage: UnreviewedAccepted
Type: UncategorizedBug

comment:2 by Jacob Walls, 9 days ago

Patch needs improvement: set

comment:3 by Jacob Walls, 8 days ago

Patch needs improvement: unset
Triage Stage: AcceptedReady for checkin

comment:4 by Jacob Walls <jacobtylerwalls@…>, 3 days ago

Resolution: fixed
Status: assignedclosed

In 7576781:

Fixed #37255 -- Avoided relying on implicit SELECT ordering in tests.

Many (~70) tests assert a specific row order for querysets whose SQL has no
ORDER BY. SQL standard does not define that order.

On PostgreSQL the rows usually arrive in physical storage order, which
on a freshly loaded table matches insertion order -- so these assertions
pass by coincidence rather than by contract. That breaks when rows are
updated or deleted and the space is reused, or when the planner picks a
different access path such as an index scan. A test like this can start
failing with no change to Django or to the test itself.

The patch made sure that most of these tests do not
rely on unspecified unordered selection order.

Note: See TracTickets for help on using tickets.
Back to Top