Opened 2 months ago

Closed 3 weeks ago

Last modified 3 weeks ago

#37297 closed Bug (fixed)

Programming error when the first JSONField path segment is an integer overflowing int4's range on PostgreSQL

Reported by: Clifford Gama Owned by: Clifford Gama
Component: Database layer (models, ORM) Version: dev
Severity: Normal Keywords: JSONField, PostgreSQL, KeyTransform, jsonb
Cc: Triage Stage: Ready for checkin
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description (last modified by Clifford Gama)

JSONField key lookup raises an unhandled ProgrammingError ("operator does not exist: jsonb -> bigint") instead of behaving like any other out-of-range/non-matching lookup, when a purely-numeric key segment decimal-parses to a value outside PostgreSQL's int4 range.

I had a JSONField whose keys are generated using uuid.uuid4().hex[:12]. When a key was all-digit, this ran into #29504, leading to flaky tests when I did something like .filter(**{key: "some value"}). That's because Django ​interprets an integer key segment as an array index rather than a text path -- but only when it's the lookup's sole segment. In this case, though, instead of getting no match, I got a ProgrammingError: the number overflowed int, and -> isn't overloaded for bigint.

operator does not exist: jsonb -> bigint
HINT:  No operator matches the given name and argument types. You
       might need to add explicit type casts.

Since Django is guessing if an all-number string is an array index, I suggest that we treat numbers outside of the allowed range as strings. An appropriate solution would be to make this match the same way as an out of range index, which means returning NULL.

See attached PR for a regression test.

Change History (8)

comment:1 by Clifford Gama, 2 months ago

comment:2 by Clifford Gama, 2 months ago

Now that I think about it, returning no match would be the consistent with the behaviour of the other database backends.

comment:3 by Clifford Gama, 2 months ago

Description: modified (diff)

comment:4 by Jacob Walls, 3 weeks ago

Triage Stage: Unreviewed → Accepted

Thanks for the report. Returning NULL seems to agree with the behavior for other missing keys/indices.

comment:5 by Jacob Walls, 3 weeks ago

Patch needs improvement: set

comment:6 by Jacob Walls, 3 weeks ago

Patch needs improvement: unset
Triage Stage: Accepted → Ready for checkin

comment:7 by Jacob Walls <jacobtylerwalls@…>, 3 weeks ago

Resolution: → fixed
Status: assigned → closed

In 66c40aa:

Fixed #37297 -- Prevented crash on JSONField key lookups when an integer key is outside PostgreSQL's int4 range.

comment:8 by Jacob Walls <jacobtylerwalls@…>, 3 weeks ago

In 7f48e26:

Refs #37297 -- Prevented crash on JSONField isnull lookups on Oracle when a key is outside its integer range.

Oracle's JSON_EXISTS() rejects an array-subscript literal this long
with ORA-40598/JZN-00211.

Note: See TracTickets for help on using tickets.
Back to Top