Opened 6 months ago

Closed 7 weeks ago

#36085 closed Bug (fixed)

SQLite backend raises exception on negative array indices in JSONField

Reported by: savanto Owned by: savanto
Component: Database layer (models, ORM) Version: 5.1
Severity: Normal Keywords: sqlite, jsonfield
Cc: Sage Abdullah Triage Stage: Ready for checkin
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description

Positive jsonfield array index queries can be constructed the usual way with kwargs (ie. Model.objects.filter(jsonfield__2=1)), while negative jsonfield array index query must be constructed using the splat operator on a dict: Model.objects.filter(**{"jsonfield__-2": 1})). When querying jsonfields in this way on the SQLite backend, the following error arises:

django.db.utils.OperationalError: bad JSON path: '$[-2]'

When constructing the query, the JSON path constructor simply appends any numerical value to the path (https://github.com/django/django/blob/5.1.4/django/db/models/fields/json.py#L155). But the SQLite backend uses a special syntax for performing negative-indexing in jsonfields (https://sqlite.org/json1.html#path_arguments) that is different from other backends: negative indices must be prepended by a literal # character.

Change History (21)

comment:2 by Tim Graham, 6 months ago

Patch needs improvement: set
Triage Stage: UnreviewedAccepted

comment:3 by Sage Abdullah, 6 months ago

Cc: Sage Abdullah added

comment:4 by hesham hatem, 5 months ago

Thanks savanto for your contributions!!

I hope you and tim graham and lay monage find my PR well done.
thanks again!!

https://github.com/django/django/pull/19182

comment:5 by Antoliny, 5 months ago

Patch needs improvement: unset

comment:6 by Jacob Walls, 5 months ago

Patch needs improvement: set

A PR was opened to start with a related refactor. It needs a little tweak.

comment:7 by savanto, 4 months ago

Patch needs improvement: unset

Since https://github.com/django/django/pull/19182 was closed without merge a few weeks ago, I incorporated the refactor work from it back into https://github.com/django/django/pull/19030 and made the suggested changes from discussions from both PRs there.

comment:8 by Sarah Boyce, 4 months ago

Patch needs improvement: set

comment:9 by Sage Abdullah, 2 months ago

Patch needs improvement: unset

comment:10 by moshe nahmias, 2 months ago

It seems like the comments were done and test passes on the PR branch but fails on the main branch and is still relevant for 5.2 and the dev version

Last edited 2 months ago by moshe nahmias (previous) (diff)

comment:11 by moshe nahmias, 2 months ago

Triage Stage: AcceptedReady for checkin

comment:12 by Sarah Boyce, 2 months ago

Needs documentation: set
Triage Stage: Ready for checkinAccepted

comment:13 by Jacob Walls, 2 months ago

Needs documentation: unset

Friendly reminder to update the ticket flags when you're ready for another review.

comment:14 by savanto, 2 months ago

Triage Stage: AcceptedReady for checkin

Oh, ok, thanks! I added documentation and squashed into a single commit, so this is ready for final review.

comment:15 by Jacob Walls, 2 months ago

Triage Stage: Ready for checkinAccepted

Sounds good. Ready for checkin is only set by reviewers FYI, I was referring to the "needs..." flags, but no worries.

comment:16 by savanto, 2 months ago

Thanks/sorry =)

comment:17 by Sarah Boyce, 8 weeks ago

Triage Stage: AcceptedReady for checkin

comment:18 by Sarah Boyce, 8 weeks ago

Patch needs improvement: set
Triage Stage: Ready for checkinAccepted

comment:19 by Sarah Boyce, 8 weeks ago

Patch needs improvement: unset
Triage Stage: AcceptedReady for checkin

comment:20 by Sarah Boyce <42296566+sarahboyce@…>, 7 weeks ago

In a8716f3c:

Refs #36085 -- Moved compile_json_path to BaseDatabaseOperations.

comment:21 by Sarah Boyce <42296566+sarahboyce@…>, 7 weeks ago

Resolution: fixed
Status: assignedclosed

In 8620a3b0:

Fixed #36085 -- Added JSONField support for negative array indexing on SQLite.

Note: See TracTickets for help on using tickets.
Back to Top