Opened 8 months ago

Closed 4 months ago

#36085 closed Bug (fixed)

SQLite backend raises exception on negative array indices in JSONField

Reported by: savanto Owned by: savanto
Component: Database layer (models, ORM) Version: 5.1
Severity: Normal Keywords: sqlite, jsonfield
Cc: Sage Abdullah Triage Stage: Ready for checkin
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description

Positive jsonfield array index queries can be constructed the usual way with kwargs (ie. Model.objects.filter(jsonfield__2=1)), while negative jsonfield array index query must be constructed using the splat operator on a dict: Model.objects.filter(**{"jsonfield__-2": 1})). When querying jsonfields in this way on the SQLite backend, the following error arises:

django.db.utils.OperationalError: bad JSON path: '$[-2]'

When constructing the query, the JSON path constructor simply appends any numerical value to the path (https://github.com/django/django/blob/5.1.4/django/db/models/fields/json.py#L155). But the SQLite backend uses a special syntax for performing negative-indexing in jsonfields (https://sqlite.org/json1.html#path_arguments) that is different from other backends: negative indices must be prepended by a literal # character.

Change History (21)

comment:2 by Tim Graham, 8 months ago

Patch needs improvement: set
Triage Stage: UnreviewedAccepted

comment:3 by Sage Abdullah, 8 months ago

Cc: Sage Abdullah added

comment:4 by hesham hatem, 7 months ago

Thanks savanto for your contributions!!

I hope you and tim graham and lay monage find my PR well done.
thanks again!!

https://github.com/django/django/pull/19182

comment:5 by Antoliny, 7 months ago

Patch needs improvement: unset

comment:6 by Jacob Walls, 7 months ago

Patch needs improvement: set

A PR was opened to start with a related refactor. It needs a little tweak.

comment:7 by savanto, 6 months ago

Patch needs improvement: unset

Since https://github.com/django/django/pull/19182 was closed without merge a few weeks ago, I incorporated the refactor work from it back into https://github.com/django/django/pull/19030 and made the suggested changes from discussions from both PRs there.

comment:8 by Sarah Boyce, 6 months ago

Patch needs improvement: set

comment:9 by Sage Abdullah, 4 months ago

Patch needs improvement: unset

comment:10 by moshe nahmias, 4 months ago

It seems like the comments were done and test passes on the PR branch but fails on the main branch

Version 0, edited 4 months ago by moshe nahmias (next)

comment:11 by moshe nahmias, 4 months ago

Triage Stage: AcceptedReady for checkin

comment:12 by Sarah Boyce, 4 months ago

Needs documentation: set
Triage Stage: Ready for checkinAccepted

comment:13 by Jacob Walls, 4 months ago

Needs documentation: unset

Friendly reminder to update the ticket flags when you're ready for another review.

comment:14 by savanto, 4 months ago

Triage Stage: AcceptedReady for checkin

Oh, ok, thanks! I added documentation and squashed into a single commit, so this is ready for final review.

comment:15 by Jacob Walls, 4 months ago

Triage Stage: Ready for checkinAccepted

Sounds good. Ready for checkin is only set by reviewers FYI, I was referring to the "needs..." flags, but no worries.

comment:16 by savanto, 4 months ago

Thanks/sorry =)

comment:17 by Sarah Boyce, 4 months ago

Triage Stage: AcceptedReady for checkin

comment:18 by Sarah Boyce, 4 months ago

Patch needs improvement: set
Triage Stage: Ready for checkinAccepted

comment:19 by Sarah Boyce, 4 months ago

Patch needs improvement: unset
Triage Stage: AcceptedReady for checkin

comment:20 by Sarah Boyce <42296566+sarahboyce@…>, 4 months ago

In a8716f3c:

Refs #36085 -- Moved compile_json_path to BaseDatabaseOperations.

comment:21 by Sarah Boyce <42296566+sarahboyce@…>, 4 months ago

Resolution: fixed
Status: assignedclosed

In 8620a3b0:

Fixed #36085 -- Added JSONField support for negative array indexing on SQLite.

Note: See TracTickets for help on using tickets.
Back to Top