Opened 10 months ago
Closed 6 months ago
#36085 closed Bug (fixed)
SQLite backend raises exception on negative array indices in JSONField
| Reported by: | savanto | Owned by: | savanto |
|---|---|---|---|
| Component: | Database layer (models, ORM) | Version: | 5.1 |
| Severity: | Normal | Keywords: | sqlite, jsonfield |
| Cc: | Sage Abdullah | Triage Stage: | Ready for checkin |
| Has patch: | yes | Needs documentation: | no |
| Needs tests: | no | Patch needs improvement: | no |
| Easy pickings: | no | UI/UX: | no |
Description
Positive jsonfield array index queries can be constructed the usual way with kwargs (ie. Model.objects.filter(jsonfield__2=1)), while negative jsonfield array index query must be constructed using the splat operator on a dict: Model.objects.filter(**{"jsonfield__-2": 1})). When querying jsonfields in this way on the SQLite backend, the following error arises:
django.db.utils.OperationalError: bad JSON path: '$[-2]'
When constructing the query, the JSON path constructor simply appends any numerical value to the path (https://github.com/django/django/blob/5.1.4/django/db/models/fields/json.py#L155). But the SQLite backend uses a special syntax for performing negative-indexing in jsonfields (https://sqlite.org/json1.html#path_arguments) that is different from other backends: negative indices must be prepended by a literal # character.
Change History (21)
comment:1 by , 10 months ago
| Has patch: | set |
|---|
comment:2 by , 10 months ago
| Patch needs improvement: | set |
|---|---|
| Triage Stage: | Unreviewed → Accepted |
comment:3 by , 10 months ago
| Cc: | added |
|---|
comment:4 by , 9 months ago
Thanks savanto for your contributions!!
I hope you and tim graham and lay monage find my PR well done.
thanks again!!
comment:5 by , 9 months ago
| Patch needs improvement: | unset |
|---|
comment:6 by , 8 months ago
| Patch needs improvement: | set |
|---|
A PR was opened to start with a related refactor. It needs a little tweak.
comment:7 by , 8 months ago
| Patch needs improvement: | unset |
|---|
Since https://github.com/django/django/pull/19182 was closed without merge a few weeks ago, I incorporated the refactor work from it back into https://github.com/django/django/pull/19030 and made the suggested changes from discussions from both PRs there.
comment:8 by , 7 months ago
| Patch needs improvement: | set |
|---|
comment:9 by , 6 months ago
| Patch needs improvement: | unset |
|---|
comment:10 by , 6 months ago
It seems like the comments were done and test passes on the PR branch but fails on the main branch and is still relevant for 5.2 and the dev version
comment:11 by , 6 months ago
| Triage Stage: | Accepted → Ready for checkin |
|---|
comment:12 by , 6 months ago
| Needs documentation: | set |
|---|---|
| Triage Stage: | Ready for checkin → Accepted |
comment:13 by , 6 months ago
| Needs documentation: | unset |
|---|
Friendly reminder to update the ticket flags when you're ready for another review.
comment:14 by , 6 months ago
| Triage Stage: | Accepted → Ready for checkin |
|---|
Oh, ok, thanks! I added documentation and squashed into a single commit, so this is ready for final review.
comment:15 by , 6 months ago
| Triage Stage: | Ready for checkin → Accepted |
|---|
Sounds good. Ready for checkin is only set by reviewers FYI, I was referring to the "needs..." flags, but no worries.
comment:17 by , 6 months ago
| Triage Stage: | Accepted → Ready for checkin |
|---|
comment:18 by , 6 months ago
| Patch needs improvement: | set |
|---|---|
| Triage Stage: | Ready for checkin → Accepted |
comment:19 by , 6 months ago
| Patch needs improvement: | unset |
|---|---|
| Triage Stage: | Accepted → Ready for checkin |
https://github.com/django/django/pull/19030