#28214 closed Bug (fixed)
IndexError for JSONField queries containing percent character in key.
Reported by: | mrsanders | Owned by: | André Ericson |
---|---|---|---|
Component: | contrib.postgres | Version: | 1.11 |
Severity: | Normal | Keywords: | |
Cc: | Triage Stage: | Accepted | |
Has patch: | no | Needs documentation: | no |
Needs tests: | no | Patch needs improvement: | no |
Easy pickings: | no | UI/UX: | no |
Description
Using Django 1.11.1 and Postgres 9.5.6.
A JSON field may legitimately contain a percent symbol in a key, for example:
{"%total": 10}
This may be queried as follows (assuming the JSONField in the Django model is called 'my_json_field'):
>>> qs=MyModel.objects.filter(**{"my_json_field__%total": 10})
However when the Queryset is evaluated, it raises an IndexError:
>>> len(qs) Traceback (most recent call last): File "<console>", line 1, in <module> File "/usr/local/lib/python3.6/site-packages/django/db/models/query.py", line 232, in __len__ self._fetch_all() File "/usr/local/lib/python3.6/site-packages/django/db/models/query.py", line 1103, in _fetch_all self._result_cache = list(self._iterable_class(self)) File "/usr/local/lib/python3.6/site-packages/django/db/models/query.py", line 53, in __iter__ results = compiler.execute_sql(chunked_fetch=self.chunked_fetch) File "/usr/local/lib/python3.6/site-packages/django/db/models/sql/compiler.py", line 886, in execute_sql raise original_exception File "/usr/local/lib/python3.6/site-packages/django/db/models/sql/compiler.py", line 876, in execute_sql cursor.execute(sql, params) File "/usr/local/lib/python3.6/site-packages/django/db/backends/utils.py", line 80, in execute return super(CursorDebugWrapper, self).execute(sql, params) File "/usr/local/lib/python3.6/site-packages/django/db/backends/utils.py", line 65, in execute return self.cursor.execute(sql, params) IndexError: tuple index out of range
...and when attempting to print the generated query, a ValueError is raised:
>>> print(qs.query) Traceback (most recent call last): File "<console>", line 1, in <module> File "/usr/local/lib/python3.6/site-packages/django/db/models/sql/query.py", line 234, in __str__ return sql % params ValueError: unsupported format character 't' (0x74) at index 1498 >>>
It seems that something isn't escaping the percent before doing a string formatting operation.
Changing the query to double-up the percent symbol causes the query to work (although it shouldn't):
>>> qs=MyModel.objects.filter(**{"my_json_field__%%total": 10}
Change History (5)
comment:1 by , 7 years ago
Triage Stage: | Unreviewed → Accepted |
---|
comment:2 by , 7 years ago
Owner: | set to |
---|---|
Status: | new → assigned |
comment:3 by , 7 years ago
comment:4 by , 5 years ago
Resolution: | → fixed |
---|---|
Status: | assigned → closed |
Summary: | IndexError for JSONField queries containing percent character in key → IndexError for JSONField queries containing percent character in key. |
This was fixed in 7deeabc7c7526786df6894429ce89a9c4b614086.
Hey,
how should we tackle this? psycopg2 documentation says "in order to include a literal % in the query you can use the %% string."
If we decide to escape it, it will break for people that already fixed this in their own code by pre-escaping the JSON keys (ie: '%%total' key).
Another approach would be to add a note in the JSONField documentation that '%' must be escaped.