Opened 3 years ago
Closed 21 months ago
#32775 closed Bug (fixed)
LiveServerTestCase fails when query expressions are used for model ordering
Reported by: | Gergely Kalmár | Owned by: | nobody |
---|---|---|---|
Component: | Testing framework | Version: | 3.2 |
Severity: | Normal | Keywords: | |
Cc: | Triage Stage: | Accepted | |
Has patch: | no | Needs documentation: | no |
Needs tests: | no | Patch needs improvement: | no |
Easy pickings: | no | UI/UX: | no |
Description
Reproducing the bug
Create a models.py
file as follows:
from django.db import models from django.db.models.functions import Lower class Order(models.Model): id = models.CharField(max_length=10, primary_key=True) class Meta: ordering = [Lower('id')] class Offer(models.Model): order = models.OneToOneField(Order, primary_key=True, on_delete=models.PROTECT)
Create a test case:
from django.contrib.staticfiles.testing import LiveServerTestCase class Test(LiveServerTestCase): def test_case(self): pass
Run the tests with ./manage.py test
:
> ./manage.py test Creating test database for alias 'default'... Traceback (most recent call last): File "./manage.py", line 17, in <module> main() File "./manage.py", line 13, in main execute_from_command_line(sys.argv) File ".../lib/python3.8/site-packages/django/core/management/__init__.py", line 419, in execute_from_command_line utility.execute() File ".../lib/python3.8/site-packages/django/core/management/__init__.py", line 413, in execute self.fetch_command(subcommand).run_from_argv(self.argv) File ".../lib/python3.8/site-packages/django/core/management/commands/test.py", line 23, in run_from_argv super().run_from_argv(argv) File ".../lib/python3.8/site-packages/django/core/management/base.py", line 354, in run_from_argv self.execute(*args, **cmd_options) File ".../lib/python3.8/site-packages/django/core/management/base.py", line 398, in execute output = self.handle(*args, **options) File ".../lib/python3.8/site-packages/django/core/management/commands/test.py", line 55, in handle failures = test_runner.run_tests(test_labels) File ".../lib/python3.8/site-packages/django/test/runner.py", line 725, in run_tests old_config = self.setup_databases(aliases=databases) File ".../lib/python3.8/site-packages/django/test/runner.py", line 643, in setup_databases return _setup_databases( File ".../lib/python3.8/site-packages/django/test/utils.py", line 179, in setup_databases connection.creation.create_test_db( File ".../lib/python3.8/site-packages/django/db/backends/base/creation.py", line 90, in create_test_db self.connection._test_serialized_contents = self.serialize_db_to_string() File ".../lib/python3.8/site-packages/django/db/backends/base/creation.py", line 136, in serialize_db_to_string serializers.serialize("json", get_objects(), indent=None, stream=out) File ".../lib/python3.8/site-packages/django/core/serializers/__init__.py", line 129, in serialize s.serialize(queryset, **options) File ".../lib/python3.8/site-packages/django/core/serializers/base.py", line 90, in serialize for count, obj in enumerate(queryset, start=1): File ".../lib/python3.8/site-packages/django/db/backends/base/creation.py", line 133, in get_objects yield from queryset.iterator() File ".../lib/python3.8/site-packages/django/db/models/query.py", line 353, in _iterator yield from self._iterable_class(self, chunked_fetch=use_chunked_fetch, chunk_size=chunk_size) File ".../lib/python3.8/site-packages/django/db/models/query.py", line 51, in __iter__ results = compiler.execute_sql(chunked_fetch=self.chunked_fetch, chunk_size=self.chunk_size) File ".../lib/python3.8/site-packages/django/db/models/sql/compiler.py", line 1162, in execute_sql sql, params = self.as_sql() File ".../lib/python3.8/site-packages/django/db/models/sql/compiler.py", line 513, in as_sql extra_select, order_by, group_by = self.pre_sql_setup() File ".../lib/python3.8/site-packages/django/db/models/sql/compiler.py", line 56, in pre_sql_setup order_by = self.get_order_by() File ".../lib/python3.8/site-packages/django/db/models/sql/compiler.py", line 372, in get_order_by resolved = expr.resolve_expression(self.query, allow_joins=True, reuse=None) File ".../lib/python3.8/site-packages/django/db/models/expressions.py", line 247, in resolve_expression c.set_source_expressions([ File ".../lib/python3.8/site-packages/django/db/models/expressions.py", line 248, in <listcomp> expr.resolve_expression(query, allow_joins, reuse, summarize) File ".../lib/python3.8/site-packages/django/db/models/expressions.py", line 678, in resolve_expression c.source_expressions[pos] = arg.resolve_expression(query, allow_joins, reuse, summarize, for_save) File ".../lib/python3.8/site-packages/django/db/models/expressions.py", line 578, in resolve_expression return query.resolve_ref(self.name, allow_joins, reuse, summarize) File ".../lib/python3.8/site-packages/django/db/models/sql/query.py", line 1754, in resolve_ref join_info = self.setup_joins(field_list, self.get_meta(), self.get_initial_alias(), can_reuse=reuse) File ".../lib/python3.8/site-packages/django/db/models/sql/query.py", line 1623, in setup_joins path, final_field, targets, rest = self.names_to_path( File ".../lib/python3.8/site-packages/django/db/models/sql/query.py", line 1537, in names_to_path raise FieldError("Cannot resolve keyword '%s' into field. " django.core.exceptions.FieldError: Cannot resolve keyword 'id' into field. Choices are: order, order_id
If there is any workaround that I can use in the meantime please let me know. Thank you!
Change History (4)
comment:1 by , 3 years ago
comment:2 by , 3 years ago
Triage Stage: | Unreviewed → Accepted |
---|
You'll want to make sure to generate migrations for the app containing these models otherwise you won't be able to reproduce.
This bug is two fold. First we likely just want to have the querysets generated during serialization order by the primary key without involving possible related object semantic
diff --git a/django/db/backends/base/creation.py b/django/db/backends/base/creation.py index 81cb34bd9f..f49c31edaf 100644 --- a/django/db/backends/base/creation.py +++ b/django/db/backends/base/creation.py @@ -129,7 +129,7 @@ def get_objects(): ): queryset = model._base_manager.using( self.connection.alias, - ).order_by(model._meta.pk.name) + ).order_by('pk') yield from queryset.iterator() # Serialize to a string out = StringIO()
The above happens to address the reported crash but it doesn't deal with the fundamental issue which is that Referent.objects.order_by('reference')
crashes if Referred.Meta.ordering
contains expressions because they won't be transposed to the origin of the queryset (e.g. Lower('id') -> Lower('order__id')
in the reported case). It can be reproduced with the same set of models and the following test
from django.test import TestCase from .models import Offer class Test(TestCase): def test_case(self): list(Offer.objects.order_by('order'))
The issue lies in SQLCompiler.find_ordering_name
but is not trivial to address as we'd need a way to prefix all field references in an Expression
so I'd suggest we raise a NotImplementedError
instead for now. Not sure when this path is covered by the suite but I would assume it would only work when MTI inheritance is involved and referenced field references don't need to be prefixed as they are inherited?
comment:3 by , 3 years ago
Thank you for the help! It seems that I can work around this by setting
DATABASES = { 'default': { ... 'TEST': {'SERIALIZE': False} } }
which is fine for now, I think. It would be great if the recommended patch for at least fixing the immediate issue could be added though in case we'd need tests to be serialized for some reason in the future.
comment:4 by , 21 months ago
Resolution: | → fixed |
---|---|
Status: | new → closed |
Fixed in 2798c937deb6625a4e6a36e70d4d60ce5faac954 (#29538).
We'll have to figure out exactly what ORM query is being constructed here. The error doesn't happen on Django's main branch after 3089018e951dcca568574c0e0ebf9d8aab112389 but I'd guess the underlying problem still exists.