AttributeError: 'tuple' object has no attribute 'extend' when subclassing BuiltinLookup and swapping the lhs and rhs params

[Julien Palard]

I tried to run django-oscar tests using django 6 and spotted an AttributeError: 'tuple' object has no attribute 'extend' that has been fixed by 787cc96ef6197d73c7d4ad96f25500910c399603, which does not exists in the stable/6.0.x branch.

[Julien Palard]

This is mentionned in Django 6.0 release notes :)

​https://docs.djangoproject.com/en/6.0/releases/6.0/#custom-orm-expressions-should-return-params-as-a-tuple

[Natalia Bidart]

Hello Julien, thank you for your ticket. The commit you reference was merged into main after the 6.0 feature freeze (this is when the stable/6.0.x branch is cut), that's why the commit is not there. Now, whether this commit should have been backported, that's another question. In the PR, Jacob explicitly said:

Not certain whether to suggest a 6.0 backport. Nothing is broken per se, it's just two examples falling outside the doc'd pattern we updated to.

Can you please provide more details about your use case and how your project is affected? Please reopen the ticket when doing so. Thank you!

[Julien Palard]

django-oscar tests are failing without this patch:

> pytest -x --pdb
================================================================= test session starts =================================================================
platform linux -- Python 3.13.12, pytest-9.0.2, pluggy-1.6.0
django: version: 6.0.2
rootdir: /home/mdk/src/django-oscar
configfile: setup.cfg
testpaths: tests/
plugins: Faker-40.1.2, django-webtest-1.9.14, django-4.11.1, xdist-3.8.0
collected 1682 items                                                                                                                                  

tests/functional/basket/test_manipulation.py F
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> traceback >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>

self = <tests.functional.basket.test_manipulation.TestAddingToBasket testMethod=test_validation_errors_in_form>

    def test_validation_errors_in_form(self):
>       product = factories.ProductFactory()
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^

tests/functional/basket/test_manipulation.py:43: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
.venv/lib/python3.13/site-packages/factory/base.py:43: in __call__
    return cls.create(**kwargs)
           ^^^^^^^^^^^^^^^^^^^^
.venv/lib/python3.13/site-packages/factory/base.py:539: in create
    return cls._generate(enums.CREATE_STRATEGY, kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
.venv/lib/python3.13/site-packages/factory/django.py:122: in _generate
    return super()._generate(strategy, params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
.venv/lib/python3.13/site-packages/factory/base.py:468: in _generate
    return step.build()
           ^^^^^^^^^^^^
.venv/lib/python3.13/site-packages/factory/builder.py:283: in build
    postgen_results[declaration_name] = declaration.declaration.evaluate_post(
.venv/lib/python3.13/site-packages/factory/declarations.py:652: in evaluate_post
    return self.call(instance, step, postgen_context)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
.venv/lib/python3.13/site-packages/factory/declarations.py:733: in call
    return step.recurse(factory, passed_kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
.venv/lib/python3.13/site-packages/factory/builder.py:228: in recurse
    return builder.build(parent_step=self, force_sequence=force_sequence)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
.venv/lib/python3.13/site-packages/factory/builder.py:270: in build
    step.resolve(pre)
.venv/lib/python3.13/site-packages/factory/builder.py:211: in resolve
    self.attributes[field_name] = getattr(self.stub, field_name)
                                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
.venv/lib/python3.13/site-packages/factory/builder.py:356: in __getattr__
    value = value.evaluate_pre(
.venv/lib/python3.13/site-packages/factory/declarations.py:67: in evaluate_pre
    return self.evaluate(instance, step, context)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
.venv/lib/python3.13/site-packages/factory/declarations.py:457: in evaluate
    return step.recurse(subfactory, extra, force_sequence=force_sequence)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
.venv/lib/python3.13/site-packages/factory/builder.py:228: in recurse
    return builder.build(parent_step=self, force_sequence=force_sequence)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
.venv/lib/python3.13/site-packages/factory/builder.py:274: in build
    instance = self.factory_meta.instantiate(
.venv/lib/python3.13/site-packages/factory/base.py:320: in instantiate
    return self.factory._create(model, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
.venv/lib/python3.13/site-packages/factory/django.py:175: in _create
    return manager.create(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
.venv/lib/python3.13/site-packages/django/db/models/manager.py:87: in manager_method
    return getattr(self.get_queryset(), name)(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
.venv/lib/python3.13/site-packages/django/db/models/query.py:669: in create
    obj.save(force_insert=True, using=self.db)
src/oscar/apps/catalogue/abstract_models.py:244: in save
    super().save(*args, **kwargs)
.venv/lib/python3.13/site-packages/django/db/models/base.py:874: in save
    self.save_base(
.venv/lib/python3.13/site-packages/django/db/models/base.py:981: in save_base
    post_save.send(
.venv/lib/python3.13/site-packages/django/dispatch/dispatcher.py:209: in send
    response = receiver(signal=self, sender=sender, **named)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
src/oscar/apps/catalogue/receivers.py:46: in post_save_set_ancestors_are_public
    instance.set_ancestors_are_public()
src/oscar/apps/catalogue/abstract_models.py:255: in set_ancestors_are_public
    self.get_descendants_and_self().update(
.venv/lib/python3.13/site-packages/django/db/models/query.py:1300: in update
    rows = query.get_compiler(self.db).execute_sql(ROW_COUNT)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
.venv/lib/python3.13/site-packages/django/db/models/sql/compiler.py:2111: in execute_sql
    row_count = super().execute_sql(result_type)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
.venv/lib/python3.13/site-packages/django/db/models/sql/compiler.py:1611: in execute_sql
    sql, params = self.as_sql()
                  ^^^^^^^^^^^^^
.venv/lib/python3.13/site-packages/django/db/models/sql/compiler.py:2074: in as_sql
    sql, params = self.compile(val)
                  ^^^^^^^^^^^^^^^^^
.venv/lib/python3.13/site-packages/django/db/models/sql/compiler.py:578: in compile
    sql, params = node.as_sql(self, self.connection)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
.venv/lib/python3.13/site-packages/django/db/models/expressions.py:1551: in as_sql
    sql, params = super().as_sql(compiler, connection)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
.venv/lib/python3.13/site-packages/django/db/models/expressions.py:1530: in as_sql
    return compiler.compile(self.expression)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
.venv/lib/python3.13/site-packages/django/db/models/sql/compiler.py:578: in compile
    sql, params = node.as_sql(self, self.connection)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
.venv/lib/python3.13/site-packages/django/db/models/expressions.py:1859: in as_sql
    return super().as_sql(compiler, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
.venv/lib/python3.13/site-packages/django/db/models/expressions.py:1829: in as_sql
    subquery_sql, sql_params = self.query.as_sql(compiler, connection)
                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
.venv/lib/python3.13/site-packages/django/db/models/sql/query.py:1323: in as_sql
    sql, params = self.get_compiler(connection=connection).as_sql()
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
.venv/lib/python3.13/site-packages/django/db/models/sql/compiler.py:795: in as_sql
    self.compile(self.where) if self.where is not None else ("", [])
    ^^^^^^^^^^^^^^^^^^^^^^^^
.venv/lib/python3.13/site-packages/django/db/models/sql/compiler.py:578: in compile
    sql, params = node.as_sql(self, self.connection)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
.venv/lib/python3.13/site-packages/django/db/models/sql/where.py:151: in as_sql
    sql, params = compiler.compile(child)
                  ^^^^^^^^^^^^^^^^^^^^^^^
.venv/lib/python3.13/site-packages/django/db/models/sql/compiler.py:578: in compile
    sql, params = node.as_sql(self, self.connection)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = ReverseStartsWith(Col(U0, catalogue.Category.path), Col(catalogue_category, catalogue.Category.path))
compiler = <SQLCompiler model=Category connection=<DatabaseWrapper vendor='postgresql' alias='default'> using=None>
connection = <DatabaseWrapper vendor='postgresql' alias='default'>

    def as_sql(self, compiler, connection):
        lhs_sql, params = self.process_lhs(compiler, connection)
        rhs_sql, rhs_params = self.process_rhs(compiler, connection)
>       params.extend(rhs_params)
        ^^^^^^^^^^^^^
E       AttributeError: 'tuple' object has no attribute 'extend'

.venv/lib/python3.13/site-packages/django/db/models/lookups.py:240: AttributeError
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> entering PDB >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>

>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> PDB post_mortem (IO-capturing turned off) >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> /home/mdk/src/django-oscar/.venv/lib/python3.13/site-packages/django/db/models/lookups.py(240)as_sql()
-> params.extend(rhs_params)
(Pdb) 

[Jacob Walls]

Thanks. Ordinarily I wouldn't clone a repo to run its test suite for a reproducer, but I gave it my best shot, and the test suite doesn't launch even after installing test dependencies, failing with ModuleNotFoundError on from oscar.defaults import *.

It's possible we need to backport this, but there are so many third-party dependencies in your traceback we really can't say it a glance if Django is at fault.

Can you share the queryset that is failing?

[Jacob Walls]

Hang on, I'm pretty close to reverse engineering a failing case.

[Jacob Walls]

Okay, I couldn't engineer a failing case. Please reopen if you can find one using no third-party packages.

I think your problem is that django-oscar is subclassing StartsWith to create a reverse starts with by ​swapping `lhs_params` and `rhs_params`.

The 6.0 version of StartsWith is safe, because it calls extend() on the params returned by process_lhs(), which is always a list. If you change the return type of an internal by swapping the implementations like this, all bets are off. You could vendor the more resilient version from main, which is pretty much the advice in the release note you linked to (always return params in a tuple from custom ORM expressions).