Validation of constraints with __len lookup crashes for ArrayFields.

[James Gillard]

I'd already posted this in the django-users groups and someone had suggested this might be a bug, so am reposting here:

I can't yet work out whether this is a Django bug or how I'm using model constraints... Just upgraded from 4.0.8 to 4.1.4 and have hit this issue when saving this model in the admin. I'd read in the release notes that these constraints would be validated on model save, and that's the code that's leading to this exception. If it's not something I've done, it seems ArrayField isn't working with this new validation of my condition. The same happens for Q(phone_numberslengte=0), and the error disappears if I comment out this condition. It seems the generated code might be wrong, as I see 12 "%s" and only 11 elements in params. All it's trying to do is ensure that an empty list isn't considered unique.

When hitting save I now get IndexError: tuple index out of range

Here's the failing model:

phone_numbers = ArrayField(models.CharField(max_length=200), default=list, blank=True)

class Meta:
    constraints = [
        models.UniqueConstraint(
           fields=['phone_numbers'],
           condition=~Q(phone_numbers__len=0),
           name='unique_email_phones',
        ),
    ]
)

And the full stack trace:

Traceback (most recent call last):
  File ".../django/core/handlers/wsgi.py", line 131, in __call__
    response = self.get_response(request)
  File ".../django/core/handlers/base.py", line 140, in get_response
    response = self._middleware_chain(request)
  File ".../django/core/handlers/exception.py", line 57, in inner
    response = response_for_exception(request, exc)
  File ".../django/core/handlers/exception.py", line 140, in response_for_exception
    response = handle_uncaught_exception(
  File ".../django/core/handlers/exception.py", line 181, in handle_uncaught_exception
    return debug.technical_500_response(request, *exc_info)
  File ".../django_extensions/management/technical_response.py", line 40, in null_technical_500_response
    raise exc_value.with_traceback(tb)
  File ".../django/core/handlers/exception.py", line 55, in inner
    response = get_response(request)
  File ".../django/core/handlers/base.py", line 197, in _get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File ".../django/contrib/admin/options.py", line 686, in wrapper
    return self.admin_site.admin_view(view)(*args, **kwargs)
  File ".../django/utils/decorators.py", line 133, in _wrapped_view
    response = view_func(request, *args, **kwargs)
  File ".../django/views/decorators/cache.py", line 62, in _wrapped_view_func
    response = view_func(request, *args, **kwargs)
  File ".../django/contrib/admin/sites.py", line 242, in inner
    return view(request, *args, **kwargs)
  File "apps/catalogue/admin.py", line 1175, in change_view
    return super().change_view(
  File ".../django_object_actions/utils.py", line 57, in change_view
    return super(BaseDjangoObjectActions, self).change_view(
  File ".../django/contrib/admin/options.py", line 1893, in change_view
    return self.changeform_view(request, object_id, form_url, extra_context)
  File ".../django/utils/decorators.py", line 46, in _wrapper
    return bound_method(*args, **kwargs)
  File ".../django/utils/decorators.py", line 133, in _wrapped_view
    response = view_func(request, *args, **kwargs)
  File ".../django/contrib/admin/options.py", line 1750, in changeform_view
    return self._changeform_view(request, object_id, form_url, extra_context)
  File ".../django/contrib/admin/options.py", line 1796, in _changeform_view
    form_validated = form.is_valid()
  File ".../django/forms/forms.py", line 205, in is_valid
    return self.is_bound and not self.errors
  File ".../django/forms/forms.py", line 200, in errors
    self.full_clean()
  File ".../django/forms/forms.py", line 439, in full_clean
    self._post_clean()
  File ".../django/forms/models.py", line 492, in _post_clean
    self.instance.full_clean(exclude=exclude, validate_unique=False)
  File ".../django/db/models/base.py", line 1472, in full_clean
    self.validate_constraints(exclude=exclude)
  File ".../django/db/models/base.py", line 1423, in validate_constraints
    constraint.validate(model_class, self, exclude=exclude, using=using)
  File ".../django/db/models/constraints.py", line 361, in validate
    if (self.condition & Exists(queryset.filter(self.condition))).check(
  File ".../django/db/models/query_utils.py", line 141, in check
    return compiler.execute_sql(SINGLE) is not None
  File ".../django/db/models/sql/compiler.py", line 1398, in execute_sql
    cursor.execute(sql, params)
  File ".../debug_toolbar/panels/sql/tracking.py", line 230, in execute
    return self._record(self.cursor.execute, sql, params)
  File ".../debug_toolbar/panels/sql/tracking.py", line 154, in _record
    return method(sql, params)
  File ".../django/db/backends/utils.py", line 103, in execute
    return super().execute(sql, params)
  File ".../django/db/backends/utils.py", line 67, in execute
    return self._execute_with_wrappers(
  File ".../django/db/backends/utils.py", line 80, in _execute_with_wrappers
    return executor(sql, params, many, context)
  File ".../django/db/backends/utils.py", line 89, in _execute
    return self.cursor.execute(sql, params)
IndexError: tuple index out of range

[James Gillard]

stack trace

[Mariusz Felisiak]

Thanks for the report! It is a long standing bug introduced in 88fc9e2826044110b7b22577a227f122fe9c1fb5 that began manifesting in Django 4.1. The following patch fixes it for me:

django/contrib/postgres/fields/array.py

@@ -297 +297 @@
         return (
             "CASE WHEN %(lhs)s IS NULL THEN NULL ELSE "
             "coalesce(array_length(%(lhs)s, 1), 0) END"
-        ) % {"lhs": lhs}, params
+        ) % {"lhs": lhs}, params * 2
 
 
 @ArrayField.register_lookup

We two lhs, so we need to pass params for both.

Would you like to prepare a patch? (a regression test is required, I'd add it to the postgres_tests.test_constraints.SchemaTests.)

[James Gillard]

Thanks for the quick feedback and suggested change, that was very helpful for a someone who's new here :)

I've submitted a ​​pull request for this.

[Mariusz Felisiak <felisiak.mariusz@…>]

In c5ed884:

Fixed #34205 -- Fixed Meta.constraints validation crash with ArrayField and len lookup.

Regression in 88fc9e2826044110b7b22577a227f122fe9c1fb5 that began
manifesting in Django 4.1.

[Mariusz Felisiak <felisiak.mariusz@…>]

In af3cfc8:

[4.1.x] Fixed #34205 -- Fixed Meta.constraints validation crash with ArrayField and len lookup.

Regression in 88fc9e2826044110b7b22577a227f122fe9c1fb5 that began
manifesting in Django 4.1.

Backport of c5ed884eabf3b2b67581c55bf6c87e721f69157f from main.