Opened 15 months ago

Closed 15 months ago

Last modified 15 months ago

#34205 closed Bug (fixed)

Validation of constraints with __len lookup crashes for ArrayFields.

Reported by: James Gillard Owned by: James Gillard
Component: Database layer (models, ORM) Version: 4.1
Severity: Release blocker Keywords:
Cc: Triage Stage: Ready for checkin
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description (last modified by James Gillard)

I'd already posted this in the django-users groups and someone had suggested this might be a bug, so am reposting here:

I can't yet work out whether this is a Django bug or how I'm using model constraints... Just upgraded from 4.0.8 to 4.1.4 and have hit this issue when saving this model in the admin. I'd read in the release notes that these constraints would be validated on model save, and that's the code that's leading to this exception. If it's not something I've done, it seems ArrayField isn't working with this new validation of my condition. The same happens for Q(phone_numberslengte=0), and the error disappears if I comment out this condition. It seems the generated code might be wrong, as I see 12 "%s" and only 11 elements in params. All it's trying to do is ensure that an empty list isn't considered unique.

When hitting save I now get IndexError: tuple index out of range

Here's the failing model:

phone_numbers = ArrayField(models.CharField(max_length=200), default=list, blank=True)

class Meta:
    constraints = [
        models.UniqueConstraint(
           fields=['phone_numbers'],
           condition=~Q(phone_numbers__len=0),
           name='unique_email_phones',
        ),
    ]
)

And the full stack trace:

Traceback (most recent call last):
  File ".../django/core/handlers/wsgi.py", line 131, in __call__
    response = self.get_response(request)
  File ".../django/core/handlers/base.py", line 140, in get_response
    response = self._middleware_chain(request)
  File ".../django/core/handlers/exception.py", line 57, in inner
    response = response_for_exception(request, exc)
  File ".../django/core/handlers/exception.py", line 140, in response_for_exception
    response = handle_uncaught_exception(
  File ".../django/core/handlers/exception.py", line 181, in handle_uncaught_exception
    return debug.technical_500_response(request, *exc_info)
  File ".../django_extensions/management/technical_response.py", line 40, in null_technical_500_response
    raise exc_value.with_traceback(tb)
  File ".../django/core/handlers/exception.py", line 55, in inner
    response = get_response(request)
  File ".../django/core/handlers/base.py", line 197, in _get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File ".../django/contrib/admin/options.py", line 686, in wrapper
    return self.admin_site.admin_view(view)(*args, **kwargs)
  File ".../django/utils/decorators.py", line 133, in _wrapped_view
    response = view_func(request, *args, **kwargs)
  File ".../django/views/decorators/cache.py", line 62, in _wrapped_view_func
    response = view_func(request, *args, **kwargs)
  File ".../django/contrib/admin/sites.py", line 242, in inner
    return view(request, *args, **kwargs)
  File "apps/catalogue/admin.py", line 1175, in change_view
    return super().change_view(
  File ".../django_object_actions/utils.py", line 57, in change_view
    return super(BaseDjangoObjectActions, self).change_view(
  File ".../django/contrib/admin/options.py", line 1893, in change_view
    return self.changeform_view(request, object_id, form_url, extra_context)
  File ".../django/utils/decorators.py", line 46, in _wrapper
    return bound_method(*args, **kwargs)
  File ".../django/utils/decorators.py", line 133, in _wrapped_view
    response = view_func(request, *args, **kwargs)
  File ".../django/contrib/admin/options.py", line 1750, in changeform_view
    return self._changeform_view(request, object_id, form_url, extra_context)
  File ".../django/contrib/admin/options.py", line 1796, in _changeform_view
    form_validated = form.is_valid()
  File ".../django/forms/forms.py", line 205, in is_valid
    return self.is_bound and not self.errors
  File ".../django/forms/forms.py", line 200, in errors
    self.full_clean()
  File ".../django/forms/forms.py", line 439, in full_clean
    self._post_clean()
  File ".../django/forms/models.py", line 492, in _post_clean
    self.instance.full_clean(exclude=exclude, validate_unique=False)
  File ".../django/db/models/base.py", line 1472, in full_clean
    self.validate_constraints(exclude=exclude)
  File ".../django/db/models/base.py", line 1423, in validate_constraints
    constraint.validate(model_class, self, exclude=exclude, using=using)
  File ".../django/db/models/constraints.py", line 361, in validate
    if (self.condition & Exists(queryset.filter(self.condition))).check(
  File ".../django/db/models/query_utils.py", line 141, in check
    return compiler.execute_sql(SINGLE) is not None
  File ".../django/db/models/sql/compiler.py", line 1398, in execute_sql
    cursor.execute(sql, params)
  File ".../debug_toolbar/panels/sql/tracking.py", line 230, in execute
    return self._record(self.cursor.execute, sql, params)
  File ".../debug_toolbar/panels/sql/tracking.py", line 154, in _record
    return method(sql, params)
  File ".../django/db/backends/utils.py", line 103, in execute
    return super().execute(sql, params)
  File ".../django/db/backends/utils.py", line 67, in execute
    return self._execute_with_wrappers(
  File ".../django/db/backends/utils.py", line 80, in _execute_with_wrappers
    return executor(sql, params, many, context)
  File ".../django/db/backends/utils.py", line 89, in _execute
    return self.cursor.execute(sql, params)
IndexError: tuple index out of range

Attachments (1)

Screenshot 2022-12-09 at 10.11.59-min.png (254.6 KB ) - added by James Gillard 15 months ago.
stack trace

Download all attachments as: .zip

Change History (9)

by James Gillard, 15 months ago

stack trace

comment:1 by James Gillard, 15 months ago

Description: modified (diff)
Summary: Arrayfield constraint issue in 4.1Arrayfield constraint validation crash in 4.1

comment:2 by James Gillard, 15 months ago

Description: modified (diff)

comment:3 by Mariusz Felisiak, 15 months ago

Severity: NormalRelease blocker
Summary: Arrayfield constraint validation crash in 4.1Validation of constraints with __len lookup crashes for ArrayFields.
Triage Stage: UnreviewedAccepted
Type: UncategorizedBug

Thanks for the report! It is a long standing bug introduced in 88fc9e2826044110b7b22577a227f122fe9c1fb5 that began manifesting in Django 4.1. The following patch fixes it for me:

  • django/contrib/postgres/fields/array.py

    diff --git a/django/contrib/postgres/fields/array.py b/django/contrib/postgres/fields/array.py
    index c247387eb7..eaff032465 100644
    a b class ArrayLenTransform(Transform):  
    297297        return (
    298298            "CASE WHEN %(lhs)s IS NULL THEN NULL ELSE "
    299299            "coalesce(array_length(%(lhs)s, 1), 0) END"
    300         ) % {"lhs": lhs}, params
     300        ) % {"lhs": lhs}, params * 2
    301301
    302302
    303303@ArrayField.register_lookup

We two lhs, so we need to pass params for both.

Would you like to prepare a patch? (a regression test is required, I'd add it to the postgres_tests.test_constraints.SchemaTests.)

comment:4 by James Gillard, 15 months ago

Owner: changed from nobody to James Gillard
Status: newassigned

comment:5 by James Gillard, 15 months ago

Has patch: set

Thanks for the quick feedback and suggested change, that was very helpful for a someone who's new here :)

I've submitted a ​pull request for this.

comment:6 by Mariusz Felisiak, 15 months ago

Triage Stage: AcceptedReady for checkin

comment:7 by Mariusz Felisiak <felisiak.mariusz@…>, 15 months ago

Resolution: fixed
Status: assignedclosed

In c5ed884:

Fixed #34205 -- Fixed Meta.constraints validation crash with ArrayField and len lookup.

Regression in 88fc9e2826044110b7b22577a227f122fe9c1fb5 that began
manifesting in Django 4.1.

comment:8 by Mariusz Felisiak <felisiak.mariusz@…>, 15 months ago

In af3cfc8:

[4.1.x] Fixed #34205 -- Fixed Meta.constraints validation crash with ArrayField and len lookup.

Regression in 88fc9e2826044110b7b22577a227f122fe9c1fb5 that began
manifesting in Django 4.1.

Backport of c5ed884eabf3b2b67581c55bf6c87e721f69157f from main.

Note: See TracTickets for help on using tickets.
Back to Top