Opened 5 months ago

Closed 5 months ago

#35395 closed Bug (fixed)

slice filter crashes on an empty dict with Python 3.12

Reported by: Tim Richardson Owned by: Tim Richardson
Component: Template system Version: 4.2
Severity: Normal Keywords:
Cc: Triage Stage: Ready for checkin
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: yes UI/UX: no

Description (last modified by Tim Graham)

I have a template with a fragment like this:

<span class="fs-6">Supplier: {{ po_metadata|get_item:dict_item.PONumber|get_item:"supplier" }} PO Date: {{ po_metadata|get_item:dict_item.PONumber|get_item:"order_date" | slice:":10" }} </span>

When using python 3.13.2, I get an exception in django/template/defaultfilters.py slice_filter()

This code does not throw an exception with 3.11

The exception is a Key Error and it happens because instead a list being passed to the filter, a dictionary is. An empty dict in my case.

In python 3.11, the breakpoints in slice_filter() are not hit. Somehow the preceeding filters are behaving differently in case of po_metadata being an empty dict.

Change History (13)

comment:1 by Tim Richardson, 5 months ago

Description: modified (diff)

comment:2 by Tim Graham, 5 months ago

Description: modified (diff)
Resolution: needsinfo
Status: newclosed

Hi Tim, You haven't provided complete steps to reproduce the issue as your snippet involves custom template filters. The issue may be in those filters. You should debug the problem and confirm that Django is at fault. If so, reopen the issue with an explanation. See TicketClosingReasons/UseSupportChannels if you need help.

comment:3 by Tim Richardson, 5 months ago

Fair enough.

The code below is the slice_filter from the 4.2.11 source.

If you run this code in python 3.11,
the filter returns
{}

(an empty dict)

If you run it in python 3.12, it raises an exception.

Traceback (most recent call last):
  File "/app/./test_filter.py", line 21, in <module>
    r = slice_filter({},":5")
        ^^^^^^^^^^^^^^^^^^^^^
  File "/app/./test_filter.py", line 13, in slice_filter
    return value[slice(*bits)]
           ~~~~~^^^^^^^^^^^^^^

KeyError: slice(None, 5, None)

def slice_filter(value, arg):
    """
    Return a slice of the list using the same syntax as Python's list slicing.
    """
    try:
        bits = []
        for x in str(arg).split(":"):
            if not x:
                bits.append(None)
            else:
                bits.append(int(x))
        return value[slice(*bits)]
    except (ValueError, TypeError) as e:
        return value  # Fail silently.




if __name__ == "__main__":
    r = slice_filter({},":5")
    print(f"{r=}")

I will try to work out why, but this at least describes the problem I think.

comment:4 by Tim Richardson, 5 months ago

the slice() function is being interpreted as key in 3.12 but not in prior python versions.

the calculation of

bits

is identical.

Python 3.12 docs say that slice objects are now hashable, so it gets treated as a key I suppose.
I wonder how it worked in prior python versions.

Including KeyError in the caught exceptions restores consistent behaviour.

def slice_filter(value, arg):
    """
    Return a slice of the list using the same syntax as Python's list slicing.
    """
    try:
        bits = []
        for x in str(arg).split(":"):
            if not x:
                bits.append(None)
            else:
                bits.append(int(x))
        return value[slice(*bits)]
    except (ValueError, TypeError, KeyError):
        return value  # Fail silently.

comment:5 by Tim Graham, 5 months ago

Easy pickings: set
Resolution: needsinfo
Status: closednew
Summary: Python 3.12 filter bug with Django 4.2.11slice filter crashes on an empty dict with Python 3.12
Triage Stage: UnreviewedAccepted

Thanks. It looks like unexpected usage since slicing dictionaries doesn't make sense. It won't hurt to fix it, but I don't think the fix will be backported so you may have to work around it in your code.

A test which passes on Python 3.11 and crashes on 3.12.

  • tests/template_tests/filter_tests/test_slice.py

    diff --git a/tests/template_tests/filter_tests/test_slice.py b/tests/template_tests/filter_tests/test_slice.py
    index 5a5dd6b155..23257b1282 100644
    a b class FunctionTests(SimpleTestCase):  
    5353    def test_fail_silently(self):
    5454        obj = object()
    5555        self.assertEqual(slice_filter(obj, "0::2"), obj)
     56
     57    def test_empty_dict(self):
     58        self.assertEqual(slice_filter({}, "1"), {})

comment:6 by Tim Richardson, 5 months ago

When I submit a patch for this, is there a preferred way I can request that it be backported to 4.2 LTS, so at least someone considers it?

comment:7 by Tim Richardson, 5 months ago

Owner: changed from nobody to Tim Richardson
Status: newassigned

comment:8 by Tim Richardson, 5 months ago

Has patch: set

comment:10 by Sarah Boyce, 5 months ago

Hello, thank you for the PR!

When I submit a patch for this, is there a preferred way I can request that it be backported to 4.2 LTS, so at least someone considers it?

Requesting on the ticket is fine. I have considered it, however I agree with Tim Graham that this doesn't qualify for a backport. 4.2 is only receiving security and data loss fixes.
slice is also documented to be for lists (https://docs.djangoproject.com/en/5.0/ref/templates/builtins/#slice), so I agree this is unexpected usage. I think you will need to make a custom template filter in your code for 4.2.

comment:11 by Sarah Boyce, 5 months ago

Triage Stage: AcceptedReady for checkin

comment:12 by Tim Richardson, 5 months ago

thanks. That was my first PR and it was a very well documented, clear and easy process.

comment:13 by Sarah Boyce <42296566+sarahboyce@…>, 5 months ago

Resolution: fixed
Status: assignedclosed

In e64d42e:

Fixed #35395 -- slice filter crashes on an empty dict with Python 3.12.

Keep consistent behaviour of slice() filter between python 3.12 and prior
versions in the case of a dict passed to the filter (catch the new to python
3.12 KeyError exception).

Note: See TracTickets for help on using tickets.
Back to Top