id	summary	reporter	owner	description	type	status	component	version	severity	resolution	keywords	cc	stage	has_patch	needs_docs	needs_tests	needs_better_patch	easy	ui_ux
30563	Optimize django.forms.widgets.Media.__add__.	David Dorothy	David Smith	"While working with another project that make extensive use of `django.forms.widgets.Media` I discovered that the fix for ticket #30153 has unintended consequences on the performance of `Media.__add__`. If the number of Media objects added grows beyond a certain point (not sure it may be machine specific) then the performance of all subsequent `Media.__add__` calls becomes terrible.

This was causing page load times of several minutes on my development machine. I agree that you need to delay as long as possible as #30153 intends but it seems that there probably should be an upper bound on this so that performance does not suddenly decrease.

Here is some sample code that can reproduce the issue:

{{{
from django.forms import Media
import datetime

def create_media(MediaClass):
    '''Creates a simple Media object with only one or two items.'''
    return MediaClass(css={'all': ['main.css']}, js=['main.js'])

start = datetime.datetime.now()
media = create_media(Media)
for i in range(100000):
    media = media + create_media(Media)
    
print('100000 additions took: %s' % (datetime.datetime.now() - start))
}}}

On my machine several runs of this code result in times between 1:35 - 1:44 (eg. 1 minute 35 seconds to 1 minute 44 seconds). However, taking away one zero from the number of media objects runs in under a second. Near as I can tell this has to do with the memory used to store these arrays and when it gets past a certain point the performance is awful. Since I am not sure if it is machine specific, for reference my machine is a i7-8700 with 64 GB RAM.

Here is a sample that has a modified Media class that does not have theses issues:

{{{
from django.forms import Media
import datetime

def create_media(MediaClass):
    '''Creates a simple Media object with only one or two items.'''
    return MediaClass(css={'all': ['main.css']}, js=['main.js'])

class CustomMedia(Media):
    def __add__(self, other):
        combined = CustomMedia()
        if len(self._css_lists) + len(other._css_lists) > 1000:
            combined._css_lists = [self._css, other._css]
        else:
            combined._css_lists = self._css_lists + other._css_lists
        
        if len(self._js_lists) + len(other._js_lists) > 1000:
            combined._js_lists = [self._js, other._js]
        else:
            combined._js_lists = self._js_lists + other._js_lists
        
        return combined

start = datetime.datetime.now()
media = create_media(CustomMedia)
for i in range(100000):
    media = media + create_media(CustomMedia)
    
print('100000 additions took: %s' % (datetime.datetime.now() - start))
}}}

With this change it again runs in under a second. If I increase the number of loops the performance seems to change at a much more expected level.

I set an upper limit on the length allowed before a merge occurs. I'm not sure if the number of additions allowed should be a setting that can be adjusted to meet specific needs or if something that is just ""reasonably"" high is sufficient. It does appear that limiting the total number of items in the list to about 1000 works best on my machine. I'm also not sure that this is the best solution.

Thanks for your consideration."	Cleanup/optimization	closed	Forms	dev	Normal	fixed	media	Matthias Kestenholz	Ready for checkin	1	0	0	0	0	0