Opened 12 years ago

Closed 12 years ago

#17892 closed Bug (fixed)

Calling RegexURLResolver.__repr__ takes a lot of time (and memory)

Reported by: milosu Owned by: nobody
Component: Core (URLs) Version: 1.4-beta-1
Severity: Normal Keywords:
Cc: Triage Stage: Accepted
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description

I have an application which has about 70K RegexURLPatterns.

When I try to access some non-existent url in this app under Django 1.4RC1, it takes a huge amount of time - I mean 1 or 2 minutes - to generate the technical 404 response and also the memory consumption goes from 220MB to something about 460MB.

The problem is in a call to RegexURLResolver.repr, which is called about 3 times during the technical 404 debug response. It looks like the whole RegexURLResolver including all its sub-patterns is being recursively traversed during a call to repr.

I had to patch RegexURLResolver.repr to cut this CPU overhead.

Patch attached, but I'm not sure how to write a test case for it.

I'm also attaching the profiling results of unpatched Django 1.4RC.

I'm upgrading from Django 1.2.3 so I'm not sure if this behaviour is present in 1.3 or if it is new to 1.4, but from my point of view this is a release blocker.

Attachments (2)

resolver_repr.diff (1.1 KB ) - added by milosu 12 years ago.
regex_repr_profile.txt (60.9 KB ) - added by milosu 12 years ago.

Download all attachments as: .zip

Change History (4)

by milosu, 12 years ago

Attachment: resolver_repr.diff added

by milosu, 12 years ago

Attachment: regex_repr_profile.txt added

comment:1 by Jannis Leidel, 12 years ago

Severity: Release blockerNormal
Triage Stage: UnreviewedAccepted

comment:2 by Claude Paroz <claude@…>, 12 years ago

Resolution: fixed
Status: newclosed

In [28fd876bae15df747d164dcae229840d2cf135ca]:

Fixed #17892 -- Do not include whole RegexURLPattern lists in RegexURLResolver repr

Thanks milosu for the report and the initial patch.

Note: See TracTickets for help on using tickets.
Back to Top