Opened 4 years ago

Closed 3 years ago

#17892 closed Bug (fixed)

Calling RegexURLResolver.__repr__ takes a lot of time (and memory)

Reported by: milosu Owned by: nobody
Component: Core (URLs) Version: 1.4-beta-1
Severity: Normal Keywords:
Cc: Triage Stage: Accepted
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no


I have an application which has about 70K RegexURLPatterns.

When I try to access some non-existent url in this app under Django 1.4RC1, it takes a huge amount of time - I mean 1 or 2 minutes - to generate the technical 404 response and also the memory consumption goes from 220MB to something about 460MB.

The problem is in a call to RegexURLResolver.repr, which is called about 3 times during the technical 404 debug response. It looks like the whole RegexURLResolver including all its sub-patterns is being recursively traversed during a call to repr.

I had to patch RegexURLResolver.repr to cut this CPU overhead.

Patch attached, but I'm not sure how to write a test case for it.

I'm also attaching the profiling results of unpatched Django 1.4RC.

I'm upgrading from Django 1.2.3 so I'm not sure if this behaviour is present in 1.3 or if it is new to 1.4, but from my point of view this is a release blocker.

Attachments (2)

resolver_repr.diff (1.1 KB) - added by milosu 4 years ago.
regex_repr_profile.txt (60.9 KB) - added by milosu 4 years ago.

Download all attachments as: .zip

Change History (4)

Changed 4 years ago by milosu

Changed 4 years ago by milosu

comment:1 Changed 4 years ago by jezdez

  • Needs documentation unset
  • Needs tests unset
  • Patch needs improvement unset
  • Severity changed from Release blocker to Normal
  • Triage Stage changed from Unreviewed to Accepted

comment:2 Changed 3 years ago by Claude Paroz <claude@…>

  • Resolution set to fixed
  • Status changed from new to closed

In [28fd876bae15df747d164dcae229840d2cf135ca]:

Fixed #17892 -- Do not include whole RegexURLPattern lists in RegexURLResolver repr

Thanks milosu for the report and the initial patch.

Note: See TracTickets for help on using tickets.
Back to Top