Opened 4 years ago

Closed 4 years ago

#30899 closed Cleanup/optimization (fixed)

Lazily compile regular expressions.

Reported by: Adam Johnson Owned by: Hasan Ramezani
Component: Core (Other) Version: dev
Severity: Normal Keywords:
Cc: me@… Triage Stage: Ready for checkin
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description

Inspired by [this article from Instagram](https://instagram-engineering.com/python-at-scale-strict-modules-c0bb9245c834)

Currently Django lazily compiles some regular expressions such as those in validators: https://github.com/django/django/blob/54ea290e5bbd19d87bd8dba807738eeeaf01a362/django/core/validators.py#L17

This is to save on import time.

There are other import-time re.compile calls throughout the codebase, these could be migrated to lazy regex compiles to save on their import time:

https://github.com/django/django/search?q=re.compile&unscoped_q=re.compile

Change History (8)

comment:1 by Mariusz Felisiak, 4 years ago

Summary: Lazily compile large regular expressionsLazily compile regular expressions.
Triage Stage: UnreviewedAccepted

comment:2 by Hasan Ramezani, 4 years ago

Owner: changed from nobody to Hasan Ramezani
Status: newassigned

I can work on this an prepare a patch.
Should I move the _lazy_re_compile function to django.utils.text?

comment:3 by Adam Johnson, 4 years ago

Yeah django.utils.text sounds like a sensible location to me. Keeping it private (starting with underscore, undocumented) for now seems wise.

comment:4 by Hasan Ramezani, 4 years ago

Has patch: set

Description provided on the PR

Last edited 4 years ago by Hasan Ramezani (previous) (diff)

comment:5 by Mariusz Felisiak, 4 years ago

Triage Stage: AcceptedReady for checkin

comment:6 by Mariusz Felisiak <felisiak.mariusz@…>, 4 years ago

In c4cba148:

Refs #30899 -- Moved _lazy_re_compile() to the django.utils.regex_helper.

comment:7 by Mariusz Felisiak <felisiak.mariusz@…>, 4 years ago

In 39a34d4:

Refs #30899 -- Made _lazy_re_compile() support bytes.

comment:8 by Mariusz Felisiak <felisiak.mariusz@…>, 4 years ago

Resolution: fixed
Status: assignedclosed

In e3d0b4d5:

Fixed #30899 -- Lazily compiled import time regular expressions.

Note: See TracTickets for help on using tickets.
Back to Top