Improve performance of import_string().

[Mariusz Felisiak]

We can improve performance of import_string() by avoiding multiple imports for the same path. Proposed by piaoxue1949 in ​PR.

[Mariusz Felisiak]

​PR

[Keryn Knight]

Here's some back of the napkin cprofile output on my laptop, in case piaoxue1949 is unable to provide before/after based on the proposed changes:

Before, as of 46c8df640cfed5dd525ac1bcf5ad7e57b7ff2571:

In [2]: get_backends()
Out[2]:
[<django.contrib.auth.backends.ModelBackend at 0x105307e50>,
 <django.contrib.auth.backends.AllowAllUsersModelBackend at 0x1053bf790>,
 <django.contrib.auth.backends.RemoteUserBackend at 0x105307940>,
 <django.contrib.auth.backends.AllowAllUsersRemoteUserBackend at 0x105307790>]

In [3]: %timeit get_backends()
45.2 µs ± 1.02 µs per loop (mean ± std. dev. of 7 runs, 10000 loops each)

In [4]: %prun for _ in range(10000): get_backends()
   1940003 function calls in 0.911 seconds
   Ordered by: internal time
   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
    80000    0.119    0.000    0.228    0.000 <frozen importlib._bootstrap>:166(_get_module_lock)
    40000    0.118    0.000    0.718    0.000 <frozen importlib._bootstrap>:1002(_find_and_load)
    80000    0.079    0.000    0.100    0.000 <frozen importlib._bootstrap>:87(acquire)
    80000    0.077    0.000    0.097    0.000 <frozen importlib._bootstrap>:112(release)
    80000    0.058    0.000    0.089    0.000 <frozen importlib._bootstrap>:58(__init__)
    80000    0.054    0.000    0.080    0.000 <frozen importlib._bootstrap>:185(cb)
    40000    0.034    0.000    0.243    0.000 <frozen importlib._bootstrap>:203(_lock_unlock_module)
    40000    0.032    0.000    0.856    0.000 module_loading.py:15(import_string)
   160000    0.030    0.000    0.030    0.000 {built-in method _thread.allocate_lock}
    40000    0.029    0.000    0.800    0.000 __init__.py:109(import_module)
    40000    0.028    0.000    0.194    0.000 <frozen importlib._bootstrap>:156(__enter__)
    40000    0.024    0.000    0.761    0.000 <frozen importlib._bootstrap>:1018(_gcd_import)
    ... many more lines, snipped for brevity

and after, using the cached_import to peek directly into sys.modules

In [3]: %timeit get_backends()
3.78 µs ± 24.1 ns per loop (mean ± std. dev. of 7 runs, 100000 loops each)

In [4]: %prun for _ in range(10000): get_backends()
   260003 function calls in 0.115 seconds

   Ordered by: internal time

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
    40000    0.024    0.000    0.061    0.000 module_loading.py:15(import_string)
    10000    0.023    0.000    0.105    0.000 __init__.py:24(_get_backends)
    40000    0.018    0.000    0.028    0.000 module_loading.py:8(cached_import)
    40000    0.017    0.000    0.078    0.000 __init__.py:20(load_backend)
    40000    0.010    0.000    0.010    0.000 {built-in method builtins.getattr}
    40000    0.010    0.000    0.010    0.000 {method 'rsplit' of 'str' objects}
        1    0.005    0.005    0.115    0.115 <string>:1(<module>)
    40000    0.004    0.000    0.004    0.000 {method 'append' of 'list' objects}
    10000    0.004    0.000    0.109    0.000 __init__.py:37(get_backends)
        1    0.000    0.000    0.115    0.115 {built-in method builtins.exec}
        1    0.000    0.000    0.000    0.000 {method 'disable' of '_lsprof.Profiler' objects}

that's the whole list. Not stripped for brevity.

[Mariusz Felisiak <felisiak.mariusz@…>]

In ecf87ad5:

Fixed #33099 -- Improved performance of import_string().

This improves performance of import_string() by avoiding multiple
imports for the same path.

Thanks Andrew Godwin and Keryn Knight for the implementation idea.