Memoize _os.safe_join()

[German M. Bravo]

I found out safe_join() was taking a big chunk of time in my project (I have many apps and the load_template_source() calls use safe_join).

Since safe_join() does a lot of repetitive things when it joins paths in the same platform, and since the number of joined paths can be sort of limited, I made a patch to memoize it.

Before:

Stats:
         1372899 function calls (1333476 primitive calls) in 1.744 seconds

   Ordered by: internal time

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
    16170    0.165    0.000    0.240    0.000 posixpath.py:312(normpath)
    15288    0.148    0.000    0.148    0.000 {posix.getcwdu}
     8085    0.073    0.000    0.073    0.000 {open}
     1018    0.051    0.000    0.051    0.000 {method 'recv' of '_socket.socket' objects}
    23375    0.048    0.000    0.068    0.000 posixpath.py:60(join)
   191264    0.046    0.000    0.046    0.000 {isinstance}
     8085    0.042    0.000    0.566    0.000 _os.py:28(safe_join)
    80087    0.035    0.000    0.035    0.000 {method 'startswith' of 'unicode' objects}
    16170    0.035    0.000    0.482    0.000 _os.py:18(abspathu)
   2468/1    0.034    0.000    1.533    1.533 base.py:740(render)
48746/40126    0.033    0.000    0.348    0.000 {getattr}
   200177    0.033    0.000    0.033    0.000 {method 'append' of 'list' objects}
8710/7896    0.029    0.000    0.062    0.000 base.py:373(__setattr__)
6884/6587    0.028    0.000    0.342    0.000 base.py:667(_resolve_lookup)
45540/43923    0.028    0.000    0.087    0.000 encoding.py:54(force_unicode)
      147    0.025    0.000    0.662    0.005 app_directories.py:52(load_template_source)

After:

Stats:
         904358 function calls (864935 primitive calls) in 1.123 seconds

   Ordered by: internal time

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
     1017    0.050    0.000    0.050    0.000 {method 'recv' of '_socket.socket' objects}
     8085    0.048    0.000    0.048    0.000 {open}
   158630    0.040    0.000    0.040    0.000 {isinstance}
   2468/1    0.032    0.000    0.888    0.888 base.py:740(render)
48746/40126    0.032    0.000    0.141    0.000 {getattr}
8710/7896    0.028    0.000    0.063    0.000 base.py:373(__setattr__)
6884/6587    0.027    0.000    0.135    0.000 base.py:667(_resolve_lookup)
    30091    0.020    0.000    0.029    0.000 {hasattr}
29370/27753    0.019    0.000    0.070    0.000 encoding.py:54(force_unicode)
       18    0.019    0.001    0.019    0.001 {method 'execute' of 'psycopg2._psycopg.cursor' objects}
     6618    0.018    0.000    0.020    0.000 __init__.py:160(__getattr__)

[anonymous]

I'm wontfixing this. If safe join is that common in your application you can memoize it yourself, I do not think it is sufficiently common in django code as to justify the memory increase (remember: memoization is a memory/speed tradeoff) for all users.

[Alex Gaynor]

That was me.