Opened 9 years ago

Last modified 9 years ago

#24224 closed Cleanup/optimization

sqlmigrate is slow — at Version 3

Reported by: TorstenRottmann Owned by: nobody
Component: Migrations Version: 1.7
Severity: Normal Keywords: migration slow
Cc: Triage Stage: Unreviewed
Has patch: no Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description (last modified by TorstenRottmann)

When requesting an sqlmigrate (or when doing a test or migrate also), it takes a very long time to compute the sql for a particular app migration. This app has some 78 models and is highly interconnected with ForeignKeys (100+).

I ran the sqlmigrate through the profiler an got these results:

         74628367 function calls (74229507 primitive calls) in 132.448 seconds

   Ordered by: internal time

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
   552977   18.515    0.000   29.690    0.000 __init__.py:292(deconstruct)
    40096   14.144    0.000   19.827    0.000 related.py:276(swappable_setting)
   318698   11.825    0.000   20.907    0.000 __init__.py:1011(__init__)
   575182    9.425    0.000   12.465    0.000 __init__.py:123(__init__)
   571729    7.252    0.000   96.117    0.000 state.py:277(construct_fields)
16773892/16722006    6.668    0.000    6.707    0.000 {getattr}
    85138    2.822    0.000    4.825    0.000 __init__.py:1092(__init__)
553122/553106    2.799    0.000    6.470    0.000 {__import__}
637429/318719    1.908    0.000    2.137    0.000 deconstruct.py:15(__new__)
  3236790    1.900    0.000    1.900    0.000 {method 'startswith' of 'unicode' objects}
3881364/3881362    1.884    0.000    4.418    0.000 {isinstance}
  5915601    1.548    0.000    1.812    0.000 options.py:283(_swapped)
    43868    1.507    0.000    1.507    0.000 base.py:33(subclass_exception)
   901714    1.491    0.000    2.534    0.000 abc.py:128(__instancecheck__)
21946/19820    1.472    0.000   24.363    0.001 base.py:61(__new__)
  2395625    1.467    0.000    1.467    0.000 {hasattr}
 11291797    1.448    0.000    1.460    0.000 {method 'get' of 'dict' objects}
   298364    1.394    0.000    6.343    0.000 __init__.py:572(contribute_to_class)
553078/553076    1.375    0.000    8.543    0.000 __init__.py:20(import_module)
...

Waiting more than a minute for a test to start is very annoying ...

My first question, when looking at the profile is: What is imported there over half a million times?

Change History (3)

comment:1 by Claude Paroz, 9 years ago

Are you able to test on 1.8alpha? Most probably a duplicate of #23745.

comment:2 by TorstenRottmann, 9 years ago

Description: modified (diff)

comment:3 by TorstenRottmann, 9 years ago

Description: modified (diff)
Note: See TracTickets for help on using tickets.
Back to Top