Opened 10 years ago
Last modified 10 years ago
#24396 new Cleanup/optimization
Migrate only required models to the "other" database
| Reported by: | Aymeric Augustin | Owned by: | nobody |
|---|---|---|---|
| Component: | Testing framework | Version: | dev |
| Severity: | Normal | Keywords: | |
| Cc: | Josh Smeaton, Michael Manfre | Triage Stage: | Accepted |
| Has patch: | no | Needs documentation: | no |
| Needs tests: | no | Patch needs improvement: | no |
| Easy pickings: | no | UI/UX: | no |
Description
It seems wasteful to create hundreds of tables on the "other" database when only a few tests of Django's test suite touch it.
I'm attaching a patch optimizing this. However it doesn't appear to save much time in practice. "Rendering model states" is the killer.
Attachments (1)
Change History (5)
by , 10 years ago
| Attachment: | 24396.patch added |
|---|
comment:1 by , 10 years ago
| Cc: | added |
|---|
comment:2 by , 10 years ago
Thanks for knocking this up. I just ran the the test suite on Oracle with this patch:
1minute 53 seconds to create the first database
27 seconds to create the second database
(The cleanup/dropping databases is also heavily weighted against the first database - I didn't measure but it would have been around 1 minute and 30 seconds respectively)
Without Patch:
2 minutes to create the first database
1 minute 35 seconds to create the second database.
About 1 minute is saved on the creation of the second database. I suspect mssql will see similar improvements, so I'll tag manfre here incase he wants to check it out.
I have done some measurements WRT Oracle and migrations but I abandoned them. Querying for table_names, view_names, and constraints takes quite a long time, and the results are never cached (clearing the cache wasn't as straightforward as it could have been).
https://github.com/jarshwah/django/tree/migrate-other-tests incase others struggle with git apply x.patch like I did :P
comment:3 by , 10 years ago
| Cc: | added |
|---|
comment:4 by , 10 years ago
| Triage Stage: | Unreviewed → Accepted |
|---|
Wall timing of test runs:
Using jarshwah's branch with this patch,
Command line to start of create "other" database is 2 minutes.
Command line to start of first test is 3 minutes 30 seconds.
Using latest master,
Command line to start of create "other" database is 2 minutes 1 second.
Command line to start of first test is 3 minutes 50 seconds.
~20 seconds speed up. This is a worthwhile improvement that will add up to a lot of saved time based upon how frequently I run the test suite.
A quick test shows that it takes 49 seconds to start running the tests with this patch vs. 53 seconds without on Postgres.
On SQLite the difference isn't measurable.