Opened 15 years ago

Closed 12 years ago

Last modified 12 years ago

#11449 closed Cleanup/optimization (wontfix)

Performance regression loading from fixtures in 1.1 & trunk

Reported by: novalis Owned by: nobody
Component: Core (Serialization) Version: 1.1-beta
Severity: Normal Keywords: perf
Cc: em@… Triage Stage: Accepted
Has patch: no Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description

I have created a test that does nothing but read in a fixture. It goes significantly slower on Django 1.1-beta and trunk than it does in Django 1.0.

1.0: Ran 1 test in 4.940s
1.1: Ran 1 test in 21.541s
trunk: Ran 1 test in 21.228s

My rejected patch in #11181 reduces cuts each of these times in half -- but does not fix the regression.

Attachments (1)

perf.zip (223.9 KB ) - added by Russell Keith-Magee 15 years ago.
Test case that apparently demonstrates the problem.

Download all attachments as: .zip

Change History (14)

comment:1 by Russell Keith-Magee, 15 years ago

I have to say this conflicts with my personal experience - I have found that tests under v1.1 run much faster than v1.0 tests, as a result of the transactional test case changes.

Would you care to share your exact test code?

comment:2 by Alex Gaynor, 15 years ago

Resolution: worksforme
Status: newclosed

It's been a month with no new information, closing as worksforme.

comment:3 by Russell Keith-Magee, 15 years ago

Resolution: worksforme
Status: closedreopened

@alex - There was a discussion on Django-users; the reporter just didn't update this ticket with the helpful details. I'll upload his test case.

by Russell Keith-Magee, 15 years ago

Attachment: perf.zip added

Test case that apparently demonstrates the problem.

comment:4 by Russell Keith-Magee, 14 years ago

milestone: 1.2
Triage Stage: UnreviewedAccepted

Provided test case is about 50% slower under 1.1 than under 1.0 in my tests; 1.2-alpha is slightly faster than 1.1, but not faster than 1.0. Worth some investigation.

comment:5 by anonymous, 14 years ago

Component: UncategorizedSerialization

comment:6 by Jacob, 14 years ago

Keywords: perf added
milestone: 1.21.3

With 1.2 in the final stages and no more information about why this is slow, I'm booting it out of the milestone. Fixture loading isn't critical in deployment, so this shouldn't block the release.

comment:7 by EmilStenstrom, 14 years ago

Cc: em@… added

In a big live project, our fixture loading is also slow... But for some reason, setting DEBUG to true at the top of our tests.py makes them fast again:

from django.conf import settings
settings.DEBUG = True

Or maybe this is some side effect of how we load our test_settting.py file when testing:

#!/usr/bin/env python
import sys
from django.core.management import execute_manager

try:
    if len(sys.argv) > 1 and sys.argv[1] == 'test':
        import test_settings as settings
    else:
        import settings
except ImportError:    
    sys.stderr.write("Error: Can't find the file 'settings.py' in the directory containing %r. It appears you've customized things.\nYou'll have to run django-admin.py, passing it your settings module.\n(If the file settings.py does indeed exist, it's causing an ImportError somehow.)\n" % __file__)
    sys.exit(1)

if __name__ == "__main__":
    execute_manager(settings)

I'm not sure, but I thought I'd throw it out there as an idea of where to start looking.

comment:8 by Julien Phalip, 13 years ago

Severity: Normal
Type: Cleanup/optimization

comment:9 by Jacob, 12 years ago

milestone: 1.3

Milestone 1.3 deleted

comment:11 by Aymeric Augustin, 12 years ago

UI/UX: unset

Change UI/UX from NULL to False.

comment:12 by Aymeric Augustin, 12 years ago

Easy pickings: unset

Change Easy pickings from NULL to False.

comment:13 by Claude Paroz, 12 years ago

Resolution: wontfix
Status: reopenedclosed

Here are the benchmark results on my machine:

1.0 16s
1.1 23.7s
1.2 20.5s
1.3 19.5s
1.4 22.2s
dev 23s

I tried to profile the execution and saw that the slowdown was in the object save/save_base method. This is not related to serialization, but simply that saving objects is slower now than in the 1.0 days.

comment:14 by Anssi Kääriäinen, 12 years ago

FWIW I think there is not much room for improvement in save_base itself, but there is plenty to improve in qs.clone() and generating the SQL string. Could you try the test with #16759 applied? I would guess you get around 20s execution time with that.

Note: See TracTickets for help on using tickets.
Back to Top