Opened 6 hours ago

Closed 2 hours ago

#35853 closed New feature (wontfix)

Disable garbage collection during transaction.atomic.

Reported by: Matej Spiller Muys Owned by:
Component: Database layer (models, ORM) Version: 5.1
Severity: Normal Keywords: gc, atomic, transaction
Cc: Triage Stage: Unreviewed
Has patch: no Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description

We noticed that occasionally atomic blocks are keeping transaction open for longer than expected.
This was traced to GC being run during atomic operation affecting the DB performance since locks are released too late.

We added tracing to GC and transactional.atomic() and we noticed that GC is keeping DB transaction longer from 0.5 to 1 second.

To avoid that we extended atomic object to call gc.disable before creating new transaction and gc.enable after commit or rollback.
This positively impacted database load. Because we use mod_wsgi with a lot of workers and horizontally scaling the app it happens quite often that GC is run inside a DB transaction making whole system slower.

It would be nice to have this buildin in Django (opt-out, opt-in) for better DB performance.

Change History (1)

comment:1 by Natalia Bidart, 2 hours ago

Component: UncategorizedDatabase layer (models, ORM)
Resolution: wontfix
Status: newclosed

Hello Matej Spiller Muys, thank you for taking the time to create this report.

It might be worth asking on the Django forum for help/input and then confirming if something new would need adding to Django (and what that would look like). We usually ask contributors to get feedback from the community through a forum discussion first, so we can confirm there is community consensus that Django should add this feature.

From my personal view, this seems like a very specific need arising from a niche use case. I believe you could implement your own atomic decorator on top of Django's and disable the garbage collector on entry and re-enable on exit? I'm not sure this applies to the broader ecosystem, and Django is a framework designed to offer robust and accurate solutions for common scenarios. But if you get positive feedback from the forum, please comment back and link the forum conversation here so we reopen the ticket.

Note: See TracTickets for help on using tickets.
Back to Top