Opened 3 years ago

Closed 3 years ago

#19861 closed Bug (fixed)

Improve connection._dirty flag handling

Reported by: akaariai Owned by: nobody
Component: Database layer (models, ORM) Version: master
Severity: Normal Keywords:
Cc: Triage Stage: Ready for checkin
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no


The connection._dirty flag currently represents a mix of things:

  • It is sometimes used for tracking if transaction management is enabled (enabled when self._dirty is not None). This doesn't work correctly, as the _dirty flag is never reset to None. And, three valued flag isn't a good idea anyways.
  • It is used to track if the connection needs commit, but only inside transaction management. When outside tx management it will not be set by Django.

Patch available here: I believe the commit resolves a bunch of subtle bugs, but it likely also causes a bunch of subtle backwards incompatibilities. One such change is that .commit() and .rollback() can now be called always, not only inside transaction management.

I tried the patch and it passes all tests on sqlite and postgresql (with autocommit off, autocommit on I had time to run just backends transactions transactions_regress and these pass).

With postgresql I also ran the tests with transaction state tracking enabled (the commented out code in psycopg2/ There is one false positive (Django believes the transaction to be in transaction, in reality it isn't) caused by test_cursor_executemany_with_empty_params_list in backends tests. executemany with empty params does nothing, but the dirty flag handling doesn't spot this. This is likely a non issue (or, if this is a issue, it is a bug in current Django, too). There are no false negatives (Django doesn't believe the connection to be in tx, but it is).

The patch also contains a bunch of fixes to tests doing various weird things with transaction management.

Change History (3)

comment:1 Changed 3 years ago by carljm

  • Needs documentation unset
  • Needs tests unset
  • Patch needs improvement unset
  • Triage Stage changed from Unreviewed to Accepted

comment:2 Changed 3 years ago by akaariai

  • Triage Stage changed from Accepted to Ready for checkin

A new version of the patch available from ​

I think this one is ready for commit now.

comment:3 Changed 3 years ago by Anssi Kääriäinen <akaariai@…>

  • Resolution set to fixed
  • Status changed from new to closed

In 50328f0a618674b7143d86acaa7016c5293e9774:

Fixed #19861 -- Transaction ._dirty flag improvement

There were a couple of errors in ._dirty flag handling:

  • It started as None, but was never reset to None.
  • The _dirty flag was sometimes used to indicate if the connection was inside transaction management, but this was not done consistently. This also meant the flag had three separate values.
  • The None value had a special meaning, causing for example inability to commit() on new connection unless enter/leave tx management was done.
  • The _dirty was tracking "connection in transaction" state, but only in managed transactions.
  • Some tests never reset the transaction state of the used connection.
  • And some additional less important changes.

This commit has some potential for regressions, but as the above list
shows, the current situation isn't perfect either.

Note: See TracTickets for help on using tickets.
Back to Top