Opened 17 years ago

Closed 17 years ago

#5106 closed (fixed)

django.db.backend.*.base refactoring

Reported by: (removed) Owned by: Malcolm Tredinnick
Component: Database layer (models, ORM) Version: dev
Severity: Keywords:
Cc: Triage Stage: Design decision needed
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description

Splitting connection pooling up into two patches; this particular ticket holds refactoring of django.db.backend.*.base.DatabaseWrapper classes to remove redundancy, and give an easier spot to control connection pooling.

Very least, the misc backend implementation are a bit redundant- deriving from a common base class is a bit saner in this case for debugging and extension reasons.

Attachments (3)

backend-refactoring.patch (15.5 KB ) - added by (removed) 17 years ago.
backend-refactoring-v2.patch (70.3 KB ) - added by (removed) 17 years ago.
version/stage 2 of the refactoring; api breaking, but molds existing old style layout into new automatically, and allows for actual OOP trickery.
backend-refactoring-v2-correct-bitrot.patch (72.2 KB ) - added by (removed) 17 years ago.
re-cut v2, since adrians core/management horked v2 from being applied.

Download all attachments as: .zip

Change History (37)

by (removed), 17 years ago

Attachment: backend-refactoring.patch added

comment:1 by Chris Beaven, 17 years ago

Triage Stage: UnreviewedDesign decision needed

The whole backends area is a bit messy. This is a good start to tidying it up.

I was tempted to push straight to checkin, but I'll let some more eyes look at it before then.

comment:2 by (removed), 17 years ago

via http://groups.google.com/group/django-developers/browse_thread/thread/ab7649e4abd4e589 , might suggest getting the first patch posted into vcs; the second is following and is far more intrusive; actual api changes (no more django.db.backend access primarily), but constricts access to backend capabilities/ops/client/introspection through the wrapper object, thus allowing for far easier/intrusive hooking/override- specifically to just layer connection pooling over a backend, rather then having to write a pooler specific to each backend.

comment:3 by anonymous, 17 years ago

If you mean that code won't have access to django.db.backend any longer (or shouldn't be accessing it), you will want to start a design discussion on django-dev first. That's a very intrusive change and will break a lot of code, so we need to discuss the change in abstract first before wading through lots of lines of code.

comment:4 by Malcolm Tredinnick, 17 years ago

Owner: changed from Adrian Holovaty to Malcolm Tredinnick

Oops... last comment was me, by the way.

I'm generally +1 on the current patch, since it looks clean. Still want to look at it once or twice more, but it'll go in soon, I would expect.

by (removed), 17 years ago

version/stage 2 of the refactoring; api breaking, but molds existing old style layout into new automatically, and allows for actual OOP trickery.

in reply to:  3 comment:5 by (removed), 17 years ago

Replying to mtredinnick:

If you mean that code won't have access to django.db.backend any longer (or shouldn't be accessing it), you will want to start a design discussion on django-dev first. That's a very intrusive change and will break a lot of code, so we need to discuss the change in abstract first before wading through lots of lines of code.

Still has access to it, just has access to a compartmentalized version of it. Already converted existing code over to it w/in django; atm, getting v2 up in the tracker (upstream for the patch is bzr branch at http://pkgcore.org/~ferringb/django/connection-pooling) prior to kicking off the discussion; it's worth noting that with a rather simple getattr, it's possible to preserve the old api (preferably firing off a warning on access).

comment:6 by George Vilches <gav@…>, 17 years ago

Since we don't have an official voting system yet, just adding a comment that I'm +1 for this patch (the first or the second, I'll be happy with either), it would make the SQL logging additions much cleaner.

by (removed), 17 years ago

re-cut v2, since adrians core/management horked v2 from being applied.

comment:7 by Adrian Holovaty, 17 years ago

I'm inclined to check in the first patch, as it's a simpler / more atomic change, and I'd rather not check in substantial patches like the second patch in a single commit (mostly because they're hard to follow). I'm on it.

comment:8 by Adrian Holovaty, 17 years ago

(In [5949]) Refactored all database backends to inherit from a common base class to remove quite a bit of duplicated code. Thanks for the patch, Brian Harring. Refs #5106

comment:9 by Adrian Holovaty, 17 years ago

First patch is checked into trunk. Working on the second patch now; I'm going to split it over several commits, for sanity's sake.

comment:10 by Adrian Holovaty, 17 years ago

Continued refactoring in [5950].

comment:11 by Adrian Holovaty, 17 years ago

(In [5951]) Refactored get_date_extract_sql() to DatabaseOperations.date_extract_sql(). Refs #5106

comment:12 by Adrian Holovaty, 17 years ago

(In [5952]) Refactored get_date_trunc_sql() to DatabaseOperations.date_trunc_sql(). Refs #5106

comment:13 by Adrian Holovaty, 17 years ago

(In [5953]) Refactored get_datetime_cast_sql() to DatabaseOperations.datetime_cast_sql(). Refs #5106

comment:14 by Adrian Holovaty, 17 years ago

(In [5955]) Refactored get_deferrable_sql() to DatabaseOperations.deferrable_sql(). Refs #5106

comment:15 by Adrian Holovaty, 17 years ago

(In [5956]) Refactored get_drop_foreignkey_sql() to DatabaseOperations.drop_foreignkey_sql(). Refs #5106

comment:16 by Adrian Holovaty, 17 years ago

(In [5957]) Refactored get_fulltext_search_sql() to DatabaseOperations.fulltext_search_sql(). Refs #5106

comment:17 by Adrian Holovaty, 17 years ago

(In [5958]) Refactored get_last_insert_id() to DatabaseOperations.last_insert_id(). Refs #5106

comment:18 by Adrian Holovaty, 17 years ago

(In [5959]) Refactored get_limit_offset_sql() to DatabaseOperations.limit_offset_sql(). Refs #5106

comment:19 by Adrian Holovaty, 17 years ago

(In [5960]) Refactored get_max_name_length() to DatabaseOperations.max_name_length(). Refs #5106

comment:20 by Adrian Holovaty, 17 years ago

(In [5961]) Refactored get_pk_default_value() to DatabaseOperations.pk_default_value(). Refs #5106

comment:21 by Adrian Holovaty, 17 years ago

(In [5962]) Refactored get_random_function_sql() to DatabaseOperations.random_function_sql(). Refs #5106

comment:22 by Adrian Holovaty, 17 years ago

(In [5963]) Refactored get_sql_flush() to DatabaseOperations.sql_flush(). Refs #5106

comment:23 by Adrian Holovaty, 17 years ago

(In [5964]) Refactored get_sql_sequence_reset() to DatabaseOperations.sequence_reset_sql(). Refs #5106

comment:24 by Adrian Holovaty, 17 years ago

(In [5965]) Refactored get_start_transaction_sql() to DatabaseOperations.start_transaction_sql(). Refs #5106

comment:25 by Adrian Holovaty, 17 years ago

(In [5966]) Refactored get_tablespace_sql() to DatabaseOperations.tablespace_sql(). Refs #5106

comment:26 by Adrian Holovaty, 17 years ago

(In [5967]) Refactored quote_name() to DatabaseOperations.quote_name(). Refs #5106

comment:27 by Adrian Holovaty, 17 years ago

(In [5972]) Moved postgresql backend DatabaseOperations class into a new module, postgresql/operations.py, so that it can be imported by both the postgresql and postgresql_psycopg2 backends. Hence the two backends no longer have a duplicated DatabaseOperations class

comment:28 by Adrian Holovaty, 17 years ago

(In [5974]) Implemented BaseDatabaseFeatures and changed all code to access it -- connection.features.foo instead of backend.foo

comment:29 by Adrian Holovaty, 17 years ago

Note that I used "features" instead of "capabilities" because the latter just takes too long to type.

comment:30 by Adrian Holovaty, 17 years ago

(In [5976]) Refactored get_query_set_class() to DatabaseOperations.query_set_class(). Also added BaseDatabaseFeatures.uses_custom_queryset. Refs #5106

comment:31 by Adrian Holovaty, 17 years ago

(In [5977]) Refactored get_field_cast_sql() to DatabaseOperations.field_cast_sql(). Refs #5106

comment:32 by Adrian Holovaty, 17 years ago

(In [5978]) Refactored get_drop_sequence() to DatabaseOperations.drop_sequence_sql(). Refs #5106

comment:33 by Adrian Holovaty, 17 years ago

(In [5982]) Refactored OPERATOR_MAPPING so that it exists as django.db.connection.operators instead of django.db.backend.OPERATOR_MAPPING. Refs #5106

comment:34 by Adrian Holovaty, 17 years ago

Resolution: fixed
Status: newclosed

OK, I think we can safely mark this as fixed. I've made 95% of the changes from the patch, with the exception of making DatabaseError, IntegrityError and the introspection and creation functionality accessible via django.db.connection object. Those changes can happen in the near future, if they need to be made. (Please open separate tickets w/ patches for those changes if you think they're worth doing.)

I ended up making small naming/implementation changes in places throughout this fix, but Brian's core design for this refactoring remained intact. Thanks for the great suggestions and patch, Brian!

Note: See TracTickets for help on using tickets.
Back to Top