Opened 7 years ago

Closed 4 years ago

#13053 closed New feature (wontfix)

Support for non-static DB connection info

Reported by: Mathias Nielsen Owned by: nobody
Component: Database layer (models, ORM) Version: master
Severity: Normal Keywords:
Cc: Triage Stage: Someday/Maybe
Has patch: no Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no


It would be great if we could specify a class or function as the source of the DB-connection information.

I am thinking something like:

    'default': {
        'INFO_FINDER': 'myproject.myDBFinderApp.MyDBFinder',
    'users': {
        'NAME': 'user_data',
        'ENGINE': 'django.db.backends.mysql',
        'USER': 'mysql_user',
        'PASSWORD': 'priv4te'

This would then get executed every time something was needed from the 'default' DB.

Where would this be useful?
In my current project I am building a webapp where each client gets their own subdomain and is free to create users inside it. It would be very useful to shard data by this subdomain since my clients never have to communicate via the webapp. This means that I have to be able to create a new DB and connect to it whenever a new client signs up. This is not possible with the current setup using

This method of dividing clients is not new at all so I guess there is many more than myself that will benefit from this, but more importantly (I think) is that it will most likely cover any other corner-case that Django is not capable of, since it will be so easy to just write your own DBFinder and link it. Any DBFinder will, of course, have to always return the connection info in the correct format just like to one normally used in settings.

I have tried to do it myself, but I just cant make any sense of it and have no idea where to do the check to see if settings give connection-info or just a place to get it.
But I cannot imagine it would be any huge amount of work for the people who wrote the thing or just about anyone with greater skills in Django-development than my own.

Attachments (1)

a_s-draft.diff (1.7 KB) - added by Mathias Nielsen 6 years ago.
Initial draft of the idea. Most likely not commit-ready.

Download all attachments as: .zip

Change History (9)

comment:1 Changed 7 years ago by Russell Keith-Magee

Needs documentation: unset
Needs tests: unset
Patch needs improvement: unset
Triage Stage: UnreviewedSomeday/Maybe

Accepting the use case; not necessarily sold on the proposed implementation approach.

It might be possible to do right now by replacing the entire settings dictionary object with an object that implements the dict interface.

The bigger issue will be getting access to useful context information (e.g., the domain where an request originates) to be able to shard on that basis. I'm not sure I see an easy way to get access to this data where it will be required.

comment:2 Changed 7 years ago by Mathias Nielsen

I will look into your suggestion on the settings dictionary object replacement.

On the subject of useful context information:
I have found something called django_globals
It is very simple. It just uses threading.local() to store values, but if it will work everywhere I have no idea.

I have been testing it on uWSGI with Cherokee in front and it seems to be behaving as one would want, but I have no idea if it is a viable method for any kind of setup. E.g. if it will share the values between requests in certain kinds of setups.

Changed 6 years ago by Mathias Nielsen

Attachment: a_s-draft.diff added

Initial draft of the idea. Most likely not commit-ready.

comment:3 Changed 6 years ago by Mathias Nielsen

And now, 8 months later, I finally get around to looking at it.
I am submitting a draft for, what I have called, ActiveSettings

This is just meant to be a hook that will let external projects get a chance to change the settings as they are being requested.
If this, or something like it, could be implemented, then I can just construct an external project that will let people solve this problem of sharding costumers and without getting in the way of anyone that will not be needing it, which I suspect is the largest part of the Django user-base.

The diff is just a draft since this is my first contribution to Django, I thought I would post it for scrutiny before doing the docs and tests. I am sure there is a lot of errors in it, but the idea should be clear.

Also, should I be taking this up on the mailing list? I read something about that in the contribution guide.

comment:4 Changed 5 years ago by Luke Plant

Type: New feature

comment:5 Changed 5 years ago by Luke Plant

Severity: Normal

comment:6 Changed 5 years ago by Aymeric Augustin

UI/UX: unset

Change UI/UX from NULL to False.

comment:7 Changed 5 years ago by Aymeric Augustin

Easy pickings: unset

Change Easy pickings from NULL to False.

comment:8 Changed 4 years ago by Aymeric Augustin

Resolution: wontfix
Status: newclosed

This is a subset of the general problem of multi-tenancy (#15089).

The approach taken by the patch doesn't work because Django maintains internal caches that depend on the values of settings. If you change the active settings between two requests handled by the same Python process, these caches may hold stale values, with undefined results.

The settings_changed signal might help, but it's only designed for tests at this time, and I'm not sure it covers all settings at this time.

I'm going to close this ticket because I don't think the solution to multi-tenancy will come from hacks in the settings.

Note: See TracTickets for help on using tickets.
Back to Top