Django

Code

Ticket #5390 (assigned)

Opened 2 years ago

Last modified 1 week ago

Add signals to ManyRelatedManager

Reported by: Ludovico Magnocavallo <ludo@qix.it> Assigned to: rvdrijst (accepted)
Milestone: Component: Database layer (models, ORM)
Version: SVN Keywords: manytomanyfield feature signals
Cc: sjulean@ideacentrum.eu, django@erik.karulf.com, vsevolod.solovyov@gmail.com, sean@gmx.org, patrick.lauber@dvio.ch, chpapa@gmail.com, thiago.salves@gmail.com, oliver@bluelavatech.com, rvdrijst, piranha@piranha.org.ua, palewire@palewire.com, maxischmeii@gmail.com, django@ulo.pe, lvscar@gmail.com, brent.hagany@gmail.com Triage Stage: Design decision needed
Has patch: 1 Needs documentation: 0
Needs tests: 0 Patch needs improvement: 0

Description

There's currently no way to run some custom code when the ManyRelatedManager? adds, removes, or clears objects. Extending the ManyRelatedManager? is very hard, since it's dynamically created each time, and the ManyRelatedObjectsDescriptor? prevents any access to it from its container class. A simple solution is to add three signals, and hook them into the dispatching mechanism when adding, removing, or deleting a related object.

A typical use case for attaching custom code to a MayRelatedManager? is blog categories: having a num_posts field in the Category models, and updating it each time a category is added/removed to/from a post, speeds up considerably a simple category list that excludes empty categories, and counts only published posts. The difference in resource usage is huge -- one query with two simple filters against one query for the list, plus one for each category (yes, you can cache the output, but think eg of a multi-blog app where posts are frequently updated, and a cache miss is very expensive), and the resulting code is much simpler.

A simple patch is attached to this ticket, I have tested it briefly against some of my models and it appears to work ok.

Attachments

related_manager.diff (2.9 kB) - added by Ludovico Magnocavallo <ludo@qix.it> on 09/10/07 11:43:53.
related_manager.2.diff (2.9 kB) - added by ludo@qix.it on 09/10/07 16:13:33.
better patch, checked against django regression tests
related.diff (2.7 kB) - added by Ludovico Magnocavallo <ludo@qix.it> on 09/11/07 10:00:08.
Cleaner variant of patch #2
models.py (3.0 kB) - added by Ludovico Magnocavallo <ludo@qix.it> on 09/11/07 10:00:32.
Test models
tests.py (1.6 kB) - added by Ludovico Magnocavallo <ludo@qix.it> on 09/11/07 10:00:52.
Test cases
m2msignals_doc.txt (1.0 kB) - added by ludo@qix.it on 01/22/08 03:53:13.
patch for the db-api docs
related.field_name.diff (4.3 kB) - added by murkt on 01/30/08 13:11:34.
related.diff with added field_name parameter (signals and dispatcher are already imported)
m2m_signals.diff (4.4 kB) - added by piranha on 08/07/08 03:40:56.
m2m signals patch, updated against signals refactoring
complete-patch.diff (16.7 kB) - added by rvdrijst on 01/23/09 11:07:41.
A complete patch with code, docs and tests for an m2m_changed signal.

Change History

09/10/07 11:43:53 changed by Ludovico Magnocavallo <ludo@qix.it>

  • attachment related_manager.diff added.

09/10/07 16:13:33 changed by ludo@qix.it

  • attachment related_manager.2.diff added.

better patch, checked against django regression tests

09/11/07 03:50:50 changed by Ludovico Magnocavallo <ludo@qix.it>

  • needs_better_patch changed.
  • needs_tests changed.
  • needs_docs changed.

Sending source_col and target_col to the dispatcher is probably unnecessary.

09/11/07 10:00:08 changed by Ludovico Magnocavallo <ludo@qix.it>

  • attachment related.diff added.

Cleaner variant of patch #2

09/11/07 10:00:32 changed by Ludovico Magnocavallo <ludo@qix.it>

  • attachment models.py added.

Test models

09/11/07 10:00:52 changed by Ludovico Magnocavallo <ludo@qix.it>

  • attachment tests.py added.

Test cases

09/11/07 10:01:52 changed by Ludovico Magnocavallo <ludo@qix.it>

I attached a cleaner patch that does away with sending the field names, and some test cases with the necessary models.

09/15/07 14:51:07 changed by PhiR

  • keywords changed from manytomanyfield to manytomanyfield feature.

09/16/07 08:25:22 changed by ubernostrum

#1069 and #2861 were duplicates.

09/16/07 13:49:14 changed by ubernostrum

#3854 was a duplicate.

10/01/07 06:25:59 changed by Silviu Julean <sjulean@ideacentrum.eu>

  • cc set to sjulean@ideacentrum.eu.

I am using this patch to monitor user-group mapping in contrib.auth, and it works without any glitches.

11/03/07 02:50:22 changed by ekarulf

  • cc changed from sjulean@ideacentrum.eu to sjulean@ideacentrum.eu, django@erik.karulf.com.

I am also using the patch to monitor contrib.auth user-group relationships with no problems.

11/03/07 03:14:16 changed by mtredinnick

  • stage changed from Unreviewed to Design decision needed.

01/18/08 12:56:32 changed by sean

I'm using the patch quite a lot and up to now it works great. I really think this functionality should go into core, simply because it complements the other signals nicely and enables a developer to fully monitor all database changes (given that the orm is used).

01/20/08 12:09:53 changed by murkt

  • cc changed from sjulean@ideacentrum.eu, django@erik.karulf.com to sjulean@ideacentrum.eu, django@erik.karulf.com, vsevolod.solovyov@gmail.com.
  • needs_docs set to 1.

Patch misses documentation. Notes about m2m signals should go into db-api.txt, section Many-to-many relationships.

01/22/08 03:53:13 changed by ludo@qix.it

  • attachment m2msignals_doc.txt added.

patch for the db-api docs

01/22/08 04:00:20 changed by Ludovico Magnocavallo <ludo@qix.it>

  • needs_docs deleted.

01/24/08 17:44:49 changed by sean

  • cc changed from sjulean@ideacentrum.eu, django@erik.karulf.com, vsevolod.solovyov@gmail.com to sjulean@ideacentrum.eu, django@erik.karulf.com, vsevolod.solovyov@gmail.com, sean@gmx.org.

As I said before the patch works, but I just found a piece of missing functionality. By using related_name it is possible to have 2 or more Many2many fields to the same model, but using the dispatcher there is no way to distinguish between the relations, or am I missing something here? A possible solution would be to send the the name of the join_table along with the signal, but that seems rather hackish.

01/30/08 12:43:51 changed by murkt

It's possible to add one parameter to RelatedManager.__init__()field_name, and send it along with the signal. With the help of field_name we can easily get field instance itself: [f for f in MyModel._meta.many_to_many if f.name == field_name][0], or access RelatedManager: getattr(instance, field_name).

01/30/08 13:11:34 changed by murkt

  • attachment related.field_name.diff added.

related.diff with added field_name parameter (signals and dispatcher are already imported)

06/14/08 07:35:08 changed by Patrick Lauber <patrick.lauber@divio.ch>

  • cc changed from sjulean@ideacentrum.eu, django@erik.karulf.com, vsevolod.solovyov@gmail.com, sean@gmx.org to sjulean@ideacentrum.eu, django@erik.karulf.com, vsevolod.solovyov@gmail.com, sean@gmx.org, patrick.lauber@dvio.ch.

06/21/08 08:01:48 changed by anonymous

  • cc changed from sjulean@ideacentrum.eu, django@erik.karulf.com, vsevolod.solovyov@gmail.com, sean@gmx.org, patrick.lauber@dvio.ch to sjulean@ideacentrum.eu, django@erik.karulf.com, vsevolod.solovyov@gmail.com, sean@gmx.org, patrick.lauber@dvio.ch, chpapa@gmail.com.

07/23/08 21:55:03 changed by anonymous

  • cc changed from sjulean@ideacentrum.eu, django@erik.karulf.com, vsevolod.solovyov@gmail.com, sean@gmx.org, patrick.lauber@dvio.ch, chpapa@gmail.com to sjulean@ideacentrum.eu, django@erik.karulf.com, vsevolod.solovyov@gmail.com, sean@gmx.org, patrick.lauber@dvio.ch, chpapa@gmail.com, thiago.salves@gmail.com.

08/07/08 03:40:56 changed by piranha

  • attachment m2m_signals.diff added.

m2m signals patch, updated against signals refactoring

08/31/08 18:58:54 changed by anonymous

  • cc changed from sjulean@ideacentrum.eu, django@erik.karulf.com, vsevolod.solovyov@gmail.com, sean@gmx.org, patrick.lauber@dvio.ch, chpapa@gmail.com, thiago.salves@gmail.com to sjulean@ideacentrum.eu, django@erik.karulf.com, vsevolod.solovyov@gmail.com, sean@gmx.org, patrick.lauber@dvio.ch, chpapa@gmail.com, thiago.salves@gmail.com, oliver@bluelavatech.com.

+1 on this, it would be so useful right now for me; too bad it isn't in yet :(

12/12/08 04:39:41 changed by rvdrijst

  • cc changed from sjulean@ideacentrum.eu, django@erik.karulf.com, vsevolod.solovyov@gmail.com, sean@gmx.org, patrick.lauber@dvio.ch, chpapa@gmail.com, thiago.salves@gmail.com, oliver@bluelavatech.com to sjulean@ideacentrum.eu, django@erik.karulf.com, vsevolod.solovyov@gmail.com, sean@gmx.org, patrick.lauber@dvio.ch, chpapa@gmail.com, thiago.salves@gmail.com, oliver@bluelavatech.com, rvdrijst.

#6778 was a duplicate

12/19/08 11:00:04 changed by rvdrijst

  • keywords changed from manytomanyfield feature to manytomanyfield feature signals.
  • needs_better_patch set to 1.
  • needs_docs set to 1.
  • milestone set to post-1.0.

I have started a discussion on django-developers concerning this ticket and the limitations I have encountered: http://groups.google.com/group/django-developers/browse_thread/thread/afe6ad7994d868ba

Hopefully, this will push development and get this feature in the 1.1alhpa scheduled for january '09.

If you are interested, please read and comment there. If discussion is fruitful, I will claim the ticket and improve the patch, docs and tests.

Additionally, not related to the functionality but to the documentation: the db-api.txt is not the right place for signal documentation (there is no talk of pre/post_save signals there either). The correct place seems docs/ref/signals.txt. Also, not only the names of the signals, but also the extra arguments will need to be described.

12/19/08 15:09:59 changed by piranha

  • cc changed from sjulean@ideacentrum.eu, django@erik.karulf.com, vsevolod.solovyov@gmail.com, sean@gmx.org, patrick.lauber@dvio.ch, chpapa@gmail.com, thiago.salves@gmail.com, oliver@bluelavatech.com, rvdrijst to sjulean@ideacentrum.eu, django@erik.karulf.com, vsevolod.solovyov@gmail.com, sean@gmx.org, patrick.lauber@dvio.ch, chpapa@gmail.com, thiago.salves@gmail.com, oliver@bluelavatech.com, rvdrijst, piranha@piranha.org.ua.

01/23/09 11:07:41 changed by rvdrijst

  • attachment complete-patch.diff added.

A complete patch with code, docs and tests for an m2m_changed signal.

01/23/09 11:07:57 changed by rvdrijst

  • owner changed from nobody to rvdrijst.
  • needs_better_patch deleted.
  • status changed from new to assigned.
  • needs_docs deleted.

I have attached a new patch that combines the changes to the code, the documentation and the tests in one file.

The biggest change is that there is now only one signal, m2m_changed, that has an 'action' parameter (based on a comment of Malcolm Tredinnick. The other parameters have changed somewhat, please discuss problems/suggestions on the django-developer thread mentioned above, where I have also posted a short explanation of the new parameters.

And thanks to Ludovico for the initial implementation.

02/25/09 13:51:44 changed by

  • milestone deleted.

Milestone post-1.0 deleted

(follow-up: ↓ 24 ) 04/25/09 20:11:22 changed by wmdmark

+1 Please include this fix in the upcoming release. I'm having to do some awful code hacks to get around this.

(in reply to: ↑ 23 ) 04/25/09 20:15:48 changed by ubernostrum

Replying to wmdmark:

+1 Please include this fix in the upcoming release. I'm having to do some awful code hacks to get around this.

You do realize the feature freeze for 1.1 has passed already, right (in fact, 1.1 is behind schedule because we delayed a bit for a late-running feature)? No new feature checkins will happen until after the release, at which point proposals for inclusion in 1.2 will open.

If you're really interested in getting a feature into Django, it's in your best interest to get involved on the developers' list and pay attention to the release schedule.

05/16/09 19:15:52 changed by palewire

  • cc changed from sjulean@ideacentrum.eu, django@erik.karulf.com, vsevolod.solovyov@gmail.com, sean@gmx.org, patrick.lauber@dvio.ch, chpapa@gmail.com, thiago.salves@gmail.com, oliver@bluelavatech.com, rvdrijst, piranha@piranha.org.ua to sjulean@ideacentrum.eu, django@erik.karulf.com, vsevolod.solovyov@gmail.com, sean@gmx.org, patrick.lauber@dvio.ch, chpapa@gmail.com, thiago.salves@gmail.com, oliver@bluelavatech.com, rvdrijst, piranha@piranha.org.ua, palewire@palewire.com.

05/19/09 07:00:42 changed by schmeii

  • cc changed from sjulean@ideacentrum.eu, django@erik.karulf.com, vsevolod.solovyov@gmail.com, sean@gmx.org, patrick.lauber@dvio.ch, chpapa@gmail.com, thiago.salves@gmail.com, oliver@bluelavatech.com, rvdrijst, piranha@piranha.org.ua, palewire@palewire.com to sjulean@ideacentrum.eu, django@erik.karulf.com, vsevolod.solovyov@gmail.com, sean@gmx.org, patrick.lauber@dvio.ch, chpapa@gmail.com, thiago.salves@gmail.com, oliver@bluelavatech.com, rvdrijst, piranha@piranha.org.ua, palewire@palewire.com, maxischmeii@gmail.com.

05/25/09 13:22:07 changed by UloPe

  • cc changed from sjulean@ideacentrum.eu, django@erik.karulf.com, vsevolod.solovyov@gmail.com, sean@gmx.org, patrick.lauber@dvio.ch, chpapa@gmail.com, thiago.salves@gmail.com, oliver@bluelavatech.com, rvdrijst, piranha@piranha.org.ua, palewire@palewire.com, maxischmeii@gmail.com to sjulean@ideacentrum.eu, django@erik.karulf.com, vsevolod.solovyov@gmail.com, sean@gmx.org, patrick.lauber@dvio.ch, chpapa@gmail.com, thiago.salves@gmail.com, oliver@bluelavatech.com, rvdrijst, piranha@piranha.org.ua, palewire@palewire.com, maxischmeii@gmail.com, django@ulo.pe.

06/12/09 02:23:47 changed by lvscar

  • cc changed from sjulean@ideacentrum.eu, django@erik.karulf.com, vsevolod.solovyov@gmail.com, sean@gmx.org, patrick.lauber@dvio.ch, chpapa@gmail.com, thiago.salves@gmail.com, oliver@bluelavatech.com, rvdrijst, piranha@piranha.org.ua, palewire@palewire.com, maxischmeii@gmail.com, django@ulo.pe to sjulean@ideacentrum.eu, django@erik.karulf.com, vsevolod.solovyov@gmail.com, sean@gmx.org, patrick.lauber@dvio.ch, chpapa@gmail.com, thiago.salves@gmail.com, oliver@bluelavatech.com, rvdrijst, piranha@piranha.org.ua, palewire@palewire.com, maxischmeii@gmail.com, django@ulo.pe, lvscar@gmail.com.

06/26/09 21:14:53 changed by bhagany

  • cc changed from sjulean@ideacentrum.eu, django@erik.karulf.com, vsevolod.solovyov@gmail.com, sean@gmx.org, patrick.lauber@dvio.ch, chpapa@gmail.com, thiago.salves@gmail.com, oliver@bluelavatech.com, rvdrijst, piranha@piranha.org.ua, palewire@palewire.com, maxischmeii@gmail.com, django@ulo.pe, lvscar@gmail.com to sjulean@ideacentrum.eu, django@erik.karulf.com, vsevolod.solovyov@gmail.com, sean@gmx.org, patrick.lauber@dvio.ch, chpapa@gmail.com, thiago.salves@gmail.com, oliver@bluelavatech.com, rvdrijst, piranha@piranha.org.ua, palewire@palewire.com, maxischmeii@gmail.com, django@ulo.pe, lvscar@gmail.com, brent.hagany@gmail.com.

Add/Change #5390 (Add signals to ManyRelatedManager)




Change Properties
Action