Opened 5 years ago

Closed 3 years ago

#17270 closed New feature (duplicate)

methods of the manager on subqueries QuerySet objects

Reported by: Wojciech Banaś <fizista@…> Owned by: fizista@…
Component: Database layer (models, ORM) Version: master
Severity: Normal Keywords: queryset manager Extends
Cc: fizista@…, 8mayday@… Triage Stage: Someday/Maybe
Has patch: yes Needs documentation: yes
Needs tests: no Patch needs improvement: no
Easy pickings: yes UI/UX: no

Description

Extends the operation methods of the manager on subqueries QuerySet objects.

The "standard manager" you can only like this:

SomeModel.object.some_method_manager().all().filter(...). ...

But this does not work:

SomeModel.object.all().some_method_manager()

And this it works with the use of this class / patches:

SomeModel.object.all().some_method_manager().filter(...).some_method_manager2(). ... 
SomeModel.object.some_method_manager().some_method_manager2().some_method_manager3(). ... 

Extension of the methods manager on all child objects "QuerySeth".

Example:

  • class model
        class Animals(models.Model):
            type = models.CharField('Type of animal') # dog, cat, elephant
            color = models.CharField('Colour Name')
            
            objects = AnimalsManager()
    
  • class manager
    class AnimalsManager(ManagerQuerySetDecorator):
    
        def get_dogs(self):
            return self.get_query_set().filter(type='dog')
        get_dogs.queryset_method = True
        
        def get_cats(self):
            return self.get_query_set().filter(type='cat')
        get_cats.queryset_method = True
        
        def get_white_animals(self):
            return self.get_query_set().filter(color='white')
        get_white_animals.queryset_method = True
        
        def get_black_animals(self):
            return self.get_query_set().filter(color='black')
        get_black_animals.queryset_method = True
        
        def get_brown_animals(self):
            return self.get_query_set().filter(color='black')

  • add models data
    Animals(type='dog', color='black').save()
    Animals(type='dog', color='bown').save()
    Animals(type='dog', color='white').save()
    Animals(type='cat', color='black').save()
    Animals(type='cat', color='bown').save()
    Animals(type='cat', color='white').save()

  • examples of actions a new manager
    animals = Animals.objects.get_black_animals().get_dogs()
    # return list black dogs

    animals = Animals.objects.get_white_animals().get_cats()
    # return list white cats
    
    # When ffff equals False, or not defined (as it is in 
    # the original model manager), we can not perform such an operation:
    animals = Animals.objects.get_cats().get_brown_animals()
    # return:
    # AttributeError: 'QuerySet' object has no attribute 'get_brown_animals'
    
    # but:
    animals = Animals.objects.get_brown_animals().get_cats()
    # return brown cats, because get_cat() present in all parent object instances.

In Annex file a class action extends the manager.

If there is interest in this patch, then add in the future full of tests, and prepare a ready-made patch for django code.

Attachments (6)

manager.py (4.3 KB) - added by Wojciech Banaś <fizista@…> 5 years ago.
New manager class
manager.2.py (3.5 KB) - added by Wojciech Banaś <fizista@…> 5 years ago.
Improved and simplified version.
manager.3.py (3.5 KB) - added by Wojciech Banaś <fizista@…> 5 years ago.
Even small changes
manager.4.py (3.6 KB) - added by Wojciech Banaś <fizista@…> 5 years ago.
Dynamic class is created only once
manager_queryset.diff (5.9 KB) - added by Wojciech Banaś <fizista@…> 5 years ago.
Another solution, tested, working, downwards compatible, django devel path
manager.5.py (2.3 KB) - added by Wojciech Banaś <fizista@…> 5 years ago.
Another solution, tested, working, downwards compatible, decorator

Download all attachments as: .zip

Change History (16)

Changed 5 years ago by Wojciech Banaś <fizista@…>

Attachment: manager.py added

New manager class

Changed 5 years ago by Wojciech Banaś <fizista@…>

Attachment: manager.2.py added

Improved and simplified version.

comment:1 Changed 5 years ago by Wojciech Banaś <fizista@…>

Easy pickings: set
Needs documentation: set

Based on the ticket 17271, I rewrote the code and got a very nice and simple patch.

It is located in the attachment manager.2.py.

Changed 5 years ago by Wojciech Banaś <fizista@…>

Attachment: manager.3.py added

Even small changes

comment:2 Changed 5 years ago by Carl Meyer

As a longer-term project, I think finding a way to unify Manager and QuerySet could be a real improvement to the ORM API, if we can find a way to do it that is both sane and backwards compatible. The technique here, dynamically creating a new QuerySet subclass on every access and adding manager methods to it, is (probably mostly) backwards compatible, but I am not at all convinced that it is sane, or an acceptable performance hit.

Changed 5 years ago by Wojciech Banaś <fizista@…>

Attachment: manager.4.py added

Dynamic class is created only once

comment:3 in reply to:  2 Changed 5 years ago by Andrey Popp

Replying to carljm:

As a longer-term project, I think finding a way to unify Manager and QuerySet could be a real improvement to the ORM API, if we can find a way to do it that is both sane and backwards compatible. The technique here, dynamically creating a new QuerySet subclass on every access and adding manager methods to it, is (probably mostly) backwards compatible, but I am not at all convinced that it is sane, or an acceptable performance hit.

Speaking of that, do you have any clue, why not make ManagerDescriptor returns QuerySet instead of Manager? Looking at sources I can't see any useful functionality which is up to Manager — most code are just produces and proxies calls to QuerySet.

comment:4 Changed 5 years ago by Andrey Popp

Cc: 8mayday@… added

comment:5 Changed 5 years ago by Aymeric Augustin

Triage Stage: UnreviewedDesign decision needed

I'm skeptical about the clarity and maintainability of code that create classes dynamically too.

comment:6 Changed 5 years ago by Luke Plant

Easy pickings: unset

For reference, an alternative solution is here: https://github.com/zacharyvoase/django-qmixin

Changed 5 years ago by Wojciech Banaś <fizista@…>

Attachment: manager_queryset.diff added

Another solution, tested, working, downwards compatible, django devel path

comment:7 Changed 5 years ago by Wojciech Banaś <fizista@…>

Easy pickings: set

Indeed, the dynamic class caused a lot of problems. I've enclosed a different solution based on various other solutions. It is downward compatible. This gives two possible ways of implementation.

  • inside model class
objects_second = PersonManagerSecond(ManagerQuerySet, ['get_fun_people', 'get_not_fun_people'])

or inside manager class

class PersonManager(models.Manager):
    def __init__(self):
        super(PersonManager, self).__init__()
        self.implement_queryset_cls(ManagerQuerySet)
        self.implement_queryset_methods('get_fun_people', 'get_not_fun_people')

For more details refer to attachment: manager_queryset.diff

This implementation passed all django tests.

Changed 5 years ago by Wojciech Banaś <fizista@…>

Attachment: manager.5.py added

Another solution, tested, working, downwards compatible, decorator

comment:8 Changed 5 years ago by Wojciech Banaś <fizista@…>

Version: SVN

comment:9 Changed 4 years ago by Jacob

Triage Stage: Design decision neededSomeday/Maybe

Marking Someday: we want to do this, but it's entirely unclear exactly how. If you've got a specific proposal, please take it to django-developers for discussion!

comment:10 Changed 3 years ago by Anssi Kääriäinen

Resolution: duplicate
Status: newclosed

I am closing this as duplicate of #20625 - that ticket has more momentum than this one.

Note: See TracTickets for help on using tickets.
Back to Top