Opened 4 years ago

Closed 2 years ago

#17270 closed New feature (duplicate)

methods of the manager on subqueries QuerySet objects

Reported by: Wojciech Banaś <fizista@…> Owned by: fizista@…
Component: Database layer (models, ORM) Version: master
Severity: Normal Keywords: queryset manager Extends
Cc: fizista@…, 8mayday@… Triage Stage: Someday/Maybe
Has patch: yes Needs documentation: yes
Needs tests: no Patch needs improvement: no
Easy pickings: yes UI/UX: no

Description

Extends the operation methods of the manager on subqueries QuerySet objects.

The "standard manager" you can only like this:

SomeModel.object.some_method_manager().all().filter(...). ...

But this does not work:

SomeModel.object.all().some_method_manager()

And this it works with the use of this class / patches:

SomeModel.object.all().some_method_manager().filter(...).some_method_manager2(). ... 
SomeModel.object.some_method_manager().some_method_manager2().some_method_manager3(). ... 

Extension of the methods manager on all child objects "QuerySeth".

Example:

  • class model
        class Animals(models.Model):
            type = models.CharField('Type of animal') # dog, cat, elephant
            color = models.CharField('Colour Name')
            
            objects = AnimalsManager()
    
  • class manager
    class AnimalsManager(ManagerQuerySetDecorator):
    
        def get_dogs(self):
            return self.get_query_set().filter(type='dog')
        get_dogs.queryset_method = True
        
        def get_cats(self):
            return self.get_query_set().filter(type='cat')
        get_cats.queryset_method = True
        
        def get_white_animals(self):
            return self.get_query_set().filter(color='white')
        get_white_animals.queryset_method = True
        
        def get_black_animals(self):
            return self.get_query_set().filter(color='black')
        get_black_animals.queryset_method = True
        
        def get_brown_animals(self):
            return self.get_query_set().filter(color='black')

  • add models data
    Animals(type='dog', color='black').save()
    Animals(type='dog', color='bown').save()
    Animals(type='dog', color='white').save()
    Animals(type='cat', color='black').save()
    Animals(type='cat', color='bown').save()
    Animals(type='cat', color='white').save()

  • examples of actions a new manager
    animals = Animals.objects.get_black_animals().get_dogs()
    # return list black dogs

    animals = Animals.objects.get_white_animals().get_cats()
    # return list white cats
    
    # When ffff equals False, or not defined (as it is in 
    # the original model manager), we can not perform such an operation:
    animals = Animals.objects.get_cats().get_brown_animals()
    # return:
    # AttributeError: 'QuerySet' object has no attribute 'get_brown_animals'
    
    # but:
    animals = Animals.objects.get_brown_animals().get_cats()
    # return brown cats, because get_cat() present in all parent object instances.

In Annex file a class action extends the manager.

If there is interest in this patch, then add in the future full of tests, and prepare a ready-made patch for django code.

Attachments (6)

manager.py (4.3 KB) - added by Wojciech Banaś <fizista@…> 4 years ago.
New manager class
manager.2.py (3.5 KB) - added by Wojciech Banaś <fizista@…> 4 years ago.
Improved and simplified version.
manager.3.py (3.5 KB) - added by Wojciech Banaś <fizista@…> 4 years ago.
Even small changes
manager.4.py (3.6 KB) - added by Wojciech Banaś <fizista@…> 4 years ago.
Dynamic class is created only once
manager_queryset.diff (5.9 KB) - added by Wojciech Banaś <fizista@…> 4 years ago.
Another solution, tested, working, downwards compatible, django devel path
manager.5.py (2.3 KB) - added by Wojciech Banaś <fizista@…> 4 years ago.
Another solution, tested, working, downwards compatible, decorator

Download all attachments as: .zip

Change History (16)

Changed 4 years ago by Wojciech Banaś <fizista@…>

New manager class

Changed 4 years ago by Wojciech Banaś <fizista@…>

Improved and simplified version.

comment:1 Changed 4 years ago by Wojciech Banaś <fizista@…>

  • Easy pickings set
  • Needs documentation set
  • Needs tests unset
  • Patch needs improvement unset

Based on the ticket 17271, I rewrote the code and got a very nice and simple patch.

It is located in the attachment manager.2.py.

Changed 4 years ago by Wojciech Banaś <fizista@…>

Even small changes

comment:2 follow-up: Changed 4 years ago by carljm

As a longer-term project, I think finding a way to unify Manager and QuerySet could be a real improvement to the ORM API, if we can find a way to do it that is both sane and backwards compatible. The technique here, dynamically creating a new QuerySet subclass on every access and adding manager methods to it, is (probably mostly) backwards compatible, but I am not at all convinced that it is sane, or an acceptable performance hit.

Changed 4 years ago by Wojciech Banaś <fizista@…>

Dynamic class is created only once

comment:3 in reply to: ↑ 2 Changed 4 years ago by andreypopp

Replying to carljm:

As a longer-term project, I think finding a way to unify Manager and QuerySet could be a real improvement to the ORM API, if we can find a way to do it that is both sane and backwards compatible. The technique here, dynamically creating a new QuerySet subclass on every access and adding manager methods to it, is (probably mostly) backwards compatible, but I am not at all convinced that it is sane, or an acceptable performance hit.

Speaking of that, do you have any clue, why not make ManagerDescriptor returns QuerySet instead of Manager? Looking at sources I can't see any useful functionality which is up to Manager — most code are just produces and proxies calls to QuerySet.

comment:4 Changed 4 years ago by andreypopp

  • Cc 8mayday@… added

comment:5 Changed 4 years ago by aaugustin

  • Triage Stage changed from Unreviewed to Design decision needed

I'm skeptical about the clarity and maintainability of code that create classes dynamically too.

comment:6 Changed 4 years ago by lukeplant

  • Easy pickings unset

For reference, an alternative solution is here: https://github.com/zacharyvoase/django-qmixin

Changed 4 years ago by Wojciech Banaś <fizista@…>

Another solution, tested, working, downwards compatible, django devel path

comment:7 Changed 4 years ago by Wojciech Banaś <fizista@…>

  • Easy pickings set

Indeed, the dynamic class caused a lot of problems. I've enclosed a different solution based on various other solutions. It is downward compatible. This gives two possible ways of implementation.

  • inside model class
objects_second = PersonManagerSecond(ManagerQuerySet, ['get_fun_people', 'get_not_fun_people'])

or inside manager class

class PersonManager(models.Manager):
    def __init__(self):
        super(PersonManager, self).__init__()
        self.implement_queryset_cls(ManagerQuerySet)
        self.implement_queryset_methods('get_fun_people', 'get_not_fun_people')

For more details refer to attachment: manager_queryset.diff

This implementation passed all django tests.

Changed 4 years ago by Wojciech Banaś <fizista@…>

Another solution, tested, working, downwards compatible, decorator

comment:8 Changed 4 years ago by Wojciech Banaś <fizista@…>

  • Version set to SVN

comment:9 Changed 2 years ago by jacob

  • Triage Stage changed from Design decision needed to Someday/Maybe

Marking Someday: we want to do this, but it's entirely unclear exactly how. If you've got a specific proposal, please take it to django-developers for discussion!

comment:10 Changed 2 years ago by akaariai

  • Resolution set to duplicate
  • Status changed from new to closed

I am closing this as duplicate of #20625 - that ticket has more momentum than this one.

Note: See TracTickets for help on using tickets.
Back to Top