id,summary,reporter,owner,description,type,status,component,version,severity,resolution,keywords,cc,stage,has_patch,needs_docs,needs_tests,needs_better_patch,easy,ui_ux 34067,django.core.Paginator wrong query slicing,Hristo Trendafilov,nobody,"A have a ListView defined like so: {{{ class AllPracticesListView(LoginRequiredMixin, PermissionRequiredMixin, ListView, ToolBoxFunctions): template_name = 'practice_list.html' model = Practice paginate_by = 6 permission_required = 'customauth.access_practices' def get_queryset(self): qs = self.get_all_practices_by_user().order_by( 'company_branch__pk', 'practice_profile__personal_profile__last_name').prefetch_related('practice_profile') # method < get_all_practices_by_user > is a toolbox method that returns: # Practice.objects.filter(company_branch__owner=self.request.user).order_by('pk') return qs }}} Models are defined like so: {{{ class Practice(models.Model): company_branch = models.ForeignKey( CompanyBranch, on_delete=models.PROTECT, related_name='practices', ) # CompanyBranch model has an owner field which ForeignKey to the user model practice_profile = models.ForeignKey( PracticeProfile, on_delete=models.PROTECT, related_name='practices', ) class PracticeProfile(models.Model): personal_profile = models.ForeignKey( PersonalProfile, on_delete=models.PROTECT, blank=True, null=True, ) # PersonalProfile model has a field called < last_name > }}} Or schematically: {{{ Practice object relations: -> CompanyBranch -> PracticeProfile -> PersonalProfile /*NULLABLE RELATION/ }}} When the code is run, the view does not display results correctly. I did like so to get the WRONG result: Added a breakpoint on {{{Paginator._get_page}}} return statement Queryset is passed okay and is available in {{{Paginator.object_list}}} But in the args of {{{Paginator._get_page}}} the queryset is totally different than expected / should be a slice from the first six elements of {{{Paginator.object_list}}} / I have tried to add a breakpoint like so [[Image(https://imgur.com/SdaQUt6)]] Then I get this result [[Image(https://imgur.com/5zzLNV0)]]. As you can see, PKs 1632, 1624, etc. are missing, objects are different/marked red and purple/, and are actually removed and never shown in the view. If you run slice manually at this point - everything is again fine [[Image(https://imgur.com/yvn8KMO)]] This happens only on NULL PersonalProfile objects. I did like so to get OKAY results: 1. Added a simple breakpoint is added on {{{Paginator.page}}} like so [[Image(https://imgur.com/d7J5BMZ)]] and that breakpoint is just run then the result is okay [[Image(https://imgur.com/ILDHAMN)]] 2. changed paginate_by to 10 3. changed {{{order_by}}} in {{{get_queryset}}} to {{{-pk}}} 4. call again the {{{page = paginator.page(page_number)}}} It is Django fault for those reasons: * `order_by` might be complicated but is supported by the framework * Chained `order_by` is also supported * queryset slicing is a feature of Django queryset API. * queryset slicing should act like list and string slicing and namely to provide you with results `[ from : to ]` indexes provided and not to change or lose data in between. Or in other words - what is passed in the `Paginator.object_list` should be just sliced `[ from : to ]`. `Paginator` should never care for the queryset, what is the ordering of the queryset, what is excluded and so on - it should just work with it. * changing `paginate_by` to `10` or greater fixes the problem. `paginate_by` is a standard and documented Django ListView setting. * changing `order_by` to some other `order_by` also fixes the problem. `order_by` should work the same or if any limitations those should be well documented * removing `order_by` also fixes the problem. P.S. That is also happening in Django 2.2",Bug,closed,Core (Other),3.2,Normal,needsinfo,"Paginator,slice,queryset",,Unreviewed,0,0,0,0,0,0