Possible Memory leak while evaluating a QuerySet

[Ramez Issac]

I discovered this as i have a 150K rows QuerySet,
I checked its existence by the not efficient

if queryset:
    pass

However, as soon as execution hits this line, memory consumption jumps big time and it never goes back to normal unless uwsgi process gets restarted.
This information has been noticed using htop showing the relevant uwsgi process.

Using query.exists() is working perfectly good.

[Tim Graham]

As ​the documentation says:

Testing a QuerySet in a boolean context, such as using bool(), or, and or an if statement, will cause the query to be executed. If there is at least one result, the QuerySet is True, otherwise False. If you only want to determine if at least one result exists (and don’t need the actual objects), it’s more efficient to use exists().

Can you provide more details if that doesn't address your concerns?

[Ramez Issac]

Thank you dear time for your quick reply,

What i'm referring to here is when i made the mistake of using bool(queryset) , the server memory jumped and never returned back to normal even after the function exit.
Something is not right here.
The only way to bring the memory back to normal was by restarting the uwsgi process running the django instance.

It's not about that the line bool(queryset) takes longer time to execute, or is heavy on memory;
But this line have a long lasting side effect on server memory consumption.

[Tim Graham]

I'm not intimately familiar with memory management in Python but unless you can provide details as to why, I'm doubtful that Django is at fault for this behavior. Maybe this post about ​Diagnosing Memory "Leaks" in Python helps.

[Claude Paroz]

You don't run with DEBUG=True, right?

[Ramez Issac]

@claudep Lol Nope ! :D

@Tim I'll review the document you attached and get back to you with more detailed/insightful information.