Field lookup of __gt and __gte on DecimalField causes memory blowup

[odonian]

Somewhere between Django alpha 1 and 2, it seems a bug was introduced where using a field lookup of gt or gte on a DecimalField causes an instance of the development server (and probably other server instances, e.g. mod_python) to rapidly grow in memory and consume system resources (primarily memory). Perhaps a race condition is being triggered. I have to kill the dev server to get my system (running Linux) under control again.

There were no such problems seen with the exact same code in the alpha 1 svn, as far as I recall. I notice some changes have been made to the DecimalField #8143, but these are pre-existing values already in the database. I'm using the trunk at revision 8336.

[Malcolm Tredinnick]

Since you can see this problem occurring, would you mind to construct a small test case that demonstrates the problem? We need to be able to repeat it before we can diagnose it.

[odonian]

Update: looks like the way to avoid troubles is to ensure that the field lookup value is of the same type, e.g. for the DecimalField where the field lookup is active, the some_value in "the_decimalfieldgte=some_value" has to be converted to a Decimal instance first.

Still, the system should fail with a warning or exception instead of gobbling up memory.

[Karen Tracey <kmtracey@…>]

I cannot recreate the problem described here. Note in describing the workaround/solution you found you neglected to include what type 'some_value' was when it caused you problems. Converting it to a Decimal fixes the problem, OK, but what type was it when you saw the problem? That would be helpful in order to track down if there is something in Django that needs fixing. I'm going to guess, though, that you ran into trouble with [8143] (which requires you to now convert floats to String/Decimal where before they just worked). If you did have a float somewhere, and were running with DEBUG on, the problem described by #6748 could have resulted in the memory/cpu blowup you describe (if you happened to have a large QuerySet variable hanging around in a local var) when Django tried to generate the debug page to report the problem with passing a float in as a Decimal value. If you think this is not what happened, and that there is a problem here that needs fixing in Django, please reopen with specifics on how to recreate the problem.

[odonian]

Replying to Karen Tracey <kmtracey@gmail.com>:

I cannot recreate the problem described here. Note in describing the workaround/solution you found you neglected to include what type 'some_value' was when it caused you problems. Converting it to a Decimal fixes the problem, OK, but what type was it when you saw the problem? That would be helpful in order to track down if there is something in Django that needs fixing. I'm going to guess, though, that you ran into trouble with [8143] (which requires you to now convert floats to String/Decimal where before they just worked). If you did have a float somewhere, and were running with DEBUG on, the problem described by #6748 could have resulted in the memory/cpu blowup you describe (if you happened to have a large QuerySet variable hanging around in a local var) when Django tried to generate the debug page to report the problem with passing a float in as a Decimal value. If you think this is not what happened, and that there is a problem here that needs fixing in Django, please reopen with specifics on how to recreate the problem.

Thanks for following up on this. Yes, the some_value was of type float that, once converted to a Decimal, worked fine. And yes, #6748 could well have applied in this situation, the QuerySet was large. Thanks again!

[Jacob]

Milestone 1.0 deleted