Opened 4 weeks ago

Closed 4 weeks ago

#36574 closed Bug (invalid)

Regression: DecimalField values are no longer quantized before written to the DB

Reported by: Aaron Mader Owned by:
Component: Database layer (models, ORM) Version: 5.2
Severity: Normal Keywords: decimal_places
Cc: Triage Stage: Unreviewed
Has patch: no Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description (last modified by Aaron Mader)

Steps to reproduce:

  1. Create a new project using the mysql backend
  2. Create this model:
    class Apple(models.Model):
        weight = models.DecimalField(max_digits=4, decimal_places=1)
    
  3. Create an instance of this record with input outside of the quantization limits
    from decimal import Decimal
    Apple.objects.create(
        weight = Decimal("1.01"),
    )
    
  4. Attempt to fetch the record by the (expected) adjusted value
    from decimal import Decimal
    Apple.objects.filter(
        weight = Decimal("1.0"),
    )
    

Result:
With django 5.2.0, the filter query fails with 0 results.
With django 5.1.11, the filter query success, returning the created record.

Specifically, this change in behaviour was introduced in django 5.2.0, with prior versions of django quantizing the input value (1.01) to an acceptable value (1.0) before writing the value to the database.
I believe that change was introduced in pull request: https://github.com/django/django/pull/18895

Change History (4)

comment:1 by Aaron Mader, 4 weeks ago

Description: modified (diff)

comment:2 by Tim Graham, 4 weeks ago

I'm not sure that the old behavior of silently discarding decimal places is justified. If you don't validate your data, there could be problems. Could you explain your use case?

Last edited 4 weeks ago by Tim Graham (previous) (diff)

comment:3 by Simon Charette, 4 weeks ago

I also share Tim's position; if input validation is not performed then it's down to how the database backend implements storeage (e.g. SQLite will allow to store pretty much anything in any field)

Do you happen to be running MySQL in non-strict mode which allows invalid input to be stored in the first place? I cannot reproduce with the following test against MySQL with strict mode enabled (which is the default). In strict mode the data is truncated as expected

  • tests/lookup/test_decimalfield.py

    diff --git a/tests/lookup/test_decimalfield.py b/tests/lookup/test_decimalfield.py
    index d938ccf649..d4adb859c7 100644
    a b def test_lt(self):  
    3535    def test_lte(self):
    3636        qs = self.queryset.filter(qty_needed__lte=0)
    3737        self.assertCountEqual(qs, [self.p1, self.p2])
     38
     39    def test_eq_truncation(self):
     40        from decimal import Decimal
     41
     42        obj = Product.objects.create(name="Product1", qty_target=Decimal("2000.003"))
     43        self.assertEqual(Product.objects.get(qty_target=Decimal("2000.00")), obj)

Note that the test do fail when MySQL strict mode is disabled and as expected on SQLite where pretty much any data type is stored as text.

I let others chime in but to me this is a case of invalid or wontfix

comment:4 by Jacob Walls, 4 weeks ago

Keywords: decimal_places added
Resolution: invalid
Status: newclosed

Agree with the above assessments.

The release note describes this bugfix as relevant only to third-party database backends, so although we don't normally document bugfixes, I could be open to an addition to indicate that silent truncation no longer happens when values are prepped for insertion, since we have this report of a user relying on it.

Note: See TracTickets for help on using tickets.
Back to Top