Tests fail with r8336

Tested on r8336, Debian Etch, Python 2.4.4

vm3:~/dev/django-trunk/tests# ./runtests.py --settings settings
======================================================================
FAIL: Doctest: regressiontests.m2m_through_regress.models.__test__.API_TESTS
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python2.4/site-packages/django/test/_doctest.py", line 2180, in runTest
    raise self.failureException(self.format_failure(new.getvalue()))
AssertionError: Failed doctest test for regressiontests.m2m_through_regress.models.__test__.API_TESTS
  File "/home/dev/django-trunk/tests/regressiontests/m2m_through_regress/models.py", line unknown line number, in API_TESTS

----------------------------------------------------------------------
File "/home/dev/django-trunk/tests/regressiontests/m2m_through_regress/models.py", line ?, in regressiontests.m2m_through_regress.models.__test__.API_TESTS
Failed example:
    management.call_command('dumpdata', 'm2m_through_regress', format='json', indent=2)
Expected:
    [
      {
        "pk": 1,
        "model": "m2m_through_regress.membership",
        "fields": {
          "person": 1,
          "price": 100,
          "group": 1
        }
      },
      {
        "pk": 2,
        "model": "m2m_through_regress.membership",
        "fields": {
          "person": 1,
          "price": 100,
          "group": 2
        }
      },
      {
        "pk": 3,
        "model": "m2m_through_regress.membership",
        "fields": {
          "person": 2,
          "price": 100,
          "group": 1
        }
      },
      {
        "pk": 1,
        "model": "m2m_through_regress.usermembership",
        "fields": {
          "price": 100,
          "group": 1,
          "user": 1
        }
      },
      {
        "pk": 2,
        "model": "m2m_through_regress.usermembership",
        "fields": {
          "price": 100,
          "group": 2,
          "user": 1
        }
      },
      {
        "pk": 3,
        "model": "m2m_through_regress.usermembership",
        "fields": {
          "price": 100,
          "group": 1,
          "user": 2
        }
      },
      {
        "pk": 1,
        "model": "m2m_through_regress.person",
        "fields": {
          "name": "Bob"
        }
      },
      {
        "pk": 2,
        "model": "m2m_through_regress.person",
        "fields": {
          "name": "Jim"
        }
      },
      {
        "pk": 1,
        "model": "m2m_through_regress.group",
        "fields": {
          "name": "Rock"
        }
      },
      {
        "pk": 2,
        "model": "m2m_through_regress.group",
        "fields": {
          "name": "Roll"
        }
      }
    ]
Got:
    [
      {
        "pk": 1,
        "model": "m2m_through_regress.membership",
        "fields": {
          "person": 1,
          "price": 100,
          "group": 1
        }
      },
      {
        "pk": 2,
        "model": "m2m_through_regress.membership",
        "fields": {
          "person": 1,
          "price": 100,
          "group": 2
        }
      },
      {
        "pk": 3,
        "model": "m2m_through_regress.membership",
        "fields": {
          "person": 2,
          "price": 100,
          "group": 1
        }
      },
      {
        "pk": 1,
        "model": "m2m_through_regress.usermembership",
        "fields": {
          "price": 100,
          "group": 1,
          "user": 2
        }
      },
      {
        "pk": 2,
        "model": "m2m_through_regress.usermembership",
        "fields": {
          "price": 100,
          "group": 2,
          "user": 2
        }
      },
      {
        "pk": 3,
        "model": "m2m_through_regress.usermembership",
        "fields": {
          "price": 100,
          "group": 1,
          "user": 3
        }
      },
      {
        "pk": 1,
        "model": "m2m_through_regress.person",
        "fields": {
          "name": "Bob"
        }
      },
      {
        "pk": 2,
        "model": "m2m_through_regress.person",
        "fields": {
          "name": "Jim"
        }
      },
      {
        "pk": 1,
        "model": "m2m_through_regress.group",
        "fields": {
          "name": "Rock"
        }
      },
      {
        "pk": 2,
        "model": "m2m_through_regress.group",
        "fields": {
          "name": "Roll"
        }
      }
    ]
----------------------------------------------------------------------
File "/home/dev/django-trunk/tests/regressiontests/m2m_through_regress/models.py", line ?, in regressiontests.m2m_through_regress.models.__test__.API_TESTS
Failed example:
    management.call_command('dumpdata', 'm2m_through_regress', format='xml', indent=2)
Expected:
    <?xml version="1.0" encoding="utf-8"?>
    <django-objects version="1.0">
      <object pk="1" model="m2m_through_regress.membership">
        <field to="m2m_through_regress.person" name="person" rel="ManyToOneRel">1</field>
        <field to="m2m_through_regress.group" name="group" rel="ManyToOneRel">1</field>
        <field type="IntegerField" name="price">100</field>
      </object>
      <object pk="2" model="m2m_through_regress.membership">
        <field to="m2m_through_regress.person" name="person" rel="ManyToOneRel">1</field>
        <field to="m2m_through_regress.group" name="group" rel="ManyToOneRel">2</field>
        <field type="IntegerField" name="price">100</field>
      </object>
      <object pk="3" model="m2m_through_regress.membership">
        <field to="m2m_through_regress.person" name="person" rel="ManyToOneRel">2</field>
        <field to="m2m_through_regress.group" name="group" rel="ManyToOneRel">1</field>
        <field type="IntegerField" name="price">100</field>
      </object>
      <object pk="1" model="m2m_through_regress.usermembership">
        <field to="auth.user" name="user" rel="ManyToOneRel">1</field>
        <field to="m2m_through_regress.group" name="group" rel="ManyToOneRel">1</field>
        <field type="IntegerField" name="price">100</field>
      </object>
      <object pk="2" model="m2m_through_regress.usermembership">
        <field to="auth.user" name="user" rel="ManyToOneRel">1</field>
        <field to="m2m_through_regress.group" name="group" rel="ManyToOneRel">2</field>
        <field type="IntegerField" name="price">100</field>
      </object>
      <object pk="3" model="m2m_through_regress.usermembership">
        <field to="auth.user" name="user" rel="ManyToOneRel">2</field>
        <field to="m2m_through_regress.group" name="group" rel="ManyToOneRel">1</field>
        <field type="IntegerField" name="price">100</field>
      </object>
      <object pk="1" model="m2m_through_regress.person">
        <field type="CharField" name="name">Bob</field>
      </object>
      <object pk="2" model="m2m_through_regress.person">
        <field type="CharField" name="name">Jim</field>
      </object>
      <object pk="1" model="m2m_through_regress.group">
        <field type="CharField" name="name">Rock</field>
      </object>
      <object pk="2" model="m2m_through_regress.group">
        <field type="CharField" name="name">Roll</field>
      </object>
    </django-objects>
Got:
    <?xml version="1.0" encoding="utf-8"?>
    <django-objects version="1.0">
      <object pk="1" model="m2m_through_regress.membership">
        <field to="m2m_through_regress.person" name="person" rel="ManyToOneRel">1</field>
        <field to="m2m_through_regress.group" name="group" rel="ManyToOneRel">1</field>
        <field type="IntegerField" name="price">100</field>
      </object>
      <object pk="2" model="m2m_through_regress.membership">
        <field to="m2m_through_regress.person" name="person" rel="ManyToOneRel">1</field>
        <field to="m2m_through_regress.group" name="group" rel="ManyToOneRel">2</field>
        <field type="IntegerField" name="price">100</field>
      </object>
      <object pk="3" model="m2m_through_regress.membership">
        <field to="m2m_through_regress.person" name="person" rel="ManyToOneRel">2</field>
        <field to="m2m_through_regress.group" name="group" rel="ManyToOneRel">1</field>
        <field type="IntegerField" name="price">100</field>
      </object>
      <object pk="1" model="m2m_through_regress.usermembership">
        <field to="auth.user" name="user" rel="ManyToOneRel">2</field>
        <field to="m2m_through_regress.group" name="group" rel="ManyToOneRel">1</field>
        <field type="IntegerField" name="price">100</field>
      </object>
      <object pk="2" model="m2m_through_regress.usermembership">
        <field to="auth.user" name="user" rel="ManyToOneRel">2</field>
        <field to="m2m_through_regress.group" name="group" rel="ManyToOneRel">2</field>
        <field type="IntegerField" name="price">100</field>
      </object>
      <object pk="3" model="m2m_through_regress.usermembership">
        <field to="auth.user" name="user" rel="ManyToOneRel">3</field>
        <field to="m2m_through_regress.group" name="group" rel="ManyToOneRel">1</field>
        <field type="IntegerField" name="price">100</field>
      </object>
      <object pk="1" model="m2m_through_regress.person">
        <field type="CharField" name="name">Bob</field>
      </object>
      <object pk="2" model="m2m_through_regress.person">
        <field type="CharField" name="name">Jim</field>
      </object>
      <object pk="1" model="m2m_through_regress.group">
        <field type="CharField" name="name">Rock</field>
      </object>
      <object pk="2" model="m2m_through_regress.group">
        <field type="CharField" name="name">Roll</field>
      </object>
    </django-objects>


----------------------------------------------------------------------
Ran 443 tests in 580.486s

FAILED (failures=1)

[Malcolm Tredinnick]

Can somebody go through the test result and work out the actual difference is here. Right now, it's a bit hard to tell what went wrong.

First diff:

*** Untitled1.txt	Wed Aug 13 18:15:49 2008 UTC
--- Untitled2.txt	Wed Aug 13 18:15:49 2008 UTC
***************
*** 32,38 ****
          "fields": {
            "price": 100,
            "group": 1,
!           "user": 1
          }
        },
        {
--- 32,38 ----
          "fields": {
            "price": 100,
            "group": 1,
!           "user": 2
          }
        },
        {
***************
*** 41,47 ****
          "fields": {
            "price": 100,
            "group": 2,
!           "user": 1
          }
        },
        {
--- 41,47 ----
          "fields": {
            "price": 100,
            "group": 2,
!           "user": 2
          }
        },
        {
***************
*** 50,56 ****
          "fields": {
            "price": 100,
            "group": 1,
!           "user": 2
          }
        },
        {
--- 50,56 ----
          "fields": {
            "price": 100,
            "group": 1,
!           "user": 3
          }
        },
        {

Second diff:

*** Untitled1.txt	Wed Aug 13 18:15:49 2008 UTC
--- Untitled2.txt	Wed Aug 13 18:15:49 2008 UTC
***************
*** 16,32 ****
          <field type="IntegerField" name="price">100</field>
        </object>
        <object pk="1" model="m2m_through_regress.usermembership">
!         <field to="auth.user" name="user" rel="ManyToOneRel">1</field>
          <field to="m2m_through_regress.group" name="group" rel="ManyToOneRel">1</field>
          <field type="IntegerField" name="price">100</field>
        </object>
        <object pk="2" model="m2m_through_regress.usermembership">
!         <field to="auth.user" name="user" rel="ManyToOneRel">1</field>
          <field to="m2m_through_regress.group" name="group" rel="ManyToOneRel">2</field>
          <field type="IntegerField" name="price">100</field>
        </object>
        <object pk="3" model="m2m_through_regress.usermembership">
!         <field to="auth.user" name="user" rel="ManyToOneRel">2</field>
          <field to="m2m_through_regress.group" name="group" rel="ManyToOneRel">1</field>
          <field type="IntegerField" name="price">100</field>
        </object>
--- 16,32 ----
          <field type="IntegerField" name="price">100</field>
        </object>
        <object pk="1" model="m2m_through_regress.usermembership">
!         <field to="auth.user" name="user" rel="ManyToOneRel">2</field>
          <field to="m2m_through_regress.group" name="group" rel="ManyToOneRel">1</field>
          <field type="IntegerField" name="price">100</field>
        </object>
        <object pk="2" model="m2m_through_regress.usermembership">
!         <field to="auth.user" name="user" rel="ManyToOneRel">2</field>
          <field to="m2m_through_regress.group" name="group" rel="ManyToOneRel">2</field>
          <field type="IntegerField" name="price">100</field>
        </object>
        <object pk="3" model="m2m_through_regress.usermembership">
!         <field to="auth.user" name="user" rel="ManyToOneRel">3</field>
          <field to="m2m_through_regress.group" name="group" rel="ManyToOneRel">1</field>
          <field type="IntegerField" name="price">100</field>
        </object>

[Russell Keith-Magee]

This looks like a nasty, order-dependent, condition dependent bug. The User ID's allocated when jarrow ran the test don't correspond to the ones that were generated when I generated the test.

This test is mostly just a sanity check -it duplicates slightly a test in serializers_regress, but it's helpful to have a test that checks the output of the serializer against a known literal (to check against the case of the serializer and the deserializer doing the right thing with respect to each other, but not with respect to what we expect to see). We could prune down the contents of the database before running the dumpdata - we are just looking to check that the m2m field isn't serialized.

I'll try to take a look at this tonight, unless someone beats me to it.

[Karen Tracey <kmtracey@…>]

Hmm, just in case this is useful I'll note I noticed this was elusive today when I could run tests without seeing this on one of my machines (Windows) but not another (Linux). The failures I initially got on Linux were slightly different (more stuff was serialized, not just different numbers), but they became unreproducible after deleting .pyc files and making sure there were absolutely no differences (not that I ever found any that should have made a difference) in my tree vs. SVN. Then I started seeing the differences documented in this ticket, sometimes. Now it seems the machine has fallen into a pattern where it will hit the reported errors if I run the full test suite, but not if I run that test individually. I never got it to fail on Windows, trying various different Pythons and different DB backends, full test suite or just that test. I found it easier to recreate on Linux but even there it seems pretty slippery.

[Russell Keith-Magee]

@karen - that sounds consistent with what I'd expect. If you just run the m2m_through_regress test, you're only going to get 2 users created, with the IDs as expected. When other tests are running, it is possible that another user is created, messing up the IDs expected in the output.

As a quick test - if you put User.objects.all().delete() at the top of the m2m_through test on a machine, does it make the test pass?

[Karen Tracey <kmtracey@…>]

Replying to russellm:

As a quick test - if you put User.objects.all().delete() at the top of the m2m_through test on a machine, does it make the test pass?

Yes, I added that (attached) and on the machine where the full test suite had been failing it passed, at least once.

[Russell Keith-Magee]

(In [8347]) Fixed #8286 -- Simplified the m2m intermediate serialization test to make it easier to debug, and less prone to failures caused by test execution order. Thanks to jarrow for the report, and Karen Tracey for her help confirming the problem.

[Jacob]

Milestone 1.0 beta deleted