Code

Opened 5 years ago

Closed 5 years ago

Last modified 5 years ago

#10960 closed (invalid)

fixture loading of M2M fields fails for child ids greater than 9

Reported by: adrian_nye Owned by: nobody
Component: Uncategorized Version: 1.0
Severity: Keywords:
Cc: Triage Stage: Unreviewed
Has patch: no Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: UI/UX:

Description

Loading data in a JSON file that includes data for both models involved in a M2M relationship,
child ids up to 9 work fine, 10 or greater cause an error on MySQL or Postgres

On Postgres:

psycopg2.IntegrityError: insert or update on table "entity_person_address" violates foreign key constraint "entity_person_address_address_id_fkey"
DETAIL: Key (address_id)=(0) is not present in table "entity_address".

On MySQL:

Cannot add or update a child row: a foreign key constraint fails - tixsa.entity_person_telephone CONSTRAIN telephone_id_refs_id_92398a FOREIGN KEY telephone_id REFERENCES entity_telephone id

To see problem, use the attached model file and attached initial_data.json. You'll see the errors above. Then search for an id of "10" and change it to "1" and the problem goes away.

Attachments (2)

models.py (9.7 KB) - added by adrian_nye 5 years ago.
initial_data.json (14.2 KB) - added by adrian_nye 5 years ago.
json initial data with "10" value causing error

Download all attachments as: .zip

Change History (6)

Changed 5 years ago by adrian_nye

Changed 5 years ago by adrian_nye

json initial data with "10" value causing error

comment:1 Changed 5 years ago by kmtracey

  • Needs documentation unset
  • Needs tests unset
  • Patch needs improvement unset
  • Resolution set to invalid
  • Status changed from new to closed

I believe you are specifying the ManyToMany fields incorrectly. As there are potentially many values for these, they need to be specified as a list. The PostgreSQL error here is more helpful than MySQL as PostgreSQL indicates specifically that it is objecting to an address ID of 0 associated with a person, which gives me the impression that for this bit of your initial data file:

       "pk": 10,
 	        "model": "entity.person",
 	        "fields": {
 	                "name_prefix": "MR",
 	                "first_name": "Valary",
 	                "middle_name": "",
 	                "last_name": "Howard",
 	                "name_suffix": "III",
 	                "address": "10",
 	                "telephone": "10",
 	                "email_address": "vhoward@interbaun.com",
 	                "email_accepted": "True",
 	                "website_url": "",
 	                "updated": "2009-04-20 15:31"
 	                }
 	        },

the address value is being interpreted as 2 values: first 1, then 0. The attempt to add a reference to an address with a pk value of 0 fails since there is no address with a pk value of 0.

I believe the correct format for specifying that (and all other ManyToManys you have) would be:

	                "address": ["10"],

comment:2 follow-up: Changed 5 years ago by adrian_nye

If you are right, then why does changing the "10" to "1" make the error message go away? The format should still be wrong and should still cause an error since the models have not changed.

comment:3 Changed 5 years ago by adrian_nye

Furthermore, if Django does indeed take a list here and fills the intermediate table automatically from the list, then this feature needs to be documented with an example.

comment:4 in reply to: ↑ 2 Changed 5 years ago by kmtracey

Replying to adrian_nye:

If you are right, then why does changing the "10" to "1" make the error message go away?

Because iterating over "1" produces "1" whereas iterating over "10" produces "1" and "0":

>>> for x in "10":
...    print x
...
1
0
>>> for x in "1":
...    print x
...
1

Which is also why I'd guess it would be fixed by changing the value to a list, as then iterating over it would yield the (single) correct value you are looking for:

>>> for x in ["10"]:
...    print x
...
10
>>>

The format should still be wrong and should still cause an error since the models have not changed.

I'd guess any iterable will do for a multi-valued field, so the format is not wrong as in completely invalid but just wrong because it isn't producing the results you want. "10" is iterable but iterating over it results in an attempt to create two address associations for this person (one to address with pk 1 and one to address with pk 0). The only reason you are seeing a database error is because address with pk 0 does not exist. I suspect you wouldn't see a database-level error with "12", since both 1 and 2 are valid address pks. But you'd get an object with 2 associated addresses (1 and 2), not the one address (12) I'm guessing you're expecting. Single-digit strings are also iterable but don't show cause a problem because iterating over them produces the same results as iterating over a list that contains them as a single item.

I'm not a serialization expert and I'm just basing the above on the error message and a brief look at a json initial data file for the aggregation_regress tests. The "authors" field specified there is for a ManyToMany field and it is specified using lists. Further the pk values specified there are integers, not strings, which seems more correct since in the DB these will be integer values. Your strings are working I'd guess because they can be cast to the right type without error. (If you had specified them all as ints instead of strings I suspect you'd have seen an error due ints not being iterable when it came time to iterate over the sequence of values for your ManyToMany fields.)

As for documenting this -- it may be that the doc on formats could use some enhancements, but I don't know enough about common usage to say for sure. Reading through the serialization doc I get the impression these files are most commonly creating by serializing existing objects, not writing the files from scratch. I don't see where we go into much detail on the actual formats used anywhere? If the usual way people create these files is either by serializing existing objects (or at least starting with what the serializer producers as an example) I don't know that it would be that useful to go into excruciating detail on the exact formats in the doc.

Add Comment

Modify Ticket

Change Properties
<Author field>
Action
as closed
as The resolution will be set. Next status will be 'closed'
The resolution will be deleted. Next status will be 'new'
Author


E-mail address and user name can be saved in the Preferences.

 
Note: See TracTickets for help on using tickets.