Opened 7 years ago

Closed 7 years ago

#6884 closed (invalid)

Deserialization in loaddata causes a TypeError

Reported by: dfrishberg Owned by: nobody
Component: Core (Serialization) Version: master
Severity: Keywords:
Cc: Triage Stage: Unreviewed
Has patch: no Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: UI/UX:

Description

line 181, django/core/serializers/xml_serializer.py:

return base.DeserializedObject(Model(**data), m2m_data)

Python functions cannot accept unicode kwargs, so when data has unicode keywords a TypeError is thrown by this line. Recommend enforcing that all dicts passed to constructors with this syntax are normalized to string keys.

Error occurred when the deserializer somehow turned the xml below into the dict below:

<object pk="1" model="scorecard.committeeassignment">
        <field type="IntegerField" name="pid">158699</field>
        <field to="scorecard.official" name="official" rel="ManyToOneRel">Office object</field>
        <field type="CharField" name="comcode1">HCAUC015</field>
        <field type="CharField" name="subcode11"><None></None></field>
        <field type="CharField" name="subcode12"><None></None></field>
        <field type="CharField" name="subcode13"><None></None></field>
        <field type="CharField" name="subcode14"><None></None></field>
        <field type="CharField" name="subcode15"><None></None></field>
        <field type="CharField" name="comcode2">HCAUC075</field>
        <field type="CharField" name="subcode21"><None></None></field>
        <field type="CharField" name="subcode22"><None></None></field>
        <field type="CharField" name="subcode23"><None></None></field>
        <field type="CharField" name="subcode24"><None></None></field>
        <field type="CharField" name="subcode25"><None></None></field>
        <field type="CharField" name="comcode3">HCAUC220</field>
        <field type="CharField" name="subcode31"><None></None></field>
        <field type="CharField" name="subcode32"><None></None></field>
        <field type="CharField" name="subcode33"><None></None></field>
        <field type="CharField" name="subcode34"><None></None></field>
        <field type="CharField" name="subcode35"><None></None></field>
        <field type="CharField" name="comcode4">HNRES</field>
        <field type="CharField" name="subcode41">HNRES05</field>
        <field type="CharField" name="subcode42">HNRES10</field>
        <field type="CharField" name="subcode43">HNRES15</field>
        <field type="CharField" name="subcode44">HNRES20</field>
        <field type="CharField" name="subcode45">HNRES25</field>
        <field type="CharField" name="comcode5">HTRA</field>
        <field type="CharField" name="subcode51">HTRA10</field>
        <field type="CharField" name="subcode52">HTRA20</field>
        <field type="CharField" name="subcode53"><None></None></field>
        <field type="CharField" name="subcode54"><None></None></field>
        <field type="CharField" name="subcode55"><None></None></field>
        <field type="CharField" name="comcode6">JCAUC031</field>
        <field type="CharField" name="comcode7">JCAUC033</field>
        <field type="CharField" name="comcode8"><None></None></field>
        <field type="CharField" name="comcode9"><None></None></field>
    </object>
{'pid': u'158699', u'comcode6': u'JCAUC031', u'comcode7': u'JCAUC033', 'id': 1, u'subcode52': u'HTRA20', u'subcode51': u'HTRA10', u'subcode13': None, u'subcode12': None, u'subcode11': None, u'subcode15': None, u'subcode14': None, u'subcode35': None, u'subcode34': None, u'subcode31': None, u'subcode33': None, u'subcode32': None, u'comcode4': u'HNRES', u'comcode5': u'HTRA', u'subcode55': None, u'subcode54': None, u'subcode53': None, u'comcode1': u'HCAUC015', u'comcode2': u'HCAUC075', u'comcode3': u'HCAUC220', u'comcode8': None, u'comcode9': None, 'official_id': u'Office object', u'subcode42': u'HNRES10', u'subcode43': u'HNRES15', u'subcode44': u'HNRES20', u'subcode45': u'HNRES25', u'subcode41': u'HNRES05', u'subcode24': None, u'subcode25': None, u'subcode22': None, u'subcode23': None, u'subcode21': None}

Change History (2)

comment:1 Changed 7 years ago by dfrishberg

  • Needs documentation unset
  • Needs tests unset
  • Patch needs improvement unset

EDIT: I think it's my fault after all. I forced unicode attributes in my model declaration. I'm leaving the ticket open, though, until someone makes the decision that django doesn't need to do more than it's already doing to enforce keyword restrictions:

class CommitteeAssignment(models.Model):
    #uid = models.IntegerField(db_column='UID', max_length=10, unique=True)
    pid = models.IntegerField(db_column='PID', max_length=10)
    official = models.ForeignKey(Official, db_column='uid', related_name='committee_assignment', to_field='office')
    for i in range(9):
        i += 1
        locals()['comcode'+unicode(i)] = models.CharField(db_column='COMCODE'+unicode(i), max_length=8, null=True, blank=True)
        if i < 6:
            for j in range(5):
                j += 1
                locals()['subcode'+unicode(i)+unicode(j)] = models.CharField(db_column='SUBCODE'+unicode(i)+unicode(j), max_length=8, null=True, blank=True)

    class Meta:
        db_table = 'committee_assignments'

comment:2 Changed 7 years ago by mtredinnick

  • Resolution set to invalid
  • Status changed from new to closed

Normal XML serialisation and then deserialisation appears to work fine. I can't see that there's an issue here. We'd be seeing reports all the time if that wasn't the case (and it works fine for me in testing).

I agree that in your example, whatever it is that you're trying to do in the model there is the cause of the problem.

Note: See TracTickets for help on using tickets.
Back to Top