Opened 10 years ago

Closed 10 years ago

#10301 closed (duplicate)

Refactor Serializer to support iteratation

Reported by: Paul Egan Owned by: nobody
Component: Core (Serialization) Version: 1.0
Severity: Keywords:
Cc: Triage Stage: Design decision needed
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no


The [source:django/trunk/django/core/serializers/ Serializer] interface forces the result to be fully materialised before use. This prevents it's use for streaming responses.

Currently the closest you can come is something like the following, but the response doesn't return until the xml document is complete:

    response = HttpResponse(content_type='text/xml')
    serializer = serializers.get_serializer("xml")()
    serializer.serialize(queryset.iterator(), stream=response)
    return response

The attached patch adds an iterator method to Serializer which yields the serialised data in chunks. This allows the serialisation of large query sets to be streamed to a response without using much memory:

    serializer = serializers.get_serializer("xml")()
    xml_iter = serializer.iterator(queryset.iterator(), chunk_size=100)
    return HttpResponse(xml_iter, content_type='text/xml')

Attachments (1)

serialzer_iterator.diff (2.6 KB) - added by Paul Egan 10 years ago.
Patch to add iterator support to Serializer

Download all attachments as: .zip

Change History (3)

Changed 10 years ago by Paul Egan

Attachment: serialzer_iterator.diff added

Patch to add iterator support to Serializer

comment:1 Changed 10 years ago by Eric Holscher

Triage Stage: UnreviewedDesign decision needed

Setting to DDN because it is a design decision. That said, I think this is a solid improvement and would be a nice thing to have in 1.1

comment:2 Changed 10 years ago by Russell Keith-Magee

Resolution: duplicate
Status: newclosed

Duplicate of #5423 (although this patch like it could be more promising).

Note: See TracTickets for help on using tickets.
Back to Top