Opened 8 years ago

Closed 7 years ago

#10301 closed (duplicate)

Refactor Serializer to support iteratation

Reported by: Paul Egan Owned by: nobody
Component: Core (Serialization) Version: 1.0
Severity: Keywords:
Cc: Triage Stage: Design decision needed
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: UI/UX:

Description

The [source:django/trunk/django/core/serializers/base.py Serializer] interface forces the result to be fully materialised before use. This prevents it's use for streaming responses.

Currently the closest you can come is something like the following, but the response doesn't return until the xml document is complete:

    response = HttpResponse(content_type='text/xml')
    serializer = serializers.get_serializer("xml")()
    serializer.serialize(queryset.iterator(), stream=response)
    return response

The attached patch adds an iterator method to Serializer which yields the serialised data in chunks. This allows the serialisation of large query sets to be streamed to a response without using much memory:

    serializer = serializers.get_serializer("xml")()
    xml_iter = serializer.iterator(queryset.iterator(), chunk_size=100)
    return HttpResponse(xml_iter, content_type='text/xml')

Attachments (1)

serialzer_iterator.diff (2.6 KB) - added by Paul Egan 8 years ago.
Patch to add iterator support to Serializer

Download all attachments as: .zip

Change History (3)

Changed 8 years ago by Paul Egan

Attachment: serialzer_iterator.diff added

Patch to add iterator support to Serializer

comment:1 Changed 8 years ago by Eric Holscher

Triage Stage: UnreviewedDesign decision needed

Setting to DDN because it is a design decision. That said, I think this is a solid improvement and would be a nice thing to have in 1.1

comment:2 Changed 7 years ago by Russell Keith-Magee

Resolution: duplicate
Status: newclosed

Duplicate of #5423 (although this patch like it could be more promising).

Note: See TracTickets for help on using tickets.
Back to Top