Code

Opened 5 years ago

Closed 5 years ago

#10301 closed (duplicate)

Refactor Serializer to support iteratation

Reported by: paulegan Owned by: nobody
Component: Core (Serialization) Version: 1.0
Severity: Keywords:
Cc: Triage Stage: Design decision needed
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: UI/UX:

Description

The [source:django/trunk/django/core/serializers/base.py Serializer] interface forces the result to be fully materialised before use. This prevents it's use for streaming responses.

Currently the closest you can come is something like the following, but the response doesn't return until the xml document is complete:

    response = HttpResponse(content_type='text/xml')
    serializer = serializers.get_serializer("xml")()
    serializer.serialize(queryset.iterator(), stream=response)
    return response

The attached patch adds an iterator method to Serializer which yields the serialised data in chunks. This allows the serialisation of large query sets to be streamed to a response without using much memory:

    serializer = serializers.get_serializer("xml")()
    xml_iter = serializer.iterator(queryset.iterator(), chunk_size=100)
    return HttpResponse(xml_iter, content_type='text/xml')

Attachments (1)

serialzer_iterator.diff (2.6 KB) - added by paulegan 5 years ago.
Patch to add iterator support to Serializer

Download all attachments as: .zip

Change History (3)

Changed 5 years ago by paulegan

Patch to add iterator support to Serializer

comment:1 Changed 5 years ago by ericholscher

  • Needs documentation unset
  • Needs tests unset
  • Patch needs improvement unset
  • Triage Stage changed from Unreviewed to Design decision needed

Setting to DDN because it is a design decision. That said, I think this is a solid improvement and would be a nice thing to have in 1.1

comment:2 Changed 5 years ago by russellm

  • Resolution set to duplicate
  • Status changed from new to closed

Duplicate of #5423 (although this patch like it could be more promising).

Add Comment

Modify Ticket

Change Properties
<Author field>
Action
as closed
as The resolution will be set. Next status will be 'closed'
The resolution will be deleted. Next status will be 'new'
Author


E-mail address and user name can be saved in the Preferences.

 
Note: See TracTickets for help on using tickets.