Opened 20 months ago

Last modified 17 months ago

#34392 assigned New feature

Allow using test client response.json() with StreamingHttpResponse

Reported by: vainu-arto Owned by: vainu-arto
Component: Testing framework Version: dev
Severity: Normal Keywords:
Cc: Carlton Gibson Triage Stage: Accepted
Has patch: yes Needs documentation: no
Needs tests: yes Patch needs improvement: no
Easy pickings: no UI/UX: no

Description

This doesn't work since the ClientMixin._parse_json() method uses response.content directly instead of using the response.getvalue() wrapper which abstracts away the difference between the Response implementations.

Change History (7)

comment:1 by vainu-arto, 20 months ago

Owner: changed from nobody to vainu-arto
Status: newassigned

comment:2 by Carlton Gibson, 20 months ago

Triage Stage: UnreviewedAccepted

OK, I think this is likely an ok addition.

getvalue() should allow for an async iterator if we're going to use it this way. (Consuming self rather than self.streaming_content would be one way, but the warning would need to be caught.)

comment:3 by Carlton Gibson, 20 months ago

Needs tests: set

in reply to:  2 comment:4 by vainu-arto, 20 months ago

Replying to Carlton Gibson:

OK, I think this is likely an ok addition.

getvalue() should allow for an async iterator if we're going to use it this way. (Consuming self rather than self.streaming_content would be one way, but the warning would need to be caught.)

Huh. Apparently trac chose not to notify me on your comment. Sorry about the delay.

I don't really see the difference you are talking about here. "Consuming self" means calling StreamingHttpResponse.iter, right? That just returns self.streaming_content, ending up in the same place as far as I can tell. Can you be more specific on what you are asking me to do instead?

comment:5 by Mariusz Felisiak, 18 months ago

Cc: Carlton Gibson added

comment:6 by Carlton Gibson, 18 months ago

"Consuming self" means calling StreamingHttpResponse.iter, right? That just returns self.streaming_content, ending up in the same place as far as I can tell.

No, that's not quite right. Rather __iter__ and __aiter__ both handle mapping the wrong kind of iterator, and issue a warning, to let you know that's likely not what you're after:

>>> import asyncio
>>> from django.conf import settings 
>>> from django.http import StreamingHttpResponse
>>>
>>> settings.configure()
>>> 
>>> async def content():
...     yield 1
...     await asyncio.sleep(1)
...     yield 2
... 
>>> r = StreamingHttpResponse(content())
>>> list(r)
/Users/carlton/Projects/Django/django/django/http/response.py:497: Warning: StreamingHttpResponse must consume asynchronous iterators in order to serve them synchronously. Use a synchronous iterator instead.
  warnings.warn(
[b'1', b'2']
>>> r = StreamingHttpResponse(content())
>>> list(r.streaming_content)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: 'async_generator' object is not iterable

That is, calling list (or join, in the case at hand here) on the response itself allows accessing the content, where doing the same on streaming_content does not.

getvalue() is currently going via streaming_content and so (currently) would hit the type error here. Moving to …join(self) would resolve that, at the cost of the warning which would need to be captured.

Make sense?

Version 0, edited 18 months ago by Carlton Gibson (next)

in reply to:  6 comment:7 by vainu-arto, 17 months ago

Replying to Carlton Gibson:

"Consuming self" means calling StreamingHttpResponse.iter, right? That just returns self.streaming_content, ending up in the same place as far as I can tell.

No, that's not quite right. Rather __iter__ and __aiter__ both handle mapping the wrong kind of iterator, and issue a warning, to let you know that's likely not what you're after:

>>> import asyncio
>>> from django.conf import settings 
>>> from django.http import StreamingHttpResponse
>>>
>>> settings.configure()
>>> 
>>> async def content():
...     yield 1
...     await asyncio.sleep(1)
...     yield 2
... 
>>> r = StreamingHttpResponse(content())
>>> list(r)
/Users/carlton/Projects/Django/django/django/http/response.py:497: Warning: StreamingHttpResponse must consume asynchronous iterators in order to serve them synchronously. Use a synchronous iterator instead.
  warnings.warn(
[b'1', b'2']
>>> r = StreamingHttpResponse(content())
>>> list(r.streaming_content)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: 'async_generator' object is not iterable

That is, calling list (or join, in the case at hand here) on the response itself allows accessing the content, where doing the same on streaming_content does not.

getvalue() is currently going via streaming_content and so (currently) would hit the type error here. Moving to …join(self) would resolve that, at the cost of the warning which would need to be captured, in ClientMixin.

Make sense?


So you are discussing a bug that currently exists in StreamingHttpResponse.getvalue() when the content is an async iterator? StreamingHttpResponse.json() doesn't currently work at all (regardless of async/sync) and as such this issue isn't caused or made worse by the code change suggested in this PR. Is fixing this only somewhat-related bug considered a prerequisite to merging this change?

Note: See TracTickets for help on using tickets.
Back to Top