Opened 14 years ago

Closed 4 years ago

#12091 closed New feature (wontfix)

Support for WSGI applications within Django

Reported by: Gustavo Narea Owned by: Gustavo Narea
Component: HTTP handling Version: dev
Severity: Normal Keywords: WSGI
Cc: Triage Stage: Accepted
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: yes
Easy pickings: no UI/UX: no

Description

With the attached patch, it will be possible to run WSGI applications from Django and even use such applications as Django views.

It depends on the patch I supplied in #8927.

Please let me know what you think about it.

Attachments (1)

wsgi-apps.diff (19.4 KB ) - added by Gustavo Narea 14 years ago.
Support for WSGI apps within Django

Download all attachments as: .zip

Change History (17)

comment:1 by James Bennett, 14 years ago

I think it'd be nice if this explained how it differs from the WSGI-oriented SoC branch, and indicated whether it builds on that work.

in reply to:  1 comment:2 by Gustavo Narea, 14 years ago

Replying to ubernostrum:

I think it'd be nice if this explained how it differs from the WSGI-oriented SoC branch, and indicated whether it builds on that work.

Hi.

I did it all from scratch and the result is quite different from what I found at: http://github.com/rfk/django/blob/f96eb6068552e228f705beda9c8181db2079a797/django/wsgi/wsgi_to_django.py

Most importantly, it's much simpler because I am not trying to reconstruct the WSGI environ if it isn't already present. I assume it's already available (see patch in #8927). Therefore it also passes the wsgi.input and wsgi.error file-like objects on to the WSGI app, without reconstructing them.

Cheers.

by Gustavo Narea, 14 years ago

Attachment: wsgi-apps.diff added

Support for WSGI apps within Django

comment:3 by Gustavo Narea, 14 years ago

I've updated the patch because I found a couple of minor bugs in the previous one and the test suite didn't have a 100% of code coverage.

comment:4 by Gustavo Narea, 14 years ago

Since it's not going to make it by v1.2 and we don't want to have a patchy Django for so long, I reimplemented this patch and others in an independent package:
http://bitbucket.org/2degrees/twod.wsgi/

If you'd like to implement this in Django, I'd recommend copying the relevant modules from twod.wsgi (which is licensed under the BSD license too) because I fixed a few tiny bugs with my original patch.

comment:5 by Russell Keith-Magee, 14 years ago

milestone: 1.2
Triage Stage: UnreviewedAccepted

Marking as accepted (since, like #8297, it's a reasonable idea). However, I'm still unclear to the extent this patch overlaps with existing WSGI efforts.

in reply to:  5 comment:6 by Gustavo Narea, 14 years ago

Replying to russellm:

Marking as accepted (since, like #8297, it's a reasonable idea).

I'm glad to hear that! Here's the documentation for that functionality: http://packages.python.org/twod.wsgi/manual/embedded-apps.html

Please let me know if you like the way it's implemented, so that I can port it from twod.wsgi into Django.

However, I'm still unclear to the extent this patch overlaps with existing WSGI efforts.

Here's my (biased) response: http://packages.python.org/twod.wsgi/about.html#twod-wsgi-versus

comment:7 by Julien Phalip, 13 years ago

Severity: Normal
Type: New feature

comment:8 by patchhammer, 13 years ago

Easy pickings: unset
Patch needs improvement: set

wsgi-apps.diff fails to apply cleanly on to trunk

comment:9 by Aymeric Augustin, 12 years ago

UI/UX: unset

Change UI/UX from NULL to False.

comment:10 by Aymeric Augustin, 11 years ago

Resolution: wontfix
Status: newclosed

I'm in favor of keeping this feature outside of Django, for the following reasons.

1) Embedding WSGI applications in Django isn't the most robust design. It's much better to dispatch at the WSGI level between Django and other applications.

2) While it's easy to just make this feature work for simple use cases, it's very difficult to make the gateway 100% WSGI-compliant (even if the recently-added StreamingHttpResponse could help abide by the buffering rules). I'm afraid this would add a significant maintainance load, while benefitting only a few users with non-trivial architectures.

3) There isn't any reason why this needs to live in core, and there are a few implementations floating in the wild. I have my own implementation of this too -- I use it in development as a replacement for uwsgi.applications (http://projects.unbit.it/uwsgi/wiki/ApplicationsDict).

comment:11 by Gustavo Narea, 11 years ago

Resolution: wontfix
Status: closednew

1) "Dispatch at the WSGI level" is ambiguous. I assume you mean dispatch before Django is reached, which wouldn't be better or even accomplish the same. Picture this example: You want to use the WSGI X-Sendfile application from a Django view because you want to (1) check that the user can download the file, (2) log who's downloading the file and (3) log when the file was downloaded. Try to do that outside of Django and you'll end up with an unmaintainable piece of code.
2) I'm not sure I understand this point well enough. What does the gateway have to do with all this? Anyway, I can assure you that this is a piece of cake and it's 100% reliable in 100% of the cases you can imagine. I've reimplemented a variant of that patch in twod.wsgi and presented demos of how you can proxy applications like Trac and the PHP-powered Wordpress via Django. What complicated situations can you imagine that wouldn't be supported?
3) I think you're confusing "embedding WSGI applications inside Django" with "routing WSGI applications". Both can be quite useful, depending on your needs, but there's a big difference. This ticket is about embedding, but you're talking about routing.

I also want to add that all the other mainstream WSGI framework have supported this outside-the-box for years.

I'll dare reopening this ticket because I think there's been a misunderstanding, and would like to prevent this ticket from going unnoticed for years again.

in reply to:  11 ; comment:12 by Aymeric Augustin, 11 years ago

Replying to Gustavo:

1) "Dispatch at the WSGI level" is ambiguous. I assume you mean dispatch before Django is reached, which wouldn't be better or even accomplish the same. Picture this example: You want to use the WSGI X-Sendfile application from a Django view because you want to (1) check that the user can download the file, (2) log who's downloading the file and (3) log when the file was downloaded. Try to do that outside of Django and you'll end up with an unmaintainable piece of code.

For such cases, I suggest to use an external Django view to WSGI mapper — twod.wsgi or one of the many other packages providing the same feature.

2) I'm not sure I understand this point well enough. What does the gateway have to do with all this? Anyway, I can assure you that this is a piece of cake and it's 100% reliable in 100% of the cases you can imagine. I've reimplemented a variant of that patch in twod.wsgi and presented demos of how you can proxy applications like Trac and the PHP-powered Wordpress via Django. What complicated situations can you imagine that wouldn't be supported?

Django is a WSGI application. If it acts as a WSGI server too, that makes it a WSGI gateway. As such, it'll be expected to honor the spec for WSGI gateways. People can be unbelievably annoying with spec-compliance bugs.

Let's take this sentence from PEP 3333:

WSGI servers, gateways, and middleware must not delay the transmission of any block; they must either fully transmit the block to the client, or guarantee that they will continue transmission even while the application is producing its next block.

twod.wsgi doesn't respect this when the write() callable is used. (To you credit, you handled the iterator case correctly.)

Describing something as a piece of cake and 100% correct isn't good for your credibility when I can find a bug through code inspection!

3) I think you're confusing "embedding WSGI applications inside Django" with "routing WSGI applications". Both can be quite useful, depending on your needs, but there's a big difference. This ticket is about embedding, but you're talking about routing.

Well, the very first example in twod.wgsi's docs is called embedding but it actually shows routing:
http://pythonhosted.org/twod.wsgi/manual/embedded-apps.html#embedding-non-django-applications

The second example is about embedding:
http://pythonhosted.org/twod.wsgi/manual/embedded-apps.html#calling-them-from-a-django-view

The concepts aren't as neatly delimited as you make it sound, and both can be implemented as external apps.

I also want to add that all the other mainstream WSGI framework have supported this outside-the-box for years.

Django isn't a "WSGI framework" per se — it uses WSGI as a deployment platform, that's all. Besides, this isn't a competition.

I'll dare reopening this ticket because I think there's been a misunderstanding, and would like to prevent this ticket from going unnoticed for years again.

Sure, we can leave it open for now, I'll let someone else make the decision.

in reply to:  12 comment:13 by Gustavo Narea, 11 years ago

Replying to aaugustin:

Replying to Gustavo:

1) "Dispatch at the WSGI level" is ambiguous. I assume you mean dispatch before Django is reached, which wouldn't be better or even accomplish the same. Picture this example: You want to use the WSGI X-Sendfile application from a Django view because you want to (1) check that the user can download the file, (2) log who's downloading the file and (3) log when the file was downloaded. Try to do that outside of Django and you'll end up with an unmaintainable piece of code.

For such cases, I suggest to use an external Django view to WSGI mapper — twod.wsgi or one of the many other packages providing the same feature.

Then why not provide it in Django and prevent the proliferation of implementations to accomplish this? That's indicative that there's a demand for this.

2) I'm not sure I understand this point well enough. What does the gateway have to do with all this? Anyway, I can assure you that this is a piece of cake and it's 100% reliable in 100% of the cases you can imagine. I've reimplemented a variant of that patch in twod.wsgi and presented demos of how you can proxy applications like Trac and the PHP-powered Wordpress via Django. What complicated situations can you imagine that wouldn't be supported?

Django is a WSGI application. If it acts as a WSGI server too, that makes it a WSGI gateway. As such, it'll be expected to honor the spec for WSGI gateways. People can be unbelievably annoying with spec-compliance bugs.

I see. Agreed.

Let's take this sentence from PEP 3333:

WSGI servers, gateways, and middleware must not delay the transmission of any block; they must either fully transmit the block to the client, or guarantee that they will continue transmission even while the application is producing its next block.

twod.wsgi doesn't respect this when the write() callable is used. (To you credit, you handled the iterator case correctly.)

Actually, the write() callable is supported just fine. twod.wsgi doesn't do anything with it because it delegates it to WebOb's Request.call_application(), which supports write() the way it's supposed to: https://github.com/Pylons/webob/blob/master/webob/request.py#L1238

Describing something as a piece of cake and 100% correct isn't good for your credibility when I can find a bug through code inspection!

The approach is a piece of cake, even if there are bugs in my implementation, which may well be the case because I'm a human! "100% correct" are your own words, isn't that right? I said "100% reliable in 100% of the cases you can imagine", and I stand by it.

And what you've found isn't a bug, unless I've misinterpreted your reply.

3) I think you're confusing "embedding WSGI applications inside Django" with "routing WSGI applications". Both can be quite useful, depending on your needs, but there's a big difference. This ticket is about embedding, but you're talking about routing.

Well, the very first example in twod.wgsi's docs is called embedding but it actually shows routing:
http://pythonhosted.org/twod.wsgi/manual/embedded-apps.html#embedding-non-django-applications
The second example is about embedding:
http://pythonhosted.org/twod.wsgi/manual/embedded-apps.html#calling-them-from-a-django-view

No, it's embedding. make_wsgi_app() returns a Django view, which makes a huge difference.

Consider this example: If the embedded WSGI app relies on REMOTE_USER for authentication and Django previously authenticated the current user, then the embedded app will reuse Django's authentication. By contrast, if you route requests to the same WSGI app outside of Django, as you suggest, the embedded WSGI app won't be able to reuse Django's authentication.

The concepts aren't as neatly delimited as you make it sound,

Well, they are and they've been for ages with other Python frameworks.

and both can be implemented as external apps.

No.

I also want to add that all the other mainstream WSGI framework have supported this outside-the-box for years.

Django isn't a "WSGI framework" per se — it uses WSGI as a deployment platform, that's all. Besides, this isn't a competition.

I didn't mean that as a call for competition or anything. I just wanted to point out that everywhere else this functionality is taken for granted and yet here we are discussing whether it should be part of Django itself, while other Django users continue creating their own implementations for the missing functionality.

It may not be its primary focus, but Django is a WSGI framework, just like TurboGears, Pylons, etc. WSGI is more just a deployment platform.

I'll dare reopening this ticket because I think there's been a misunderstanding, and would like to prevent this ticket from going unnoticed for years again.

Sure, we can leave it open for now, I'll let someone else make the decision.

Thanks! I wonder whether this should be taken to the mailing list.

comment:15 by Tim Graham, 9 years ago

Summary: [PATCH] Support for WSGI applications within DjangoSupport for WSGI applications within Django

comment:16 by Mariusz Felisiak, 4 years ago

Resolution: wontfix
Status: newclosed

After reconsideration I think we shouldn't move it forward. It's quite niche, can live in a 3rd party package, and not worth the additional complexity. Moreover there was no consensus on the mailing list. Thanks for you efforts!

Note: See TracTickets for help on using tickets.
Back to Top