Code

Opened 4 years ago

Last modified 12 months ago

#12091 new New feature

[PATCH] Support for WSGI applications within Django

Reported by: Gustavo Owned by: Gustavo
Component: HTTP handling Version: master
Severity: Normal Keywords: WSGI
Cc: Triage Stage: Accepted
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: yes
Easy pickings: no UI/UX: no

Description

With the attached patch, it will be possible to run WSGI applications from Django and even use such applications as Django views.

It depends on the patch I supplied in #8927.

Please let me know what you think about it.

Attachments (1)

wsgi-apps.diff (19.4 KB) - added by Gustavo 4 years ago.
Support for WSGI apps within Django

Download all attachments as: .zip

Change History (15)

comment:1 follow-up: Changed 4 years ago by ubernostrum

  • Needs documentation unset
  • Needs tests unset
  • Patch needs improvement unset

I think it'd be nice if this explained how it differs from the WSGI-oriented SoC branch, and indicated whether it builds on that work.

comment:2 in reply to: ↑ 1 Changed 4 years ago by Gustavo

Replying to ubernostrum:

I think it'd be nice if this explained how it differs from the WSGI-oriented SoC branch, and indicated whether it builds on that work.

Hi.

I did it all from scratch and the result is quite different from what I found at: http://github.com/rfk/django/blob/f96eb6068552e228f705beda9c8181db2079a797/django/wsgi/wsgi_to_django.py

Most importantly, it's much simpler because I am not trying to reconstruct the WSGI environ if it isn't already present. I assume it's already available (see patch in #8927). Therefore it also passes the wsgi.input and wsgi.error file-like objects on to the WSGI app, without reconstructing them.

Cheers.

Changed 4 years ago by Gustavo

Support for WSGI apps within Django

comment:3 Changed 4 years ago by Gustavo

I've updated the patch because I found a couple of minor bugs in the previous one and the test suite didn't have a 100% of code coverage.

comment:4 Changed 4 years ago by Gustavo

Since it's not going to make it by v1.2 and we don't want to have a patchy Django for so long, I reimplemented this patch and others in an independent package:
http://bitbucket.org/2degrees/twod.wsgi/

If you'd like to implement this in Django, I'd recommend copying the relevant modules from twod.wsgi (which is licensed under the BSD license too) because I fixed a few tiny bugs with my original patch.

comment:5 follow-up: Changed 4 years ago by russellm

  • milestone 1.2 deleted
  • Triage Stage changed from Unreviewed to Accepted

Marking as accepted (since, like #8297, it's a reasonable idea). However, I'm still unclear to the extent this patch overlaps with existing WSGI efforts.

comment:6 in reply to: ↑ 5 Changed 4 years ago by Gustavo

Replying to russellm:

Marking as accepted (since, like #8297, it's a reasonable idea).

I'm glad to hear that! Here's the documentation for that functionality: http://packages.python.org/twod.wsgi/manual/embedded-apps.html

Please let me know if you like the way it's implemented, so that I can port it from twod.wsgi into Django.

However, I'm still unclear to the extent this patch overlaps with existing WSGI efforts.

Here's my (biased) response: http://packages.python.org/twod.wsgi/about.html#twod-wsgi-versus

comment:7 Changed 3 years ago by julien

  • Severity set to Normal
  • Type set to New feature

comment:8 Changed 3 years ago by patchhammer

  • Easy pickings unset
  • Patch needs improvement set

wsgi-apps.diff fails to apply cleanly on to trunk

comment:9 Changed 2 years ago by aaugustin

  • UI/UX unset

Change UI/UX from NULL to False.

comment:10 Changed 13 months ago by aaugustin

  • Resolution set to wontfix
  • Status changed from new to closed

I'm in favor of keeping this feature outside of Django, for the following reasons.

1) Embedding WSGI applications in Django isn't the most robust design. It's much better to dispatch at the WSGI level between Django and other applications.

2) While it's easy to just make this feature work for simple use cases, it's very difficult to make the gateway 100% WSGI-compliant (even if the recently-added StreamingHttpResponse could help abide by the buffering rules). I'm afraid this would add a significant maintainance load, while benefitting only a few users with non-trivial architectures.

3) There isn't any reason why this needs to live in core, and there are a few implementations floating in the wild. I have my own implementation of this too -- I use it in development as a replacement for uwsgi.applications (http://projects.unbit.it/uwsgi/wiki/ApplicationsDict).

comment:11 follow-up: Changed 13 months ago by Gustavo

  • Resolution wontfix deleted
  • Status changed from closed to new

1) "Dispatch at the WSGI level" is ambiguous. I assume you mean dispatch before Django is reached, which wouldn't be better or even accomplish the same. Picture this example: You want to use the WSGI X-Sendfile application from a Django view because you want to (1) check that the user can download the file, (2) log who's downloading the file and (3) log when the file was downloaded. Try to do that outside of Django and you'll end up with an unmaintainable piece of code.
2) I'm not sure I understand this point well enough. What does the gateway have to do with all this? Anyway, I can assure you that this is a piece of cake and it's 100% reliable in 100% of the cases you can imagine. I've reimplemented a variant of that patch in twod.wsgi and presented demos of how you can proxy applications like Trac and the PHP-powered Wordpress via Django. What complicated situations can you imagine that wouldn't be supported?
3) I think you're confusing "embedding WSGI applications inside Django" with "routing WSGI applications". Both can be quite useful, depending on your needs, but there's a big difference. This ticket is about embedding, but you're talking about routing.

I also want to add that all the other mainstream WSGI framework have supported this outside-the-box for years.

I'll dare reopening this ticket because I think there's been a misunderstanding, and would like to prevent this ticket from going unnoticed for years again.

comment:12 in reply to: ↑ 11 ; follow-up: Changed 13 months ago by aaugustin

Replying to Gustavo:

1) "Dispatch at the WSGI level" is ambiguous. I assume you mean dispatch before Django is reached, which wouldn't be better or even accomplish the same. Picture this example: You want to use the WSGI X-Sendfile application from a Django view because you want to (1) check that the user can download the file, (2) log who's downloading the file and (3) log when the file was downloaded. Try to do that outside of Django and you'll end up with an unmaintainable piece of code.

For such cases, I suggest to use an external Django view to WSGI mapper — twod.wsgi or one of the many other packages providing the same feature.

2) I'm not sure I understand this point well enough. What does the gateway have to do with all this? Anyway, I can assure you that this is a piece of cake and it's 100% reliable in 100% of the cases you can imagine. I've reimplemented a variant of that patch in twod.wsgi and presented demos of how you can proxy applications like Trac and the PHP-powered Wordpress via Django. What complicated situations can you imagine that wouldn't be supported?

Django is a WSGI application. If it acts as a WSGI server too, that makes it a WSGI gateway. As such, it'll be expected to honor the spec for WSGI gateways. People can be unbelievably annoying with spec-compliance bugs.

Let's take this sentence from PEP 3333:

WSGI servers, gateways, and middleware must not delay the transmission of any block; they must either fully transmit the block to the client, or guarantee that they will continue transmission even while the application is producing its next block.

twod.wsgi doesn't respect this when the write() callable is used. (To you credit, you handled the iterator case correctly.)

Describing something as a piece of cake and 100% correct isn't good for your credibility when I can find a bug through code inspection!

3) I think you're confusing "embedding WSGI applications inside Django" with "routing WSGI applications". Both can be quite useful, depending on your needs, but there's a big difference. This ticket is about embedding, but you're talking about routing.

Well, the very first example in twod.wgsi's docs is called embedding but it actually shows routing:
http://pythonhosted.org/twod.wsgi/manual/embedded-apps.html#embedding-non-django-applications

The second example is about embedding:
http://pythonhosted.org/twod.wsgi/manual/embedded-apps.html#calling-them-from-a-django-view

The concepts aren't as neatly delimited as you make it sound, and both can be implemented as external apps.

I also want to add that all the other mainstream WSGI framework have supported this outside-the-box for years.

Django isn't a "WSGI framework" per se — it uses WSGI as a deployment platform, that's all. Besides, this isn't a competition.

I'll dare reopening this ticket because I think there's been a misunderstanding, and would like to prevent this ticket from going unnoticed for years again.

Sure, we can leave it open for now, I'll let someone else make the decision.

comment:13 in reply to: ↑ 12 Changed 13 months ago by Gustavo

Replying to aaugustin:

Replying to Gustavo:

1) "Dispatch at the WSGI level" is ambiguous. I assume you mean dispatch before Django is reached, which wouldn't be better or even accomplish the same. Picture this example: You want to use the WSGI X-Sendfile application from a Django view because you want to (1) check that the user can download the file, (2) log who's downloading the file and (3) log when the file was downloaded. Try to do that outside of Django and you'll end up with an unmaintainable piece of code.

For such cases, I suggest to use an external Django view to WSGI mapper — twod.wsgi or one of the many other packages providing the same feature.

Then why not provide it in Django and prevent the proliferation of implementations to accomplish this? That's indicative that there's a demand for this.

2) I'm not sure I understand this point well enough. What does the gateway have to do with all this? Anyway, I can assure you that this is a piece of cake and it's 100% reliable in 100% of the cases you can imagine. I've reimplemented a variant of that patch in twod.wsgi and presented demos of how you can proxy applications like Trac and the PHP-powered Wordpress via Django. What complicated situations can you imagine that wouldn't be supported?

Django is a WSGI application. If it acts as a WSGI server too, that makes it a WSGI gateway. As such, it'll be expected to honor the spec for WSGI gateways. People can be unbelievably annoying with spec-compliance bugs.

I see. Agreed.

Let's take this sentence from PEP 3333:

WSGI servers, gateways, and middleware must not delay the transmission of any block; they must either fully transmit the block to the client, or guarantee that they will continue transmission even while the application is producing its next block.

twod.wsgi doesn't respect this when the write() callable is used. (To you credit, you handled the iterator case correctly.)

Actually, the write() callable is supported just fine. twod.wsgi doesn't do anything with it because it delegates it to WebOb's Request.call_application(), which supports write() the way it's supposed to: https://github.com/Pylons/webob/blob/master/webob/request.py#L1238

Describing something as a piece of cake and 100% correct isn't good for your credibility when I can find a bug through code inspection!

The approach is a piece of cake, even if there are bugs in my implementation, which may well be the case because I'm a human! "100% correct" are your own words, isn't that right? I said "100% reliable in 100% of the cases you can imagine", and I stand by it.

And what you've found isn't a bug, unless I've misinterpreted your reply.

3) I think you're confusing "embedding WSGI applications inside Django" with "routing WSGI applications". Both can be quite useful, depending on your needs, but there's a big difference. This ticket is about embedding, but you're talking about routing.

Well, the very first example in twod.wgsi's docs is called embedding but it actually shows routing:
http://pythonhosted.org/twod.wsgi/manual/embedded-apps.html#embedding-non-django-applications
The second example is about embedding:
http://pythonhosted.org/twod.wsgi/manual/embedded-apps.html#calling-them-from-a-django-view

No, it's embedding. make_wsgi_app() returns a Django view, which makes a huge difference.

Consider this example: If the embedded WSGI app relies on REMOTE_USER for authentication and Django previously authenticated the current user, then the embedded app will reuse Django's authentication. By contrast, if you route requests to the same WSGI app outside of Django, as you suggest, the embedded WSGI app won't be able to reuse Django's authentication.

The concepts aren't as neatly delimited as you make it sound,

Well, they are and they've been for ages with other Python frameworks.

and both can be implemented as external apps.

No.

I also want to add that all the other mainstream WSGI framework have supported this outside-the-box for years.

Django isn't a "WSGI framework" per se — it uses WSGI as a deployment platform, that's all. Besides, this isn't a competition.

I didn't mean that as a call for competition or anything. I just wanted to point out that everywhere else this functionality is taken for granted and yet here we are discussing whether it should be part of Django itself, while other Django users continue creating their own implementations for the missing functionality.

It may not be its primary focus, but Django is a WSGI framework, just like TurboGears, Pylons, etc. WSGI is more just a deployment platform.

I'll dare reopening this ticket because I think there's been a misunderstanding, and would like to prevent this ticket from going unnoticed for years again.

Sure, we can leave it open for now, I'll let someone else make the decision.

Thanks! I wonder whether this should be taken to the mailing list.

Add Comment

Modify Ticket

Change Properties
<Author field>
Action
as new
The owner will be changed from Gustavo to anonymous. Next status will be 'assigned'
as The resolution will be set. Next status will be 'closed'
Author


E-mail address and user name can be saved in the Preferences.

 
Note: See TracTickets for help on using tickets.