Opened 2 years ago

Closed 23 months ago

Last modified 17 months ago

#20537 closed Bug (fixed)

`request_finished` signal not being called with Django 1.5 and uWSGI < 1.2.6(?)

Reported by: jaap3 Owned by: aaugustin
Component: Documentation Version: 1.5
Severity: Normal Keywords: uwsgi postgres mysql idle connections
Cc: alanjds Triage Stage: Ready for checkin
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: yes UI/UX: no

Description

As discovered on the django-users mailinglist in the thread request_finished signal not being called with Django 1.5.1 with uwsgi

The version of uWSGI in the Ubuntu/Debian apt repositories (1.0.3) doesn't call close on the wsgi application, this in turn causes the request_finished signal to never fire resulting in open database connections.

According to this blogpost uWSGI >= 1.2.6 should be the first version of uWSGI that supports the expected behavior. But the same blogpost warns about other wsgi middleware potentially misbehaving.

I'm wondering why Django < 1.5 didn't show this behavior and what the best course for Django regarding this issue is.

Should there be a warning in the docs about the minimum required uWSGI version? Or should the previous behavior be restored somehow?

Change History (17)

comment:1 Changed 2 years ago by aaugustin

  • Needs documentation unset
  • Needs tests unset
  • Patch needs improvement unset

This is a bug in uWSGI, I've hit it in production too. Thanks for filing this ticket.

Django < 1.5 didn't exhibit this behavior because it closed the database connection earlier. But code could reopen the connection later on, before the end of the request, in the case of streaming responses. That was a bug, I fixed it, and I still think it's the right thing to do.

Since Django 1.5 provides explicit support for streaming responses, the closing of the database connection must be performed after the content is fully sent, ie. in close().

I think the best we can do is document this bug of uWSGI in the deployment docs and in the 1.5 release notes.

comment:2 Changed 2 years ago by claudep

  • Component changed from Uncategorized to Documentation
  • Triage Stage changed from Unreviewed to Accepted

comment:3 Changed 2 years ago by jaap3

  • Owner changed from nobody to jaap3
  • Status changed from new to assigned

I think the following pages need at least a note about the known issue with uWSGI. I can confirm the fist version that should no longer show this behavior is 1.2.6 and up (uWSGI 1.2.6 announcement.

Release notes - request-finished

How to use Django with uWSGI

Maybe include a more general description of the problem on these pages since the issue might crop up with "badly implemented" wsgi middleware as well.

Signals - request-finished

How to deploy with WSGI

I'll try to whip up a patch.

comment:4 Changed 2 years ago by jaap3

There's a note for the request-finished signal that says:

When a view returns a streaming response, this signal is sent only after the entire response is consumed by the client (strictly speaking, by the WSGI gateway).

If I'm understanding correctly this is not entirely true. This behavior occurs regardless of a streaming response, but was implemented to facilitate streaming responses. Going to slightly alter that note as well.

comment:5 Changed 2 years ago by aaugustin

Well, technically the sentence is correct :) But it implies that the behavior is different for regular responses, which isn't true :(

I'm not a native speaker, I struggle with writing good docs in English, improvements are welcome.

comment:6 Changed 2 years ago by jaap3

  • Easy pickings set
  • Has patch set

I've struggled a bit as well, but I think I've hit the mark in this pull request.

I've folded the note in the signals page under the "Changed in Django 1.5" as both of those were trying to say the same thing.

comment:7 Changed 2 years ago by aaugustin

  • Owner changed from jaap3 to aaugustin

Thank you. I'll review your proposal.

comment:8 Changed 23 months ago by timo

  • Triage Stage changed from Accepted to Ready for checkin

Looks good to me, left one comment on the pull request. I'll let Aymeric review it as well.

comment:9 Changed 23 months ago by aaugustin

PR looks good, thank you, I'll push it (with negligible edits).

comment:10 Changed 23 months ago by Aymeric Augustin <aymeric.augustin@…>

  • Resolution set to fixed
  • Status changed from assigned to closed

In 3ce1d303daa92ecb1bc3ba6344f1ec9746a79c70:

Warned that request_finished isn't sent by some buggy setups.

Older versions of uWSGI and Sentry's middleware do not adhere to
the WSGI spec and cause the request_finished signal to never
fire. Added notes to the appropriate places in the docs.

Fixed #20537.

comment:11 Changed 23 months ago by Aymeric Augustin <aymeric.augustin@…>

In 60353458ae7105ea943dc425954403c581b05255:

[1.5.x] Warned that request_finished isn't sent by some buggy setups.

Older versions of uWSGI and Sentry's middleware do not adhere to
the WSGI spec and cause the request_finished signal to never
fire. Added notes to the appropriate places in the docs.

Fixed #20537.

Backport of 3ce1d30.

Conflicts:

docs/howto/deployment/wsgi/index.txt

comment:12 Changed 23 months ago by jaap3

Cool, thanks!

comment:13 Changed 21 months ago by anonymous

Bug exists in Django 1.5.2 and uwsgi 1.9.12

comment:14 Changed 21 months ago by aaugustin

Do you have evidence of that behavior?

comment:15 Changed 21 months ago by tomek@…

I found it after first test deployment. Symptoms are most suitable for this ticket.
Everything works fine for devserver,gunicorn and [uwsgi] processes=1

$uname -a
Linux ip-10-55-75-70 3.2.0-40-virtual #64-Ubuntu SMP Mon Mar 25 22:07:32 UTC 2013 i686 i686 i386 GNU/Linux
$ django-admin.py --version
1.5.2
$cat booking.ini
uid = djangoapps
gid = djangoapps
vhost = true
plugins = http,python
master = true
enable-threads = false
processes = 1
socket = 127.0.0.1:3031
;wsgi-file = /home/djangoapps/projects/booking/booking/wsgi.py
module=booking.wsgi:application
virtualenv = /home/djangoapps/virtualenvs/booking
chdir = /home/djangoapps/projects/booking
pythonpath = /home/djangoapps/projects/booking
env = DJANGO_SETTINGS_MODULE=booking.settings.test
touch-reload = /home/djangoapps/projects/booking/reload
daemonize = /var/log/uwsgi/app/booking.log

#settings.py
DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.mysql', # Add 'postgresql_psycopg2', 'mysql', 'sqlite3' or 'oracle'.
        'NAME': 'booking',
        # The following settings are not used with sqlite3:
        'USER': 'root',
        'PASSWORD': 'test',
        'HOST': '127.0.0.1',
        'PORT': '3306',                      # Set to empty string for default.
    }
}

CACHES = {
    'default': {
        'BACKEND': 'django.core.cache.backends.memcached.MemcachedCache',
        'LOCATION': '127.0.0.1:11211',
    }
}


SESSION_ENGINE = 'django.contrib.sessions.backends.cache'
MIDDLEWARE_CLASSES = (
    'django.middleware.common.CommonMiddleware',
    'django.contrib.sessions.middleware.SessionMiddleware',
    'django.middleware.csrf.CsrfViewMiddleware',
    'django.contrib.auth.middleware.AuthenticationMiddleware',
    'django.contrib.messages.middleware.MessageMiddleware',
    # Uncomment the next line for simple clickjacking protection:
    # 'django.middleware.clickjacking.XFrameOptionsMiddleware',
)

INSTALLED_APPS = (
    'django.contrib.auth',
    'django.contrib.contenttypes',
    'django.contrib.sessions',
    'django.contrib.sites',
    'django.contrib.messages',
    'django.contrib.staticfiles',
    #'stacks',
    #'mptt',
    #'menus',
    'south',
    #'sekizai',
    'reversion',
    # Uncomment the next line
)

#views.py
def index(request)
    """Works for
    [uwsgi]processes=1 
    gunicorn,dev server
    """
    user = authenticate(username='username', password='password1')
    login(request, user)
    return HttpResponseRedirect('/test_login/')

def index2(request)
    """Works for:
    [uwsgi]processes>1
    gunicorn, dev server
    """
    user = authenticate(username='username', password='password1')
    login(request, user)
    return test_login(request)

comment:16 Changed 17 months ago by alanjds

  • Cc alanjds added

Just to clarify, this signal is sent before or after middleware processing?

comment:17 Changed 17 months ago by aaugustin

After.

Note: See TracTickets for help on using tickets.
Back to Top