Opened 8 years ago

Closed 4 years ago

Last modified 4 years ago

#10402 closed Bug (needsinfo)

mysql backend connections left idle open

Reported by: DirkDatzert Owned by: nobody
Component: Database layer (models, ORM) Version: 1.4
Severity: Normal Keywords:
Cc: pembo13@… Triage Stage: Accepted
Has patch: no Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description

Hi,

I have a related issue which behaves like ticket #9878 .

I query a model.objects during an AJAX jQuery Autocomplete call which receives an simple http-request, does an orm-related query of the db and reponses with a single http-response. Example view and urls are attached.

I run Django-1.0.X SVN 9684 with python-mysql 1.2.2 oder 1.2.3b1, Python 2.4.2 on SLES10 and apache2-mod_python-3.1.3 under apache2-2.2.3.

After one request the mysql db connection is left over in established mode on the client side and visible under SHOW PROCESSLIST via mysql cli as sleeping.

Some threads in the internet told that this maybe a bug in python-mysql in relation to mod_python but an update of python-mysql to latest beta didn't resolve this. I have a second application running under the same apache2 construct on the same host which didn't shows this idle connections. I connection is opened and closed correctly after a single HTTP-request.

Regards,
Dirk

Attachments (1)

jquery-autocomplete.txt (2.3 KB) - added by DirkDatzert 8 years ago.
jquery autocomplete views.py/urls.py

Download all attachments as: .zip

Change History (11)

Changed 8 years ago by DirkDatzert

Attachment: jquery-autocomplete.txt added

jquery autocomplete views.py/urls.py

comment:1 Changed 8 years ago by Jacob

Triage Stage: UnreviewedDesign decision needed

comment:2 Changed 6 years ago by Chris Beaven

Resolution: needsinfo
Severity: Normal
Status: newclosed
Type: Bug

This is a rather old ticket, and seems to hinge around a very specific setup.

If someone can verify this is still an issue for them then feel free to reopen.

comment:3 Changed 4 years ago by Arthur Pemberton

Easy pickings: unset
UI/UX: unset

I have the following environment:

  • Python 2.6.6
  • Django-1.4.1
  • Apache httpd-2.2.15-26
  • mod_wsgi
  • MySQL-python-1.2.3-0.3

While investigating a large number of aborted connection on a mysql server (large relatively to the amount of activity I was expecting, ~18 abortions per hour) I stumbled upon my Django app showing sleeping two sleeping connections in the SHOW PROCESSLIST; With multiple refreshes, I saw an increment of the aborted connections counter after each connection disappeared from the list.

comment:4 Changed 4 years ago by Arthur Pemberton

Cc: pembo13@… added
Resolution: needsinfo
Status: closednew

comment:5 Changed 4 years ago by Arthur Pemberton

Version: 1.01.4

comment:6 Changed 4 years ago by Aymeric Augustin

Resolution: worksforme
Status: newclosed
Triage Stage: Design decision neededAccepted

Unfortunately I have absolutely no idea where to start to diagnose this... If you're using Django 1.5 or earlier the code that closes connections at the end of each request is extremly straightforward -- it just calls connection.close()!

Connection handling code was overhauled in master to support persistent connections. If you can reproduce the bug on master (with CONN_MAX_AGE set to 0) have a look at this code:
https://github.com/django/django/blob/master/django/db/__init__.py#L69-L80
and try to figure out why it doesn't work as expected on your setup?

comment:7 Changed 4 years ago by anonymous

I upgraded to django 1.5 yesterday and definitely see an increase in the idle connections. Here is a graph from RDS http://screencast.com/t/zSYesJef . Running "sudo netstat --all --program | grep 'mysql'" shows me that almost all of these were spawned from uwsgi which I used to serve django (they aren't from command line processes.

comment:8 Changed 4 years ago by anonymous

Resolution: worksforme
Status: closednew

comment:9 Changed 4 years ago by Jacob

Resolution: needsinfo
Status: newclosed

I'm fairly sure that this is related to your particular setup and not anything specific in Django. For example, if you've got code that connects to the database in your uwsgi config before uwsgi forks, that'll leak database connections. There isn't a ton that we can do about that; those connections get created "outside" Django essentially. In our experimentation, we couldn't make this happen unless we did something along those lines. So I think this is a problem with your configuration, not with Django itself.

It's possible I'm wrong, and if you do think this is a Django bug and can provide more information to help us reproduce it please feel free to reopen. But we're going to need a fair bit of information -- ideally a very small app that easily recreates the issue. And please do try to reproduce against Django master; connection handling's completely changed, and we won't be making any further improvements to the version shipping in 1.5.

comment:10 Changed 4 years ago by Arthur Pemberton

Jacob, since I'm on 1.4, would my input on this possible issue be useless until I can reproduce in master?

Note: See TracTickets for help on using tickets.
Back to Top