python-mysql-1.2.0 has a bug with threading, raising 'ReferenceError: weakly-referenced object no longer exists'

[wbyoung@…]

I'm getting the following error every so often from a revision checked out on Jan 02, 2007. Sorry... I updated before checking the revision number. Jan 02 puts it somewhere between r4270 and r4274.

2007-01-10 16:36:40: (mod_fastcgi.c.2502) FastCGI-stderr: Traceback (most recent call last):
  File "build/bdist.linux-i686/egg/flup/server/fcgi_base.py", line 558, in run
  File "build/bdist.linux-i686/egg/flup/server/fcgi_base.py", line 1112, in handler
  File "/usr/lib/python2.4/site-packages/django/core/handlers/wsgi.py", line 193, in __call__
    response = middleware_method(request, response)
  File "/usr/lib/python2.4/site-packages/django/contrib/sessions/middleware.py", line 81, in process_response
    session_key = request.session.session_key or Session.objects.get_new_session_key()
  File "/usr/lib/python2.4/site-packages/django/contrib/sessions/models.py", line 21, in get_new_session_key
    self.get(session_key=session_key)
  File "/usr/lib/python2.4/site-packages/django/db/models/manager.py", line 67, in get
    return self.get_query_set().get(*args, **kwargs)
  File "/usr/lib/python2.4/site-packages/django/db/models/query.py", line 211, in get
    obj_list = list(clone)
  File "/usr/lib/python2.4/site-packages/django/db/models/query.py", line 103, in __iter__
    return iter(self._get_data())
  File "/usr/lib/python2.4/site-packages/django/db/models/query.py", line 430, in _get_data
    self._result_cache = list(self.iterator())
  File "/usr/lib/python2.4/site-packages/django/db/models/query.py", line 172, in iterator
    cursor.execute("SELECT " + (self._distinct and "DISTINCT " or "") + ",".join(select) + sql, params)
  File "/usr/lib/python2.4/site-packages/MySQLdb/cursors.py", line 137, in execute
    self.errorhandler(self, exc, value)
  File "/usr/lib/python2.4/site-packages/MySQLdb/connections.py", line 33, in defaulterrorhandler
    raise errorclass, errorvalue
ReferenceError: weakly-referenced object no longer exists

Let me know if there's any more info you'd like.

[mir@…]

Hi,

the problem with this bug is, if we cannot reproduce the bug, we cannot fix it. And I use mysql and fastcgi myself all the time, and it just works!

Can you please give us some details how to reproduce the bug? I have some feeling that it's also related to fastcgi, but I cannot reproduce it.

• Is it reproduceable on your side?
• Are you using fastcgi with different processes, or with threading?
• What's your version of the mysql python module, and what mysql version do you use?
• Version of flup?
• Anything else that could help?

I'm closing the bug in the meantime. Please reopen it when you reply!

• Michael

[wbyoung@…]

Info:

• flup revision 2305
• fcgi-2.4.0 with threading
• mysql Ver 14.12 Distrib 5.0.27, for redhat-linux-gnu (i686) using readline 5.0
• mysql-python 1.2.1_p2

Here are some time stamps for frequency:

2007-01-17 10:27:03:
2007-01-17 11:55:23:
2007-01-17 13:26:54:
2007-01-17 14:13:51:
2007-01-17 16:47:06:
2007-01-17 18:13:44:
2007-01-18 03:19:09:
2007-01-18 05:01:25:
2007-01-18 06:43:56:
2007-01-18 06:48:57:
2007-01-18 08:15:19:
2007-01-18 08:28:43:
2007-01-18 09:59:43:
2007-01-18 13:40:09:
2007-01-18 13:44:30:
2007-01-18 16:03:45:

So yes it's reproducable... it happens quite a bit, but the only times that I've seen an error it only takes reloading the page to stop the error. I don't know steps to reproduce it. And it happens on multiple pages.... not just one.

There are actually two other weak references that seem to be popping up.... they seem to be related so I included occurrences of those in the timestamps.

  File "build/bdist.linux-i686/egg/flup/server/fcgi_base.py", line 558, in run
  File "build/bdist.linux-i686/egg/flup/server/fcgi_base.py", line 1112, in handler
  File "/usr/lib/python2.4/site-packages/django/core/handlers/wsgi.py", line 193, in __call__
    response = middleware_method(request, response)
  File "/usr/lib/python2.4/site-packages/django/contrib/sessions/middleware.py", line 89, in process_response
    datetime.datetime.now() + datetime.timedelta(seconds=settings.SESSION_COOKIE_AGE))
  File "/usr/lib/python2.4/site-packages/django/contrib/sessions/models.py", line 29, in save
    s.save()
  File "/usr/lib/python2.4/site-packages/django/db/models/base.py", line 175, in save
    (backend.quote_name(self._meta.db_table), backend.quote_name(self._meta.pk.column)), [pk_val])
  File "/usr/lib/python2.4/site-packages/MySQLdb/cursors.py", line 137, in execute
    self.errorhandler(self, exc, value)
  File "/usr/lib/python2.4/site-packages/MySQLdb/connections.py", line 33, in defaulterrorhandler
    raise errorclass, errorvalue
ReferenceError: weakly-referenced object no longer exists


  File "build/bdist.linux-i686/egg/flup/server/fcgi_base.py", line 558, in run
  File "build/bdist.linux-i686/egg/flup/server/fcgi_base.py", line 1112, in handler
  File "/usr/lib/python2.4/site-packages/django/core/handlers/wsgi.py", line 193, in __call__
    response = middleware_method(request, response)
  File "/usr/lib/python2.4/site-packages/django/contrib/sessions/middleware.py", line 89, in process_response
    datetime.datetime.now() + datetime.timedelta(seconds=settings.SESSION_COOKIE_AGE))
  File "/usr/lib/python2.4/site-packages/django/contrib/sessions/models.py", line 29, in save
    s.save()
  File "/usr/lib/python2.4/site-packages/django/db/models/base.py", line 204, in save
    ','.join(placeholders)), db_values)
  File "/usr/lib/python2.4/site-packages/MySQLdb/cursors.py", line 137, in execute
    self.errorhandler(self, exc, value)
  File "/usr/lib/python2.4/site-packages/MySQLdb/connections.py", line 33, in defaulterrorhandler
    raise errorclass, errorvalue
ReferenceError: weakly-referenced object no longer exists

Thanks for looking into it.

[anonymous]

Hi, thanks for the fast reply!

Could you try to switch to non-threading fastcgi with multiple processes? This just *looks* to be a threading bug. It might be that threading is not supported with this mysql lib. With the logs you see, I'd strongly advise not to use this setup in production.

Please don't mind the closed status of this ticket, this is only due to the reopen problem. I'm currently keeping a watch list of tickets ...

[mir@…]

Args, that previous Mr. Anonymous was me.

[wbyoung@…]

I'm running it preforked now. I'll let you know if I get the same errors or not.

I'm on a production server with very little resources, though, and would prefer to run the fcgi as threaded instead of forked. Do you think that there might not be this same issue with PostgreSQL?

[mir@…]

If django with threading-flup uses one global connection object and multiple cursors created from the global connection object, this is not going to work, see:

​http://forums.mysql.com/read.php?50,104391,104688#msg-104688

Regarding threading and PostgreSQL, there's a lot to say why you shouldn't do threading with python on Linux, but please ask on the user list. I'd like to keep the ticket clean and separate from user support, as a long ticket with sidelines will be difficult to read.

Before I put this ticket forward, I'm going to find out about whether there's a global connection object or whether I can verify this, but this might take until Monday or Tuesday.

[wbyoung@…]

FYI: No errors with the preforked method.

[anonymous]

#2919 marked as duplicate. #2919 is about lighthttp/fastcgi/threaded/mysql and has a similar backtrace. I'm putting this now to 'Accepted'. I have not yet verified it, but with multiple sources under different http servers, this seems to be serious with mysql and threading.

[mir@…]

Aha! Found this in the mailing list archives: ​http://wolfram.kriesing.de/blog/index.php/2006/multithreading-with-mysqldb-and-weakrefs

python-mysql-1.2.0 has a problem with multithreading. Please upgrade. 1.2.1p2 has been reported to solve this. Thanks, Wolfram Kriesing!

Are you two able to test this?

@core: Should we add a warning to the docs?

[wbyoung@…]

Sorry, I never specified a web server, but I'm on lighttpd, too.

And my error was with mysql-python 1.2.1p2, so maybe it's still a problem with 1.2.1p2.

[mir@…]

Replying to wbyoung@mcdonogh.org:

Sorry, I never specified a web server, but I'm on lighttpd, too.

And my error was with mysql-python 1.2.1p2, so maybe it's still a problem with 1.2.1p2.

Args, what a strange bug.

Are you absolutely 120 % sure that you are actually using the 1.2.1p2 module? The blog entry from Wolfram points to exactly the line that, according to your logs, is responsible for the error message. I find this hard to believe. Perhaps we can check this:

• Where is python-mysql installed?
• What are the contents of lines 35-45 in the cursors.py file within MySQLdb?
• Please give us the output of (please follow exactly!)

python manage.py shell
import MySQLdb
MySQLdb.version_info
MySQLdb.__path__

• Another detail that might be interesting: What's your operating system?

[wbyoung@…]

nevermind. you're right. i did have a previous version of python-mysql installed. i updated and set things up as threaded again. i'll let you know if i have any more problems, but you can probably close this now.

[Marc Fargas <telenieko@…>]

For the record:

• python-mysql-1.2.0 has a problem with multithreading.
• Please upgrade. 1.2.1p2 has been reported to solve this.
• source: ​http://wolfram.kriesing.de/blog/index.php/2006/multithreading-with-mysqldb-and-weakrefs

Already closed, removing "reopen" keyword.

[anonymous]

Marc, Please read comment #9.

I'd still like to see it documented that you shouldn't use python-mysql-1.2.0 with threading, so I hope for Jacob's magic to reopen this ticket.

[mir@…]

the last one was me. I promise to start using trac's session thingie from now on ;-)

[Marc Fargas <telenieko@…>]

Adds a small note on install.txt on the DB bindings section.

[Marc Fargas <telenieko@…>]

Sorry mir, forgot about comment 9 while reading the ticket :) [I've got the memory of a mosquito!]
Added a patch with a comment for that on install.txt hope it's enough if documentation about that is to be added ;)

[Jacob]

(In [4621]) Fixed #3279: Added a note about using MySQLdb 1.2.1 or later. Thanks, Marc Fargas.