Code

Opened 3 years ago

Closed 18 months ago

Last modified 9 months ago

#16484 closed Bug (duplicate)

Duplicate entry sessions error

Reported by: hash.3g@… Owned by: nobody
Component: contrib.sessions Version: 1.3
Severity: Normal Keywords:
Cc: Triage Stage: Accepted
Has patch: no Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description

Hi, Sometimes we receive "duplicate entry" error from django sessions. See Traceback.

Traceback (most recent call last):

File "/usr/local/lib/python2.6/dist-packages/django/core/handlers/base.py", line 178, in get_response
  response = middleware_method(request, response)
File "/usr/local/lib/python2.6/dist-packages/django/contrib/sessions/middleware.py", line 36, in process_response
  request.session.save()
File "/usr/local/lib/python2.6/dist-packages/django/contrib/sessions/backends/db.py", line 63, in save
  obj.save(force_insert=must_create, using=using)
File "/usr/local/lib/python2.6/dist-packages/django/db/models/base.py", line 460, in save
  self.save_base(using=using, force_insert=force_insert, force_update=force_update)
File "/usr/local/lib/python2.6/dist-packages/django/db/models/base.py", line 553, in save_base
  result = manager._insert(values, return_id=update_pk, using=using)
File "/usr/local/lib/python2.6/dist-packages/django/db/models/manager.py", line 195, in _insert
  return insert_query(self.model, values, **kwargs)
File "/usr/local/lib/python2.6/dist-packages/django/db/models/query.py", line 1436, in insert_query
  return query.get_compiler(using=using).execute_sql(return_id)
File "/usr/local/lib/python2.6/dist-packages/django/db/models/sql/compiler.py", line 791, in execute_sql
  cursor = super(SQLInsertCompiler, self).execute_sql(None)
File "/usr/local/lib/python2.6/dist-packages/django/db/models/sql/compiler.py", line 735, in execute_sql
  cursor.execute(sql, params)
File "/usr/local/lib/python2.6/dist-packages/django/db/backends/mysql/base.py", line 86, in execute
  return self.cursor.execute(query, args)
File "/usr/lib/pymodules/python2.6/MySQLdb/cursors.py", line 166, in execute
  self.errorhandler(self, exc, value)
File "/usr/lib/pymodules/python2.6/MySQLdb/connections.py", line 35, in defaulterrorhandler
  raise errorclass, errorvalue
IntegrityError: (1062, “Duplicate entry ‘e110c3788ccb21c76865285446102338’ for key 'PRIMARY'”)

Periodically we do cleaning of out-of-date sessions, but it does not help from current error.

Attachments (1)

my.cnf (4.3 KB) - added by anonymous 3 years ago.

Download all attachments as: .zip

Change History (11)

comment:1 Changed 3 years ago by aaugustin

  • Needs documentation unset
  • Needs tests unset
  • Patch needs improvement unset
  • Resolution set to needsinfo
  • Status changed from new to closed

From code inspection, I don't see how this can happen: in the call to obj.save(force_insert=must_create, using=using), must_create is False, so the session should just be updated if it already exists. Here, you're hitting a naked raise, which means that the original author thought this should never happen.

I can only imagine one cause for a race condition: if two requests running in two separate processes open a transaction, then commit: under some isolation levels, this may fail.

Could you provide more information about your database setup so we can try to reproduce this?

Last edited 3 years ago by aaugustin (previous) (diff)

Changed 3 years ago by anonymous

comment:2 Changed 3 years ago by hash.3g@…

  • Resolution needsinfo deleted
  • Status changed from closed to reopened

Here's my settings for project, and I attached my.cnf file if needed

DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.mysql', # Add 'postgresql_psycopg2', 'postgresql', 'mysql', 'sqlite3' or 'oracle'.
        'NAME': 'dou',                      # Or path to database file if using sqlite3.
        'USER': 'hash',                      # Not used with sqlite3.
        'PASSWORD': '*******',                  # Not used with sqlite3.
        'HOST': '',                      # Set to empty string for localhost. Not used with sqlite3.
        'PORT': '',                      # Set to empty string for default. Not used with sqlite3.
        'OPTIONS': {"init_command": "SET storage_engine=INNODB;"}
    }
}

comment:3 Changed 3 years ago by aaugustin

Which version of MySQL are you using?

comment:4 Changed 3 years ago by anonymous

I am using 5.1.49

comment:5 Changed 3 years ago by aaugustin

  • Triage Stage changed from Unreviewed to Accepted

It's really hard to tell what happens here :/

I will accept the ticket because the traceback proves the error.

comment:6 Changed 2 years ago by sidmitra.del@…

I think i can duplicate the error. I have the same Mysql version(with INNODB).

File "/home/ubuntu/.virtualenvs/app/lib/python2.6/site-packages/django/core/handlers/base.py", line 178, in get_response
  response = middleware_method(request, response)
 File "/home/ubuntu/.virtualenvs/app/lib/python2.6/site-packages/django/contrib/sessions/middleware.py", line 36, in process_response
  request.session.save()
 File "/home/ubuntu/.virtualenvs/app/lib/python2.6/site-packages/django/contrib/sessions/backends/cached_db.py", line 31, in save
  super(SessionStore, self).save(must_create)
 File "/home/ubuntu/.virtualenvs/app/lib/python2.6/site-packages/django/contrib/sessions/backends/db.py", line 63, in save
  obj.save(force_insert=must_create, using=using)
 File "/home/ubuntu/.virtualenvs/app/lib/python2.6/site-packages/django/db/models/base.py", line 460, in save
  self.save_base(using=using, force_insert=force_insert, force_update=force_update)
 File "/home/ubuntu/.virtualenvs/app/lib/python2.6/site-packages/django/db/models/base.py", line 553, in save_base
  result = manager._insert(values, return_id=update_pk, using=using)
 File "/home/ubuntu/.virtualenvs/app/lib/python2.6/site-packages/django/db/models/manager.py", line 195, in _insert
  return insert_query(self.model, values, **kwargs)
 File "/home/ubuntu/.virtualenvs/app/lib/python2.6/site-packages/django/db/models/query.py", line 1436, in insert_query
  return query.get_compiler(using=using).execute_sql(return_id)
 File "/home/ubuntu/.virtualenvs/app/lib/python2.6/site-packages/johnny/cache.py", line 344, in newfun
  ret = original(cls, *args, **kwargs)
 File "/home/ubuntu/.virtualenvs/app/lib/python2.6/site-packages/django/db/models/sql/compiler.py", line 791, in execute_sql
  cursor = super(SQLInsertCompiler, self).execute_sql(None)
 File "/home/ubuntu/.virtualenvs/app/lib/python2.6/site-packages/johnny/cache.py", line 293, in newfun
  return original(cls, *args, **kwargs)
 File "/home/ubuntu/.virtualenvs/app/lib/python2.6/site-packages/django/db/models/sql/compiler.py", line 735, in execute_sql
  cursor.execute(sql, params)
 File "/home/ubuntu/.virtualenvs/app/lib/python2.6/site-packages/django/db/backends/mysql/base.py", line 86, in execute
  return self.cursor.execute(query, args)
 File "/home/ubuntu/.virtualenvs/app/lib/python2.6/site-packages/MySQLdb/cursors.py", line 174, in execute
  self.errorhandler(self, exc, value)
 File "/home/ubuntu/.virtualenvs/app/lib/python2.6/site-packages/MySQLdb/connections.py", line 36, in defaulterrorhandler
  raise errorclass, errorvalue
 IntegrityError: (1062, “Duplicate entry ‘e4d75e04ee5995bd2e03d25a697a00d1’ for key 'PRIMARY'”)
 

Here're my session settings:

SESSION_ENGINE                      = 'django.contrib.sessions.backends.cached_db'
SESSION_COOKIE_AGE                  = 2 * 60 * 60
SESSION_EXPIRE_AT_BROWSER_CLOSE     = True

Note: I use johnnny-cache alongside

comment:7 Changed 2 years ago by aaugustin

#18344 may describe the same problem.

comment:8 Changed 18 months ago by aaugustin

  • Resolution set to duplicate
  • Status changed from reopened to closed

This is suspiciously similar to #18557.

The conclusion on the mailing list was to use the READ COMMITTED isolation level (recommended), or database-level autocommit of every query (not recommended).

MySQL defaults to REPEATABLE READ, which appears to make such situations intractable.

comment:9 follow-up: Changed 9 months ago by jokerejoker

This is still a problem for Django 1.5.1

  File "/usr/local/lib/python2.7/dist-packages/Django-1.5.1-py2.7.egg/django/core/handlers/base.py", line 187, in get_response
    response = middleware_method(request, response)

  File "/usr/local/lib/python2.7/dist-packages/Django-1.5.1-py2.7.egg/django/contrib/sessions/middleware.py", line 38, in process_response
    request.session.save()

  File "/usr/local/lib/python2.7/dist-packages/Django-1.5.1-py2.7.egg/django/contrib/sessions/backends/db.py", line 57, in save
    obj.save(force_insert=must_create, using=using)

  File "/usr/local/lib/python2.7/dist-packages/Django-1.5.1-py2.7.egg/django/db/models/base.py", line 546, in save
    force_update=force_update, update_fields=update_fields)

  File "/usr/local/lib/python2.7/dist-packages/Django-1.5.1-py2.7.egg/django/db/models/base.py", line 650, in save_base
    result = manager._insert([self], fields=fields, return_id=update_pk, using=using, raw=raw)

  File "/usr/local/lib/python2.7/dist-packages/Django-1.5.1-py2.7.egg/django/db/models/manager.py", line 215, in _insert
    return insert_query(self.model, objs, fields, **kwargs)

  File "/usr/local/lib/python2.7/dist-packages/Django-1.5.1-py2.7.egg/django/db/models/query.py", line 1661, in insert_query
    return query.get_compiler(using=using).execute_sql(return_id)

  File "/usr/local/lib/python2.7/dist-packages/Django-1.5.1-py2.7.egg/django/db/models/sql/compiler.py", line 937, in execute_sql
    cursor.execute(sql, params)

  File "/usr/local/lib/python2.7/dist-packages/Django-1.5.1-py2.7.egg/django/db/backends/mysql/base.py", line 122, in execute
    six.reraise(utils.IntegrityError, utils.IntegrityError(*tuple(e.args)), sys.exc_info()[2])

  File "/usr/local/lib/python2.7/dist-packages/Django-1.5.1-py2.7.egg/django/db/backends/mysql/base.py", line 120, in execute
    return self.cursor.execute(query, args)

  File "/usr/lib/python2.7/dist-packages/MySQLdb/cursors.py", line 174, in execute
    self.errorhandler(self, exc, value)

  File "/usr/lib/python2.7/dist-packages/MySQLdb/connections.py", line 36, in defaulterrorhandler
    raise errorclass, errorvalue

IntegrityError: (1062, "Duplicate entry '2v4qsr4c75qbdkn27u3dypfig46fca2z' for key 'PRIMARY'")

comment:10 in reply to: ↑ 9 Changed 9 months ago by anonymous

Replying to jokerejoker:

This is still a problem for Django 1.5.1

There was no claim above that this would be "fixed" by Django 1.5.1, since it's suspected the underlying problem is MySQL's default "reapeatable read" transaction isolation level. There is no way (aside from including a commit within get_and_create itself, which is completely unacceptable) to "fix" this inside of Django. The fix is to change your transaction isolation level to "read committed".

Add Comment

Modify Ticket

Change Properties
<Author field>
Action
as closed
as The resolution will be set. Next status will be 'closed'
The resolution will be deleted. Next status will be 'new'
Author


E-mail address and user name can be saved in the Preferences.

 
Note: See TracTickets for help on using tickets.