Django

Code

Ticket #1751 (assigned)

Opened 2 years ago

Last modified 5 days ago

unique_together does not work when any of the listed fields contains a FK

Reported by: Olive Assigned to: gwilson (accepted)
Milestone: 1.0 Component: Admin interface
Version: Keywords: fk unique_together
Cc: nslater@gmail.com, gary.wilson@gmail.com, cathy@bentbacktulips.co.uk, larlet@gmail.com Triage Stage: Accepted
Has patch: 0 Needs documentation: 0
Needs tests: 0 Patch needs improvement: 0

Description

An IntegrityError is raised when saving the following kind of object:

class DocTitle(models.Model):
    sequence    = models.IntegerField(_('sequence'))
    title_en    = models.CharField(_('title'),maxlength=200)
    
    def __repr__(self):
        return self.title_en
    
    class Meta:
        verbose_name = _('documentation title')
        verbose_name_plural = _('documentation titles')
        ordering = ['sequence']
        
    class Admin:
        list_display = ('title_en',)
             
class DocChunk(models.Model):
    application = models.ForeignKey(Application,edit_inline=True,num_in_admin=1)
    title       = models.ForeignKey(DocTitle)
    text        = models.TextField(_('text'),core=True)
    
    def __repr__(self):
        return self.title.title_en
    
    class Meta:
        verbose_name = _('Document chunk')
        verbose_name_plural = _('Document chunks')
        ordering = ['title']
        unique_together = (("application","title"),)

Attachments

Change History

05/03/06 04:25:32 changed by Olive

I mean, when saving an "application" object, of course

05/08/06 08:21:49 changed by Olive

  • severity changed from major to blocker.

In fact the problem occures if there is at least one ForeignKey? field specified in unique_together list (no matter if it is an edit_inline or not)

05/08/06 09:54:53 changed by adrian

  • priority changed from high to normal.
  • severity changed from blocker to normal.

07/20/06 17:32:01 changed by anonymous

Yeah, what's the deal with this?

07/20/06 17:37:15 changed by anonymous

  • summary changed from unique_together does not work when one of the fields has edit_inline set to True to unique_together does not work when any of the fields contains a FK or has edit_inline set to True.

07/20/06 17:38:03 changed by anonymous

  • summary changed from unique_together does not work when any of the fields contains a FK or has edit_inline set to True to unique_together does not work when any of the listed fields contains a FK.

08/08/06 17:05:59 changed by Gary Wilson <gary.wilson@gmail.com>

  • cc set to gary.wilson@gmail.com.

I am using unique_together with ForeignKey fields in a model that looks like:

class Course(models.Model):
    """A course number and title combination."""
    
    department = models.ForeignKey(Department)
    number = models.CharField('course number', maxlength=6)
    title = models.CharField('course title', maxlength=100)
    
    class Meta:
        # Don't allow duplicate courses.
        unique_together = (("department", "number", "title"),)

I have no problems saving Department or Course objects. I also have no problems when I add edit_inline=True in the department field as long as at least one other field has core=True. It's only if I fail to add a core=True to any field that I get the following IntegrityError:

Traceback (most recent call last):
File "/usr/lib/python2.4/site-packages/django/core/handlers/base.py" in get_response
  74. response = callback(request, *callback_args, **callback_kwargs)
File "/usr/lib/python2.4/site-packages/django/contrib/admin/views/decorators.py" in _checklogin
  55. return view_func(request, *args, **kwargs)
File "/usr/lib/python2.4/site-packages/django/views/decorators/cache.py" in _wrapped_view_func
  39. response = view_func(request, *args, **kwargs)
File "/usr/lib/python2.4/site-packages/django/contrib/admin/views/main.py" in add_stage
  254. new_object = manipulator.save(new_data)
File "/usr/lib/python2.4/site-packages/django/db/models/manipulators.py" in save
  198. new_rel_obj.save()
File "/usr/lib/python2.4/site-packages/django/db/models/base.py" in save
  203. ','.join(placeholders)), db_values)
File "/usr/lib/python2.4/site-packages/django/db/backends/util.py" in execute
  12. return self.cursor.execute(sql, params)

  IntegrityError at /admin/courses/department/add/
  ERROR: duplicate key violates unique constraint "courses_course_department_id_key" INSERT INTO "courses_course" ("department_id","number","title") VALUES (6,'','')

I also have a model that uses unique_together using two fields that are both ForeignKeys and I don't experience any problem there with or without edit_inline=True.

(follow-up: ↓ 18 ) 08/28/06 01:43:02 changed by favo@exoweb.net

Notice that "This validation needs to occur after html2python to be effective." but validator always run before html2python. so I change ForeignKey? to use field.name instead of field.attname.

Index: db/models/manipulators.py
===================================================================
--- db/models/manipulators.py   (revision 4643)
+++ db/models/manipulators.py   (working copy)
@@ -283,7 +283,7 @@
         # This is really not going to work for fields that have different
         # form fields, e.g. DateTime.
         # This validation needs to occur after html2python to be effective.
-        field_val = all_data.get(f.attname, None)
+        field_val = all_data.get(f.name, None)
         if field_val is None:
             # This will be caught by another validator, assuming the field
             # doesn't have blank=True.

08/28/06 04:15:14 changed by favo@exoweb.net

New version: unique together fields could be null=True now.

Index: db/models/manipulators.py
===================================================================
--- db/models/manipulators.py   (revision 4643)
+++ db/models/manipulators.py   (working copy)
@@ -283,13 +283,13 @@
         # This is really not going to work for fields that have different
         # form fields, e.g. DateTime.
         # This validation needs to occur after html2python to be effective.
-        field_val = all_data.get(f.attname, None)
-        if field_val is None:
+        field_val = f.get_manipulator_new_data(all_data)
+        if not f.null and field_val is None:
             # This will be caught by another validator, assuming the field
             # doesn't have blank=True.
             return
         if isinstance(f.rel, ManyToOneRel):
-            kwargs['%s__pk' % f.name] = field_val
+            kwargs[f.get_validator_unique_lookup_type()] = field_val
         else:
             kwargs['%s__iexact' % f.name] = field_val
     try:

11/08/06 08:55:39 changed by anonymous

  • cc changed from gary.wilson@gmail.com to gary.wilson@gmail.com, cathy@bentbacktulips.co.uk.

11/28/06 10:38:00 changed by Noah Slater

  • cc changed from gary.wilson@gmail.com, cathy@bentbacktulips.co.uk to nslater@gmail.com, gary.wilson@gmail.com, cathy@bentbacktulips.co.uk.

I can confirm that the first patch submitted by favo works.

01/10/07 19:59:34 changed by Gary Wilson <gary.wilson@gmail.com>

Maybe the problem is backend specific? For anyone seeing this problem, which database backend are you using?

My comments from above were on postgresql.

01/10/07 21:19:37 changed by anonymous

Same here, PostgreSQL.

(follow-up: ↓ 15 ) 01/11/07 03:47:00 changed by Cathy Young

I was on MySQL when I hit this problem.

(in reply to: ↑ 14 ) 01/23/07 17:29:05 changed by anonymous

I had this same problem until I switched the order of my fields in the unique_together clause. In my example, using 'slug' first threw an integrity error, but having 'site' first validates correctly. (I'm working in 91-bugfixes with PostgreSQL)

 slug = meta.SlugField(prepopulate_from=('name',))
 site = meta.ForeignKey(Site)

 class META:
     unique_together=(('site', 'slug'),)

01/23/07 18:47:29 changed by Simon G. <dev@simon.net.nz>

  • keywords set to fk unique_together.
  • stage changed from Unreviewed to Accepted.

02/27/07 00:37:23 changed by Gary Wilson <gary.wilson@gmail.com>

#2019 marked as a duplicate of this.

(in reply to: ↑ 8 ) 07/03/07 01:38:03 changed by John Shaffer <jshaffer2112@gmail.com>

Replying to favo@exoweb.net:

I change ForeignKey? to use field.name instead of field.attname.

This change was made in [4028].

08/27/07 10:58:44 changed by Gábor Farkas <gabor@nekomancer.net>

i'm not sure what's the status of this ticket, but maybe a clean reproducible test-case will help.

this is with from-svn-django (r6022), postgresql8, "postgresql" driver.

from django.db.models import *

class A(Model):
    name = CharField(maxlength=200)

    class Admin: pass

class B(Model):
    name = CharField(maxlength=200)

    class Admin: pass

class C(Model):
    name = CharField(maxlength=200)
    a = ForeignKey(A)
    b = ForeignKey(B)

    class Meta:
        unique_together = (("a","b"),)

    class Admin: pass

in the admin:

  1. create an instance of A
  2. go to create an instance of C, and fill out the "name" field, and select an "a" object, but do not select a "b" object
  3. save the instance

you will get an exception saying:

ERROR: invalid input syntax for integer: 
"" SELECT "main_c"."id","main_c"."name","main_c"."a_id","main_c"."b_id" 
FROM "main_c" INNER JOIN "main_a" AS "main_c__a" ON "main_c"."a_id" = "main_c__a"."id" 
WHERE ("main_c"."b_id" = '' AND "main_c__a"."id" ILIKE '1')

(isn't that ILIKE '1' quite nice for a numeric-key? :))

btw. this works fine in sqlite3, probably because it simply accepts this syntax...

08/27/07 11:02:59 changed by Gábor Farkas <gabor@nekomancer.net>

the stacktrace:

Traceback (most recent call last):
File "/home/gabor/src/django/django/core/handlers/base.py" in get_response
  77. response = callback(request, *callback_args, **callback_kwargs)
File "/home/gabor/src/django/django/contrib/admin/views/decorators.py" in _checklogin
  55. return view_func(request, *args, **kwargs)
File "/home/gabor/src/django/django/views/decorators/cache.py" in _wrapped_view_func
  39. response = view_func(request, *args, **kwargs)
File "/home/gabor/src/django/django/contrib/admin/views/main.py" in add_stage
  257. errors = manipulator.get_validation_errors(new_data)
File "/home/gabor/src/django/django/oldforms/__init__.py" in get_validation_errors
  61. errors.update(field.get_validation_errors(new_data))
File "/home/gabor/src/django/django/oldforms/__init__.py" in get_validation_errors
  378. self.run_validator(new_data, validator)
File "/home/gabor/src/django/django/oldforms/__init__.py" in run_validator
  368. validator(new_data.get(self.field_name, ''), new_data)
File "/home/gabor/src/django/django/utils/functional.py" in _curried
  3. return _curried_func(*(args+moreargs), **dict(kwargs, **morekwargs))
File "/home/gabor/src/django/django/db/models/manipulators.py" in manipulator_validator_unique_together
  303. old_obj = self.manager.get(**kwargs)
File "/home/gabor/src/django/django/db/models/manager.py" in get
  69. return self.get_query_set().get(*args, **kwargs)
File "/home/gabor/src/django/django/db/models/query.py" in get
  261. obj_list = list(clone)
File "/home/gabor/src/django/django/db/models/query.py" in __iter__
  114. return iter(self._get_data())
File "/home/gabor/src/django/django/db/models/query.py" in _get_data
  482. self._result_cache = list(self.iterator())
File "/home/gabor/src/django/django/db/models/query.py" in iterator
  189. cursor.execute("SELECT " + (self._distinct and "DISTINCT " or "") + ",".join(select) + sql, params)
File "/home/gabor/src/django/django/db/backends/util.py" in execute
  19. return self.cursor.execute(sql, params)
File "/home/gabor/src/django/django/db/backends/postgresql/base.py" in execute
  47. return self.cursor.execute(smart_str(sql, self.charset), self.format_params(params))

  ProgrammingError at /admin/main/c/add/
  ERROR: invalid input syntax for integer: "" SELECT "main_c"."id","main_c"."name","main_c"."a_id","main_c"."b_id" FROM "main_c" INNER JOIN "main_a" AS "main_c__a" ON "main_c"."a_id" = "main_c__a"."id" WHERE ("main_c"."b_id" = '' AND "main_c__a"."id" ILIKE '2')

02/04/08 10:17:23 changed by david

  • cc changed from nslater@gmail.com, gary.wilson@gmail.com, cathy@bentbacktulips.co.uk to nslater@gmail.com, gary.wilson@gmail.com, cathy@bentbacktulips.co.uk, larlet@gmail.com.

02/28/08 12:36:26 changed by Simon Ditner <simon-django@uc.org>

I found that the above patches weren't effective for 0.96, and also did not handle BooleanField correctly with sqlite3, as the value is stored as True/False as opposed to 'on' -- so the generated <name>__iexact = 'on' would not match results correctly. I wrote the following patch for 0.96 that seems to resolve this issue:

Index: manipulators.py
===================================================================
--- manipulators.py
+++ manipulators.py
@@ -278,6 +278,7 @@

 def manipulator_validator_unique_together(field_name_list, opts, self, field_data, all_data):
     from django.db.models.fields.related import ManyToOneRel
+    from django.db.models.fields import BooleanField
     from django.utils.text import get_text_list
     field_list = [opts.get_field(field_name) for field_name in field_name_list]
     if isinstance(field_list[0].rel, ManyToOneRel):
@@ -294,7 +295,15 @@
             # doesn't have blank=True.
             return
         if isinstance(f.rel, ManyToOneRel):
-            kwargs['%s__pk' % f.name] = field_val
+            if field_val is '':
+                kwargs['%s__isnull' % f.name] = True
+            else:
+                kwargs['%s__pk' % f.name] = field_val
+        elif isinstance(f, BooleanField):
+            if field_val in ('on', 'True', 'true', True):
+               kwargs['%s' % f.name] = True
+            else:
+               kwargs['%s' % f.name] = False
         else:
             kwargs['%s__iexact' % f.name] = field_val
     try:

06/25/08 13:40:29 changed by gwilson

  • milestone set to 1.0.

08/15/08 10:37:59 changed by cmarshal

  • owner changed from nobody to cmarshal.
  • status changed from new to assigned.

08/15/08 11:58:55 changed by cmarshal

I've tried the above scenarios and the only way I am currently able to produce an IntegrityError with unique_together FKs is by adding an actual duplicate -- but this is the issue in #8209.

I'm going to look into #8209 and then see if this is resolved..

08/15/08 16:00:28 changed by cmarshal

  • owner changed from cmarshal to nobody.
  • status changed from assigned to new.

I cannot reproduce this one so I'll release it. Perhaps someone else can resolve?

08/16/08 16:40:49 changed by gwilson

  • owner changed from nobody to gwilson.
  • status changed from new to assigned.

Add/Change #1751 (unique_together does not work when any of the listed fields contains a FK)




Change Properties
Action