Code

Opened 6 years ago

Closed 3 months ago

#8548 closed Bug (fixed)

Lengthy verbose_name results in fatal mysql 'warning' error during syncdb.

Reported by: samt@… Owned by: marcelor
Component: Core (Management commands) Version: 1.4
Severity: Normal Keywords:
Cc: thepapermen, markus@… Triage Stage: Accepted
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: yes
Easy pickings: no UI/UX: no

Description (last modified by mtredinnick)

class Thingie(models.Model)
#fields go here
  class Meta:
    verbose_name = 'A String Greater Than 50 Characters Long, Probably For The Admin'

Running 'python manage.py syncdb' results in MySQLdb trying to run the following:

INSERT INTO `auth_permission` (`name`, `content_type_id`, `codename`) VALUES (%s, %s, %s)(u'A String Greater Than 50 Characters Long, Probably For The Admin', 11, u'add_thingie')

The field name has a length restriction of 50 characters. This results in a warning 'error' being generated (_mysql_exceptions.Warning: Data truncated for column 'name' at row 1) but no further useful information. This also halts syncdb, which is also called during the test suite.

Suggestions (since I'm nowhere near qualified to contribute, yet):

  • Maybe just catch the warning and ignore it? This seems to be a display-only field, so it probably won't hurt.
  • Or maybe catch the warning and explain the cause?
  • Model validator could complain about verbose_name being too long?
  • Increase the size of the field?
  • Maybe just let google index this ticket so that the next person to google the error sees why it happened?
Traceback (most recent call last):
  File "manage.py", line 11, in <module>
    execute_manager(settings)
  File "/usr/local/python-x/lib/python2.5/site-packages/django/core/management/__init__.py", line 334, in execute_manager
    utility.execute()
  File "/usr/local/python-x/lib/python2.5/site-packages/django/core/management/__init__.py", line 295, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/usr/local/python-x/lib/python2.5/site-packages/django/core/management/base.py", line 77, in run_from_argv
    self.execute(*args, **options.__dict__)
  File "/usr/local/python-x/lib/python2.5/site-packages/django/core/management/base.py", line 96, in execute
    output = self.handle(*args, **options)
  File "/usr/local/python-x/lib/python2.5/site-packages/django/core/management/base.py", line 178, in handle
    return self.handle_noargs(**options)
  File "/usr/local/python-x/lib/python2.5/site-packages/django/core/management/commands/syncdb.py", line 101, in handle_noargs
    emit_post_sync_signal(created_models, verbosity, interactive)
  File "/usr/local/python-x/lib/python2.5/site-packages/django/core/management/sql.py", line 205, in emit_post_sync_signal
    interactive=interactive)
  File "/usr/local/python-x/lib/python2.5/site-packages/django/dispatch/dispatcher.py", line 148, in send
    response = receiver(signal=self, sender=sender, **named)
  File "/usr/local/python-x/lib/python2.5/site-packages/django/contrib/auth/management/__init__.py", line 28, in create_permissions
    defaults={'name': name, 'content_type': ctype})
  File "/usr/local/python-x/lib/python2.5/site-packages/django/db/models/manager.py", line 84, in get_or_create
    return self.get_query_set().get_or_create(**kwargs)
  File "/usr/local/python-x/lib/python2.5/site-packages/django/db/models/query.py", line 331, in get_or_create
    obj.save()
  File "/usr/local/python-x/lib/python2.5/site-packages/django/db/models/base.py", line 282, in save
    self.save_base(force_insert=force_insert, force_update=force_update)
  File "/usr/local/python-x/lib/python2.5/site-packages/django/db/models/base.py", line 356, in save_base
    result = manager._insert(values, return_id=update_pk)
  File "/usr/local/python-x/lib/python2.5/site-packages/django/db/models/manager.py", line 126, in _insert
    return insert_query(self.model, values, **kwargs)
  File "/usr/local/python-x/lib/python2.5/site-packages/django/db/models/query.py", line 884, in insert_query
    return query.execute_sql(return_id)
  File "/usr/local/python-x/lib/python2.5/site-packages/django/db/models/sql/subqueries.py", line 308, in execute_sql
    cursor = super(InsertQuery, self).execute_sql(None)
  File "/usr/local/python-x/lib/python2.5/site-packages/django/db/models/sql/query.py", line 1641, in execute_sql
    cursor.execute(sql, params)
  File "/usr/local/python-x/lib/python2.5/site-packages/django/db/backends/util.py", line 20, in execute
    return self.cursor.execute(sql, params)
  File "/usr/local/python-x/lib/python2.5/site-packages/MySQL_python-1.2.2-py2.5-macosx-10.5-i386.egg/MySQLdb/cursors.py", line 168, in execute
  File "/usr/local/python-x/lib/python2.5/site-packages/MySQL_python-1.2.2-py2.5-macosx-10.5-i386.egg/MySQLdb/cursors.py", line 82, in _warning_check
  File "/usr/local/python-x/lib/python2.5/warnings.py", line 62, in warn
    globals)
  File "/usr/local/python-x/lib/python2.5/warnings.py", line 102, in warn_explicit
    raise message
_mysql_exceptions.Warning: Data truncated for column 'name' at row 1

Attachments (4)

validate_verbose_8564.diff (629 bytes) - added by adamv 6 years ago.
Initial stab at a patch.
validate_verbose_8564.2.diff (676 bytes) - added by marcelor 5 years ago.
Modified patch with more informative error message.
ticket8548.tgz (2.4 KB) - added by marcelor 5 years ago.
Simple project to test the patch.
validate_verbose_name-8548-test.diff (1.5 KB) - added by marcelor 5 years ago.
Patch to add a test in modeltests/invalid_models

Download all attachments as: .zip

Change History (27)

comment:1 Changed 6 years ago by mtredinnick

  • Component changed from Database wrapper to django-admin.py
  • Description modified (diff)
  • milestone set to post-1.0
  • Needs documentation unset
  • Needs tests unset
  • Patch needs improvement unset
  • Triage Stage changed from Unreviewed to Accepted

(Fixed description. The preview button is your friend.)

This problem should be caught by the validate command for django-admin.py. That's the most logical fix.

Changed 6 years ago by adamv

Initial stab at a patch.

comment:2 Changed 6 years ago by adamv

  • Has patch set
  • Needs tests set

comment:3 Changed 6 years ago by mtredinnick

  • Patch needs improvement set

Looks like a reasonable start, but just telling the user that they made a mistake is not particularly helpful. They are going to be wondering how to fix it or what the correct maximum would be. Let's give them that information in the error message.

comment:4 Changed 5 years ago by anonymous

  • Owner changed from nobody to anonymous
  • Status changed from new to assigned

The longest prefixes for the string value in the column auth_permission.name are "Can change " and "Can delete ", both with 11 characters. The column maximum length is 50 so the maximum length of Meta.verbose_name is 39.

Modified the patch using 39 as the upper limit. Modified the error message to truncate the verbose_name value to the first 39 characters and informed the user about that limit.

I attach the modified patch and a Test Project with two models, one has a verbose_name length 39 (allowed max) and the other with length 40.

comment:5 Changed 5 years ago by marcelor

  • Owner changed from anonymous to marcelor
  • Status changed from assigned to new

Sorry, forgot to login before submitting.

Changed 5 years ago by marcelor

Modified patch with more informative error message.

Changed 5 years ago by marcelor

Simple project to test the patch.

comment:6 Changed 5 years ago by marcelor

  • Status changed from new to assigned

comment:7 Changed 5 years ago by marcelor

Where in the test suite can i add tests for this patch?

comment:8 Changed 5 years ago by russellm

The invalid_models regression test is the right place to check that the right validation errors are thrown.

comment:9 Changed 5 years ago by anonymous

  • milestone post-1.0 deleted

Milestone post-1.0 deleted

Changed 5 years ago by marcelor

Patch to add a test in modeltests/invalid_models

comment:10 Changed 5 years ago by marcelor

  • Needs tests unset

comment:11 Changed 5 years ago by marcelor

Thanks russelm.

comment:12 Changed 5 years ago by guettli

Related #4748 "Increase length of names of permissions"

comment:13 Changed 4 years ago by anonymous

any changes?

comment:14 Changed 3 years ago by lukeplant

  • Severity set to Normal
  • Type set to Bug

comment:15 Changed 3 years ago by thepapermen

  • Cc thepapermen added
  • Easy pickings unset

comment:16 Changed 3 years ago by anonymous

  • UI/UX unset

The resolution to ticket #4748 is nonsensical. How would *increasing* the length of the field break backwards compatibility? Any working application in production up to this point logically must have all auth_permissions character lengths of below 50 right now, making it a decidedly backwards compatible change. If those apps that are in production add an auth permission that has a length of greater than 50 in the future, then yes, they would have to manually change it in the DB, but that is preferable to making EVERY user with this issue manually change their DB.

This seems to me to be a case where a developer is hiding behind a technicality to push off a change that fixes broken functionality.

comment:17 Changed 3 years ago by jbonnett

  • Resolution set to worksforme
  • Status changed from assigned to closed

I have tried to replicate this behavior on trunk and so far I am having no luck. The test model fails to raise a error on sqlite and mysql. If there is no longer a error, providing a error nicer error message is moot. If I am wrong and this is still a problem, by all means let me know, i would be happy to take a crack at updating the patch.

comment:18 Changed 3 years ago by kmtracey

  • Resolution worksforme deleted
  • Status changed from closed to reopened

This is still re-creatable by me with a brand new MySQL database. I don't imagine it would ever be re-creatable on sqlite: sqlite simply doesn't enforce length restrictions. On MySQL whether it is a warning or an error condition is determined by the strictness mode set. If MySQL is configured to consider it just a warning situation, it's possible that the data will just get truncated with a warning being printed at the console; Django only turns MySQL warnings into exceptions when the DEBUG setting is True.

comment:19 Changed 2 years ago by mdeboard

  • Version changed from 1.0-beta-1 to 1.3

I can verify this problem is persistent in Django 1.3, MySQL 5.1.58. Specifically I encounter it:

  1. Trying to load the fixture from django-countria
  2. Saving tags using Django-taggit
  3. Any place where the specified field length is shorter than the attempted input.

The error handling for this particular issue is very frustrating. The only solution (afaik) is to either reconfigure MySQL to consider it a warning or to subclass the models in question and increase the max_length of the field. This is fine, however a more explanatory error message to alert users to this fact is called for, in my opinion.

Thanks,

comment:20 Changed 15 months ago by cdoussin@…

  • Version changed from 1.3 to 1.4

This is also an issue with long model names of 6-7 words and up (and verbose names automatically generated from the model name). For example I just hit the issue with "PremiumAppSubscriptionIncludedIssueCreditProduct".

When you look at the MySQL table, the length of the fields doesn't make sense for long model names: name is half as short as codename, when the codename values are bound to be a bit shorter than the verbose name values.

mysql> describe auth_permission;
+-----------------+--------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+-----------------+--------------+------+-----+---------+----------------+
| id | int(11) | NO | PRI | NULL | auto_increment |
| name | varchar(50) | NO | | NULL | |
| content_type_id | int(11) | NO | MUL | NULL | |
| codename | varchar(100) | NO | | NULL | |
+-----------------+--------------+------+-----+---------+----------------+
4 rows in set (0.00 sec)

As a result, I would be in favour of increasing the length of the "name" field.

comment:21 Changed 13 months ago by aaugustin

  • Status changed from reopened to new

comment:22 Changed 13 months ago by keimlink

  • Cc markus@… added

comment:23 Changed 3 months ago by claudep

  • Resolution set to fixed
  • Status changed from new to closed

I think that the fix for #18866 also resolved this issue. #8162 is used to track permission name length.

Add Comment

Modify Ticket

Change Properties
<Author field>
Action
as closed
as The resolution will be set. Next status will be 'closed'
The resolution will be deleted. Next status will be 'new'
Author


E-mail address and user name can be saved in the Preferences.

 
Note: See TracTickets for help on using tickets.