RunSQL Bug Django 1.7

[Aryeh Hillman]

Django migrations are wonderful, but I believe I have found a bug. The essence of what I am trying to do here: rename all tables starting with the name "hello" (i.e. hello_1, hello2_, hello_3, ...) to start with the name "goodbye (goodbye_1, goodbye_2, goodbye_3, ...), which could be very helpful when renaming an application.

Here's how to reproduce (there is probably a more minimal case):

1. Install this function at a shell for PostgreSQL (very handy for

CREATE FUNCTION exec(text) returns text language plpgsql volatile
  AS $f$
    BEGIN
      EXECUTE $1;
      RETURN $1;
    END;
$f$;

source: ​http://wiki.postgresql.org/wiki/Dynamic_DDL

2. Write a RunSQL operation to help rename some tables

class Migration(migrations.Migration):
...
    operations = [
        migrations.RunSQL(
            sql="""select exec(format('alter table %I rename to %I', tablename, regexp_replace(tablename, '^hello, 'goodbye'))) from pg_tables where tablename like 'hello%';"""
        ),
    ]
...

3. Run the migration. I got this error:

Traceback (most recent call last):
  File "./manage.py", line 10, in <module>
    execute_from_command_line(sys.argv)
  File "/Users/abhillman/stable_env/lib/python2.7/site-packages/django/core/management/__init__.py", line 385, in execute_from_command_line
    utility.execute()
  File "/Users/abhillman/stable_env/lib/python2.7/site-packages/django/core/management/__init__.py", line 377, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/Users/abhillman/stable_env/lib/python2.7/site-packages/django/core/management/base.py", line 288, in run_from_argv
    self.execute(*args, **options.__dict__)
  File "/Users/abhillman/stable_env/lib/python2.7/site-packages/django/core/management/base.py", line 338, in execute
    output = self.handle(*args, **options)
  File "/Users/abhillman/stable_env/lib/python2.7/site-packages/django/core/management/commands/migrate.py", line 160, in handle
    executor.migrate(targets, plan, fake=options.get("fake", False))
  File "/Users/abhillman/stable_env/lib/python2.7/site-packages/django/db/migrations/executor.py", line 63, in migrate
    self.apply_migration(migration, fake=fake)
  File "/Users/abhillman/stable_env/lib/python2.7/site-packages/django/db/migrations/executor.py", line 97, in apply_migration
    migration.apply(project_state, schema_editor)
  File "/Users/abhillman/stable_env/lib/python2.7/site-packages/django/db/migrations/migration.py", line 107, in apply
    operation.database_forwards(self.app_label, schema_editor, project_state, new_state)
  File "/Users/abhillman/stable_env/lib/python2.7/site-packages/django/db/migrations/operations/special.py", line 69, in database_forwards
    schema_editor.execute(statement)
  File "/Users/abhillman/stable_env/lib/python2.7/site-packages/django/db/backends/schema.py", line 98, in execute
    cursor.execute(sql, params)
  File "/Users/abhillman/stable_env/lib/python2.7/site-packages/django/db/backends/utils.py", line 81, in execute
    return super(CursorDebugWrapper, self).execute(sql, params)
  File "/Users/abhillman/stable_env/lib/python2.7/site-packages/django/db/backends/utils.py", line 65, in execute
    return self.cursor.execute(sql, params)
IndexError: list index out of range

Not sure what's going on here. Could it be the custom PGSQL function?

[Aryeh Hillman]

Okay. I looked into this a bit more. I seemed to be able to resolve my issue by escaping the '%' character (i.e. convert all instances of '%%' to '%'). Inspecting the stack trace, I am not entirely clear on why this is happening. Perhaps this is the correct behavior on django's behalf. Any enlightenment on this issue would be much appreciated. In addition, I wonder if some warning message or other error message might be appropriate or helpful in this sort of situation.

[Claude Paroz]

I think I encountered something similar while working on an unrelated issue.
Could you try applying the following diff and tell us if this changes anything for you?

diff --git a/django/db/backends/schema.py b/django/db/backends/schema.py
index b7e285d..e2d5b54 100644
--- a/django/db/backends/schema.py
+++ b/django/db/backends/schema.py
@@ -95,7 +95,7 @@ class BaseDatabaseSchemaEditor(object):
         # Log the command we're running, then run it
         logger.debug("%s; (params %r)" % (sql, params))
         if self.collect_sql:
-            self.collected_sql.append((sql % tuple(map(self.quote_value, params))) + ";")
+            self.collected_sql.append((sql % tuple(map(self.quote_value, params or []))) + ";")
         else:
             with self.connection.cursor() as cursor:
                 cursor.execute(sql, params)
@@ -268,7 +268,7 @@ class BaseDatabaseSchemaEditor(object):
             "table": self.quote_name(model._meta.db_table),
             "definition": ", ".join(column_sqls)
         }
-        self.execute(sql, params)
+        self.execute(sql, params or None)
         # Add any index_togethers
         for field_names in model._meta.index_together:
             fields = [model._meta.get_field_by_name(field)[0] for field in field_names]

[Tim Graham]

I think #23426 is related.

[Claude Paroz]

I'd even say it's a duplicate.