Custom SQL with Windows or Mac end-of-lines fail with Python 3
|Reported by:||adsworth||Owned by:||adsworth|
|Has patch:||yes||Needs documentation:||yes|
|Needs tests:||no||Patch needs improvement:||no|
In core.managment.sql.custom_sql_for_model. The execution of custom sql fails when the sql file has windows or mac line endings. The sql file is opened with universal newline so that windows or mac end-of-lines are automatically translated to unix end-of-lines. In python 2.7 if the universal newline mode passed to open() the file is opened in text mode, even if binary mode is set, as is the case here. In Python 3 universal newline only works if the file is opened in text mode. Specifying b and U in the mode apparently opens it in binary mode and thus ignores the universal newline mode.
The python re module only uses \n as end-of-line. So the \r in the window or mac sql file causes the regex to fail. Subsequently multiple SQL statements are passed to the backend. This fails in e.g. SQLite.
The attached patch changes the regex which splits the files.
Another option would be to use different open() calls depending on python the version. If the file is opened in text mode in Python 3 the decode method is missing on the returned string. In python 3 the encoding parameter should probably be used in that case.
Change History (9)
Changed 5 years ago by adsworth
comment:1 Changed 5 years ago by adsworth
- Needs documentation unset
- Needs tests unset
- Owner changed from nobody to adsworth
- Patch needs improvement unset
comment:2 Changed 5 years ago by aaugustin
- Needs documentation set
- Triage Stage changed from Unreviewed to Accepted