Opened 5 years ago

Closed 3 years ago

#16737 closed Bug (fixed)

inspectdb does not handle non-ascii column names correctly

Reported by: moof@… Owned by: nobody
Component: Database layer (models, ORM) Version: 1.3
Severity: Normal Keywords: inspectdb
Cc: Triage Stage: Accepted
Has patch: no Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description

When a database I am inspecting has a columnname of "TamañoFuente", inspectdb fails as it is unable to write "\xf1" as a valid ascii character.

It probably requires a proper #encoding: header, and some sort of logic to make proper python identifiers out of such names.

Attachments (1)

16737-test.diff (1.7 KB) - added by Claude Paroz 4 years ago.
Test of a non-ascii column name

Download all attachments as: .zip

Change History (6)

comment:1 Changed 5 years ago by Aymeric Augustin

Keywords: inspectdb added
Needs documentation: unset
Needs tests: unset
Patch needs improvement: unset
Triage Stage: UnreviewedAccepted

Non-ASCII identifiers are only supported by Python 3, so we'll have to convert the column name to ASCII.

To fix this, we have to:

  • make sure inspectdb writes the file in utf-8
  • add # coding: utf-8 at the top of the file
  • convert the column name to an ASCII field name, for instance with: unicodedata.normalize('NFKD', column_name).encode('ascii', 'ignore')
  • set db_column on the field

comment:2 Changed 5 years ago by Ramiro Morales

Component: ORM aggregationDatabase layer (models, ORM)

comment:3 Changed 5 years ago by Claude Paroz

The latest patch in #12460 addresses this issue.

comment:4 Changed 4 years ago by Claude Paroz

Current code is outputting the db_column name with the \x escape notation (for ex. db_column='tama\xf1o'), due to the repr() call on the value. Is this acceptable?

Changed 4 years ago by Claude Paroz

Attachment: 16737-test.diff added

Test of a non-ascii column name

comment:5 Changed 3 years ago by Claude Paroz <claude@…>

Resolution: fixed
Status: newclosed

In 8c41bd93c2412ddeb9dbe8bd73e1d3c3427d171b:

Fixed #16737 -- Support non-ascii column names in inspectdb

Thanks moof at metamoof.net for the report.

Note: See TracTickets for help on using tickets.
Back to Top