CSV Enconding Enhance

[Camilo Nova]

Hi, i has to make a function to export a model data to a CSV file, i follow the docs, and works great, but later, when users began to use special characters in a field of the model, the function fails, this is the original function:

writer = csv.writer(response)
writer.writerow(['ID', 'NIT', 'Nombre', 'Direccion', 'Telefono', 'Ciudad'])
for cliente in Cliente.objects.all():
    writer.writerow([   cliente.id, cliente.nit, 
                        cliente.nombre,
                        cliente.direccion_principal, 
                        cliente.telefono_principal, 
                        cliente.ciudad
                    ])

When someone save a value like "niños por la paz en día" the function raises a " Exception Value: 'ascii' codec can't encode character u'\xd1' in position 15: ordinal not in range(128) ", i look in many places for the answer, and found using enconde() may works, and eventually i solve the problem like this:

writer = csv.writer(response)
writer.writerow(['ID', 'NIT', 'Nombre', 'Direccion', 'Telefono', 'Ciudad'])
for cliente in Cliente.objects.all():
    writer.writerow([   cliente.id, cliente.nit, 
                        cliente.nombre.encode('utf-8'),
                        cliente.direccion_principal.encode('utf-8'), 
                        cliente.telefono_principal, 
                        cliente.ciudad
                    ])

Using encode('utf-8') for the fields that maybe user has special characters. You guys, can do this better or inside django, the encode thing by default, just for saving time and make sure the code works everywhere?

Thanks

[Luke Plant]

Django deliberately uses unicode internally everywhere (where appropriate). In the past it did actually use bytestrings, it's not going back. (Just because you want UTF-8 everywhere doesn't mean everyone does).

You can probably clean up your example using a utility like the following:

def encodelist(*items):
    return [unicode(x).encode("utf-8") for x in items]
...
writer.writerow(encodelist(cliente.id, cliente.nit, cliente.nombre, cliente.direccion_principal, cliente.telefono_principal, cliente.ciudad))

Yep, it's still a pain, but its the best we can do with Python 2.x. You've stumbled on one of the biggest warts in Python string handling, it's addressed in Python 3.0.

[Alex Gaynor]

Reformatted for readability, please use the preview button folks.