Porting 2.x code so that it runs on 2.x and 3.x from a single codebase

When porting Django so that it works from a single codebase in Python 2.x and 3.x, the following is a rough-and-ready guide which I (Vinay Sajip) followed, and these guidelines should also apply to the work of porting Django apps to work on the same basis.

• Do run 2to3 on the codebase, but pipe the output to a file so that you can examine what changes need to be made, But don't run 2to3 to make inplace changes to your code, as the resulting code will typically not run under Python 2.x. Go through the piped output to see where you need to make changes. These will typically fall into a number of categories, as described below.

• In any module which uses Unicode or bytes literals (u'foo' or b'bar'), do insert from django.utils.py3 import u, b at the top of the module in the appropriate place, and do replace u'foo' with u('foo') and b'bar' with b('bar') throughout the source. The same applies to the constants with double quotes (u"foo" or b"bar", which should be replaced by u("foo") or b("bar") respectively). Note: this is needed for Python 2.5 compatibility only. For compatibility only with Python 2.6 and later, add from __future__ import unicode_literals in all modules where you are using Unicode literals, and remove the u prefix from the Unicode literals, as you don't need it. If you need native strings, use from django.utils.py3 import n and replace those literals with n(...). This will return str on both 2.x and 3.x. If you need byte strings, use b'foo'. Unfortunately, it's hard to know where native strings are required: for example, some 2.x C-based APIs expect native strings and behave unexpectedly when passed Unicode strings.

• If you need to make any code conditional on 2.x vs. 3.x, you can do from django.utils.py3 import PY3 and use PY3 as a condition (as you might expect, it's a bool which is True on 3.x and False on 2.x.

• If you see any long constants (such as 5L), do from django.utils.py3 import long_type and replace e.g. 5L with long_type(5).

• If you see (int, long) or (long, int) in the source, do from django.utils.py3 import integer_types and replace the tuple with integer_types. If long and int appear in a tuple along with other values, remove them from the tuple and replace the tuple with tuple + integer_types.

• If you see octal constants (such as 0777), replace them with the hex value (0x1ff for the 0777 case), and if possible, place the octal constant in a comment so anyone can see what it was originally. Note: this is only needed for 2.5 compatibility. For 2.6+ compatibility, you can use the 3.x octal form 0o777.

• If you see code of the type except ExceptionTypeOrTupleOfExceptionTypes, name_to_bind_to:, see if name_to _bind_to is used in the scope (exception handling clause, or later in the same function or method). If not used, just change the except: statement to except ExceptionTypeOrTupleOfExceptionTypes: and you're done. If used, do one more thing: put the code name_to_bind_to = sys.exc_info()[1] just before name_to_bind_to is first used, and make sure that the sys module is imported. Note: this is only needed for 2.5 compatibility. For 2.6+ compatibility, you can use the 3.x form except ExceptionTypeOrTupleOfExceptionTypes as name_to_bind_to. Note that in 3.x, names bound like this are not in the local scope, and so not visible outside the exception clause.

• If unicode occurs in the source (i.e. not in comments), do from django.utils.py3 import text_type and replace unicode with text_type.

• If basestring occurs in the source (i.e. not in comments), do from django.utils.py3 import string_types and replace basestring with string_types.

• If str occurs in the source (i.e. not in comments), you may need to do from django.utils.py3 import binary_type and replace str with binary_type. However, don't use this blindly: str() is sometimes used to convert something to text for display, and these occurrences of str() shouldn't need changing.

• If you have classes with metaclasses, change them to use the form MyClassWithMetaClass(with_metaclass(MetaClass, BaseClass): where you can omit BaseClass if it is object. Remove the __metaclass__ = XXX statement from the class body (replacing it with pass if that's all there was in the class body).

• If you need to reraise an exception, do from django.utils.py3 import reraise and invoke it using the form reraise(type, instance, traceback).

• You will see that 2to3 puts list() around calls to methods named keys, values or items. This is sometimes but not always necessary, so examine each on a case-by-case basis. In particular, if values is called on a queryset, you don't want to wrap with list().

• There are functions iterkeys, itervalues and iteritems defined in django.utils.py3. Import them if there is any reference to those methods, and replace using the pattern x.iterkeys() -> iterkeys(x) (where x might be an expression rather than just a name).

• Replace map calls with the list comprehensions suggested by 2to3. If you see map(None, ...) you will need to import and use izip_longest from django.utils.py3.

• If you see xrange in the source, just do from django.utils.py3 import xrange.

• If you use anything from urllib, urllib2, urlparse, you will need to import the actual functions from django.utils.py3. You can grep the instances of those calls in the ported Django code to see how it's done.

• If you're using StringIO, BytesIO, or pickle, import them from django.utils.py3.

• If a class has a next() method which is used for iteration, add a line __next__ = next just after it in the class. If it has a __nonzero__ method, add __bool__ = __nonzero__ in the same way. Where you see x.next() used to iterate, do from django.utils.py3 import next and replace using the pattern x.next() -> next(x) (where x might be an expression rather than just a name).

• If you define rich comparisons, review how they work in 2.x and 3.x and adapt your code accordingly.

• If you use exec or execfile, use the versions exec_ and execfile_ from django.utils.py3.

• Use django.utils.py3.maxsize in place of sys.maxint.

• Review __repr__, __str__ and __unicode__ methods to see what needs changing. On Python 3.x, __str__() should not return bytes, and neither should __repr__().

• Take a look at django.utils.py3 to see what else you might need to change in your code: everything in there is needed somewhere in the porting process.