Porting 2.x code so that it runs on 2.x and 3.x from a single codebase (WIP)

Please Note: This is work in progress, I am working on the draft, and I will remove "I am working on the draft" when I have finished working on the draft.

---

When porting Django so that it works from a single codebase in Python 2.x and 3.x, the following is a rough-and-ready guide which I (Vinay Sajip) followed, and should also apply to the work of porting Django apps to work on the same basis.

• Do run 2to3 on the codebase, but pipe the output to a file so that you can examine what changes need to be made, But don't run 2to3 to make inplace changes to your code, as the resulting code will typically not run under Python 2.x. Go through the piped output to see where you need to make changes. These will typically fall into a number of categories, as described below.

• In any module which uses Unicode or bytes literals (u'foo' or b'bar'), do insert from django.utils.py3 import u, b at the top of the module in the appropriate place, and do replace u'foo' with u('foo') and b'bar' with b('bar') throughout the source. The same applies to the constants with double quotes (u"foo" or b"bar", which should be replaced by u("foo") or b("bar") respectively).

• If you need to make any code conditional on 2.x vs. 3.x, you can do from django.utils.py3 import PY3 and use PY3 as a condition (as you might expect, it's a bool which is True on 3.x and False on 2.x.

• If you see any long constants (such as 5L), do from django.utils.py3 import long_type and replace e.g. 5L with long_type(5).

• If you see (int, long) or (long, int) in the source, do from django.utils.py3 import integer_types and replace the tuple with integer_types. If long and int appear in a tuple along with other values, remove them from the tuple and replace the tuple with tuple + integer_types.

• If you see octal constants (such as 0777), replace them with the hex value (0x1ff for the 0777 case), and if possible, place the octal constant in a comment so anyone can see what it was originally.
• If you see code of the type except ExceptionTypeOrTupleOfExceptionTypes, name_to_bind_to:, see if name_to _bind_to is used in the scope (exception handling clause, or later in the same function or method). If not used, just change the except: statement to except ExceptionTypeOrTupleOfExceptionTypes: and you're done. If used, do one more thing: put the code name_to_bind_to = sys.exc_info()[1] just before name_to_bind_to is first used, and make sure that the sys module is imported.

• If unicode occurs in the source (i.e. not in comments), do from django.utils.py3 import text_type and replace unicode with text_type.

• If basestring occurs in the source (i.e. not in comments), do from django.utils.py3 import string_types and replace basestring with string_types.

• If str occurs in the source (i.e. not in comments), you may need to do from django.utils.py3 import binary_type and replace str with binary_type. However, don't use this blindly: str() is sometimes used to convert something to text for display, and these occurrences of str() shouldn't need changing.

• If you have classes with metaclasses, change them to use the form MyClassWithMetaClass(with_metaclass(MetaClass, BaseClass): where you can omit BaseClass if it is object. Remove the __metaclass__ = XXX statement from the class body (replacing it with pass if that's all there was in the class body).

• If you need to reraise an exception, do from django.utils.py3 import reraise and invoke it using the form reraise(type, instance, traceback).

• You will see that 2to3 puts list() around calls to methods named keys, values or items. This is sometimes but not always necessary, so examine each on a case-by-case basis. In particular, if values is called on a queryset, you don't want to wrap with list().

• There are functions iterkeys, itervalues and iteritems defined in django.utils.py3. Import them if there is any reference to those methods, and replace using the pattern x.iterkeys() -> iterkeys(x) (where x might be an expression rather than just a name).

• Replace map calls with the list comprehensions suggested by 2to3. If you see map(None, ...) you will need to import and use izip_longest from django.utils.py3.

• If you see xrange in the source, just do from django.utils.py3 import xrange.

• If you use anything from urllib, urllib2, urlparse, you will need to import the actual functions from django.utils.py3. You can grep the instances of those calls in the ported Django code to see how it's done.

• If you're using StringIO or BytesIO, import them from django.utils.py3.

• If a class has a next() method which is used for iteration, add a line __next__ = next just after it in the class. If it has a __nonzero__ method, add __bool__ = __nonzero in the same way. Where you see x.next() used to iterate, do from django.utils.py3 import next and replace using the pattern x.next() -> next(x) (where x might be an expression rather than just a name).

• If you define rich comparisons, review how they work in 2.x and 3.x and adapt your code accordingly.