Opened 10 years ago

Last modified 10 years ago

#21721 closed Bug

Python 3.4 support — at Version 15

Reported by: Marc Tamlyn Owned by: nobody
Component: Python 3 Version: dev
Severity: Release blocker Keywords:
Cc: Florian Apolloner, berker.peksag@…, django@… Triage Stage: Accepted
Has patch: no Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description (last modified by Marc Tamlyn)

Python 3.4 support should be added in master ready for 1.7, and probably also in 1.6.X as Py3.4 will be out before Django 1.7. As Py3.4 comes with bundled pip and venv, and is a prime target for new python developers and those running classes, it is very important for Django to support it.

I've run the test suite (on sqlite) against 1.6.X and master. A test run on 1.6.X can be seen at https://gist.github.com/mjtamlyn/8195542

The current state of things is as follows:

  • There are a couple of warnings printed at startup time. These relate to API changes in plistlib (which is OSX specific) and codecs (universal newlines reading of files has been deprecated).
  • Our usage of HTMLParser should now always specify the value of convert_charrefs as it's default will change in Py3.5. This is slightly problematic as Py2.7 does not have this kwarg so we can't universally apply it. Perhaps the best option is a six-like wrapper.
  • django.utils.module_loading. module_has_submodule has some issues with eggs. sys.meta_path is giving us importlib as a finder. importlib.find_module is deferred to importlib.find_spec (new in py3.4), which throws an error (ImportError: spec missing loader).
  • There is a failing test in the mail library regarding encoding in the mail module. Florian seemed to know about this.
  • There is a significant issue with signal deregistration. I've been able to ascertain that something is up in the django.dispatch.saferef module, but I don't have a sufficient understanding to work out what's wrong. There seems to be no test which tests this code directly and failures are thrown up at random during tests, test teardown and/or test suite teardown. They show up either as NoneType is not callable or as catching classes that do not inherit from BaseException is not allowed.
  • U mode has been deprecated for files. We use it in makemessages and in sql, both for management commands. It seems unnecessary in makemessages, in the sql we will have to be more careful as at present we split on \n, which may need to be \r\n on windows. However, it may well be fine anyway it probably doesn't matter if we have trailing \r characters on the sql statements.
  • There are quite a lot of ResourceWarnings thrown for unclosed files (and sockets in the mail tests)

Change History (16)

comment:1 by Marc Tamlyn, 10 years ago

Loic pointed out on IRC a more significant issue with module_loading - which is that the code we use for acquiring a global import lock has also been deprecated. See http://docs.python.org/dev/library/imp.html#imp.acquire_lock

comment:2 by loic84, 10 years ago

#21628 tracks the specific issue of the deprecation of the imp module.

comment:3 by Tim Graham, 10 years ago

Triage Stage: UnreviewedAccepted

in reply to:  description comment:4 by Florian Apolloner, 10 years ago

Cc: Florian Apolloner added

Replying to mjtamlyn:

  • There is a failing test in the mail library regarding encoding in the mail module. Florian seemed to know about this.

Not an issue, just use an up2date 3.4 (beta 2 at least!).

  • There is a significant issue with signal deregistration. I've been able to ascertain that something is up in the django.dispatch.saferef module, but I don't have a sufficient understanding to work out what's wrong. There seems to be no test which tests this code directly and failures are thrown up at random during tests, test teardown and/or test suite teardown. They show up either as NoneType is not callable or as catching classes that do not inherit from BaseException is not allowed.

No, it has nothing to do with saferef, the main issue is that weakref und onDelete don't work properly on 3.4 during interpreter shutdown; to get this right you basically have to use finalize: https://dpaste.de/ZeyD (That patch needs a little bit of cleanup but is otherwise ready to go)

comment:5 by Florian Apolloner, 10 years ago

Okay, my patch doesn't really work yet since it doesn't disconnect all receivers, gotta fix that *gg* http://bugs.python.org/issue19255 might allow us to use weakref's onDelete again.

by Florian Apolloner, 10 years ago

Attachment: broken_builtins.diff added

comment:6 by Florian Apolloner, 10 years ago

The attached patch fixes the None is not callable issues, TypeError: catching classes that do not inherit from BaseException is not allowed still occurs with this variant.

comment:7 by Florian Apolloner, 10 years ago

That patch is still pretty much unstable, there appear to be a few issues with saferef due to all the weakrefing :(

comment:8 by Florian Apolloner, 10 years ago

Last edited 10 years ago by Florian Apolloner (previous) (diff)

comment:9 by Florian Apolloner, 10 years ago

And I got rid of saferef completely: https://github.com/apollo13/django/commit/7a6c28977bf258b27c5bc2a19417f913c541e2dd -- Please test as much as possible :)

comment:10 by Marc Tamlyn, 10 years ago

I ran some tests with that commit. The good news is that python 3.4b2 seems to be fine - the only failures now are the egg loading related failures and (obviously) the tests for saferef. There were no nasty warnings in interpreter shutdown. A number of other warnings have also disappeared, although not when you run with -Wall.

The bad news is that python 2.7 doesn't work very well. Basically all the tests in dispatch.tests fail. https://gist.github.com/mjtamlyn/8304266. The same errors also happen on py3.3.

comment:11 by anonymous, 10 years ago

Oh, I just tested handlers+signals and had pyc files around :( Will rework my patch later on

comment:14 by Anssi Kääriäinen, 10 years ago

Any chance you can run djangobench for this? IIRC model_init_self_signals will benchmark signal sending.

comment:15 by Marc Tamlyn, 10 years ago

Description: modified (diff)

Signals branch now merged- thanks Florian.

I've opened a PR for the HTMLParser warnings - https://github.com/django/django/pull/2164

I've updated the description with the current outstanding issues. There are now only two test failures, relating to loading from eggs. Everything else is warnings.

Note: See TracTickets for help on using tickets.
Back to Top