Don't fetch Field's choices on form/manipulator creation

[Ivan Sagalaev <Maniac@…>]

Currently creation of a default manipulator for a model causes actual fetching of all content of every related parent models which tends to be slow. They are used to fill <select>s created for ForeignKeys and ManyToManyFields but this may be unnecessary if those <select>s never rendered. This can be easily fixed by turning a method that gets choices for <select>s (Field.get_choices()) into a generator that will hit database only when (and if) needed.

Patch follows.

[Ivan Sagalaev <Maniac@…>]

Patch

[Gary Wilson <gary.wilson@…>]

Marking this as accepted, but new feature will probably not get added to Manipulators. Does newforms exhibit the same problem? If so, it would be great if someone could submit a patch for newforms.

[Ivan Sagalaev <Maniac@…>]

Gary, this code lives in db field, not old form fields, so it works for newforms too. Actually it was created after concerns in django-developers about newforms fetching related values like old forms.

[Malcolm Tredinnick]

If we ever need it for verifying this problem, Joe Heck put a test case in #2638.

[Malcolm Tredinnick]

A small comment about this change (since I've looked at it, but don't have cycles to complete the work at the moment)...

I looked at this patch a couple of weeks ago. It wasn't immediately clear where the performance improvement appeared. It's definitely an improvement for some uses, but are we hitting those uses in newforms?

So we need a test to demonstrate how this is triggered, I think. It's a slight backwards incompatible change (since the returned type of get_choices changes). It wasn't apparent when I moved Joe Heck's example from #2638 over to newforms, for example.

[Ivan Sagalaev <Maniac@…>]

To clarify... You want a regression test showing that it works properly or a test showing a performance gain? I'm not sure how to do the latter. And the former is show by any newforms test that renders a <select>.

[Ivan Sagalaev <Maniac@…>]

Ok, I've tested it a bit. The problem really doesn't affect newforms since Field.formfield calls get_choices only if a field has explicit 'choices' attribute set. ForeignKeys don't have choices and are fetched by formfield's widget only at display time. Oldforms still have a problem though.

[James Bennett]

Per comments above this is not a problem in newforms, and the old forms system is deprecated, so I'm marking this wontfix.