Attached is a refactoring chopping off a good chunk of dispatch's overhead; hard to test it standalone, so testing was done for simple iteration over records and isolating the diff in the profiling for the refactoring.
Short version, attached patch shows 5x boost in dispatch speed; for iteration (just that, no rendering) over 53k records, time drops from ~15.5s to ~11.6s just by cleaning up dispatch.*
The internal data structs dispatch is using are still fairly innefficient- scaling will be an issue the more receivers there are for dispatched signals. So... can clean that up further, but there isn't any documentation re: the expectations of dispatcher. For example- order of connection to a signal, must the order be maintained for notifications at a specific sender/signal level? If I connect (in this order) func f1, func f2, both to object("foo") signal Any, must they be notified in order f1,f2?
I realize pydispatcher is a seperate project- that said, they've already moved onto 2.0, and there are several 1.0.x's for pydispatcher released already- djangos' bundled copy is still at 1.0.0. Whats the intention? Intending on keeping it in sync, or is it just a fork point?
Either way, the attached refactoring is not covered by tests at the moment- mainly since there are no tests in django for dispatcher. Would like to get feedback on the expectations of dispatcher (f1/f2 from above for example) prior to writing those tests, since the implementation (and google for that matter) lack any material on it.
Finally, due to the massive number of calls to send (thus to getAllReceivers and friends), normal seperation (getAllReceivers using getReceivers for example) was intentionally broken down a bit- in other words, inlined a few functions. The codes invoked enough that the overheads seperation really isn't worth it, especially if you're not actually using signals for the most part (thus dispatcher is pretty much pure time waste).
