Django will serve an exception traceback if your 404 handler raises an exception. The relevant part of django.core.handlers.base follows:
except http.Http404, e:
if settings.DEBUG:
from django.views import debug
return debug.technical_404_response(request, e)
else:
callback, param_dict = resolver.resolve404()
return callback(request, **param_dict)
If resolve404() raises any exception (such as an invalid block tag in the 404 template, or if the user has overriden handler404), Django does not suppress the exception and serve a 500 page; instead it simply serves the traceback. Note that this happens even if DEBUG is set to False.
This might catch someone by surprise if they launch their site without checking if 404 pages work with DEBUG turned off (i.e., they would see a traceback from this issue, but be expecting it).