random error KeyError(u'SERVER_NAME',) with uwsgi/nginx setup.

[Damien Nozay]

Here is a traceback:

Environment:


Request Method: GET
Request URL: http://myhostname/api/testcase-results/?testrun=5946

Django Version: 1.6.5
Python Version: 2.7.7
Installed Applications:
('django.contrib.admin',
 'django.contrib.auth',
 'django.contrib.contenttypes',
 'django.contrib.sessions',
 'django.contrib.messages',
 'django.contrib.staticfiles',
 'haystack',
 'corsheaders',
 'rest_framework',
 'builds',
 'bugzilla',
 'testruns')
Installed Middleware:
('corsheaders.middleware.CorsMiddleware',
 'django.contrib.sessions.middleware.SessionMiddleware',
 'django.middleware.common.CommonMiddleware',
 'django.middleware.csrf.CsrfViewMiddleware',
 'django.contrib.auth.middleware.AuthenticationMiddleware',
 'django.contrib.messages.middleware.MessageMiddleware',
 'django.middleware.clickjacking.XFrameOptionsMiddleware')


Traceback:
File "/opt/nginx/test_dashboard/tools/lib/python2.7/site-packages/django/core/handlers/base.py" in get_response
  112.                     response = wrapped_callback(request, *callback_args, **callback_kwargs)
File "/opt/nginx/test_dashboard/tools/lib/python2.7/site-packages/rest_framework/viewsets.py" in view
  78.             return self.dispatch(request, *args, **kwargs)
File "/opt/nginx/test_dashboard/tools/lib/python2.7/site-packages/django/views/decorators/csrf.py" in wrapped_view
  57.         return view_func(*args, **kwargs)
File "/opt/nginx/test_dashboard/tools/lib/python2.7/site-packages/rest_framework/views.py" in dispatch
  401.             response = self.handle_exception(exc)
File "/opt/nginx/test_dashboard/tools/lib/python2.7/site-packages/rest_framework/views.py" in dispatch
  398.             response = handler(request, *args, **kwargs)
File "/opt/nginx/test_dashboard/tools/lib/python2.7/site-packages/rest_framework_extensions/etag/decorators.py" in inner
  37.                 kwargs=kwargs,
File "/opt/nginx/test_dashboard/tools/lib/python2.7/site-packages/rest_framework_extensions/etag/decorators.py" in process_conditional_request
  64.             response = view_method(view_instance, request, *args, **kwargs)
File "/opt/nginx/test_dashboard/tools/lib/python2.7/site-packages/rest_framework_extensions/etag/mixins.py" in list
  15.         return super(ListETAGMixin, self).list(request, *args, **kwargs)
File "/opt/nginx/test_dashboard/tools/lib/python2.7/site-packages/rest_framework_extensions/cache/decorators.py" in inner
  34.                 kwargs=kwargs,
File "/opt/nginx/test_dashboard/tools/lib/python2.7/site-packages/rest_framework_extensions/cache/decorators.py" in process_cache_response
  53.             response = view_method(view_instance, request, *args, **kwargs)
File "/opt/nginx/test_dashboard/tools/lib/python2.7/site-packages/rest_framework_extensions/cache/mixins.py" in list
  16.         return super(ListCacheResponseMixin, self).list(request, *args, **kwargs)
File "/opt/nginx/test_dashboard/tools/lib/python2.7/site-packages/rest_framework/mixins.py" in list
  98.         return Response(serializer.data)
File "/opt/nginx/test_dashboard/tools/lib/python2.7/site-packages/rest_framework/serializers.py" in data
  572.                 self._data = self.to_native(obj)
File "/opt/nginx/test_dashboard/tools/lib/python2.7/site-packages/rest_framework/serializers.py" in to_native
  351.             value = field.field_to_native(obj, field_name)
File "/opt/nginx/test_dashboard/tools/lib/python2.7/site-packages/rest_framework/serializers.py" in field_to_native
  410.             return [self.to_native(item) for item in value.all()]
File "/opt/nginx/test_dashboard/tools/lib/python2.7/site-packages/rest_framework/serializers.py" in to_native
  351.             value = field.field_to_native(obj, field_name)
File "/opt/nginx/test_dashboard/tools/lib/python2.7/site-packages/rest_framework/serializers.py" in field_to_native
  410.             return [self.to_native(item) for item in value.all()]
File "/opt/nginx/test_dashboard/tools/lib/python2.7/site-packages/rest_framework/serializers.py" in to_native
  351.             value = field.field_to_native(obj, field_name)
File "/opt/nginx/test_dashboard/tools/lib/python2.7/site-packages/rest_framework/serializers.py" in field_to_native
  417.         return self.to_native(value)
File "/opt/nginx/test_dashboard/tools/lib/python2.7/site-packages/rest_framework/serializers.py" in to_native
  351.             value = field.field_to_native(obj, field_name)
File "/opt/nginx/test_dashboard/tools/lib/python2.7/site-packages/rest_framework/relations.py" in field_to_native
  548.             return self.get_url(obj, view_name, request, format)
File "/opt/nginx/test_dashboard/tools/lib/python2.7/site-packages/rest_framework/relations.py" in get_url
  573.             return reverse(view_name, kwargs=kwargs, request=request, format=format)
File "/opt/nginx/test_dashboard/tools/lib/python2.7/site-packages/rest_framework/reverse.py" in reverse
  19.         return request.build_absolute_uri(url)
File "/opt/nginx/test_dashboard/tools/lib/python2.7/site-packages/django/http/request.py" in build_absolute_uri
  115.                                          self.get_host(), self.path)
File "/opt/nginx/test_dashboard/tools/lib/python2.7/site-packages/django/http/request.py" in get_host
  62.             host = self.META['SERVER_NAME']

Exception Type: KeyError at /api/testcase-results/
Exception Value: u'SERVER_NAME'

Here is META

uwsgi.node = 'myhostname'
SERVER_NAME = 'myhostname'
HTTP_HOST = 'myhostname'

I am using jQuery to do a lot (1000x) of requests to load historical data.
sometimes the requests fail miserably when they shouldn't
server_name is set correctly in the nginx.conf (less than 1% of the requests fail).

bash-4.1$ uwsgi --version
2.0.6

here is the uwsgi.ini

[uwsgi]
# variables
projectname = test_dashboard
base = /opt/nginx/test_dashboard
logs = /var/log/nginx
venv = tools

# config
master = true
protocol = uwsgi
env = DJANGO_SETTINGS_MODULE=%(projectname).local_settings
pythonpath = %(base):%(base)/%(venv)/lib/site-packages
module = %(projectname).wsgi
socket = %(base)/server.sock
daemonize = %(logs)/uwsgi.log
max-requests = 400 # respawn processes after serving N requests

processes = 4
threads = 4
stats = 127.0.0.1:9191

[Damien Nozay]

filed ​https://github.com/unbit/uwsgi/issues/715 on uwsgi just in case.

[Collin Anderson]

This is certainly a threading problem, and actually, it looks like a problem with rest_framework. Doing a quick skim of their code, it doesn't appear to be thread-safe. I'd recommend not deploying with threads. Feel free to re-open if you can confirm rest_framework is thread-safe.

[Damien Nozay]

Replying to collinanderson:

This is certainly a threading problem, and actually, it looks like a problem with rest_framework. Doing a quick skim of their code, it doesn't appear to be thread-safe. I'd recommend not deploying with threads. Feel free to re-open if you can confirm rest_framework is thread-safe.

can you please clarify? which part appeared not thread-safe when you skimmed through the code?

[Collin Anderson]

Good point. I'll open a ticket in django-rest-framekwork and explain it there. ​https://github.com/tomchristie/django-rest-framework/issues/1870