id summary reporter owner description type status component version severity resolution keywords cc stage has_patch needs_docs needs_tests needs_better_patch easy ui_ux 30114 ValidationError sometimes raised for valid UUIDs with mod_wsgi Jerry Vinokurov nobody "Our project uses UUIDs as primary keys on a number of models. This works as expected for the majority of the time, but at some point, we run into the following error: {{{ ValidationError: [""'f0c9eb37-a073-47b5-bbb1-589920939c5e' is not a valid UUID.""] }}} This is in fact a valid UUID and also a UUID in our system. We use Sentry for our error reporting, which gives the entire stack trace: {{{ ValidationError: [""'670e5eaf-d231-305e-bb5b-8c8fb0262574' is not a valid UUID.""] File ""django/core/handlers/exception.py"", line 34, in inner response = get_response(request) File ""django/core/handlers/base.py"", line 126, in _get_response response = self.process_exception_by_middleware(e, request) File ""django/core/handlers/base.py"", line 124, in _get_response response = wrapped_callback(request, *callback_args, **callback_kwargs) File ""python3.6/contextlib.py"", line 52, in inner return func(*args, **kwds) File ""django/contrib/admin/options.py"", line 604, in wrapper return self.admin_site.admin_view(view)(*args, **kwargs) File ""django/utils/decorators.py"", line 142, in _wrapped_view response = view_func(request, *args, **kwargs) File ""django/views/decorators/cache.py"", line 44, in _wrapped_view_func response = view_func(request, *args, **kwargs) File ""django/contrib/admin/sites.py"", line 223, in inner return view(request, *args, **kwargs) File ""django/utils/decorators.py"", line 45, in _wrapper return bound_method(*args, **kwargs) File ""django/utils/decorators.py"", line 142, in _wrapped_view response = view_func(request, *args, **kwargs) File ""django/contrib/admin/options.py"", line 1792, in changelist_view 'selection_note': _('0 of %(cnt)s selected') % {'cnt': len(cl.result_list)}, File ""django/db/models/query.py"", line 250, in __len__ self._fetch_all() File ""django/db/models/query.py"", line 1188, in _fetch_all self._prefetch_related_objects() File ""django/db/models/query.py"", line 723, in _prefetch_related_objects prefetch_related_objects(self._result_cache, *self._prefetch_related_lookups) File ""django/db/models/query.py"", line 1569, in prefetch_related_objects obj_list, additional_lookups = prefetch_one_level(obj_list, prefetcher, lookup, level) File ""django/db/models/query.py"", line 1682, in prefetch_one_level prefetcher.get_prefetch_queryset(instances, lookup.get_current_queryset(level))) File ""django/contrib/contenttypes/fields.py"", line 194, in get_prefetch_queryset ret_val.extend(ct.get_all_objects_for_this_type(pk__in=fkeys)) File ""django/db/models/query.py"", line 268, in __iter__ self._fetch_all() File ""django/db/models/query.py"", line 1186, in _fetch_all self._result_cache = list(self._iterable_class(self)) File ""django/db/models/query.py"", line 54, in __iter__ results = compiler.execute_sql(chunked_fetch=self.chunked_fetch, chunk_size=self.chunk_size) File ""django/db/models/sql/compiler.py"", line 1052, in execute_sql sql, params = self.as_sql() File ""django/db/models/sql/compiler.py"", line 464, in as_sql where, w_params = self.compile(self.where) if self.where is not None else ("""", []) File ""django/db/models/sql/compiler.py"", line 390, in compile sql, params = node.as_sql(self, self.connection) File ""django/db/models/sql/where.py"", line 81, in as_sql sql, params = compiler.compile(child) File ""django/db/models/sql/compiler.py"", line 390, in compile sql, params = node.as_sql(self, self.connection) File ""django/db/models/lookups.py"", line 355, in as_sql return super().as_sql(compiler, connection) File ""django/db/models/lookups.py"", line 163, in as_sql rhs_sql, rhs_params = self.process_rhs(compiler, connection) File ""django/db/models/lookups.py"", line 339, in process_rhs sqls, sqls_params = self.batch_process_rhs(compiler, connection, rhs) File ""django/db/models/lookups.py"", line 231, in batch_process_rhs pre_processed = super().batch_process_rhs(compiler, connection, rhs) File ""django/db/models/lookups.py"", line 51, in batch_process_rhs _, params = self.get_db_prep_lookup(rhs, connection) File ""django/db/models/lookups.py"", line 186, in get_db_prep_lookup if self.get_db_prep_lookup_value_is_iterable else File ""django/db/models/lookups.py"", line 185, in [get_db_prep_value(v, connection, prepared=True) for v in value] File ""django/db/models/fields/__init__.py"", line 2316, in get_db_prep_value value = self.to_python(value) File ""django/db/models/fields/__init__.py"", line 2330, in to_python params={'value': value}, }}} Ok, so what's going on near line 2330 in `django/db/models/fields/__init__.py` in the `to_python` function? {{{ 2325: return uuid.UUID(value) 2326: except (AttributeError, ValueError): 2327: raise exceptions.ValidationError( 2328: self.error_messages['invalid'], 2329: code='invalid', 2330: params={'value': value}, 2331: ) 2332: return value 2333: def formfield(self, **kwargs): 2334: return super().formfield(**{ }}} And what's going on one frame before that, near line 2316 in `django/db/models/fields/__init__.py` in `get_db_prep_value`: {{{ 2312: def get_db_prep_value(self, value, connection, prepared=False): 2313: if value is None: 2314: return None 2315: if not isinstance(value, uuid.UUID): 2316: value = self.to_python(value) 2317: if connection.features.has_native_uuid_field: 2318: return value 2319: return value.hex }}} Ok, so what's happening is that if the value that we're trying to convert is not an instance of `uuid.UUID`, then we convert it to a Python value, which calls `to_python`. This makes sense if `value` is a string, but... Sentry is telling me that `value` ''is already'' a UUID: `value UUID('f0c9eb37-a073-47b5-bbb1-589920939c5e')` The check on line 2315 in `get_db_prep_value` should catch the fact that this is a UUID, but it doesn't. I don't understand why at all. Sentry is telling me that this is a UUID but it's failing the instance check; as a result line 2325 in `to_python` attempts to convert the value to a UUID, but `value` is already a UUID, so it throws a `ValidationError`. That's the details of the error. The very weird thing about it is that this error is totally inconsistent in when it shows up. We'll be running for days in a production environment without any problems and then all of a sudden this error will crop up. Once that happens, any attempt to deserialize a UUID fails with this error. We are running our application on an EC2 instance in AWS managed by Elastic Beanstalk. A deployment of the application, which restarts it, makes this error go away. But sooner or later it crops up again and brings everything down. The worst thing is that I can't seem to replicate this error in local development. Our DB runs on an RDS instance and I can point my local development environment to that DB but I can't replicate the error. I'm truly baffled by this. Does anyone have any idea how this can happen? It very much ''seems'' like a bug to me, hence this report, but I honestly don't know. I'm just hoping to get some guidance or ideas about what is going on here. " Bug closed Database layer (models, ORM) 2.1 Normal invalid validationerror, uuid Unreviewed 0 0 0 0 0 0