id	summary	reporter	owner	description	type	status	component	version	severity	resolution	keywords	cc	stage	has_patch	needs_docs	needs_tests	needs_better_patch	easy	ui_ux
32913	test_glob inside utils_tests.test_autoreload.WatchmanReloaderTests is flaky	Abhyudai	Nick Pope	"The test sometimes raises an exception. The `traceback` on my local setup:

{{{#!sh
ERROR: test_glob (utils_tests.test_autoreload.WatchmanReloaderTests)                                                                                                                                                                                                          
----------------------------------------------------------------------                                                                                                                                                                                                        
Traceback (most recent call last):                                                                                                                                                                                                                                              File ""../django/django/utils/autoreload.py"", line 529, in update_watches                                
    self._update_watches()                                                                                                                                                                                                                                                      File ""../django/django/utils/autoreload.py"", line 518, in _update_watches                               
    self._watch_root(root)                                                                                                                                                                                                                                                      File ""../django/django/utils/autoreload.py"", line 438, in _watch_root                                   
    result = self.client.query('watch-project', str(root.absolute()))                                                                                                                                                                                                           File ""../django/venv/lib/python3.8/site-packages/pywatchman/__init__.py"", line 1052, in query
    res = self.receive()                                                                                                                                                                                                                                                        File ""../django/venv/lib/python3.8/site-packages/pywatchman/__init__.py"", line 960, in receive
    result = self.recvConn.receive()                                                                                                                                                                                                                                          
  File ""../django/venv/lib/python3.8/site-packages/pywatchman/__init__.py"", line 695, in receive                                                                                                                                                     buf = [self.transport.readBytes(sniff_len)]                                                                                        
  File ""../django/venv/lib/python3.8/site-packages/pywatchman/__init__.py"", line 344, in readBytes                                                                                                                                                   raise SocketTimeout('timed out waiting for response')                                                                              
pywatchman.SocketTimeout: timed out waiting for response, while executing ('watch-project', '/tmp/django_p9emwuc2/tmpdx4bsuny')                                                                                                                                                                                            
The above exception was the direct cause of the following exception:                                                                                                                                                                                                                                                       
Traceback (most recent call last):                                                                                                                                                                                                                                              File ""/usr/lib/python3.8/unittest/case.py"", line 60, in testPartExecutor                                                             
    yield                                                                                                                                                                                                                                                                       File ""/usr/lib/python3.8/unittest/case.py"", line 676, in run                                                                         
    self._callTestMethod(testMethod)                                                                                                                                                                                                                                            File ""/usr/lib/python3.8/unittest/case.py"", line 633, in _callTestMethod                                                             
    method()                                                                                                                                                                                                                                                                  
  File ""/usr/lib/python3.8/unittest/mock.py"", line 1325, in patched                                                                                                                                                                                                               return func(*newargs, **newkeywargs)     
  File ""../django/tests/utils_tests/test_autoreload.py"", line 542, in test_glob                                                                                                                                                                  
    with self.tick_twice():                                                                                                                                                                                                                                                   
  File ""/usr/lib/python3.8/contextlib.py"", line 113, in __enter__                                                                                                                                                                                                             
    return next(self.gen)                                                                                                                                                                                                                                                       File ""../django/tests/utils_tests/test_autoreload.py"", line 531, in tick_twice
    next(ticker)                                                                                                                                                                                                                                                              
  File ""../django/django/utils/autoreload.py"", line 556, in tick                                          
    self.update_watches()                                                                                                                                                                                                                                                     
  File ""../django/django/utils/autoreload.py"", line 532, in update_watches                                
    if self.check_server_status(ex):                                                                                                                                                                                                                                          
  File ""../django/django/utils/autoreload.py"", line 584, in check_server_status
    raise WatchmanUnavailable(str(inner_ex)) from inner_ex                                                                             
django.utils.autoreload.WatchmanUnavailable: timed out waiting for response, while executing ('watch-project', '/tmp/django_p9emwuc2/tmpdx4bsuny')
}}}

Till now, what I have observed is this only occurs when at least all the tests inside the `test_utils.test_autoreloader.WatchmanReloaderTest` class are run at once. I made a script that you may use to verify this:

{{{#!sh
#!/usr/bin/bash

iteration=0
while true; do
    iteration=$((iteration + 1))

    ./runtests.py utils_tests.test_autoreloader.WatchmanReloaderTests
    if [[ $? -ne 0 ]]; then
        echo -e ""-----------\nit failed in the $iteration iteration--------------------\n""
        break
    fi
    echo -e ""-------------\nrunning the $iteration iteration\n-----------------\n""
done
}}}

The flakiness is highly rare when executing only running the tests under `WatchmanReloaderTests`, in my case it sometimes took close to 2000 operations to fail. Although, when the whole test suite is run at once, the problems seems to occur more frequently(it occurred twice out of something like 10 times on my end)"	Cleanup/optimization	closed	Testing framework	dev	Normal	fixed		Tom Forbes	Ready for checkin	1	0	0	0	0	0