Opened 4 months ago

Closed 4 months ago

Last modified 4 months ago

#30361 closed Bug (fixed)

Watchman timing out when loading server

Reported by: Jacob Green Owned by: Jacob Green
Component: Utilities Version: 2.2
Severity: Normal Keywords:
Cc: Tobias Kunze Triage Stage: Ready for checkin
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description

When starting the server on my project using ./manage.py runserver, I get the error message:

Error connecting to Watchman: timed out waiting for response, while executing ('watch-project', 'my_app')

I am using Watchman v4.9 on an Ubuntu 16.04 Docker image hosted on Mac OS X 10.14.4.

My research shows that there is a default 1 second timeout in pywatchman that Django is using. In my tests, for our particular project it takes between 1 and 2 seconds to start the initial root watchman query, and thus will always time out when started with Django.

My suggestion is that we increase the pywatchman client timeout higher than 1 second

Change History (14)

comment:2 Changed 4 months ago by Daniel Hepper

Jacob, just wanted to let you know that the two URLs you posted are identical. I assume that was not intentional.

comment:3 Changed 4 months ago by Tobias Kunze

I had this issue occur on my machine, but I can't reproduce it. Is it reproducible for you, Jacob? If so, could you check if this solves the issue? If it does, I'll submit it as PR.

comment:4 Changed 4 months ago by Tobias Kunze

Cc: Tobias Kunze added

comment:5 Changed 4 months ago by Peter J. Farrell

We are seeing the same issue especially the container starts (less often if it's just a server reload). It appears the timeout on pywatchman needs to be configurable from Django in this case as 1.0 seconds is too low in certain circumstances.

comment:6 Changed 4 months ago by Tobias Kunze

A more generous default timeout might be a better choice than a configuration option, in my opinion. In any case it would be great if somebody who runs into this consistently could test my patch or provide one of their own.

comment:7 Changed 4 months ago by Tobias Kunze

Triage Stage: UnreviewedAccepted

comment:8 Changed 4 months ago by Jacob Green

I run into the issue consistently on our codebase so I can repro it. Tobias, how would you recommend that I test against your patch? I know how to install Django from pypi but I don't know how to install it from a fork.

Also I may be misreading but in the commit that you linked it seems that you lowered the timeout to 0.2s instead of raising it above 1.0s. Perhaps that was unintentional?

comment:9 Changed 4 months ago by James Addison

I've just found that adding a .watchmanconfig file to add the ignore_dirs directive seemed to help (I assume this would allow watchman to start more quickly). See https://facebook.github.io/watchman/docs/config.html#ignore_dirs

{
  "ignore_dirs": ["node_modules", ".envs", "bin", "lib", "include", ".git", ".idea", "templates", "static_assets"]
}

It certainly isn't a fix - but as mentioned in the Django docs, it's worth investigating as a bandaid.

comment:10 Changed 4 months ago by Jacob Green

I tried out this change (increasing timeout to 5 seconds) and that fixed my issue:

https://github.com/django/django/compare/stable/2.2.x...Jacobkg:Jacobkg-patch-1

I'm happy to submit this as a patch if there is interest

comment:11 Changed 4 months ago by Jacob Green

Component: UncategorizedUtilities
Has patch: set
Owner: changed from nobody to Jacob Green
Status: newassigned

I submitted a Pull Request, with the proposal that this be backported onto 2.2 since it improves the functionality of a feature released as part of 2.2

PR

comment:12 Changed 4 months ago by felixxm

Triage Stage: AcceptedReady for checkin

comment:13 Changed 4 months ago by Mariusz Felisiak <felisiak.mariusz@…>

Resolution: fixed
Status: assignedclosed

In ed3c590:

Fixed #30361 -- Increased the default timeout of watchman client to 5 seconds and made it customizable.

Made the default timeout of watchman client customizable via
DJANGO_WATCHMAN_TIMEOUT environment variable.

comment:14 Changed 4 months ago by Mariusz Felisiak <felisiak.mariusz@…>

In e4576319:

[2.2.x] Fixed #30361 -- Increased the default timeout of watchman client to 5 seconds and made it customizable.

Made the default timeout of watchman client customizable via
DJANGO_WATCHMAN_TIMEOUT environment variable.

Backport of ed3c59097a01ed3f32f8a8bed95307fb5c181251 from master

Note: See TracTickets for help on using tickets.
Back to Top