Opened 3 years ago

Closed 3 years ago

Last modified 6 months ago

#23523 closed Uncategorized (needsinfo)

pyinotify yelling no space available

Reported by: Mattia Procopio Owned by: nobody
Component: Uncategorized Version: 1.8
Severity: Normal Keywords:
Cc: alexandervlpl Triage Stage: Unreviewed
Has patch: no Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description (last modified by Claude Paroz)

don't know why, but in the last days whenever I try to run manage.py runserver I got a pyinotify error, while I'm still able to surf to my pages and work with them I see a nice traceback on console, what's going on?

Traceback:

September 19, 2014 - 10:04:00
Django version 1.7, using settings 'uwn.settings.local'
Starting development server at http://127.0.0.1:8000/
Quit the server with CONTROL-C.
[2014-09-19 10:04:00,066 pyinotify ERROR] add_watch: cannot watch /usr/lib64/python2.7/site-packages/numpy/core/_methods.py WD=-1, Errno=No space left on device (ENOSPC)
[2014-09-19 10:04:00,067 pyinotify ERROR] add_watch: cannot watch /usr/lib/python2.7/site-packages/rest_framework/permissions.py WD=-1, Errno=No space left on device (ENOSPC)
[2014-09-19 10:04:00,067 pyinotify ERROR] add_watch: cannot watch /usr/lib64/python2.7/site-packages/django/utils/termcolors.py WD=-1, Errno=No space left on device (ENOSPC)
[2014-09-19 10:04:00,067 pyinotify ERROR] add_watch: cannot watch /usr/lib64/python2.7/site-packages/django/shortcuts.py WD=-1, Errno=No space left on device (ENOSPC)
[2014-09-19 10:04:00,067 pyinotify ERROR] add_watch: cannot watch /usr/lib64/python2.7/shutil.py WD=-1, Errno=No space left on device (ENOSPC)
[2014-09-19 10:04:00,067 pyinotify ERROR] add_watch: cannot watch /usr/lib64/python2.7/lib-dynload/_heapq.so WD=-1, Errno=No space left on device (ENOSPC)
[2014-09-19 10:04:00,067 pyinotify ERROR] add_watch: cannot watch /usr/lib64/python2.7/lib-dynload/_randommodule.so WD=-1, Errno=No space left on device (ENOSPC)
[2014-09-19 10:04:00,067 pyinotify ERROR] add_watch: cannot watch /usr/lib64/python2.7/site-packages/numpy/fft/fftpack.py WD=-1, Errno=No space left on device (ENOSPC)
[remaining messages cut]

Change History (15)

comment:1 Changed 3 years ago by Baptiste Mispelon

Sorry if that's obvious, but have you checked that you still have space left on your root partition (or wherever /usr/lib64 is mounted)?

comment:2 Changed 3 years ago by Mattia Procopio

yep, I have just 23% used

➜  django git:(django-support-testing) df /usr/lib64 
Filesystem              1K-blocks     Used Available Use% Mounted on
/dev/mapper/fedora-root  51475068 10943904  37893340  23% /

comment:4 Changed 3 years ago by Mattia Procopio

➜  patients_database git:(big_field) df -i
Filesystem                  Inodes  IUsed      IFree IUse% Mounted on
/dev/mapper/fedora-root    3276800 354869    2921931   11% /
devtmpfs                   2039988    511    2039477    1% /dev
tmpfs                      2042567     11    2042556    1% /dev/shm
tmpfs                      2042567    694    2041873    1% /run
tmpfs                      2042567     15    2042552    1% /sys/fs/cgroup
tmpfs                      2042567     30    2042537    1% /tmp
/dev/mapper/fedora-home   11395072 238864   11156208    3% /home
/dev/sda5                   128016     83     127933    1% /boot
/dev/sda4               4294967295   2886 4294964409    1% /boot/efi

comment:5 Changed 3 years ago by Baptiste Mispelon

What about sysctl -n fs.inotify.max_user_watches? (https://github.com/seb-m/pyinotify/wiki/Frequently-Asked-Questions)

comment:6 Changed 3 years ago by Claude Paroz

Description: modified (diff)

Shortened the description, we don't need all messages...

comment:7 Changed 3 years ago by Mattia Procopio

yes, I can increase to a bigger number. But this is not a real solution to the problem. I just want to understand if this is a django problem or not. If I try to run into a virtual env I don't get the error

comment:8 Changed 3 years ago by Aymeric Augustin

Resolution: needsinfo
Status: newclosed

We're also trying to understand if this is a problem with Django or with your system.

Considering that it works in a virtualenv and that we haven't pinpointed the root cause, I'm going to close the ticket as "needsinfo". Please reopen if you understand why you're getting these messages. Then we could attempt to provide at least a better error message.

comment:9 Changed 2 years ago by alexandervlpl

Version: 1.71.8

I'm suddenly having the exact same problem with a Django 1.8 project that was running perfectly a few days ago. Moreover, starting manage.py runserver in a different project on the same system does not give me these errors.

For the record I have one partition, 64% used. I even wrote a large file to it with dd to be absolutely sure there is space on the device. Disk is healthy.

EDIT: Yes, increasing max_user_watches as mentioned above does resolve this:

sudo sysctl -n -w fs.inotify.max_user_watches=16384

Maybe a note should be made about this in the Django docs?

Last edited 2 years ago by alexandervlpl (previous) (diff)

comment:10 Changed 2 years ago by alexandervlpl

Cc: alexandervlpl added

comment:11 Changed 2 years ago by Mattia Procopio

which OS you use?

comment:12 Changed 2 years ago by alexandervlpl

OS: Ubuntu 14.04 LTS. Updated my comment (increasing max_user_watches does resolve this).

comment:13 Changed 2 years ago by Mattia Procopio

I tried but after couple of days I get the same, let me know how it will go

comment:14 Changed 13 months ago by Aman Verma

Hello, did anyone figure out the source of this issue ? I do not have sudo access on my machine, I can't increase number of watches. Is their an alternative solution ?

comment:15 Changed 6 months ago by sverker wahlin

Having the same problem on a fresh Fedora 26 installation. I have roughly 250 files in my entire devserver directory.
I do, however, have a couple of symbolic links. Not sure if that's causing any form of system panic/nervousness.
I have 8192 as max_user_watches.
I also have a nextcloud installation running. One of those folders contain over 30 000 files. I'm not sure if that has added inotify signals for all of the files, but it is a possible source.

Upping the max_user_watches to 16384 fixed it, for now. Will see if it creeps up again.

Note: See TracTickets for help on using tickets.
Back to Top