Opened 3 years ago

Last modified 3 years ago

#32708 closed Bug

Django cron file lock breaks with django 3.2 — at Initial Version

Reported by: François Dailloux Owned by: nobody
Component: File uploads/storage Version: 3.2
Severity: Normal Keywords: file lock, too many connections, django_cron
Cc: Hasan Ramezani Triage Stage: Unreviewed
Has patch: no Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description

Hello everyone !

After I upgraded django to 3.2, I noticed that I had the following error popping up

django.db.utils.OperationalError: FATAL: sorry, too many clients already

I then inspected the postgresql connections, and my processes, and it turned out, that was my django cron jobs that were running multiple times at the same time, thus opening more connections than necessary.

[Django cron has a lock mecanism to avoid this,](https://github.com/Tivix/django-cron/blob/v0.5.1/django_cron/backends/lock/file.py) that uses the lock function from django :
from django.core.files import locks

But in django3.2 with latest django_cron version, that check totally fails and we can run a cron multiple times. I looked up a bit the code to see what's the issue.

In django 3.2 [the code for that lock function is ](https://github.com/django/django/blob/main/django/core/files/locks.py)

        def lock(f, flags):
            try:
                fcntl.flock(_fd(f), flags)
                return True
            except BlockingIOError:
                return False

        def unlock(f):
            fcntl.flock(_fd(f), fcntl.LOCK_UN)
            return True

So BlockingIOError are catched, and the function then returns false.

[This changes the behavior from 3.1.7 : ](https://github.com/django/django/blob/stable/3.1.x/django/core/files/locks.py)

        def lock(f, flags):
            ret = fcntl.flock(_fd(f), flags)
            return ret == 0

        def unlock(f):
            ret = fcntl.flock(_fd(f), fcntl.LOCK_UN)
            return ret == 0


in previous behaviour, checking for «ret==0» was indeed useless, because when flock fails, it raises a IOError,

But in 3.1, the lock function didn't catch the error, and django_cron made good use of that behaviour.

I believe that lock function is not documented though, so it's up to you to decide,

whether it's Django or Django_cron that needs fixing.

have a good day !

Change History (0)

Note: See TracTickets for help on using tickets.
Back to Top