Opened 5 years ago

Closed 5 years ago

Last modified 5 years ago

#30647 closed Bug (fixed)

[FATAL] FileNotFoundError with runserver command inside Docker container

Reported by: Bobby Steinbach Owned by: Tom Forbes
Component: Core (Other) Version: 2.2
Severity: Release blocker Keywords: docker docker-compose autoreload
Cc: Tom Forbes Triage Stage: Accepted
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description

Summary
Trying to run the development server in a container with volume-mounted source is throwing a FileNotFoundError. I've verified that the issue is consistently reproducible with Django==2.2.3 and not present in Django==2.1.4.

Trace

**INFO** /code/publications/models.py changed, reloading.
**INFO** Watching for file changes with StatReloader
Performing system checks...

Traceback (most recent call last):
  File "manage.py", line 21, in <module>
    main()
  File "manage.py", line 17, in main
    execute_from_command_line(sys.argv)
  File "/usr/local/lib/python3.6/site-packages/django/core/management/__init__.py", line 381, in execute_from_command_line
    utility.execute()
  File "/usr/local/lib/python3.6/site-packages/django/core/management/__init__.py", line 375, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/usr/local/lib/python3.6/site-packages/django/core/management/base.py", line 323, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/usr/local/lib/python3.6/site-packages/django/core/management/commands/runserver.py", line 60, in execute
    super().execute(*args, **options)
  File "/usr/local/lib/python3.6/site-packages/django/core/management/base.py", line 364, in execute
    output = self.handle(*args, **options)
  File "/usr/local/lib/python3.6/site-packages/django/core/management/commands/runserver.py", line 95, in handle
    self.run(**options)
  File "/usr/local/lib/python3.6/site-packages/django/core/management/commands/runserver.py", line 102, in run
    autoreload.run_with_reloader(self.inner_run, **options)
  File "/usr/local/lib/python3.6/site-packages/django/utils/autoreload.py", line 587, in run_with_reloader
    start_django(reloader, main_func, *args, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/django/utils/autoreload.py", line 572, in start_django
    reloader.run(django_main_thread)
  File "/usr/local/lib/python3.6/site-packages/django/utils/autoreload.py", line 289, in run
    autoreload_started.send(sender=self)
  File "/usr/local/lib/python3.6/site-packages/django/dispatch/dispatcher.py", line 175, in send
    for receiver in self._live_receivers(sender)
  File "/usr/local/lib/python3.6/site-packages/django/dispatch/dispatcher.py", line 175, in <listcomp>
    for receiver in self._live_receivers(sender)
  File "/usr/local/lib/python3.6/site-packages/django/utils/translation/reloader.py", line 16, in watch_for_translation_changes
    absolute_path = path.absolute()
  File "/usr/local/lib/python3.6/pathlib.py", line 1129, in absolute
    obj = self._from_parts([os.getcwd()] + self._parts, init=False)
FileNotFoundError: [Errno 2] No such file or directory

Configuration
Dockerfile

FROM python:3.6.7-alpine3.7

RUN mkdir /code

WORKDIR /code

RUN apk add postgresql-dev libffi-dev build-base musl-dev
RUN apk add linux-headers

ADD requirements.txt .

RUN pip install -r requirements.txt

EXPOSE 3031

ADD cs /code

docker-compose

version: '3.7'
services:
  db:
    image: postgres
    volumes:
      - ./pg_data:/var/lib/postgresql/data
    ports:
      - "5432:5432"
    environment:
      POSTGRES_PASSWORD: postgres
      POSTGRES_USER: postgres
      POSTGRES_DB: postgres

  app:
    build:
      context: .
    volumes:
      - ./cs/:/code/
    ports:
      - "8000:8000"
    env_file: .env
    command: ["python", "manage.py", "runserver", "0.0.0.0:8000"]

Change History (13)

comment:1 by Mariusz Felisiak, 5 years ago

Cc: Tom Forbes added

comment:2 by Carlton Gibson, 5 years ago

First glance, this looks like some Docker weirdness:

  File "/usr/local/lib/python3.6/site-packages/django/utils/translation/reloader.py", line 16, in watch_for_translation_changes
    absolute_path = path.absolute()
  File "/usr/local/lib/python3.6/pathlib.py", line 1129, in absolute
    obj = self._from_parts([os.getcwd()] + self._parts, init=False)
FileNotFoundError: [Errno 2] No such file or directory

That's a standard library call raising the error, so why's that not working?

@steinbachr it would be helpful if you could put a breakpoint in there and try to work out exactly what's going on. (That call should work. Why isn't it? Is there an obvious something that would?)

Why the regression between 2.1 and 2.2? We were using os.path previously I guess... Still, this should be something that works, so a bit more digging is needed.

comment:3 by Carlton Gibson, 5 years ago

Component: UncategorizedCore (Other)

comment:4 by Tom Forbes, 5 years ago

Is this while using Docker for Mac? Could it be related to this Docker issue, as it seems it’s being thrown in the cwd call:

https://github.com/docker/for-mac/issues/1509

Version 0, edited 5 years ago by Tom Forbes (next)

comment:5 by Carlton Gibson, 5 years ago

Resolution: needsinfo
Status: newclosed

I'm thinking to close this as needsinfo for the moment. I'm very happy to re-open it if we can show that Django is at fault. Or if we want to provide a workaround in any case... (— Tom, happy to follow your lead there.)

@steinbachr please do add the extra information. I do mean the "very happy to re-open" :) Thanks.

comment:6 by Bobby Steinbach, 5 years ago

@Tom Forbes on my machine it is happening all the time. I can change my requirements to downgrade to Django==2.1.4 and rebuild the image, resulting in a working container. Then, I can upgrade to Django==2.2.3 and rebuild, resulting in a broken container, consistently.

Some system information:

Mac OS Version: 10.13.6 (High Sierra)
Docker version 18.09.2, build 6247962
docker-compose version 1.23.2, build 1110ad01

Let me know if there's any additional info I can provide to help

comment:7 by Tom Forbes, 5 years ago

Resolution: needsinfo
Status: closednew
Triage Stage: UnreviewedAccepted

I'm going to re-open this Carlton, if that's OK. While this isn't Django's fault, it is a regression since 2.1 and it apparently does happen reproducibly.

Patch: https://github.com/django/django/pull/11584

Bobby: I would absolutely love if you could find some time to test this patch. It should be pretty simple, if you can run:

python -mpip install https://github.com/orf/django/archive/test-file-not-found.zip

Unfortunately this will be a 100+mb tarball (as it includes the whole repository history), but it's the latest 2.2x branch with my patch applied. Please let me know if the issue still occurs!

comment:8 by Tom Forbes, 5 years ago

Has patch: set
Owner: changed from nobody to Tom Forbes
Status: newassigned

comment:9 by Mariusz Felisiak, 5 years ago

Severity: NormalRelease blocker

comment:10 by Carlton Gibson, 5 years ago

I'm going to re-open this Carlton, if that's OK.

Absolutely Tom! That was the hope. 🙂

in reply to:  7 comment:11 by Bobby Steinbach, 5 years ago

Tom, I'll give that a try now and report back.

Replying to Tom Forbes:

I'm going to re-open this Carlton, if that's OK. While this isn't Django's fault, it is a regression since 2.1 and it apparently does happen reproducibly.

Patch: https://github.com/django/django/pull/11584

Bobby: I would absolutely love if you could find some time to test this patch. It should be pretty simple, if you can run:

python -mpip install https://github.com/orf/django/archive/test-file-not-found.zip

Unfortunately this will be a 100+mb tarball (as it includes the whole repository history), but it's the latest 2.2x branch with my patch applied. Please let me know if the issue still occurs!

comment:12 by Mariusz Felisiak <felisiak.mariusz@…>, 5 years ago

Resolution: fixed
Status: assignedclosed

In fc756942:

Fixed #30647 -- Fixed crash of autoreloader when extra directory cannot be resolved.

comment:13 by Mariusz Felisiak <felisiak.mariusz@…>, 5 years ago

In 4d6449e1:

[2.2.x] Fixed #30647 -- Fixed crash of autoreloader when extra directory cannot be resolved.

Backport of fc75694257b5bceab82713f84fe5a1b23d641c3f from master.

Note: See TracTickets for help on using tickets.
Back to Top