Opened 17 months ago

Closed 17 months ago

Last modified 16 months ago

#21274 closed Bug (needsinfo)

django-admin's --template option fails to download from https via proxy

Reported by: blueyed Owned by: nobody
Component: HTTP handling Version: master
Severity: Normal Keywords:
Cc: Triage Stage: Unreviewed
Has patch: no Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: no UI/UX: no

Description

Using Squid as $https_proxy, the following will fail:

django-admin.py startproject -v3 --template=https://github.com/twoscoops/django-twoscoops-project/archive/develop.zip --extension=py,rst,html project

Rendering project template files with extensions: .html, .rst, .py
Rendering project template files with filenames: 
Downloading https://github.com/twoscoops/django-twoscoops-project/archive/develop.zip
Extracting /tmp/django_project_template_IKLof6_download/develop.zip
Traceback (most recent call last):
  File "/home/daniel/.virtualenvs/TMM/bin/django-admin.py", line 8, in <module>
    execfile(__file__)
  File "/home/daniel/projects/TMM/src/django/django/bin/django-admin.py", line 5, in <module>
    management.execute_from_command_line()
  File "/home/daniel/projects/TMM/src/django/django/core/management/__init__.py", line 397, in execute_from_command_line
    utility.execute()
  File "/home/daniel/projects/TMM/src/django/django/core/management/__init__.py", line 390, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/home/daniel/projects/TMM/src/django/django/core/management/base.py", line 244, in run_from_argv
    self.execute(*args, **options.__dict__)
  File "/home/daniel/projects/TMM/src/django/django/core/management/base.py", line 291, in execute
    output = self.handle(*args, **options)
  File "/home/daniel/projects/TMM/src/django/django/core/management/commands/startproject.py", line 31, in handle
    super(Command, self).handle('project', project_name, target, **options)
  File "/home/daniel/projects/TMM/src/django/django/core/management/templates.py", line 122, in handle
    base_subdir)
  File "/home/daniel/projects/TMM/src/django/django/core/management/templates.py", line 205, in handle_template
    return self.extract(absolute_path)
  File "/home/daniel/projects/TMM/src/django/django/core/management/templates.py", line 299, in extract
    archive.extract(filename, tempdir)
  File "/home/daniel/projects/TMM/src/django/django/utils/archive.py", line 49, in extract
    with Archive(path) as archive:
  File "/home/daniel/projects/TMM/src/django/django/utils/archive.py", line 58, in __init__
    self._archive = self._archive_cls(file)(file)
  File "/home/daniel/projects/TMM/src/django/django/utils/archive.py", line 181, in __init__
    self._archive = zipfile.ZipFile(file)
  File "/usr/lib/python2.7/zipfile.py", line 766, in __init__
    self._RealGetContents()
  File "/usr/lib/python2.7/zipfile.py", line 807, in _RealGetContents
    raise BadZipfile, "File is not a zip file"
zipfile.BadZipfile: File is not a zip file

Looking at the tempfile, it contains HTML from a Squid3 error page.

From looking at Squid3's logfile, urlretrieve appears to do a GET request to the https URL, while wget will use CONNECT.

I am using Django's current master and Python 2.7.5+ (Ubuntu 13.10, in a virtualenv).

As a workaround I can unset $https_proxy.

Change History (4)

comment:1 Changed 17 months ago by timo

  • Needs documentation unset
  • Needs tests unset
  • Patch needs improvement unset

Do you have a suggested fix (or an idea of how complex the fix might be)? Depending on the answer, it may be more effort than it's worth to maintain this code in Django - I'm unfamiliar with Squid so I could be completely wrong. One could also download the zip using something like wget and then use a local path to the zip file, right?

comment:2 Changed 17 months ago by timo

  • Resolution set to needsinfo
  • Status changed from new to closed

Closing as needsinfo in absence of a proposal for what should be done here. Feel free to reopen if you'd like to provide more details.

comment:3 Changed 16 months ago by blueyed

I have not investigated much into fixing this - I stopped when I arrived at the Python2/3-compatibility layer (django.utils.six).

I have fixed some proxy issue in an urllib based Python program before (IIRC), but cannot provide a fix here.

I agree that downloading the file via wget and/or disabling the https proxy is a good enough workaround. But I also believe that there should be a more descriptive error, instead of just a traceback and "File is not a zip file".
"

comment:4 Changed 16 months ago by aladagemre

Problem should be that an additional dot (.) at the end of the URL is added when proxy is active.
Here is the downloaded file:

<head>
<title>Error response</title>
</head>
<body>
<h1>Error response</h1>
<p>Error code 400.
<p>Message: bad url https://github.com/rdegges/django-skel/zipball/master.
<p>Error code explanation: 400 = Bad request syntax or unsupported method.
</body>
Note: See TracTickets for help on using tickets.
Back to Top