Update installation instructions for GeoDjango dependencies on Windows.

[Adam Starrh]

I'm setting GeoDjango up on Windows and discovered that GDAL is (at least currently) incompatible with 64-bit Python. Using 32-bit Python and 32-bit OSGeo4Win, I was finally able to get it working. I suggest this be mentioned in the docs to circumvent future headaches for other users.

Pull request:
​https://github.com/django/django/pull/11995

---

The installation instructions for dependencies of GeoDjango on Windows are outdated.

Issues that need to be addressed include:

• It seems that there may be some as yet undetermined issue wit 64-bit setups.
• Not referencing only 9.x versions of PostgreSQL which are > 3 years old.
• Consider referring to other geospatial database backends. Why should Windows users be forced to use PostGIS?
• Replace outdated instructions for installation of psycopg2. We have binary wheels now that are more up to date.
• Using OSGeo4W can lead to a newer version of GEOS than is supported by Django with no option to downgrade.
  • Consider installing individual dependencies for GDAL, GEOS & Proj.

[Adam Starrh]

Basically, this has happened to me a few times over the years. As a casual programmer, when following the setup docs I habitually install the 64-bit versions of everything because I figure they should match the OS/chipset. Inevitably, something fails mysteriously and I spend a lot of time trying to figure out the problem before converting everything over to 32-bit where things end up running more smoothly. Because it isn't an obvious problem, and because it seems to manifest itself in different ways, it takes a while to realize each time.

I was unable to run migrations after following the setup instructions on a fresh installation of Windows 10 and choosing the 64-bit versions of everything. I started over, this time choosing the 32-bit installations of python and OSGeo4Win, and everything worked smoothly.

This is not the first time I've had this experience. It would really be helpful as a novice programmer to at least be advised that the 64-bit version of Python may be problematic and will likely require steps that are outside the scope of the installation instructions if the user is to ultimately get it working.

[Nick Pope]

So as I said on the PR, I do not feel that we should be encouraging users to install 32-bit. It seems like a backward step and I would rather we address making this work for 64-bit.
(I will admit that I don't use Windows, but I will endeavour to help where I can.)

So my understanding is that the documentation for installing dependencies for GeoDjango on Windows is at ​https://docs.djangoproject.com/en/stable/ref/contrib/gis/install/#windows.

The first requirement there is to install Python on Windows by following ​https://docs.djangoproject.com/en/stable/howto/windows/. Django 2.2 doesn't currently support the recently release 3.8, so let's go with 3.7.5 available from ​https://www.python.org/downloads/release/python-375/.

I note that the next steps suggests is that Windows users ought to be using PostgreSQL & PostGIS. This seems a bit restrictive as I'm sure the other geospatial database backends will work. But let's run with it. (I note that the documentation here ought to be updated to drop references to 9.x - version 10, 11 and 12 are available, so 9.6 is >3 years old.) Django 2.2 only supports up to 11, so let's go with 11.5, available from ​https://www.enterprisedb.com/downloads/postgres-postgresql-downloads. I note that EnterpriseDB have stopped providing 32-bit builds for Windows for PostgreSQL >= 11. I will assume at this stage that the instructions for PostGIS are the same or similar to those described.

According to the documentation, we will also need the latest version of psycopg2 from ​http://www.stickpeople.com/projects/python/win-psycopg/. Unfortunately that page no longer seems to work for me and times out. The Internet Archive Wayback Machine gives ​this version of the page, but the latest version there is only built for Python 3.6 and is psycopg2 2.6.2 when 2.8.4 is available. I would suggest that this documentation is updated to point to ​https://www.lfd.uci.edu/~gohlke/pythonlibs/#psycopg or, better yet, recommend installing ​psycopg2-binary from PyPI which wasn't available when this was written."

Next we come to OSGeo4W. The instructions point us to ​https://trac.osgeo.org/osgeo4w/ and using the 64-bit link we get an installer. I note the following restrictions under the "Notes" heading:

• "32bit/64bit don't contain the same set of packages/versions ​(compare) (purple is 32 bit only)"
• "Don't blindly do a full install in Advanced mode. Some components require third party additions that need to be installed manually (see the wiki page of the package). Without the additions the components might renderer the installation unuseable."

So after all of that I would expect that you are using the following installers:

• ​https://www.python.org/ftp/python/3.7.5/python-3.7.5-amd64.exe
• ​https://get.enterprisedb.com/postgresql/postgresql-11.5-2-windows-x64.exe
• ​https://files.pythonhosted.org/packages/e7/0c/aca85d48bd72557251ee87ac5b8566ad17eb197725247357d90d0891216e/psycopg2_binary-2.8.4-cp37-cp37m-win_amd64.whl
• ​http://download.osgeo.org/osgeo4w/osgeo4w-setup-x86_64.exe

For OSGeo4W I presume that you are using "Express Install" as instructed and not "Advanced mode". I don't know if that could cause any problems. There is also a problem that you have stumbled into that OSGeo4W installs a newer version of GDAL than Django 2.2 officially supports. This does make me wonder whether we'd be better off documenting installation of the actual dependencies of GeoDjango - that is GDAL, GEOS and Proj - although some of those just point back to OSGeo4W :(

We may also be bumping into issues with some packages compiled with MSVC and others with MinGW64. Do you have a specific error message or traceback, by the way?

Based on the above, I've updated this ticket to reflect the need for a wider review of the installation of dependencies for GeoDjango on Windows.

[Mariusz Felisiak]

Nick, I still don't see where is an issue, except that http://www.stickpeople.com/projects/python/win-psycopg/ link is outdated.

There is also a problem that you have stumbled into that OSGeo4W installs a newer version of GDAL than Django 2.2 officially supports

I don't think that is true, OSGeo4W contains: gdal200dll, gdal201dll, gdal202dll, gdal203dll all of them are supported by Django 2.2.

It can be hard to fix an issue without details.

[Nick Pope]

Replying to felixxm:

Nick, I still don't see where is an issue, except that http://www.stickpeople.com/projects/python/win-psycopg/ link is outdated.

That and referring to PostgreSQL 9.x seems overly restrictive. There haven't been a great many updates to this section of the documentation other than cosmetic in years and my impression from my comment above is that it could be made easier.

There is also a problem that you have stumbled into that OSGeo4W installs a newer version of GDAL than Django 2.2 officially supports

I don't think that is true, OSGeo4W contains: gdal200dll, gdal201dll, gdal202dll, gdal203dll all of them are supported by Django 2.2.

I was alluding to the comments starting at ​https://github.com/django/django/pull/10962#issuecomment-548004287, but if it installs the older versions too, then ...

It can be hard to fix an issue without details.

Indeed. I asked for some more details at the end...

Anyway. As I said, I don't use Windows. And I feel like I'm being unhelpful here, so I'll back off.

[Mariusz Felisiak]

Replying to Nick Pope:

Anyway. As I said, I don't use Windows. ...

Me neither, but it seems to me that without details we'll not be able to fix this ticket.

[Carlton Gibson]

OK, I will have a look at this, and see if I can step through it to find improvements. (I'll leave it unassigned in case anyone else wants to do that too.)

I would rather we address making this work for 64-bit.

Absolutely. We're just bottling up issues for later otherwise.

[Adam Starrh]

Hello,

When following the instructions in the documentation, only the latest version of the gdal dll is placed in the /bin/ directory via the OSGeo4Win installer. There are no older versions that are made available.

I don't have immediate access to the traces but the general result was that the utility that searches for the .dll was unable to find it, even when it was present. I should note that I identified two separate issues while setting this up.

The first was a repeat of an issue I encountered earlier:

https://code.djangoproject.com/ticket/28237

In this case, gdal204.dll was missing from Django's libgdal.py file and I had to enter it manually. However, this time, it continued to return the same GDALexception as before,
which led me to the second issue. I tried setting the GDAL_LIBRARY_PATH manually in the settings and when attempting to run the migrations, it reported: "this is not a valid win32 application". This finally tipped me off to the issue. I rolled python and OSGeo4Win back to the 32-bit versions, and that resolved it. This was the 4th or so time over the years that I have chosen 64-bit python in the installation process and had to go back and convert to the 32-bit. This is why I am suggesting an amendment to the instructions at this step in the process, as it creates hours of headaches each time.

Aside from these issues with GDAL, the instructions as-written work pretty well for getting casual coders like myself up and running quickly. The techs all work together nicely and having the process simple and straightforward is part of what I believe makes Django so accessible. The OSGeo4Win is certainly a convenient package that enables users to skip a lot of steps installing individual dependencies that they would have to endure otherwise. If a full revision is necessary, I'd hope that this is kept in mind.

64-bit Python isn't necessary for most users and creates a lot of these kinds of problems, so much so that Windows users seem to typically avoid using it. If someone has a good reason to opt for 64-bit Python, then hopefully they are prepared to handle these kinds of issues independently. The same goes for anyone with a preference toward another geospatial db library. Having more options included in the instructions makes the process more confusing for new users and more complicated to support.

The only other real trip-up is the outdated link. At one time psycopg2 wouldn't install via pip, and this was the preferred workaround. That is no longer an issue as the package will successfully install via pip, so long as the user has Visual Studio C++ installed (which should probably be mentioned).

I am away from the system in question until early next week. I can post the full tracebacks at that time and can also try the alternate gdal version suggested in the PR if it would still be helpful.

[David Smith]

​PR

[Carlton Gibson]

This will be fixed for Django 3.1, which has compatibility with GDAL 3 and 3.1 — These are automatically installed by the latest installers.

From ​the PR description:

One of the trip-ups is that OSGeo4W installs a specific version of GDAL (currently 3.0), unless we go the 'advanced' route. Therefore to support older versions (e.g. Django 3.0) we would need to explore this. I've included a screenshot of what we would need to navigate (this screen is about half a dozen "next"s in to the install process)

Given the imminence of Django 3.1, the difficulty of _describing_ the advanced steps, and the time this has been outstanding, we'll not try to resolve the issue for 3.0 and below with GDAL 2.x. Anyone hitting this will need to explore the advanced install if they're not able to update to Django 3.1.

[Carlton Gibson <carlton@…>]

In 7f4db2d8:

Fixed #30933 -- Updated instructions for installing GeoDjango on Windows.

[Carlton Gibson <carlton.gibson@…>]

In 928f6a90:

[3.1.x] Fixed #30933 -- Updated instructions for installing GeoDjango on Windows.

Backport of 7f4db2d82b1385d1026c441e25abd4cb19fcccc6 from master