= Contents = The [http://code.djangoproject.com/browser/django/branches/gis gis] branch intends to add a contrib app allowing for Geographic-enabled fields and queries. * [GeoDjango#Background Background] * [GeoDjango#WhatsGIS What's GIS?] * [GeoDjango#UsefulCode Useful Code] * [GeoDjango#UsefulData Useful Data] * [GeoDjango#FAQ FAQ] * [GeoDjango#Implementation Implementation] * [GeoDjango#Phase1 Phase 1] * [GeoDjango#Phase2 Phase 2] * [GeoDjango#Phase3 Phase 3] * [GeoDjango#DesignIssues Design Issues] * [GeoDjango#Collaboration Collaboration] * [GeoDjango#Example Example] * [GeoDjango#GeographicModels Geographic Models] * [GeoDjango#Usingsyncdb Using syncdb] * [GeoDjango#SpatialQueries Spatial Queries] * [GeoDjango#Installation Installation] * [GeoDjango#Django GeoDjango Branch from SVN] ('''required''') * [GeoDjango#GEOS GEOS] ('''required''') * [GeoDjango#PROJ.4 PROJ.4] ('''required''') * [GeoDjango#PostGIS PostGIS] ('''required''') * [GeoDjango#GDAL GDAL] * [GeoDjango#ModelAPI Model API] * [GeoDjango#Fields Fields] * [GeoDjango#FieldKeywords Field Keywords] * [GeoDjango#CreatingandSavingModelswithGeometryFields Creating and Saving Models with Geometry Fields] * [GeoDjango#DatabaseAPI Database API] * [GeoDjango#PostGISOperatorFieldLookupTypes PostGIS Operator Field Lookup Types] * [GeoDjango#PostGISGEOSFunctionFieldLookupTypes PostGIS GEOS Function Field Lookup Types] * [GeoDjango#ExtraInstanceMethods Extra Instance Methods] = Background = '''Note:''' The content herein is a loosely structured collection of notes and links that we have found useful, not necessarily what will be supported in the future. == What's GIS? == * [http://cfis.savagexi.com/articles/category/gis Series of blog posts] giving intro to GIS; choice quote from [http://cfis.savagexi.com/articles/2006/04/20/on-coordinate-systems#comments an early post]: "If you feel like ending a conversation with a developer then simply bring up the topic of character encodings ... [o]r ... coordinate systems. ... So in the spirit of Tim Bray's and Joel Spolsky's wonderful writeups of character encodings, I thought I'd put together a basic survival guide to coordinate systems over my next few posts and then tie it back to Google Maps." * More on [http://en.wikipedia.org/wiki/Map_projection map projections], including why people can't agree on just one (utf-8). * [http://en.wikipedia.org/wiki/Geodesy geodesy] the field of science for this stuff. == Useful Code == * [http://postgis.refractions.net/ PostGIS], the [http://www.opengis.org/docs/99-049.pdf OpenGIS SQL Types (pdf)] implementation for Postgresql * [http://geos.refractions.net/ GEOS], low-level C++ port of [http://www.jump-project.org/project.php?PID=JTS Jave Topology Suite], used by PostGIS * [http://zcologia.com/news/14/python-geos-module/ PyGEOS, now outdated due to PCL, below], and shedloads more stuff on "python, geospatial, and the web" * [http://gispython.org/ Python Cartographic Library] - [http://trac.gispython.org/projects/PCL actively maintained], huge GIS package, GPL. * There are direct SWIG Python bindings in GEOS 3.0.0RC1 and above. In other words, the entire GEOS API may be called directly from Python. * [http://www.initd.org/tracker/psycopg/wiki/GeoTypes GeoTypes] is a type (and conversion) library for PostGIS via psycopg. * [http://exogen.case.edu/projects/geopy/ Geopy] * Calculates distances using (very accurate) [http://www.movable-type.co.uk/scripts/LatLongVincenty.html Vincenty], and uses the [http://en.wikipedia.org/wiki/World_Geodetic_System WGS 84] datum by default. * Has utility functions for unit of measurement (UOM) conversions (e.g. meters -> kilometers, kilometers -> miles, etc.) * Excellent GeoCoding capabilites. Has interfaces for Google, Yahoo, Microsoft Live, MediaWiki, and [http://www.geocoder.us/ Geocoder.us]. * [http://www.gdal.org/ GDAL/OGR], a library for fiddling with raster geo images. * Has a Python interface. A SWIG interface is in development, but not yet stable (no access to ''full'' API). * [http://shapelib.maptools.org/ shapelib] and [http://www.gdal.org/ogr/ogr2ogr.html ogr2ogr] are useful for ESRI shapefile manipulations. [http://www.esri.com/library/whitepapers/pdfs/shapefile.pdf ESRI shapefiles] are a lingua frana GIS format. * [http://search.cpan.org/~sderle/Geo-Coder-US/ Geo::Coder::US] An excellent ''Perl'' library for GeoCoding that powers [http://www.geocoder.us/ Geocoder.us]. Users can create their own Geographic databases using the Census Bureau's TIGER/Line data (see below). * [https://svn.greenpeace.org/projects/custard/browser/doc/geodata GeoRosetta], CC-BY-SA licensed, quality-controlled, collection of geocoding data. Not yet released to public(?). * [http://mapserver.gis.umn.edu/ MapServer]: University of Minnesota (UMN) "open source development environment for building spatially-enabled internet applications." * [http://www.mapnik.org/ MapNik]: C++ and Python toolkit for developing mapping applications. Claimed benefits over MapServer: "It uses the AGG library and offers world class anti-aliasing rendering with subpixel accuracy for geographic data. It is written from scratch in modern C++ and doesn't suffer from design decisions made a decade ago." ''See'' [http://www.mapnik.org/faq/ MapNik FAQ]. * Ruby on Rails * [http://www.ivygis.org/ IvyGIS]: Google-maps type displays with RoR and UMN's MapServer * [http://thepochisuperstarmegashow.com/ProjectsDoc/spatialadapter-doc/index.html Spatial Adapter for Rails]: A plugin for Rails which manages the MySql Spatial and PostGIS geometric columns in a transparent way (that is like the other base data type columns). This might have some useful techniques for when we try to support other spatial extensions other than PostGIS. * [http://cartographer.rubyforge.org/ Cartographer] GMaps plugin == Useful Data == * [http://www.census.gov/geo/www/tiger/tiger2006se/tgr2006se.html TIGER/Line]: "The TIGER/Line files are extracts of selected geographic and cartographic information from the [http://www.census.gov/ Census Bureau's] TIGER® (Topologically Integrated Geographic Encoding and Referencing) database." This data is useful in creating your own geocoding database service. Currently 2006 Second Edition is the latest. Note: The Census Bureau will be [http://www.census.gov/geo/www/tiger/tgrshp.html providing SHP files] in Fall, 2007. = FAQ = * Place your questions '''here'''. * '''Q:''' When dealing with points (say, degrees) from, do they need to be converted to be useful on the back-end data, assuming -that- data is in degrees? Is it enough to have the same datum and origin? (Reading the intro above is likely to answer the question.) * My (JDunck) reading indicates yes. Given the same coordinate system (i.e. datum, origin, and axes), degrees are useful without conversion. * '''Q:''' Can this implementation work with [http://dev.mysql.com/doc/refman/5.0/en/spatial-extensions.html MySQL spatial-extensions]. If not, it's planned? * No. It is (now) planned, ''see'' Phase 3 below. From the last time I (jbronn) checked, MySQL's spatial capabilities have improved. However, we're going to focus our efforts on PostGIS until things are worked out a bit more. As a spatial database PostGIS it is more standards compliant (OpenGIS consortium), more widely used, and has more features (e.g. coordinate transformation, {{{geometry_columns}}} and {{{spatial_ref_sys}}} tables). It is definitely something I would want to implement in the future since I do ''like'' MySQL. = Implementation = == Phase 1 == * Create Geometry-enabled fields and manager. Status: complete as of r4788. * Allow for Geometry-enabled queries. Status: complete as of r4788. == Phase 2 == * '''Pending''' * Add geometry-enabled routines to the fields that call directly on GEOS routines -- like area(), centroid(), etc. (partially complete as of r4884. ''See'' [GeoDjango#ExtraInstanceMethods Extra Instance Methods] section below) * Add as much from the PostGIS API as possible. * Support for a mapping framework (e.g. Google Maps/Earth, Yahoo Maps, MS Live, etc.) * Admin fields and forms (WKT field currently as of r4884, but we want widgets to view and manipulate geographic objects). * Utilities for importing raster data (SHP files first) directly into Django models. * '''Complete''' * PostGIS indexing capability == Phase 3 == * Support MySQL databases. == Design Issues == * Mapping JS framework -- do we want to support OpenLayers, the Google Maps API, the Yahoo API? * GEOS * GEOS is no longer maintained by Sean Gillies. ''See'' Sean's message on the [http://geos.refractions.net/pipermail/geos-devel/2007-March/002851.html GEOS-Devel Mailing List] (Mar. 5, 2007) * Might consider either using PCL or implement a {{{ctypes}}} wrapper for the routines that we need -- can't really port PCL code here because it is GPL (Django is licensed under BSD). * WMS Server * I'm not satisfied with any of the current WMS/WFS implementations. One implemented in Django would be desirable, e.g., {{{django.contrib.gis.wms}}}. Thoughts anyone? * MapNik is modern, but very early on in development and lacks documentation. == Collaboration == * PCL (Python Cartographic Library), now part of [http://www.gispython.org/ GIS Python], has done a lot of good work already. Let's apply the DRY principle. * Strong opportunities for collaboration with regards to: * Mapping framework * Utilities * Database representation ideas * WMS/WMF Framework * GEOS support, Sean Gilles (lead developer of PCL) looking for help maintaining Python/SWIG interface to GEOS. If SWIG interface no longer maintained, might have to move to PCL for up-to-date GEOS library support. = Example = == Geographic Models == Here is an example of how the model API currently works (assume this example is in geo_app/models.py): {{{ from django.contrib.gis.db import models class District(models.Model, models.GeoMixin): name = models.CharField(maxlength=35) num = models.IntegerField() poly = models.PolygonField() objects = models.GeoManager() class School(models.Model, models.GeoMixin): name = models.CharField(maxlength=35) point = models.PointField(index=True) objects = models.GeoManager() }}} '''Notes''': The {{{GeoMixin}}} class allows for [GeoDjango#ExtraInstanceMethods extra instance methods]. The {{{index}}} keyword is used to indicate that a GiST index be created for the School {{{PointField}}}s fields. == Using syncdb == Use the {{{manage.py}}} to invoke {{{syncdb}}} like you normally would: {{{ $ python manage.py sqlall geo_app BEGIN; CREATE TABLE "geo_app_school" ( "id" serial NOT NULL PRIMARY KEY, "name" varchar(35) NOT NULL ); CREATE TABLE "geo_app_district" ( "id" serial NOT NULL PRIMARY KEY, "name" varchar(35) NOT NULL, "num" integer NOT NULL ); SELECT AddGeometryColumn('geo_app_school', 'point', 4326, 'POINT', 2); CREATE INDEX "geo_app_school_point_id" ON "geo_app_school" USING GIST ( "point" GIST_GEOMETRY_OPS ); SELECT AddGeometryColumn('geo_app_district', 'poly', 4326, 'MULTIPOLYGON', 2); COMMIT; $ python manage.py syncdb geo_app }}} '''Note:''' The geometry columns are created outside of the {{{CREATE TABLE}}} statements by the {{{AddGeometryColumn}}}. This is done according to the OpenGIS specfication. ''See'' ''See'' Open GIS Consortium, Inc., ''[http://www.opengis.org/docs/99-049.pdf OpenGIS Simple Feature Specification For SQL]'', Document 99-049 (May 5, 1999), at Ch. 2.3.8 (Geometry Values and Spatial Reference Systems, pg. 39). == Spatial Queries == After a geographic model has been created, the PostGIS additions to the API may be used. Geographic queries are done by normally by using {{{filter()}}} and {{{exclude()}}} on geometry-enabled models using geographic lookup types (''see'' the [GeoDjango#DatabaseAPI Database API] below for lookup types). In the following example, the {{{bbcontains}}} lookup type is used which is the same as the PostGIS {{{&&}}} operator. It looks to see if the ''bounding box'' of the polygon contains the specific point. The next example uses the PostGIS {{{Contains()}}} function, which calls GEOS library to test if the ''polygon'' actually contains the specific point, not just the bounding box. {{{ >>> from geo_app.models import District, School >>> qs1 = District.objects.filter(poly__bbcontains='POINT(-95.362293 29.756539)') >>> qs2 = District.objects.filter(poly__contains='POINT(-95.362293 29.756539)') }}} Both spatial queries and normal queries using {{{filter()}}} may be used in the same query. For example, the following query set will only show school districts that have 'Houston' in their name and contain the given point within their polygon boundary: {{{ >>> qs = District.objects.filter(name__contains='Houston').filter(poly__contains='POINT(-95.362293 29.756539)') }}} Or combine both the bounding box routines (less accurate, fast) with the GEOS routines (most accurate, slower) to get a query that is both fast and accurate: {{{ >>> qs = District.objects.filter(poly__bbcontains='POINT(-95.362293 29.756539)').filter(poly__contains='POINT(-95.362293 29.756539)') }}} = Installation = Installation of the GeoDjango module will also require the installation of existing open source geographic libraries and a spatial database (currently only PostGIS). This section will describe the installation process for these libraries. Initially, these instructions will pertain only to a Linux platform (particularly Debian or Ubuntu). Mac & Windows support will be considered later; however, these instructions will most likely work through the Mac shell. Don't hold your breath for Windows support. == Django == * GeoDjango exists in the {{{gis}}} branch from SVN: {{{ $ svn co http://code.djangoproject.com/svn/django/branches/gis django_gis $ ln -s django_gis /path/to/site-packages/django }}} == GEOS == * Latest [http://geos.refractions.net/ GEOS] version is 3.0.0RC4 * Also requires SWIG >= 1.3.28. (Ubuntu Dapper comes with 1.3.27.) * If there's trouble locating your python, include PYTHON=/path/to/your/python. {{{ $ ./configure --enable-python $ make # make install }}} == PROJ.4 == * Latest [http://proj.maptools.org/ PROJ.4] version is 4.5.0 {{{ $ ./configure $ make # make install }}} * Should install datum shift files (for funky local coordinate systems) -- but I need to personally figure out how to do that first. == PostGIS == * Latest [http://postgis.refractions.net/download/ PostGIS] version is 1.2.1 * First build & install PostGIS. We are currently using v8.1 of PostgreSQL. {{{ $ ./configure --with-geos --with-proj $ make # make install }}} * Next, create a role and database for your application, and allow it to access PostGIS functionality: {{{ # su - postgres $ psql postgres=# CREATE ROLE LOGIN; postgres=# \q $ createdb -O $ createlang plpgsql $ psql -d -f /usr/local/share/lwpostgis.sql $ psql -d -f /usr/local/share/spatial_ref_sys.sql $ psql =# GRANT SELECT, UPDATE, INSERT, DELETE ON geometry_columns TO ; =# GRANT SELECT ON spatial_ref_sys TO ; }}} * Finally, update your {{{settings.py}}} to reflect the name and user for the spatially enabled database. So far, we only plan to support the psycopg2 backend, thus: {{{DATABASE_ENGINE='postgresql_psycopg2'}}}. == GDAL == * Optional, but highly useful for coordinate transformations and reading/writing ''both'' vector (e.g. SHP) and raster (e.g. TIFF) geographic data. * Latest [http://www.gdal.org/download.html GDAL] version is 1.4.0. Configure with GEOS and Python support, then make and install: {{{ $ ./configure --with-geos --with-python $ make # make install }}} * ''Note'': This is done without the 'next generation' SWIG Python bindings. I've had trouble getting them to work, and the rumor is this only works on Windows. The compilation flag to enable these is {{{--with-ngpython}}}, but our packages currently only use the old bindings. = Model API = == Fields == The following geometry-enabled fields are available: * {{{PointField}}} * {{{LineStringField}}} * {{{PolygonField}}} * {{{MultiPointField}}} * {{{MultiLineStringField}}} * {{{MultiPolygonField}}} * {{{GeometryCollectionField}}} == Field Keywords == * Field keywords are used during model creation, for example: {{{ from django.contrib.gis.db import models class Zip(models.Model, models.GeoMixin): code = models.IntegerField() poly = models.PolygonField(srid=-1, index=True) object = models.GeoManager() }}} * {{{srid}}} * Sets the SRID (Spatial Reference System Identity) of geometry to the given value. Defaults to 4326 (WGS84). ''See'' Open GIS Consortium, Inc., ''[http://www.opengis.org/docs/99-049.pdf OpenGIS Simple Feature Specification For SQL]'', Document 99-049 (May 5, 1999), at Ch. 3.2.5 (SQL Textual Representation of Geometry, pg. 53) * {{{index}}} * If set to True, will create a GiST index for the given geometry. Update the index with the PostgreSQL command {{{VACUUM ANALYZE}}} (may take a while to execute depending on how large your geographic-enabled tables are). == Creating and Saving Models with Geometry Fields == Here is an example of how to create a geometry object (assuming the {{{Zip}}} model example above): {{{ >>> from zipcode.models import Zip >>> z = Zip(code=77096, poly='POLYGON(( 10 10, 10 20, 20 20, 20 15, 10 10))') >>> z.save() }}} Geometries are represented as '''strings''' in either of the formats WKT (Well Known Text) or HEXEWKB (PostGIS specific, essentially a WKB geometry in hexadecimal). For example: * WKT Polygon: {{{'POLYGON(( 10 10, 10 20, 20 20, 20 15, 10 10))'}}} * ''See'' Open GIS Consortium, Inc., ''[http://www.opengis.org/docs/99-049.pdf OpenGIS Simple Feature Specification For SQL]'', Document 99-049 (May 5, 1999), at Ch. 3.2.5 (SQL Textual Representation of Geometry, pg. 53). * HEXEWKB Polygon: '{{{0103000000010000000 ... 00000000000002440'}}} * ''See'' [http://postgis.refractions.net/docs/ch04.html#id2904792 "PostGIS EWKB, EWKT and Canonical Forms"], PostGIS documentation at Ch. 4.1.2. = Database API = '''Note:''' The following database lookup types can only be used with on geographic fields with {{{filter()}}}. Filters on 'normal' fields (e.g. {{{CharField}}}) may be chained with those on geographic fields. Thus, geographic queries take the following form (assuming the {{{Zip}}} model used in the [GeoDjango#ModelAPI Model API] section above): {{{ >>> qs = Zip.objects.filter(__=) >>> qs = Zip.objects.exclude(...) }}} == PostGIS Operator Field Lookup Types == * ''See generally'', [http://postgis.refractions.net/docs/ch06.html#id2854381 "Operators", PostGIS Documentation at Ch. 6.2.2] * '''Note:''' This API is subject to some change -- we're open to suggestions. * {{{overlaps_left}}} * Returns true if A's bounding box overlaps or is to the left of B's bounding box. * PostGIS equivalent "{{{&<}}}" * {{{overlaps_right}}} * Returns true if A's bounding box overlaps or is to the right of B's bounding box. * PostGIS equivalent "{{{&>}}}" * {{{left}}} * Returns true if A's bounding box is strictly to the left of B's bounding box. * PostGIS equivalent "{{{<<}}}" * {{{right}}} * Returns true if A's bounding box is strictly to the right of B's bounding box. * PostGIS equivalent "{{{>>}}}" * {{{overlaps_below}}} * Returns true if A's bounding box overlaps or is below B's bounding box. * PostGIS equivalent "{{{&<|}}}" * {{{overlaps_above}}} * Returns true if A's bounding box overlaps or is above B's bounding box. * PostGIS equivalent "{{{|&>}}}" * {{{strictly_below}}} * Returns true if A's bounding box is strictly below B's bounding box. * PostGIS equivalent "{{{<<|}}}" * {{{strictly_above}}} * Returns true if A's bounding box is strictly above B's bounding box. * PostGIS equivalent "{{{|>>}}}" * {{{same_as}}} * The "same as" operator. It tests actual geometric equality of two features. So if A and B are the same feature, vertex-by-vertex, the operator returns true. * PostGIS equivalent "{{{~=}}}" * {{{contained}}} * Returns true if A's bounding box is completely contained by B's bounding box. * PostGIS equivalent "{{{@}}}" * {{{bbcontains}}} * Returns true if A's bounding box completely contains B's bounding box. * PostGIS equivalent "{{{~}}}" * {{{bboverlaps}}} * Returns true if A's bounding box overlaps B's bounding box. * PostGIS equivalent "{{{&&}}}" == PostGIS GEOS Function Field Lookup Types == * ''See generally'' [http://postgis.refractions.net/docs/ch06.html#id2615853 "Geometry Relationship Functions", PostGIS Documentation at Ch. 6.1.2]. * This documentation will be updated completely with the content from the aforementioned PostGIS docs. * {{{distance}}} * Return the cartesian distance between two geometries in projected units. * PostGIS equivalent {{{Distance(geometry, geometry)}}} * {{{equals}}} * Requires GEOS * Returns 1 (TRUE) if the given Geometries are "spatially equal". * Use this for a 'better' answer than '='. equals('LINESTRING(0 0, 10 10)','LINESTRING(0 0, 5 5, 10 10)') is true. * PostGIS equivalent {{{Equals(geometry, geometry)}}}, OGC SPEC s2.1.1.2 * {{{disjoint}}} * Requires GEOS * Returns 1 (TRUE) if the Geometries are "spatially disjoint". * PostGIS equivalent {{{Disjoint(geometry, geometry)}}} * {{{intersects}}} * PostGIS equivalent {{{Intersects(geometry, geometry)}}} * {{{touches}}} * PostGIS equivalent {{{Touches(geometry, geometry)}}} * {{{crosses}}} * PostGIS equivalent {{{Crosses(geometry, geometry)}}} * {{{overlaps}}} * PostGIS equivalent {{{Overlaps(geometry, geometry)}}} * {{{contains}}} * PostGIS equivalent {{{Contains(geometry, geometry)}}} * {{{intersects}}} * PostGIS equivalent {{{Intersects(geometry, geometry)}}} * {{{relate}}} * PostGIS equivelent {{{Relate(geometry, geometry)}}} == Extra Instance Methods == A model with geometry fields will get the following methods: == get_GEOM_wkt == For every geometry field, the model object will have a {{{get_GEOM_wkt}}} method, where {{{GEOM}}} is the name of the geometry field. For example (using the {{{School}}} model from above): {{{ >>> skool = School.objects.get(name='PSAS') >>> print skool.get_point_wkt() POINT(-95.460822 29.745463) }}} == get_GEOM_centroid == For every geometry field, the model object will have a {{{get_GEOM_centroid}}} method, where {{{GEOM}}} is the name of the geometry field. This routine will return the centroid of the geometry. For example (using the {{{District}}} model from above): {{{ >>> dist = District.objects.get(name='Houston ISD') >>> print dist.get_poly_centroid() POINT(-95.231713 29.723235) }}} == get_GEOM_area == For every geometry field, the model object will have a {{{get_GEOM_area}}} method, where {{{GEOM}}} is the name of the geometry field. This routine will return the area of the geometry. {{{ >>> dist = District.objects.get(name='Houston ISD') >>> print dist.get_poly_area() 0.08332 }}} '''Note''': The units system needs to be figured out here, since I don't know what these units represent.