id	summary	reporter	owner	description	type	status	component	version	severity	resolution	keywords	cc	stage	has_patch	needs_docs	needs_tests	needs_better_patch	easy	ui_ux
30489	Django RasterField deserialization bug with pixeltype flags	Ivor Bosloper	Ivor Bosloper	"After inserting some raster data with raster2pgsql into a Django model table with a RasterField column, I get a `list index out of range` when querying the table with a Django Queryset. 

{{{
...
File ""django/contrib/gis/db/models/fields.py"" in from_db_value
  360.         return connection.ops.parse_raster(value)
File ""django/contrib/gis/db/backends/postgis/operations.py"" in parse_raster
  369.         return from_pgraster(value)
File ""django/contrib/gis/db/backends/postgis/pgraster.py"" in from_pgraster
  57.         pixeltype = POSTGIS_TO_GDAL[pixeltype]
}}}

It turns out the `pixeltype` value used is 39 while the `POSTGIS_TO_GDAL` list is only 16 elements long. The database field contains valid data but can not be deserialized with Django. 

**Steps for reproduction:**

{{{
# Django model
class RasterModel(models.Model):
    rast = models.RasterField(srid=4326)

# raw sql, single pixel raster with nodata bit set
insert into app_rastermodel values(1, REPLACE('01 0000 0100 0000000000000000 0000000000000000 0000000000000000 0000000000000000 0000000000000000 0000000000000000 E6100000 0100 0100 6 2 03 03', ' ', '')::raster);

# query generating Exception
RasterModel.objects.get(pk=1)
}}}


**Analysis**: if we look at the [https://trac.osgeo.org/postgis/wiki/WKTRaster/RFC/RFC1_V0SerialFormat Raster specification], the pixeltype is a byte of which the 4 highest bits are flags and the lowest 4 bits are the real pixeltype. Quoting the specification:

{{{
 Pixel type and storage flag
 ---------------------------

 Pixel type specifies type of pixel values in a band.
 Storage flag specifies whether the band data is stored
 as part of the datum or is to be found on the server's
 filesytem.

 There are currently 11 supported pixel value types, so 4
 bits are enough to account for all. We'll reserve
 the upper 4 bits for generic flags and define upmost as
 storage flag:
 
 #define BANDTYPE_FLAGS_MASK 0xF0
 #define BANDTYPE_PIXTYPE_MASK 0x0F

 #define BANDTYPE_FLAG_OFFDB     (1<<7)
 #define BANDTYPE_FLAG_HASNODATA (1<<6)
 #define BANDTYPE_FLAG_ISNODATA  (1<<5)
 #define BANDTYPE_FLAG_RESERVED3 (1<<4)
}}}

However, Django deserialization code only considers a single flag (`BANDTYPE_FLAG_HASNODATA`, bit 6, value 64):

{{{
# django/contrib/gis/db/backends/postgis/pgraster.py
def from_pgraster(data):
        ...
        # Subtract nodata byte from band nodata value if it exists
        has_nodata = pixeltype >= 64
        if has_nodata:
            pixeltype -= 64
       ...
}}}

The erroneous pixeltype 39 in my example actually had the `BANDTYPE_FLAG_ISNODATA` (bit 5, value 32) bit set which indicates all rastervalues are nodata.

I have created (my first django) patch and hope somebody can assist me in getting it correct and merged."	Bug	closed	GIS	dev	Normal	fixed	RasterField	Hasan Ramezani	Ready for checkin	1	0	0	0	0	0