Talk:OGC Standards
From Intamap
INTAMAP Discussion
This section can be used by the INTAMAP partners to discuss the OGC standards.
--Matt Williams 14:13, 19 October 2006 (CET)
I have spent ages discussing with various people and thinking about which of the various standards we should use (if any). It is my personal feeling that the WFS best suites our needs for a couple of reasons:
- The WFS is the only OGC WS that has a method to allow a user to upload data - an integral part of our system!
- A WMS doesn't really provide a method for describing the Features in great detail so may be too restricted when it comes to describing the uncertainty.
The WCS sounds like an ideal interface for a lot of our needs but the lack of an Insert method is somewhat annoying!
Therefore I think that our system will mainly be a WFS-T which allows the user to insert data encoded as GML Observations and then also request grids etc via the GetFeature method (which will use automatic interpolation to derive the Feature). However, we might also tag a WMS and WCS on to our system so that a user can call the GetMap operation and recieve an image rather than a whole load of GML or use the GetCoverage operation of a WCS. Allowing users to do this will mean that we have to ensure that some data is present before they can call the GetMap or GetCoverage operations.
--Edzer Pebesma 20:13, 31 October 2006 (CET)
I have very limited experience (if any) with GML, but a lot with rather large grids. Grids for the EU on a 1km x 1km basis have 3300 x 4000 cells, say 13 megapixels. What follows is a sample session in R to generate a 100 x 100 grid with standard normal attributes:
> library(sp) > out = data.frame(expand.grid(x=1:100,y=1:100),z=rnorm(10000)) > gridded(out)=~x+y > out.pol = as.SpatialPolygons.SpatialPixels(out) > out.d = SpatialPolygonsDataFrame(out.pol, out@data,match.ID=F) > object.size(out.d) # size of object in memory, in bytes [1] 20961792 > object.size(out) [1] 283212 > library(maptools) Loading required package: foreign > writePolyShape(out.d, "out") # writes out.shp, out.dbf etc > library(rgdal) Geospatial Data Abstraction Library extensions to R successfully loaded > writeGDAL(out, "out.tif") # writes grid as a GeoTIFF file
Then I converted the shapefile with
ogr2ogr -f GML out.gml out.shp
which creates only GML2.0. The interesting output file sizes are:
-rw-r--r-- 1 edzer edzer 3974171 Oct 31 20:35 out.gml -rw-r--r-- 1 edzer edzer 40282 Oct 31 20:35 out.tif
Say Europe is 50% sea, then from 10.000 cells to 7.000.000 is a factor 700, and the GML 2.0 file would become 2.8 Gb. The GeoTIFF file (4 bytes float) would still be 13x4 = 52Mb, btw. In my 100x100 example, GeoTIFF:GML sizes are 1:100, if we have a lot of sea it may be somewhat less severe.
Should we consider the alternative to have WFS-T to receive measurement data, and WCS to serve the results? I don't like it, but performance will be an issue in our real-time service.
--Dan Cornford 21:13, 1 November 2006 (CET)
It is true that GML is very verbose; this is in some ways a strength, and in some ways a weakness! There are developing standards for compression of XML, which we might like to explore, but I fundamentally agree that we need to look carefully at what we communicate from our WFS. For sure I forsee a WCS also being implemented, and this makes perfect sense - and fits well within our proposed architecture; it might well be that we modify this somewhat to directly connect the WCS to the R interpolation engine.
The way that I envisage the overall system working is that requests for 7 million values will be relatively rare? I think we need to discuss this more. I had envisaged users requesting information in detail only for small areas; providing uncertainty estimates for very large grids (even means for such grids) will need us to look very carefully at these issues. I think we need to be pragmatic, and offer as much flexibility as possible in our WFS, WMS and WCS.
--Matt Williams 09:49, 15 February 2007 (CET)
The file size for a GML 3.1.1 grid of size 100x100 is in fact a mere 350kb and this is before compression. I don't feel that the size of the data will cause any problems. Predicting over a grid of 3300x4000 would produce a GML3.1 file of approximately 245MB - this is of course unzipped and can be greatly reduced (although this will take some time).
