New Proposed Architecture
From Intamap
Contents |
New Proposed Architecture
As you may or may not know I attended the Sensors Anywhere training in Muenster last week. Talking to various people there I learnt a lot about new (to me) OGC Web Services which sound ideal for the purposes of INTAMAP. During this meeting I drafted a new architecture based around these new services which I will try to explain below.
Architecture Overview
SOS - Sensor Observation Service SOS Specification
WPS - Web Processing Service WPS Specification
Sensor Observation Service
The idea behind the Sensor Observation Service is to provide a standardized interface to sensor data. While this service is not directly relevant to INTAMAP as a real time automatic mapping service it may benefit BfS and/or EURDEP. The general impression I got from the training was that this particular service (in conjunction with Sensor Alert Service & Sensor Planning Service) will become very popular as more institutions want to share their sensor data.
A Sensor Observation Service provides an API for managing deployed sensors and retrieving sensor data and specifically “observation” data. Whether from in-situ sensors (e.g., water monitoring) or dynamic sensors (e.g., satellite imaging), measurements made from sensor systems contribute most of the geospatial data by volume used in geospatial systems today. The general scenario for in-situ sensors is where deployed sensors of various types are grouped into several constellations that are then accessed through some service (e.g., SOS).
SOS has three mandatory “core” operations: GetObservation, DescribeSensor, and GetCapabilities. The GetObservation operation provides access to sensor observations and measurement data via a spatio-temporal query that can be filtered by phenomena. The DescribeSensor operation retrieves detailed information about the sensors making those measurements and the platforms that carry the sensors. The GetCapabilities operation provides the means to access SOS service metadata. Several optional, nonmandatory operations have also been defined. There are two operations to support transactions, RegisterSensor and InsertObservation, and six enhanced operations, including GetResult, GetFeatureOfInterest, GetFeatureOfInterestTime, DescribeFeatureOfInterest, DescribeObservationType, and DescribeResultModel. Used in conjunction with other OGC specifications, the SOS provides a broad range of interoperable capability for discovering, binding to and interrogating individual sensors, sensor platforms, or networked constellations of sensors in real-time, archived or simulated environments.
I won’t go into detail about the operations here (unless requested!) as the full specification can be found here. However, I included the Sensor Observation Service in my architecture diagram because I think that as they become more popular it would be extremely beneficial for INTAMAP to support them.
Web Processing Service
During the first prototype of INTAMAP (WFS) I noticed that it wasn’t a particularly elegant method of doing things. We had a cumbersome method of having to insert data via the Insert request – be assigned an ID value – and then call a GetFeature request with numerous filters on for a simple interpolation.
A Web Processing Service, I feel, offers a much better solution whereby you can ‘execute’ any number of processes on data. This data can be included in the execute request, or, if specified, located on the server.
A WPS can be configured to offer any sort of GIS functionality to clients across a network, including access to pre-programmed calculations and/or computation models that operate on spatially referenced data. A WPS may offer calculations as simple as subtracting one set of spatially referenced numbers from another (e.g., determining the difference in influenza cases between two different seasons), or as complicated as a global climate change model. The data required by the WPS can be delivered across a network, or available at the server.
This interface specification provides mechanisms to identify the spatially-referenced data required by the calculation, initiate the calculation, and manage the output from the calculation so that it can be accessed by the client. This Web Processing Service is targeted at processing both vector and raster data.
Operations
There are only 3 operations in a WPS these are: GetCapabilities, DescribeProcess and Execute. Below is a description of each operation.
GetCapabilites
Like all OGC Compliant Web Services, a WPS must have a GetCapabilities operation. This operation returns a complete description of the WPS including all 'processes' provided by the server. Each process process can describe its data inputs and outputs as well as 2 additional (useful?) properties. The storeSupported and statusSupported properties.
The storeSupported property, if true, means that all data outputs can be stored on the server for retrival via URI.
The statusSupported property means that a process can return immediately (without data). Then, via a URL, it can provide a status report of the process including percent complete. This is useful for processes that take a long time to complete.
DescribeProcess
This operation is similar to the DescribeFeature operation of a WFS. It is used when a client wishes to learn more about a particular operation mentioned in the GetCapabilities request. Below is an example response to a DescribeProcess request, taken from the specification.
<?xml version="1.0" encoding="UTF-8"?> <ProcessDescriptions xmlns="http://www.opengeospatial.net/wps" xmlns:wps="http://www.opengeospatial.net/wps" xmlns:ows="http://www.opengeospatial.net/ows" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.opengeospatial.net/wps ..\wpsDescribeProcess.xsd"> <ProcessDescription processVersion="2" storeSupported="true" statusSupported="false"> <ows:Identifier>Buffer</ows:Identifier> <ows:Title>Create a buffer around a polygon.</ows:Title> <ows:Abstract>Create a buffer around a single polygon. Accepts the polygon as GML and provides GML output for the buffered feature.</ows:Abstract> <ows:Metadata xlink:title="spatial" /> <ows:Metadata xlink:title="geometry" /> <ows:Metadata xlink:title="buffer" /> <ows:Metadata xlink:title="GML" /> <DataInputs> <Input> <ows:Identifier>InputPolygon</ows:Identifier> <ows:Title>Polygon to be buffered</ows:Title> <ows:Abstract>URI to a set of GML that describes the polygon.</ows:Abstract> <ComplexData defaultFormat="text/XML" defaultEncoding="base64" defaultSchema="http://foo.bar/gml/3.1.0/polygon.xsd"> <SupportedComplexData> <Format>text/XML</Format> <Encoding>UTF-8</Encoding> <Schema>http://foo.bar/gml/3.1.0/polygon.xsd</Schema> </SupportedComplexData> </ComplexData> <MinimumOccurs>1</MinimumOccurs> </Input> <Input> <ows:Identifier>BufferDistance</ows:Identifier> <ows:Title>Buffer Distance</ows:Title> <ows:Abstract>URI to a GML resource file</ows:Abstract> <LiteralData> <SupportedUOMs defaultUOM="meters"/> <ows:AnyValue/> </LiteralData> <MinimumOccurs>1</MinimumOccurs> </Input> </DataInputs> <ProcessOutputs> <Output> <ows:Identifier>BufferedPolygon</ows:Identifier> <ows:Title>Buffered Polygon</ows:Title> <ows:Abstract>GML stream describing the buffered polygon feature.</ows:Abstract> <ComplexOutput defaultFormat="text/XML" defaultEncoding="base64" defaultSchema="http://foo.bar/gml/3.1.0/polygon.xsd"> <SupportedComplexData> <Format>text/XML</Format> <Encoding>UTF-8</Encoding> <Schema>http://foo.bar/gml/3.1.0/polygon.xsd</Schema> </SupportedComplexData> </ComplexOutput> </Output> </ProcessOutputs> </ProcessDescription> </ProcessDescriptions>
This example process is a simple GIS buffer operation. The important parts of a DescribeProcess response are the DataInputs and ProcessOutputs properties.
DataInputs
The DataInputs property can contain 0 or more Input features.
Each Input feature can be 1 of 3 types: ComplexData, LiteralValue or BoundingBoxType.
A ComplexData type can contain any type of XML schema including GML. They can also contain a link to data stored on the WPS server, however, this isn't properly documented in the specification :(
A LiteralValue contains a literal value! These types could be useful for describing the resolution of the desired coverage perhaps?
A BoundingBoxType contains a bounding box - useful for specifying the bounds of a coverage.
ProcessOutputs
Like the DataInputs property, the ProcessOutputs property can contain 0 or more Output properties.
The Output properties are identical to the Input properties and can contain 1 of the 3 data types mentioned above. It is extremely likely that we will use the ComplexData type and embed a GML coverage in here, although this may change depending on the uncertainty schema.
Execute
The Execute operation is where it all happens. A client will call this operation for a specific process offered by the WPS according to the ProcessDescription for that particular process. The inputs can be embedded in the request or referenced locally on the server. However, the referencing of local datasets isn't documented in the specification apart from on page 'viii' where it says:
Version 0.1.0 included the ability to identify local datasets that could be used as inputs to a process. The current verion 0.4.0 allows a local payload to be used by using a "reference" URL that starts with the prefix "cid:", but this is not documented. The ability for a server to provide local datasets should be reinstated, or an alternative specified
Not very conclusive :(
However, the ability for a WPS to store data from a request will hugely benefit us if a client wishes to make numerous requests on a single dataset. Therefore, my idea was that when a user sends up data during a request we can store it and return an ID (starting with cid) back as one of the ProcessOutputs. This will enable the client to make future requests using that ID. However, they are not obliged to do so.
Some benefits
Why bother changing you might ask, well below I have listed a few benefits/ideas of the new architecture in an attempt to convince you.
The whole idea of this proposed architecture came about at the SWE training in Muenster when I listened to people talk about their sensor networks and how they were going to implement a SOS. Also, during the training there was a demonstration of a client for a SOS that showed a simple nearest-neighbour interpolation of the sensor data returned from a SOS. During this demonstration I noticed that it took a ridiculous amount of time (half a minute, maybe longer) to render and that it also generated a lot of interest.
So I started thinking that if the whole Sensor Web Enablement initiative took off then it would be a fantastic opportunity for INTAMAP if it was already ‘aware’ of these services, in particular Sensor Observation Services. One interesting feature would be to allow users of INTAMAP to embed information about a particular SOS within an interpolation request. The idea behind this is that as more users store their data in an SOS they won’t want to download their data then send it to us to be interpolated. So, as an extra feature, we can allow users to send information about an SOS to us and we can download the data from the SOS directly, thus saving transfer times.
During the meeting with Brian he raised a very interesting point from a client developer’s view. His idea was that a developer could write a client that provided various interpolated data (i.e. temperature, wind speed, rainfall etc) in a single client without ever handling any of data – it could all be done by referencing various SOS’s.
Another benefit of INTAMAP being a WPS is that it simplifies the process of actually using it. As it stands in our system a perspective client would have to read up about our system and discover the fact that you need to Insert data and remember an ID number and then request a particular feature via the GetFeature request. Only once he had understood this could he start developing his client. I know that it isn’t the most complex of systems, but the fact still remains that he would have to research how to use it. I think it is important that INTAMAP remains as simple to use as possible, this means that implementing it as a WPS will enable a user to read the Capabilities document and understand how to use it (providing we offer useful names for our processes).
