Habitats deliverable 4.4.1

European CommissionInformation Society and Media DELIVERABLEProject Acronym: HABITATSGrant Agreement number: CIP- ICT-PSP-2009-3-250455Project Title: Social Validation of INSPIRE Annex III Data Structures in EU Habitats D-4.4.1 HABITATS Service ToolkitDocument identifier: D4-4-1-HABITATS service toolkitDate: 24.02.2012Nature: PDissemination level: PUWP Lead Partner: Graz University of TechnologyRevision Final Project co-funded by the European Commission within the ICT Policy Support Programme Dissemination LevelPU Public XRE Restricted to a group specified by the consortium (including the Commission Services)CO Confidential, only for members of the consortium (including the Commission Services)This project is partially funded under the ICT Policy Support Programme as part of the Competitiveness andInnovation Framework Programme by the European Commission http://ec.europa.eu/ict_pspThis document reflects only the authors views and the European Community is not liable for any use that mightbe made of the information contained herein. © HABITATS Consortium, 2010

Date: 24/02/2012 HABITATS service toolkitDoc. Identifier: D4-4-1Project: Social Validation of INSPIRE Annex III Data Structures in EU Habituas Abstract: This deliverable presents the background of invoking services and their relevance to the HABITATS projects and examines the basic networking architecture and specific tools that are considered. The focus of this intermediate report is on the application of these aspects within the Reference Laboratory while the final report (D4.4.2) will also include the invoking of services at the level of the different pilots. Key Words: Invoke, service toolkit, architecture design, workflow management, Reference LaboratoryAuthors: (in alphabetical order) Raitis Berzins Peteris Bruns Jachym Cepicky Karel Charvat (HSRS) Patrick Höfler (TUG) Lisa Maurer (TUG) Michal Sredl Martin Vlk Premysl Vohnout Statement of originality: This deliverable contains original unpublished work except where clearly indicated otherwise. Acknowledgement of previously published material and of the work of others has been made through appropriate citation, quotation or both.02/04/2012 -2-

Date: 24/02/2012 HABITATS service toolkitDoc. Identifier: D4-4-1Project: Social Validation of INSPIRE Annex III Data Structures in EU HabituasRevision History Revision Date Organization Description 01 28.11.2011 TUG First structure, document focus 02 10.12.2011 HRSR Additions, invoking services, extensions 03 05.01.2012 TUG Additions, comments 04 01.02.2012 HRSR Comments, additions 05 08.02.2012 NSI Comments and suggestions 06 9.2.2012 TUG Editing 07 9.2.2012 MAC Editing suggestions. 08 14.2.2012 HSRS WMS coordinate transformation added, small comments provided, answer on Ana comments 09 20.2.2012 TUG Summary, final editing02/04/2012 -3-

Date: 24/02/2012 HABITATS service toolkitDoc. Identifier: D4-4-1Project: Social Validation of INSPIRE Annex III Data Structures in EU Habituas Project Officer: Krister Olson European Commission DG Information Society and Media Project Officer Address: DG INFSO – E06 Office: EUFO – 01/177 L – 2920 LUXEMBOURG Phone: +(352) 43 0134332 Fax: +(352) E-mail: Krister.olson@ec.europa.eu Project Manager: Mariano Navarro de la Cruz Address: C/ Julian Camarillo, 6b, 28037, Madrid, Spain Phone: + 34 91 322 65 21 Fax: + 34 91 322 63 23 E-mail: mnc@tragsa.es02/04/2012 -4-

Date: 24/02/2012 HABITATS service toolkitDoc. Identifier: D4-4-1Project: Social Validation of INSPIRE Annex III Data Structures in EU Habituas TABLE OF CONTENTS1. INTRODUCTION ......................................................................................................................................... - 3 - 1.1. OBJECTIVE AND CONTEXT OF THIS DOCUMENT ........................................................................................ - 3 - 1.2. ORGANISATION OF THE DOCUMENT.......................................................................................................... - 3 -2. INVOKING SERVICES ............................................................................................................................... - 4 - 2.1. INVOKING SERVICE REQUIREMENTS AND RECOMMENDATIONS ............................................................. - 5 -3. HABITATS NETWORKING ARCHITECTURE AND REFERENCE LABORATORY .................... - 7 - 3.1. HABITATS NETWORKING ARCHITECTURE................................................................................................. - 7 - 3.2. REFERENCE LAB ....................................................................................................................................... - 8 -4. HABITATS INVOKING SERVICES ....................................................................................................... - 10 - 4.1. INVOKING OF DISCOVERY SERVICES........................................................................................................ - 10 - 4.2. INVOKING OF VISUALISATION SERVICES ................................................................................................. - 11 - 4.2.1. HSLayers 3.3 .................................................................................................................................. - 11 - 4.2.2. Invoking of WFS and WCS ............................................................................................................. - 17 - 4.2.3. Filter Encoding Filtering WFS Layers ........................................................................................... - 21 - 4.3. WPS INVOKING ...................................................................................................................................... - 26 - 4.4. HSLAYERS SOS CLIENT ......................................................................................................................... - 29 - 4.5. HSLAYERS EMBED COMPONENT ............................................................................................................ - 30 -5. PROCESSING WORKFLOW MANAGEMENT.................................................................................... - 33 - 5.1. ORCHESTRATION ENVIRONMENT ............................................................................................................ - 33 - 5.1.1. Workflow Management System ....................................................................................................... - 33 - 5.1.2. Business Process Execution Language........................................................................................... - 33 - 5.2. ENGINES AND WORK-FLOW MANAGERS .................................................................................................. - 33 - 5.2.1. Apache ODE ................................................................................................................................... - 34 - 5.2.2. Orchestra ........................................................................................................................................ - 34 - 5.2.3. Taverna Server ............................................................................................................................... - 35 - 5.3. WORKFLOW DESIGNERS.......................................................................................................................... - 36 - 5.3.1. 52°North WPS Workflow Modeller and Orchestration API ........................................................... - 36 - 5.4. ECLIPSE BPEL ..................................................................................................................................... - 37 - 5.4.1. HUMBOLDT Workflow Design and Construction Service ............................................................ - 38 - 5.4.2. Taverna Workbench........................................................................................................................ - 40 -6. CONCLUSION............................................................................................................................................ - 43 -7. REFERENCES ............................................................................................................................................ - 44 -02/04/2012 -2-

Date: 24/02/2012 HABITATS service toolkitDoc. Identifier: D4-4-1Project: Social Validation of INSPIRE Annex III Data Structures in EU Habituas1. INTRODUCTION1.1. OBJECTIVE AND CONTEXT OF THIS DOCUMENTThe aim of WP4 (Network service architecture) is to define SDI network services to enable trans-European sharing of habitats-related spatial data between public authorities and other stakeholders inthe community (HABITATS DoW 2010) and sharing of such data across regions and across countries.This work builds on the data modelling of WP3 and also refers to the user requirements and scenariosdeveloped in WP2 and in cooperation with the pilots of WP5. The prototype versions of the singleservices are on the other hand again relevant for the developments in the social validation process andwill therefore provide valuable feedback.The development of the network service architecture process of WP4 is initiated through a state of theart analysis of existing SDI, to find out more about already existing infrastructures and to examinehow data should be shared and what services are required to enable sharing. When designing thenetworking architecture, a set of specific networking service applets was deployed and tested for datasharing within the project. Also the potential for re-use of existing application was taken into account.The Task 4.4 deals with the tools for invoking of Geospatial Services that arise within the HABITATSnetwork architecture, interlinking different data sources and also interlinking data sources fromdifferent INSPIRE thematic areas. This is first version of the document, the update and final versionwill be released in Month 27.1.2. ORGANISATION OF THE DOCUMENTThe document is divided into five chapters” • Introduction describing general problem of invoking services and implementation of invoking services in INSPIRE (Chapter 2) • Basic description of Habitats Networking Architecture and Habitats Reference Laboratory (Chapter 3) • Principles of basic services that are invoked in reference laboratory (Chapter 4) • Analysis and Processing of Workflow Management for Spatial Data (Chapter 5) • Conclusion and tasks for next period of development inside of Habitats (Chapter 6)02/04/2012 -3-

Date: 24/02/2012 HABITATS service toolkitDoc. Identifier: D4-4-1Project: Social Validation of INSPIRE Annex III Data Structures in EU Habituas2. INVOKING SERVICESThe definition of spatial data services included in the INSPIRE directive1 is the following: ‘spatial dataservices’ means the operations which may be performed, by invoking a computer application, on thespatial data contained in spatial data sets or on the related metadata. ISO 19119 define also taxonomyfor Geospatial services2: • Geographic human interaction services • Geographic model/information management services • Geographic workflow/task management services • Geographic processing services o Geographic processing services – spatial o Geographic processing services – thematic o Geographic processing services – temporal o Geographic processing services – metadata • Geographic communication services • Geographic system management services3From INSPIRE Networking architecture, there are basic Networking services 1. Discovery Service (discovery): Is a service that makes it possible to search for spatial data sets and services on the basis of the content of the corresponding metadata and to display the content of the metadata. 2. View Service (view): Is a service that makes it possible, as a minimum, to display, navigate, zoom in and out, pan or overlay viewable spatial data sets and to display legend information and any relevant content of metadata. 3. Download Service (download): Is a service that enables copies of spatial data sets, or parts of such sets, to be downloaded and, where practicable, accessed directly. 4. Transformation Service (transformation): Is a service that enables spatial data sets to be transformed with a view to achieving interoperability. 5. Invoke Spatial Data Service (invoke): Is a service that allows defining both the data inputs and data outputs expected by the spatial service and a workflow point of view 4The INSPIRE Spatial Data Service and Invoke Service – Draft, implements rules defining that Invokeservice has to be accessible via Internet and offers a mean to invoke the linked spatial data services.Invoke shall support in order to allow clients invoking spatial data services. Taking into account thepotentially wide diversity of interfaces and protocols, invoke services are services that allow access tosufficient service metadata to enable the activation or execution of the spatial data service. Thedocument updated the basic INSPIRE architecture scheme and defined sets of requirements forINSPIRE Invoking services.1 Commission of the European Communities, Directive 2007/2/EC of the European Parliament and ofthe Council of 14 March 2007 establishing an Infrastructure for Spatial Information in the EuropeanCommunity (INSPIRE). 2007, Commission of the European Communities: Brussels.2 INSPIRE Invoke Services, Roberto Lucchi, Michel Millot, European Commission, Joint ResearchCentre, Institute for Environment and Sustainability3 HABITATS, CIP- ICT-PSP-2009-3-250455, Social Validation of INSPIRE Annex III DataStructures in EU Habitats, D-3.2a – HABITATS - 2504554 INSPIRE Spatial Data Services and Invoke Service – Draft, implementing rules,Draft_IR_SDS_and_Invoke_1.0.doc02/04/2012 -4-

Date: 24/02/2012 HABITATS service toolkitDoc. Identifier: D4-4-1Project: Social Validation of INSPIRE Annex III Data Structures in EU HabituasThe requirements are divided into two groups of requirements: • IR Requirement - Are requirements that are reflected in the Implementing Rule on interoperability of spatial data sets and services are shown using this style. •SDS Requirement - Requirements that are not reflected in the Implementing Rule on interoperability of spatial data sets and services are shown using this style.Document INSPIRE Spatial Data Services and Invoke Service define also set of recommendation2.1. INVOKING SERVICE REQUIREMENTS AND RECOMMENDATIONS • IR Requirement 1 The implementing rules are restricted to spatial data services that relate to spatial data sets in themes in Annex I-III, or to their related metadata. • Recommendation 1 There shall be no other requirements applicable to ALL spatial data services than the establishment of discovery metadata. • Recommendation 2 A spatial data service in this context shall have clearly defined interfaces for machine-to-machine communication. A Geographic Information System or other systems, understood as a set of tools for collecting, processing and storing spatial data, should not be considered an invokable spatial data service from the perspective of the relevant Implementing Rules. But any specific functionality included in it, and with a well-defined and exposed interface, could be an invokable spatial data service. • IR Requirement 2 Interoperability arrangements in the INSPIRE context shall be related to invokable spatial data services. • IR Requirement 3 Requirements for interoperability arrangements are only mandatory for spatial data services operating upon harmonised data (i.e. spatial data sets conformant to the regulation for IDSS). • IR Requirement 4 A spatial data service conformant to interoperability arrangement shall support coordinate reference systems according to Annex II.1 of the Commission Regulation (EC) No 1089/2010 . • IR Requirement 5 The default temporal reference system referred to in point 5 of part B of the Annex to Commission Regulation (EC) No 1205/2008 shall be used, unless other temporal reference systems are specified for a specific spatial data theme in Annex I-III. • IR Requirement 6 A spatial data service conformant to the interoperability arrangement shall be available 99% of time. • IR Requirement 7 A spatial data service conforming to interoperability arrangement returning spatial objects as part of the output, shall encode those spatial objects according to Article 7 of Commission Regulation (EU) No 1089/2010 of 23 November 2010 implementing02/04/2012 -5-

Date: 24/02/2012 HABITATS service toolkitDoc. Identifier: D4-4-1Project: Social Validation of INSPIRE Annex III Data Structures in EU Habituas Directive 2007/2/EC of the European Parliament and of the Council as regards interoperability of spatial data sets and services. • IR Requirement 8 All spatial data services conformant to the interoperability arrangements shall include a Get Service Metadata operation. • IR Requirement 9 Newly developed spatial data services operating upon harmonised data or their metadata shall be conformant with interoperability arrangements. • IR Requirement 10 Any harmonised spatial data service shall follow the interoperability arrangements. • IR Requirement 11 Any harmonised spatial data service shall have minimal performance criteria defined in the same way as network services, i.e. performance, capacity, and availability. The values will depend upon the character of the type of service. • Recommendation 5 The gazetteer service should be related to harmonised datasets conforming to Addresses, Geographical names and Administrative boundaries. i.e. Location instances should be fetched from these three themes, and correspondingly the Location type should be either an address, a geographical name, or an administrative polygon. • IR Requirement 12 A registry service shall be compliant with ISO 19135:2005, Geographic information -- Procedures for item registration.02/04/2012 -6-

Date: 24/02/2012 HABITATS service toolkitDoc. Identifier: D4-4-1Project: Social Validation of INSPIRE Annex III Data Structures in EU Habituas3. HABITATS NETWORKING ARCHITECTURE AND REFERENCE LABORATORY3.1. HABITATS NETWORKING ARCHITECTUREThe HABITATS Networking Architecture has the goal of defining a system able to ensure theinteroperability and security of provided data and services. In particular, since integration with theINSPIRE initiatives is needed, it is based on: • A methodological approach able to define a system architecture that is scalable and adaptable to the specifications and standards currently being defined; • The adoption of a Service Oriented Architecture based on Web Services and SOAP technology.The platform neutral networking architecture of HABITATS and the basic set of networking servicescan be seen as an extension of existing INSPIRE services, for the management, discovery, sharing andprocessing of spatial planning data by public and commercial sector, NGOs, citizens, private sector,education and science, and all those who play an important role in biodiversity and see regionprotection and also exploitation. (HABITATS D4.2.1 2010)As already described in D4.1, the architecture design is realised on the basis of a Reference Model ofOpen distributed Processing (RM-ODP) (ISO/IEC 10746-1) The use of RM-ODP will give us twoopportunities: To define the basic design of the solution as platform neutral and to support differentlocal implementation, and to build on positive experiences of previous European research projects, asthis methodology is used by most European (mainly research) projects and some recommendationsalready exist. The architecture design provides an overall conceptual framework for building geo-processing services for biodiversity, sea region protection and for effective management and utilisationof sensitive areas. (HABITATS D4.2.1 2010)The HABITATS networking services will be ultimately applied on two levels: The HABITATSReference Laboratory (see also chapter 3) as a central portal with the support of global data, but alsosupporting cross scenarios implementations, and the HABITATS pilot applications, asimplementations of single HABITATS pilot cases, which will also be used for testing the sharing oflocal data and metadata. (HABITATS D4.3.1 2011)The inter-linkage of the Reference Lab and the pilot implementations is visualized in the illustrationbelow (taken from HABITATS D4.2.2 2011).02/04/2012 -7-

Date: 24/02/2012 HABITATS service toolkitDoc. Identifier: D4-4-1Project: Social Validation of INSPIRE Annex III Data Structures in EU HabituasThe reference laboratory has the following roles:• To offer a possibility for testing new services• To offer access to global data for pilots• To support implementation of cross-pilot scenarios• To make the Habitats services discoverable for external platformsThe pilot implementations will implement only the functionality required by the respective pilot users,the Reference Laboratory will implement the full functionality. (HABITATS D4.2.2 2011)The RM-ODP divides all processes of architecture design into five generic and complementary steps,which are called viewpoints of the system and its environment. These viewpoints are the: • Enterprise viewpoint: Focusing on the analysis of pilot scenarios and the definition of a limited number of generic use cases, • Information viewpoint: Focusing on the basic data and metadata sets that could be shared among the different scenarios • Computational viewpoint: Focusing on generic components that could be reused for more scenarios • Engineering viewpoint: Defining a generic scheme that can be reused for all pilot implementations • Technology viewpoint: Defining basic architecture modifications for single scenarios and suggesting potential technical implementations3.2. REFERENCE LABThe reference laboratory, as already mentioned in the previous chapter, is the central hub of theHABITATS Networking Architecture. It consists of several layers, which are (HABITATS D4.2.22011):02/04/2012 -8-

Date: 24/02/2012 HABITATS service toolkitDoc. Identifier: D4-4-1Project: Social Validation of INSPIRE Annex III Data Structures in EU Habituas• Data layers – management data and files on storage, eventually guarantee access to externalsensors• Server (engine layer) – defines tools, which guarantee basic services on the server side – supplyingservice• Client layer – is client side of Web services, which guarantee access of users to services• Application layer is some form of wrapping elementary client services into application or intosuch form, which could be used by other Web tools• Presentation layer contain such web tools, which allow to combine and publish single objects fromthe application level as part of Web presentationThe illustration below (taken from HABITATS D4.2.2 2011) shows the different layers of theHABITATS Networking Architecture.02/04/2012 -9-

Date: 24/02/2012 HABITATS service toolkitDoc. Identifier: D4-4-1Project: Social Validation of INSPIRE Annex III Data Structures in EU Habituas4. HABITATS INVOKING SERVICESIn Habitats we are dealing with the broader understanding of Invoking Services. We will consider thisas a possibility to invoke any type of geospatial services according to ISO19119 classification withplatform. This means running services without the necessity to have any application on the client side.In this first version of the deliverable we are dealing with invoking service using ReferenceLaboratory. In the final version (D4.4.2. - due June 2012) we will extend this approach also to the pilotplatform.4.1. INVOKING OF DISCOVERY SERVICESThe reference laboratory uses its own catalogue, but there are also possibilities to invoke anothercatalogue from a remote platform into the system. There are two possibilities: • To harvest metadata into the reference laboratory • To provide direct search of remote cataloguesFor both possibilities it is necessary to register the remote catalogue in the metadata system of thereference laboratory. This can be done using the Import functionality of the remote catalogue services.After registration of these services, this catalogue could be used for harvesting or multi-searching onthe side of the Reference Laboratory discovery services.02/04/2012 - 10 -

Date: 24/02/2012 HABITATS service toolkitDoc. Identifier: D4-4-1Project: Social Validation of INSPIRE Annex III Data Structures in EU Habituas4.2. INVOKING OF VISUALISATION SERVICESVisualisation services WMS (similar to WPS5 or SOS) are invoked through HSlayers on ReferenceLaboratory4.2.1. HSLayers6 3.3HSLayers combines capabilities of ExtJS (part of Sencha7) and OpenLayers and several helpingscripts to establish truly Web GIS applications. The development started in 2007. In 2009, after 2years of development, it was released under conditions of the GNU General Public License 3.OpenLayers8 is a JavaScript toolkit for the creation of mapping applications in the web browsers.OpenLayers is in some ways more powerful than Google Maps toolkit: • It can show maps based on various raster and vector formats. • It has connectors to many standards and quasi-standards such as MapServer, OGC Web Mapping Service, ArcIMS, simple Image layer, GML, GeoRSS, KML, Text and others and Google, Yahoo and VirtualEarth for commercial data providers. • The user – creator of mapping applications – does not need to take care about differences between various web browsers and their JavaScript implementation or between various data formats.ExtJS is a multi-browser JavaScript library for building rich internet applications. It consists ofcustomisable User Interface widgets, ready to be used by designers of Graphical User Interface,similar to desktop widgets, which among others are text field and text area input controls, date fieldswith a pop-up date-picker, numeric fields, list box, radio and checkbox buttons, wysiwyg html editor,text grids, suitable for spreadsheets, trees, tab panels, toolbars, menus and sliders. ExtJS was originally5 http://opengeospatial.org/standards/wps6 http://bnhelp.cz/hslayers7 http://sencha.com8 http://openlayers.org/02/04/2012 - 11 -

Date: 24/02/2012 HABITATS service toolkitDoc. Identifier: D4-4-1Project: Social Validation of INSPIRE Annex III Data Structures in EU Habituasbuilt as an add-on library extension of Yahoo UI, but now it is a standalone project. In HSLayers, weare still using ExtJS version 3.x.HSLayers features are coming up from OpenLayers and therefore their characteristics are as follows: • Portrayal of various types of data: • Raster: OGC WMS(-T), Image (PNG, JPEG, GIF), … • Vector: OGC WFS(-T), GML, GeoRSS, KML, GPX, GeoJSON, … • Data sources from commercial servers: Google Maps, Virtual Earth, Yahoo Maps, … • The user interface (use control) adheres to current conventions in web map portals. • Information about queried objects in text bubbles.HSLayers additional functions include: • Dynamic adding of OGC (Open Geospatial Consortium) services into map - clients for WMS, WFS, WCS, KML, GeoRSS and others. • Basic WFS filtering • Transformation (warping) of services, with different coordinate reference system. • Portrayal of independent data sources on the client side. Map composition is composed on the basis of requests to various servers. It is thus not necessary to install a map server. • Saving of map composition according to WMC (Web Map Context) OGC specification on user computer for repeated future use or for sharing between users. • Extension of compute functions based on WPS (Web Processing Service ) OGC service - 9 according to user needs – generic WPS Client available. • Multilingual environment • Map requests to various types of data stored on various servers, with automatic processing of results • Work with micro-formats • Search on the map using various Gazetteers services. • Connection of the application with catalogue client (OGC CSW) in the geoportal, which enables display of the searched service from catalogue directly on the map. • Vector editing function including snapping to chosen layers on the server. • Possibilities for advanced configuration of user requests • Advanced measuring of length and surfaces • Printing of map compositions - possibility of large print outs (up to A0 format), user configuration of print settings using HTML templates. PDF output as well as various image formats.4.2.1.1. LayerSwitcherLayerSwitcher has now a two panel interface, which makes it easy for the user to manipulate a largelist of layers in the map application. The first panel represents the “logical view” on the layer list –layers can be organized into folders and so user can keep logical structure of displayed data. Thesecond panel represents the “physical view” - order of layers, displayed in the map window in thestack. Usually, aerial photos are organized at the bottom of the stack, and cadaster maps are displayedat the top of the stack and their organization into folders does not make much sense.9 http://en.wikipedia.org/wiki/Web_Processing_Service02/04/2012 - 12 -

Date: 24/02/2012 HABITATS service toolkitDoc. Identifier: D4-4-1Project: Social Validation of INSPIRE Annex III Data Structures in EU HabituasThis version of HSLayers also brings the ability to hide used components to side panel of theapplication (dock) and whenever needed, it can be undocked again into a separate window.Users on smaller screen resolutions can enlarge some tool they need to use and so they are notlimited to the size of the side panel.4.2.1.2. CopyrightsHSlayers also support publishing of information about copyrights related to single services.Information is taken from metadata02/04/2012 - 13 -

Date: 24/02/2012 HABITATS service toolkitDoc. Identifier: D4-4-1Project: Social Validation of INSPIRE Annex III Data Structures in EU Habituas4.2.1.3. Invoking of view (WMS) servicesThe WMS services can be invoked into the visualisation client in two ways: The first is to discoverWMS services from catalogues and visualise WMS services directly from catalogues.The second possibility is to add the WMS url directly to the visualisation client HSlayers. The URLcould be added using OWS services.02/04/2012 - 14 -

Date: 24/02/2012 HABITATS service toolkitDoc. Identifier: D4-4-1Project: Social Validation of INSPIRE Annex III Data Structures in EU Habituas4.2.1.4. WMS coordinate transformationIn the real-life, we are often facing a problem, how to display map from the server, which does notsupport coordinate system of the displayed map. This is implemented with help of Proxy4OWS(described later in this document). It is assumed, that Capabilities document is already parsed, it isexpecting GetMap request from the client to Proxy4OWS directly. The GetMap request is expected tohave - next to original WMS parameters - also three add-on options: owsService - this is going to be WMS owsUrl - URL of the original service, which is expecting to handle the GetMap request fromCRS - CRS of the original coordinate system, from which shall the result of GetMap be transformed to.Proxy4OWS generates MapServers mapfile on-the-fly. Only one layer is attached to the mapfile -layer of type WMS. MapServer then formulates the necessary request, fetches the data from remoteserver and provides image transformation on them. Result is always little bit distorted, because theresolution is not always fine enough, but the it can be used and displayed in the mapping application.WMS Sequence diagram.02/04/2012 - 15 -

Date: 24/02/2012 HABITATS service toolkit Doc. Identifier: D4-4-1 Project: Social Validation of INSPIRE Annex III Data Structures in EU HabituasWMS transformation result - left map coordinate system, right - transformed result fromEPSG:4326 source. Thanks to Proxy4OWS, we can now display seam-less data from several WMS resources, which do not support coordinate system of the map, displayed in users browser. 4.2.1.5. Invoking of Map Compositions – Web Map Context The important new issue is the support for Web Map Context (WMC). Web Map Context (WMC) describes how to save a map view comprised of many different layers from different Web Map Servers. A context can be encoded and saved so that Web maps created by users can be automatically reconstructed and augmented by the authoring user or by other users in the future. A Context document is structured using eXtensible Markup Language (XML). Potential uses for context include creating default initial views for Web maps for different hazards, saving the state of a users work on a viewer client to preserve information such as how geospatial layers are added or modified, and saving the state of a client session for sharing with other users. This mechanism is valuable for efficiently communicating across shift transitions. Also, context documents can be catalogued and discovered for reuse by others; this allows analysts to benefit from lessons learned in previous episodes. 10 In URM there is now implemented strong support for discovery and defining new WMC based on information displayed on portal. The system allows to: • Define WMC on the base current composition on portal 10 Orchestra http://www.eu-orchestra.org/TUs/Standards/en/html/Unit4_learningObject6.html 02/04/2012 - 16 -

Date: 24/02/2012 HABITATS service toolkitDoc. Identifier: D4-4-1Project: Social Validation of INSPIRE Annex III Data Structures in EU Habituas • Save composition on local disk • Save composition with metadata on server • Open composition from local disk • Open composition from server • Open composition from remote servers using metadata descriptionThe implementation of the WMC concept presents a new way to the future upcoming solution, whenthe system will support easier collaboration and sharing of results. It also supports the reuse of resultsof work done on portal by other applications.4.2.1.6. PrintingHSLayers includes printing setup, so that content of the map can be printed at any printer or used inother desktop GIS workstation. The printing client enables the user, to choose between printing a mapto a pre-defined template or saving the content of the map into a raster image.When the user makes a choice, that he wants to create a raster image with the map’s current content,he can either directly click the button, and a copy of the map window will be displayed, according tothe selected image format (which can be one of PNG, JPEG, GIF and geo-referenced GeoTIFF). Thedesired scale and region can be set as well.When a user chooses to print a map to a pre-defined template, a new box is drawn, representing thepaper box. Users can move the paper over the map and define the desired region. The size of the paperbox is always adjusted according to the selected scale. Additional information can be added as well(map title, description, icon). The map is then layed out according to selected pre-defined template toPDF or HTML output. The template is prepared as a HTML page.Printing is provided by a server script, which is able to work with standard WMS services, tiled-layer,vector data and other inputs.4.2.2. Invoking of WFS and WCSFor the invoking of WFS and WCS for visualisation proxy4ows is used. When working with largevector datasets, we are usually facing limits of current browsers. Google Maps 1 for example has02/04/2012 - 17 -

Date: 24/02/2012 HABITATS service toolkitDoc. Identifier: D4-4-1Project: Social Validation of INSPIRE Annex III Data Structures in EU Habituaslimited the number of displaying vector features to 1000. In OpenLayers 2, there is no limit for thenumber of features used, but displaying e.g. cadastral map makes browsers often freeze. An advantageof working with vector data directly is (among others), the direct possibility of editing vector data, amore interactive way of user experience and speed (when dealing with reasonable amounts of data).While vector data can be displayed using current technologies, some popular raster data formats, suchas GeoTIFF, cannot be. According to W3C 3, only few formats for raster data are supported, none ofthem is really used or usable in GIS, usually because of the compression method they are using. In theinternet, raster graphics format focus on possibly low amount of data transferred from the server to theclient, while losing the accuracy of the data being transferred. In GIS, we are focusing on data quality,regardless of file size.Raster and vector data are usually distributed using OGC OWS standards. Vector and raster data aredistributed via OGC Web Feature and Raster Service respectively. Both services are offering lists ofdatasets and metadata.Another problem might occur, when some OGC Web Mapping Service does not offer a coordinatereference system, in which the web mapping application is configured. Some middle-ware has to beestablished between the map application and the server, which will transform the images from servercoordinate reference system to the one accepted by the client.Proxy4ows is a server script, which is between the client mapping application and OGC OWS server(WCS, WFS or WMS). It is transforming OGC WMS request types from the client, into WCS or WFSrequests, so that the target server can accept them. On the way back, it downloads the data distributedby the server (raster or vector), creates image and sends it back to the client.It also transforms the GetCapabilities request-response pair, so that the (WMS) client can deal with it.As result, data is processed on the server, into the form, common web browser mapping applicationare able to display without big demands on system resources.HSlayers.Layer.WCS and WFS are derived from Layer WMS. As proxy between the client (the map)and the WFS|WCS server, OWSProxy is placed. OWSProxy transforms WMS requests coming fromthe client into WFS|WCS calls and also responses are transformed to WMS responses or images,displayable in the web browser.Proxy4ows can also deal with OGC WMS service in the way, that it transforms the coordinatereference system of the original service into the client-preferred one in the case the server does notsupport the coordinate system of the client. The resulting image is usually slightly distorted, butdisplayable.Proxy4ows is written in the Python programming language. The following libraries are used:MapServer 4:4 http://mapserver.org02/04/2012 - 18 -

Date: 24/02/2012 HABITATS service toolkitDoc. Identifier: D4-4-1Project: Social Validation of INSPIRE Annex III Data Structures in EU Habituas MapServer is the core of Proxy4ows. Proxy4ows generates the Map object configuration and after that dispatch method of MapServer is used, which deals with the request, downloads all necessary data from servers and generates the resulting image. From the client-point of view, it is used for working directly with vector data (deals as OGC WFS client).GDAL/OGR 5 GDAL is used for reading OGC WFS and OGC WCS service metadata, so that the WMS response from Proxy4ows to the client, has all of the necessary information. While for OGC WFS service, MapServer is directly acting as a client, OGC WCS is configured as GDAL data source. Also for WMS transformation service, WMS interface of GDAL is used, as it is able to deal with tiled requests, preserving the large dataset exceptions issue.OWSLib 6 OWSLib is Python library, which acts as OWS client. With the help of OWSLib, metadata of particular target services are being collected easily. Once again: Proxy4ows acts as WMS server to the client and acts as WFS, WCS or even WMS client to the target server.GetCapabilities When a GetCapabilities request arrives from the client, Proxy4ows checks for an existing cached directory with mapfile, or creates a new one, if it doesn’t exist.GetMap When a GetMap request arrives, an image is generated based on the previously generated mapfile, using OWSDispatch method. At this point, a WFS filter is applied, if the target server is WFS.In both cases, Proxy4ows is looking for existing MapServer MapFile (it creates one, if it does notexist) and lets MapServer do the work. Proxy4ows takes care of the proper MapFile configuration.5 http://gdal.org6 http://sourceforge.net/apps/trac/owslib/wiki02/04/2012 - 19 -

Date: 24/02/2012 HABITATS service toolkitDoc. Identifier: D4-4-1Project: Social Validation of INSPIRE Annex III Data Structures in EU HabituasWhen a WMS GetCapabilities request arrives, OWSProxy generates a MapFile with list of layers,corresponding to either feature types (WFS) or coverages (WCS) of the target service. For configuringthe MapFile properly, GDAL, OGR and OWSLib libraries are used.WFS Layers are configured, using MapServer as WFS Client, while WCS layers are using GDAL asWCS Client.Then MapFile is generated, OWSDispatch method is called and the Capabilities response arrives.02/04/2012 - 20 -

Date: 24/02/2012 HABITATS service toolkitDoc. Identifier: D4-4-1Project: Social Validation of INSPIRE Annex III Data Structures in EU HabituasWhen the WMS GetMap request arrives, OWSProxy finds the existing MapFile (creates it anew, if itdoes not exist) and performs OWSDispatch function of MapServer, which generates the output imageand sends it back to the server.4.2.2.1. InvocationProxy4ows was originally designed as a WMS server. But it can also be used as WFS or WCS server,so that it can only transform original data into a coordinate system unsupported by the target server.Therefore, the client can use basically any type of OWS request using proper parameters. In additionto this, two more parameters will have to be appended to the request URL: • owsservice notes the target server service type (WCS, WFS or WMS) • owsurl is the URL of original server4.2.2.2. Further developmentCurrently, when full featured Capabilities response is sent back from the server to the client, for someservers, with high amounts of data, this can take a long time.For WFS for example, GetCapabilities, DescribeFeatureType and sometimes even GetFeature requestsare to be called, before all necessary metadata are filled to MapFile. This certainly makes theCapabilities response come back after a long time. In the future, we would like to eliminate this, sothat only the smallest amount of requests (GetCapabilities) would have to be requested, and WMSCapabilities response could possibly be returned faster. Of course, when a user chooses the particularlayer to be displayed, additional non-standard calls would have to be done, in order to get additionattributes, so that the client has all of the necessary information (which can be normally taken from theCapabilities document).Also up to now, virtually no configuration caching is created. For each GetCapabilities request, newtemporary MapFile configuration is established. A proper way would be to use already existingMapFile for a particular service if it is already generated, and to evaluate the difference between thecached version and the potentially upgraded service.Proxy4ows is not caching any data. It is open for further investigation, if it would make sense to pre-cache some original data at the Proxy4ows side, in order to make the performance better.4.2.3. Filter Encoding Filtering WFS LayersFilter Encoding is an Open Geospatial Consortium standard11 defining the syntax of expressions usedto filter data provided by the geospatial web services. It describes a rich predicate language thatenables to filter data based on their IDs, type-specific properties, geospatial properties, and to combinefilters using logical expressions, call server-defined functions, encode arithmetical expressions andmore.Filter Encoding is defined in XML. Often it is referred to as FE or FES, with S standing for Standardor Specification.4.2.3.1. FE Examples:12Simple comparison filter can look like that: <Filter> <PropertyIsLessThan>11 http://www.opengeospatial.org/standards/filter12 All the examples in here are using FES 1.1.0, unless specified otherwise02/04/2012 - 21 -

Date: 24/02/2012 HABITATS service toolkitDoc. Identifier: D4-4-1Project: Social Validation of INSPIRE Annex III Data Structures in EU Habituas <PropertyName>DEPTH</PropertyName> <Literal>20</Literal> </PropertyIsLessThan> </Filter>Example 1.Spatial filter can, among other things, define a polygon that the desired features should overlap: <Filter> <Overlaps> <PropertyName>Geometry</PropertyName> <gml:Polygon srsName="http://www.opengis.net/gml/srs/epsg.xml#63266405"> <gml:outerBoundaryIs> <gml:LinearRing> <gml:posList> ... </gml:posList> </gml:LinearRing> </gml:outerBoundaryIs> </gml:Polygon> </Overlaps> </Filter>Example 2.Logical filter allows to combine filters: <Filter> <And> <PropertyIsLessThan> ... </PropertyIsLessThan> <Overlaps> ... </Overlaps> </And> </Filter>Example 3.More examples can be found in the Filter Encoding standard.4.2.3.2. Filter Encoding and WFSFilter Encoding was originally designed as part of the Web Feature Service standard, but then it wasseparated into a standalone document as it can serve also for filtering of other services, such as WebCoverage Service, Gazetteer or Web Registries .02/04/2012 - 22 -

Date: 24/02/2012 HABITATS service toolkitDoc. Identifier: D4-4-1Project: Social Validation of INSPIRE Annex III Data Structures in EU HabituasWhen filtering data is provided by Web Feature Service, several WFS operations are involved:GetCapabilitiesAs a part of GetCapabilities response, Filter_Capabilities of the server are announced. This sectionspecifies the filter capabilities of the particular server. It can look as follows: <ogc:Filter_Capabilities> <ogc:Spatial_Capabilities> <ogc:GeometryOperands> <ogc:GeometryOperand>gml:Point</ogc:GeometryOperand> <ogc:GeometryOperand>gml:LineString</ogc:GeometryOperand> <ogc:GeometryOperand>gml:Polygon</ogc:GeometryOperand> <ogc:GeometryOperand>gml:Envelope</ogc:GeometryOperand> </ogc:GeometryOperands> <ogc:SpatialOperators> <ogc:SpatialOperator name="BBOX"/> <ogc:SpatialOperator name="Equals"/> <ogc:SpatialOperator name="Disjoint"/> <ogc:SpatialOperator name="Intersect"/> </ogc:SpatialOperators> </ogc:Spatial_Capabilities> <ogc:Scalar_Capabilities> <ogc:LogicalOperators/> <ogc:ComparisonOperators> <ogc:ComparisonOperator>LessThan</ogc:ComparisonOperator> <ogc:ComparisonOperator>GreaterThan</ogc:ComparisonOperator> <ogc:ComparisonOperator>LessThanEqualTo</ogc:ComparisonOperator> <ogc:ComparisonOperator>GreaterThanEqualTo</ogc:ComparisonOperator> <ogc:ComparisonOperator>EqualTo</ogc:ComparisonOperator> <ogc:ComparisonOperator>NotEqualTo</ogc:ComparisonOperator> <ogc:ComparisonOperator>Like</ogc:ComparisonOperator> <ogc:ComparisonOperator>Between</ogc:ComparisonOperator> </ogc:ComparisonOperators> <ogc:ArithmeticOperators> <ogc:SimpleArithmetic/> <ogc:Functions> <ogc:FunctionNames> <ogc:FunctionName nArgs="1">SIN</ogc:FunctionName> <ogc:FunctionName nArgs="1">COS</ogc:FunctionName> <ogc:FunctionName nArgs="1">TAN</ogc:FunctionName> </ogc:FunctionNames> </ogc:Functions>02/04/2012 - 23 -

Date: 24/02/2012 HABITATS service toolkitDoc. Identifier: D4-4-1Project: Social Validation of INSPIRE Annex III Data Structures in EU Habituas </ogc:ArithmeticOperators> </ogc:Scalar_Capabilities> <ogc:Id_Capabilities> <ogc:EID/> <ogc:FID/> </ogc:Id_Capabilities> </ogc:Filter_Capabilities>Example 4.DescribeFeatureTypeProperties that can be used to filter features of a specific type are advertised in the response to theDescribeFeatureType request. In the example below, four properties can be used to filter the featuresof the type location: DEPTH, SURFACE, AREA and CODE.<complexType name="locationType"> <complexContent> <extension base="gml:AbstractFeatureType"> <sequence> <element name="msGeometry" type="gml:SurfacePropertyType" minOccurs="0" maxOccurs="1"/> <element name="DEPTH" type="Integer"/> <element name="SURFACE" type="Character"/> <element name="AREA" type="Real"/> <element name="CODE" type="Character"/> </sequence> </extension></complexContent></complexType>Example 5.GetFeatureWhen the filter capabilities of the server are known as well as the properties of the particular featuretype, the filter can be created following the FE standard. Consider the filter from Example 1, this timeextended with the namespaces. It selects all the features whose DEPTH property is less than 20. To theGetFeature request the FILTER parameter is added and the filter is provided as the value:http://localhost/cgi-bin/ows?REQUEST=GetFeature&VERSION=1.1.0&SERVICE=WFS&TYPENAME=location&FILTER=<ogc:Filterxmlns:ogc="http://www.opengis.net/ogc"><ogc:PropertyIsLessThan><ogc:PropertyName>DEPTH</ogc:PropertyName><ogc:Literal>20</ogc:Literal></ogc:PropertyIsLessThan></ogc:Filter>Example 6.02/04/2012 - 24 -

Date: 24/02/2012 HABITATS service toolkitDoc. Identifier: D4-4-1Project: Social Validation of INSPIRE Annex III Data Structures in EU HabituasIn the Habitats Geoportal, Filter Encoding is used to filter WFS layers. To do it, first add a WFS layerto your map. Then right-click on the layer name in the Layer Switcher and select Filter:The filter window appears with a form for simple comparison filter. From the first drop-down listselect a property that will be used for filtering (these have been parsed out from theDescribeFeatureType response). From the second drop-down list select the operator that will be used.(The list of available operators has been parsed out from the GetCapabilities response.) In the text fieldthe value to compare it with is entered. Click Apply and the layer will be filtered.02/04/2012 - 25 -

Date: 24/02/2012 HABITATS service toolkitDoc. Identifier: D4-4-1Project: Social Validation of INSPIRE Annex III Data Structures in EU Habituas4.2.3.3. ImplementationImplemenation of the Filter Encoding is shown on the following illustration:The processing works as follows:WFS layers are displayed as WMS in the HSLayers Web Client (see Proxy4ows for details).1. WFS GetCapabilities request is sent directly to the WFS server. Capabilities document is parsed andfilter capabilities are saved.2. WFS DescribeFeatureTypes request is sent directly to the WFS server. Reply is parsed andproperties of the feature type are saved.3. User creates the filter in the GUI, saved information from steps 1-2. is used.4. User-created filter is written to FES and new WMS request is sent to Proxy4ows, accompanied withthe filter in an additional parameter.5. Proxy4ows takes the filter, creates new WFS request involving the filter, and sends it to the WFSserver. Server replies with filtered layer.6. Proxy4ows transforms the returned WFS layer to WMS and sends it to the client.4.2.3.4. Next StepsIn terms of standards, FES 1.0.0 and FES 1.1.0 is supported. From various types of filters, only thecomparison filter is currently supported by the GUI. As next steps, more filter types can be added.That covers logical filter enabling to combine various filters, spatial filter, expression editor, filteringbased on feature IDs, support for server-side functions and filtering based on time.4.3. WPS INVOKINGIn HSLayers, a new class WPSClient was introduced. The class implements generic OGC WPS clientwith graphical user interface. HSLayers WPSClient performes GetCapabilities request on the serverand creates a list of available processes. Processes are rendered into a drop-down menu. When a userchooses the process he wants to run, DescribeProcess is called. Based on ProcessDescription response,02/04/2012 - 26 -

Date: 24/02/2012 HABITATS service toolkitDoc. Identifier: D4-4-1Project: Social Validation of INSPIRE Annex III Data Structures in EU Habituasa generic input form is generated. After all input data is specified, and when users click the button, anExecute request is called, and when it is finished, an execute response is parsed and outputs of theform are filled.Complex input and BoundingBox input are relatively easy to be implemented. Writing the complexdata handler in the web environment is a real challenge. Generic HSLayers WPS Client. Form isgenerated automatically.4.3.1.1. HSLayers WPSCLient ComplexData handlerFirst, input is identified as raster or vector. Then, the map layers are scanned andHSLayers.Layer.WCS (for rasters) HSLayers.Layer.WFS and OpenLayers. Layer.Vector (for vectors)are identified and added to the drop-down select box. The Custom URL option is attached, for customraster or data link pasting (e.g. for direct WFS or WCS request) and, if the input data shall be vectors,also Custom drawings option is added, so that the user can define some geometry on the map by hand.When an Execute request is called, HSLayers collects URLs for HSLayers.Layer.WFS orHSLayers.Layer.WCS with GetFeature or GetCoverage request types – the client is not sending thedata, but only reference to the data (which is possible according to OGC WPS).HSLayers.Layer.WFS and HSLayers.Layer.WCS are special types of layers, used in HSLayers. Theyare based on OpenLayers.Layer.WMS class, but they are not working with WCS or WFS serverdirectly, but communicating via server-side proxy called Proxy4ows 13[6] (more on other place).Proxy4ows is sitting between the client (generic web browser) and the server (WCS or WFS) and istransforming the requests and responses into a form that can be handled in the web environment.GeoTIFFs are transformed into PNG and JPEGs, GMLs are transformed into picture (PNG) form.Using this approach, there is virtually no limit to the amount of features, which can be displayed in theclient or any image format issues.13 http://redmine.ccss.cz/project/owsproxy02/04/2012 - 27 -

Date: 24/02/2012 HABITATS service toolkitDoc. Identifier: D4-4-1Project: Social Validation of INSPIRE Annex III Data Structures in EU HabituasHSlayers.Layer.WCS and WFS are derived from Layer WMS. As a proxy between the client (themap) and the WFS|WCS server, Proxy4ows is placed. Proxy4ows transforms the WMS requestscoming from the client into WFS|WCS calls and also the responses are transformed to WMS responsesor images, displayable in the web browser.Features in vector layers (which are handling vector data, not raster representation) are transformedinto the format, desired by the user, according to the server-supported MimeType (this is very vagueestimation, see above). Usually this is GML and the data are then sent to the server as part of theXML-encoded request.When a response arrives, HSLayers WPSClient does prefer reference to output data, so it can behandled more easily (as Reference=True). When vector data are send back (usually in GML format),they are added to the map as features of OpenLayers.Layer.Vector. The GML is parsed usingOpenLayers tools. Direct link for downloading is also provided. Raster data (usually GeoTIFFs) areonly available for downloading.Client does not send a direct link to the original WFS or WCS server, but the link to Proxy4ows. Oneof key features of Proxy4ows is, that it ensures the data it is providing will be in the same projection asthe web mapping application.Further development is moving towards a closer usage of already described Proxy4ows. Raster andVector data will be stored on the HSLayers-server and client will consume their in web browser usablerasterized form (PNG image).GetCapabilities and DescribeProcess are parsed directly with OpenLayers tools. For the Executerequest with complex data, a reference to Proxy4ows is used. The WPS Server gets reference toProxy4ows and not to the original server, because Proxy4ows makes sure, resulting data will have thesame coordinate reference system as the map application has.As a result, HSLayers does contain a generic OGC WPS 1.0.0 version client. The client is capable toparse capabilities, to generate automatically from based on ProcessDescription response and is able tosubmit Execute requests and to parse the response.02/04/2012 - 28 -

Date: 24/02/2012 HABITATS service toolkitDoc. Identifier: D4-4-1Project: Social Validation of INSPIRE Annex III Data Structures in EU HabituasIt works with raster and vector data displayed in the mapping application. It is also possible to submitexternal dataset (using URL). The result can either be added to the map or can be downloaded andstoreed on the local hard drive.The displaying of output vector or raster complex data is only limited. If e.g. GeoTIFF is returnedback as a result, it can be only downloaded. Proxy4ows will be modified, so it does accept GeoTIFFfile and produces PNG out of it.The development of the input form, based on ProcessDescription document, needs to be more robust.Also the literal value type of data needs to be controlled according to input type (integer, string, date,...).Even some patches based on this are already accepted in OpenLayers, a more robust WPSExecutepatch will be prepared and submitted into OpenLayers code base.4.4. HSLAYERS SOS CLIENTThe SOS client in HSLayers is a component which can be used for browsing data from any OpenGISSensor Observation Service (OGC SOS) compliant services. The component can be used together withmap application based on HSLayers, or independently with any non map application.Th actual version of component supports only operations from OGC SOS Core Profile which must beimplemented in every OGC SOS compliant services.Operations supported in the actual version are: • GetCapabilities - returns a service description containing information about the service interface and the available sensor data • DescribeSensor - returns a description of one specific sensor, sensor system or data producing procedure • GetObservation - provides pull-based access to sensor observations and measurement-data via a spatio-temporal query that can be filtered by phenomena and value constraintsFuture versions of components will contain also operations from OGC SOS Enhanced Profile and willoffer more functionality for working with data from OGC SOS services.4.4.1.1. How does it work • User invokes HSLayers SOS Client UI dialog • User inputs URL of required OGC SOS • HSLayers SOS Client sends GetCapabilities request to OGC SOS, parses its response and displays available information about OGC Service (name, abstract) in UI • User selects offering and all parameters for required observations (procedure, observed property, date-time interval) • User invokes getting observations • HSLayers SOS Client sends GetObservation request with all passed parameters to OGC SOS, parses its response and display all obtained data in table and chart • If HSLayers SOS Client is used within map application based on HSLayers, user can displays location of obtained observations in map02/04/2012 - 29 -

Date: 24/02/2012 HABITATS service toolkitDoc. Identifier: D4-4-1Project: Social Validation of INSPIRE Annex III Data Structures in EU Habituas4.5. HSLAYERS EMBED COMPONENTHSLayers contains the component Embed for inserting map into any HTML pages. User can create amap in a Geoportal or in any map application which is based on HSLayers and contains the Embedcomponent. Users can also define parameters which affect how the map will be look in the targetHTML page.There are three types of a resulting inserted map window: • Pure HTML – this type is based on pure HTML and does not contain any other UI components • Simple ExtJS – this type uses ExtJS library for generating UI container • Advanced ExtJS – this type uses ExtJS library also as Simple ExtJS type and also contains another UI components (tree with list of all layers in map)02/04/2012 - 30 -

Date: 24/02/2012 HABITATS service toolkitDoc. Identifier: D4-4-1Project: Social Validation of INSPIRE Annex III Data Structures in EU HabituasThe HSLayers Embed component consists of several parts on the client and on the server side. • Embed Generator – this part is responsible for displaing the UI form to enter the parameters affecting the resulting inserted map window. When the user enters all required parameters, the actual map is serialized and sent to the Status Manager on the server. The Status Manager is an external service which is responsible for saving actual state of any component and getting it back later. • Embed Client – this part is responsible for rendering map windows in target HTML page in dependence on parameters entered by user. • Embed Script – this part is responsible for rendering main HTML part of map window and for calling Embed Client in target HTML page.4.5.1.1. How does it workGenerating HTML code 1. User invokes UI to enter the parameters of inserted map window. 2. User inputs parameters affecting map window (type of map window, size of window). 3. Embed Generator serialize actual map and send it to Status Manager. 4. Status Manager returns URL for later accessing state of actual map (when it will be rendered in target HTML page). 5. Embed Generator returns complete HTML code. User can paste this HTML code to any HTML page.02/04/2012 - 31 -

Date: 24/02/2012 HABITATS service toolkitDoc. Identifier: D4-4-1Project: Social Validation of INSPIRE Annex III Data Structures in EU Habituas4.5.1.2. Rendering map window in target HTML page 1. Target HTML page renders initial JavaScript code which creates new IFRAME element. URL of this IFRAME element is set to Embed Script and contains all parameters. 2. When IFRAME is activated, the Embed Script is called. The Embed script generates the main part of HTML code for IFRAME. This HTML code contains references for all required JavaScript files and Cascade Style Sheets (CSS) files. These files are different in dependence on type of map window. 3. HTML code in IFRAME calls the Embed Client for creating the map window. 4. Then it reads the state of the map from the Status Manager and restores the complete map from this state.02/04/2012 - 32 -

Date: 24/02/2012 HABITATS service toolkitDoc. Identifier: D4-4-1Project: Social Validation of INSPIRE Annex III Data Structures in EU Habituas5. PROCESSING WORKFLOW MANAGEMENTThe objective of this chapter is to provide information about available orchestration and workflowmanagement tools. It also explains the tool functionalities and possibilities to integrate and use withHabitats services for complex chained service orchestration and deployment.5.1. ORCHESTRATION ENVIRONMENTWe assume that we will use Open Geospatial Consortium (OGC) compliant Web Processing Service1.0.0. standard services provided by GeoServer WPS extension, 52north WPS solution and byWPS asinput sources in orchestration and work flow management.Another assumption is that in the future different service chaining will be able to be developed also bynon technical and IT highly skilled personnel.5.1.1. Workflow Management SystemWorkflow Management System (WMS) is a piece of software that provides an infrastructure to setup,execute, and monitor scientific workflows. In other words, the WMS provide an environment where inexperiments can be defined and executed.An important function of a WMS during the workflow execution or enactment is the coordination ofoperation of individual components that constitute the workflow – the process also often referred to asorchestration.As research becomes more data-intensive and more reliant on the use of computers, larger volumes ofexperimentation data are recorded quicker and with greater precision. This trend has spurred asignificant increase in the complexity of scientific simulation software. Many tools only perform asmall well-defined task, thus necessitating that several of them are joined in a pipeline to model auseful experiment.Additional difficulties arise from the need to deal with the incompatible data formats that variousservices produce or consume. It is evident that considerable amount of computer science knowledge isrequired to overcome the outlined problems. However, domain scientists across disciplines do nothave sufficient relevant expertise.Scientific workflows and WMSs have emerged to solve this problem and provide an easy-to-use wayof specifying the tasks that have to be performed during a specific in silico experiment. The need tocombine several tools into a single research analysis still holds, but technical details of workflowexecution are now delegated to Workflow Management Systems.5.1.2. Business Process Execution LanguageBusiness Process Execution Language (BPEL), short for Web Services Business Process ExecutionLanguage (WS-BPEL) is an OASIS standard XML based executable language for specifying actionswithin business processes with web services. Processes in Business Process Execution Languageexport and import information by using exclusively web service interfaces. It defines a set of basiccontrol structures like conditions or loops as well as elements to invoke web services and receivemessages from services. BPELs messaging facilities depend on the use of the Web ServicesDescription Language (WSDL) 1.1 to describe outgoing and incoming messages. Message structurescan be manipulated, assigning parts or the whole of them to variables that can in turn be used to sendother messages.5.2. ENGINES AND WORK-FLOW MANAGERSTo make tests we collected several workflow execution engines and managers. From the collectedengines the biggest part is using BPEL and only one of the solutions is oriented on data flowexecution.02/04/2012 - 33 -

Date: 24/02/2012 HABITATS service toolkitDoc. Identifier: D4-4-1Project: Social Validation of INSPIRE Annex III Data Structures in EU Habituas5.2.1. Apache ODE Name: Apache ODE + Eclipse Plugin Version: 1.3.5 Platform: Java Standards: WS-BPEL 2.0 Vendor: Apache Software Foundation, OASIS Licence Apache ODE: Apache Software License (ASL), version 2.0. Licence Eclipse BPEL: Incubation phase, version 0.5 Homepage: http://ode.apache.org/Apache ODE (Orchestration Director Engine) executes business processes written following the WS-BPEL standard. It talks to web services, sending and receiving messages, handling data manipulationand error recovery as described by your process definition. It supports both long and short livingprocess executions to orchestrate all the services that are part of your application. ODE comes witheasy to use web-based management interface (Figure 1).Ode can be deployed in three different environments: As a simple Web Service in Axis 2, ODE is bundled in a WAR than can be deployed in any application server and is invoked using plain SOAP/HTTP. As a JBI service assembly, ODE is bundled in a ZIP that can be deployed in any JBI container and is invoked using the NMR. SMX4 OSGi bundle5.2.2. Orchestra Name: Orchestra Version: 4.7.1 Platform: Java Standards: WS-BPEL 2.0 Vendor: OW2 - Object Web Licence: LGPL 2.1 Homepage: http://orchestra.ow2.org/Orchestra is a complete solution to handle long-running, service oriented processes. It provides out ofthe box orchestration functionalities to handle complex business processes. It is based on the OASISstandard BPEL.02/04/2012 - 34 -

Date: 24/02/2012 HABITATS service toolkitDoc. Identifier: D4-4-1Project: Social Validation of INSPIRE Annex III Data Structures in EU HabituasIn Orchestra server is bundled with web based workflow management Console (Figure 2).5.2.3. Taverna Server Name: Taverna Server Version: 2.2a1 Platform: Java Standards: - Vendor: myGrid, OMII-UK Licence: LGPL Homepage: http://www.taverna.org.uk/Taverna Server enables you to set up a dedicated server for executing workflows remotely. Server isusing workflows designed by Taverna Workbench. The Server exposes REST and SOAP APIs; eithercan be used to access the functionality of the Server.As demonstration manager the Taverna Demonstrator interface is used written in Ruby language. Thedemonstrator is simple GUI to manage workflow execution and can be used as inspiration for customand more complicated workflow management solution development. (Figure 3) Taverna Demonstratorcan not be used in a production environment.02/04/2012 - 35 -

Date: 24/02/2012 HABITATS service toolkitDoc. Identifier: D4-4-1Project: Social Validation of INSPIRE Annex III Data Structures in EU Habituas5.3. WORKFLOW DESIGNERSAn integral part of orchestration engines and workflow managers are designers to prepare executableworkflow documents.5.3.1. 52°North WPS Workflow Modeller and Orchestra tion API Name: 52°North WPS Workflow Modeller Version: WPS 2.0-RC6, Revision 1647 Platform: independent, language Java Vendor: 52north Licence:GPL Homepage: http://52north.org/The open source software initiative 52°North is an international network of partners from research,industry and public administration. Its mission is to foster the development of new concepts andtechnologies in Geo-informatics through a common innovation process.All software developed within this collaborative development process is published under an opensource license.The 52°North partners have a long and outstanding record in the Geo-IT domain. They are activelycontributing to the development of international standards, e.g. at W3C, ISO, OGC or INSPIRE.The Geoprocessing community aims at designing a pluggable web service architecture fororchestrating and executing geo-processes, as well as researching innovative spatio-temporal dataanalysis processing techniques.02/04/2012 - 36 -

Date: 24/02/2012 HABITATS service toolkitDoc. Identifier: D4-4-1Project: Social Validation of INSPIRE Annex III Data Structures in EU HabituasThe WPS Workflow Modeller allows the visual modeling of geoprocessing workflows. In alightweight Browser environment, WPS services can be easily chained and equipped with literal orcomplex data inputs from i.e. other OGC services such as WFS.This graphical WPS Workflow Modeller is based on the WPS Orchestration API which allows theprogrammatically chaining of WPS in a very straight forward manner.The work was has been conducted in cooperation with Sejong University through funding from theSeoul R&BD program(10540)The Workflow Modeller uses a standard Openlayers implementation to visualize input and resultlayers (Figure 4). Along with this standard implementation comes the default controls for panning andzooming as well as a layer switcher to turn on/off overlay layer and to select base layers (see section1.1). In the following the specific functions of the Workflow Modeller to interact with the map aredescribed.5.4. ECLIPSE BPEL Name: Eclipse BPEL Version: WS-BPEL 2.0 Platform: independent, language Java Vendor: The Eclipse Foundation Licence: Eclipse Public License (EPL) Homepage: http://www.eclipse.org/bpel/BPEL Project adds comprehensive support to Eclipse for the definition, authoring, editing, deploying,testing and debugging of WS-BPEL 2.0 processes. WS-BPEL (Web Services Business ProcessExecution Language), or BPEL, is a vendor-neutral specification being developed by OASIS tospecify business processes as a set of interactions between web services.The goal of the BPEL Project is to add comprehensive support to Eclipse for the definition, authoring,editing, deploying, testing and debugging of WS-BPEL 2.0 processes. WS-BPEL (Web ServicesBusiness Process Execution Language), or BPEL, is a vendor-neutral specification being developed byOASIS to specify business processes as a set of interactions between web services. By providing thesetools, this project aims to build a community around support for BPEL in Eclipse.02/04/2012 - 37 -

Date: 24/02/2012 HABITATS service toolkitDoc. Identifier: D4-4-1Project: Social Validation of INSPIRE Annex III Data Structures in EU HabituasThe implementation will be extensible to third-party vendors in a number of ways. The editor will beextensible to support new activity types, property pages for extensibility of existing constructs, anextensible palette, and product-specific branding capabilities. The runtime deployment framework willbe extensible so that third parties may add support for a variety of runtime engines. The model willsupport extensions to provide new activities or attributes, and the validator will allow for validation ofthese extensions.The Key Features are: Designer. A GEF-based editor that provides a graphical means to author BPEL processes (Figure 5). Model. An Eclipse Modeling Framework (EMF) model that represents the WS-BPEL 2.0 specification. Validation. A validator which operates on the EMF model and produces errors and warnings based on the specification. Runtime Framework. An extensible framework which will allow for deployment and execution of BPEL processes from the tools into a BPEL engine.5.4.1. HUMBOLDT Workflow Design and Construction Service Name: HUMBOLDT Workflow Design and Construction Service Version: svn Platform: independent, language Java Short description Vendor: NATURE-SDI project Licence: Homepage: http://community.esdi-humboldt.eu/wiki/5The HUMBOLDT Workflow Design and Construction Service consists of two sub-systems, agraphical user interface, called the Workflow Editor, allowing users to compose geo-processingcomponents into workflows. Second, a web service, called the Workflow Repository Service, thataccepts requests for workflows and delivers them to clients (in HUMBOLDT, the Mediator Service).02/04/2012 - 38 -

Date: 24/02/2012 HABITATS service toolkitDoc. Identifier: D4-4-1Project: Social Validation of INSPIRE Annex III Data Structures in EU HabituasBasic Functionality of the WDCS: Allow users to visually compose the workflow graph out of geo-processing components (out of WPS / java processes / WSDL?) and data sources Manual Workflow Definition, Automated Execution Allow users to register processes to the system Exports such workflows in different workflow dialects via a WSDL / SOAP interface (priority: the dialect used by the HUMBOLDT Mediator Service) Help the user in the composition process, by: type checking inputs and outputs when connecting type checking user entered input values Some small graphical features that make the composition process easierThe Graphical User Interface of the WDCS, called the Workflow Editor, can be used by users toregister processing components and to specify chains / compositions of such components (i.e.workflows). The GUI offers therefore quite a similar functionality as e.g. the GUI of the ArcGISModel Builder or a BPEL Workflow Designer but additionally provides assistance to the user in thecomposition process by e.g. comparing the input/output type definitions of the components to beconnected. For example, this prevents users from connecting components processing raster data withprocessing component accepting vector data and hence reduces the risk of runtime errors whenexecuting the composition. Currently, the processes to be composed can either be exposed as OGCWPS Processes or directly implemented in java and accessible to the HUMBOLDT Mediator Service.The Workflow Editor acts as a client application to the Workflow Repository and allows users todeploy the workflows such that they will be subsequently offered via the web service interface of theWorkflow Service.The geo-processing workflows to be built with the Workflow Editor follow the structure of dataflowgraphs, a well-know concept in software engineering. A dataflow graph is a graph G = (V,E) withV=GD being a set of nodes and E= (D×GG) a set of directed edges. G is the set of processing nodesassociated with computational components, in the case of the workflow editor, geo-processingcomponents. D is the set of data-providing nodes associated with data providing components. Adirected edge E connects a node associated with a data providing or computational component toanother computational component. In case a computational component has multiple inputs (oftencalled input ports), there can be multiple edges pointing to it, each one associated with a differentinput. In case a processing node has multiple outputs or the output shall be pushed as input to multipleinput ports of a single or multiple other processing nodes, several edges from such a processing nodemight exist.The computational components represented by the nodes G in a dataflow graph are required to befunction-like in the sense that they generate the same output given the same input and do not dependon some changing states (such as a global variable that is not a direct parameter). Further, the directedges represent the dataflow between nodes. A node in a dataflow graph can be “executed”, if the dataof all other input nodes that provide the input via a dataflow edge is available. Due to the function-likenature of the processing nodes, the outcome of a whole dataflow graph of such components isdetermined solely by the input passed to it and can therefore be treated similar as a simple or atomicnode and composed.02/04/2012 - 39 -

Date: 24/02/2012 HABITATS service toolkitDoc. Identifier: D4-4-1Project: Social Validation of INSPIRE Annex III Data Structures in EU HabituasThis project collects the HUMBOLDT Service Integration Framework sub-projects, specifically: the Mediator Service (MS), a harmonization work-flow execution engine, the Workflow Construction and Design Service (WDCS), an harmonisation needs analysis and workflow construction component, the Model Repository (MR), a conceptual schema and mappings repository, the Context Service (CS), a service for managing product descriptions for transformation results and the Information Grounding Service (IGS), a bridge component to existing catalogue services.5.4.2. Taverna Workbench Name: Taverna Workbench Version: 2.2 Platform: independent, language Java Vendor: myGrid, OMII-UK Licence: LGPL Homepage: http://www.taverna.org.uk/Taverna is an open source and domain independent Workflow Management System – a suite of toolsused to design and execute scientific workflows and aid in silico experimentation.Taverna has been created by the myGrid team and funded through the OMII-UK. The project hasguaranteed funding until 2014.The Taverna suite is written in Java and includes the Taverna Engine (used for enacting workflows)that powers both the Taverna Workbench (the desktop client application, Figure 7) and the TavernaServer (which allows remote execution of workflows). Taverna is also available as a Command LineTool for a quick execution of workflows from a terminal.Taverna allows you to define how your data flows between the services, without having to worry howyou are going to invoke these services. It will automate and pipeline processing of data.02/04/2012 - 40 -

Date: 24/02/2012 HABITATS service toolkitDoc. Identifier: D4-4-1Project: Social Validation of INSPIRE Annex III Data Structures in EU HabituasSuite of tools to design, edit and execute workflows Workflow design and execution in Taverna Workbench Command line execution of workflows Remote execution of workflows on a Taverna server Invoke workflows from the InternetWide range of services and extensible architecture Service discovery Various service types available: WSDL-style and RESTful Web services, BioMart, BioMoby, SoapLab, R, Beanshell, Excel and csv spreadsheets, pyWPS Service creation for external tools or Java libraries Extensible service plug-in architecture for adding new service typesSecure Support for secure services Secure management of users’ credentialsVersatile Workbench Tabs for finding, designing and executing workflows Fully graphical workflow design Drag and drop workflow components Comprehensive undo/redo Built-in help facility Annotations for describing workflows, services, inputs, outputs Workflow validation and debuggingCreate your own or start from existing workflows Easy design of new workflows Load existing workflows (from a disk, myExperiment or a URL) View workflow layout and logic Modify existing workflows02/04/2012 - 41 -

Date: 24/02/2012 HABITATS service toolkitDoc. Identifier: D4-4-1Project: Social Validation of INSPIRE Annex III Data Structures in EU Habituas Load workflows in off-line mode (when disconnected from the Internet) Nested workflows (sub workflows) Workflow validation during design time for debugging while composing a workflow Built-in detection when a service’s interface changes or a service go off-line during design timeFind workflows created by others and share yours Full myExperiment search options for browsing workflows Publish workflows on myExperiment for use by othersExecute and debug your workflows Execute workflows Remember previously used workflow inputs Save workflow input values used to a file Load workflow input values from a file Pipelining and streaming of data Implicit iteration of service calls Conditional and repeated calling of services Customizable looping over a service Failover and retry of service calling Parallel execution and configurable number of concurrent threads Improved error handling and reporting for debugging during run time Monitor workflow execution Pause/resume or cancel workflow execution Manage previous runs and workflow results View intermediate results and debug workflows at run time Filter and save intermediate and final workflow resultsTrack workflow runs and results Record workflow execution provenance Review provenance of previous workflow runs02/04/2012 - 42 -

Date: 24/02/2012 HABITATS service toolkitDoc. Identifier: D4-4-1Project: Social Validation of INSPIRE Annex III Data Structures in EU Habituas6. CONCLUSIONThis report gives a brief overview of the tools for invoking Geospatial Services that arise within theHABITATS network architecture. Invoking services are described and a few examples of invokingservices in the Reference Laboratory are demonstrated, and possibilities of Workflow management areanalysed.Basically the architecture design is realised on the basis of a Reference Model of Open distributedProcessing (RM-ODP), while the HABITATS networking services will be ultimately applied on thetwo levels of the Reference Laboratory as a central portal and on the level of the pilot cases for localdata testing and sharing. This first version of the deliverable examines the invoking services used inthe Reference Laboratory. As invoking strongly depends on the used platform, this first report mainlyreflects the developments in the reference Laboratory in order to support developers to implement orinvoke their services.In the next phase (which will be reported in D4.4.2) the focus will be on invoking the services at thelevel of the pilots and on the implementation of workflow management systems for the purpose ofHABITATS.02/04/2012 - 43 -

Date: 24/02/2012 HABITATS service toolkitDoc. Identifier: D4-4-1Project: Social Validation of INSPIRE Annex III Data Structures in EU Habituas7. REFERENCESGüting 2005: Ralf Hartmut Güting, Markus Schneider. Moving Objects Databases.2005, ISBN 978-0120887996.HABITATS DoW 2010: Description of Work, “Social Validation of INSPIRE Annex II DataStructures in EU Habitats”HABITATS D4.2.1 2010: D4.2.1- INSPIRE Networking Architecture Design, 2010HABITATS D4.2.2 2011: D4.2.2 – INSPIRE Networking Architecture Design, 2011HABITATS D4.3.1 2011: D4.3.1 – HABITATS networking servicesISO/IEC 10746-1 Information technology – Open Distributed Processing – Reference model:Overview, ISO (1998)TAVERNA 2011: Taverna, What is a Workflow Management System? Taverna home page 2011.(http://www.taverna.org.uk/introduction/what-is-a-workflow-management-system/ ).02/04/2012 - 44 -

Habitats deliverable 4.4.1

by Karel Charvat on Apr 05, 2012

Accessibility

Categories

Upload Details

Usage Rights

2 Embeds 267

Statistics

Habitats deliverable 4.4.1 Document Transcript

http://www.habitats.cz	266
http://www.linkedin.com	1