Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

FIWARE From Open Data to Open APIs


Published on

Slides for the talk at CeBIT'15 Public Sector Park. It includes a description of FIWARE, Smart Cities activities and Open Data features related to CKAN, NGSI (context broker) and their integration for real time open data from cities.

Published in: Technology

FIWARE From Open Data to Open APIs

  1. 1. Follow @FIWARE on Twitter! FIWARE: From Open Data to Open APIs Sergio Garcia Gomez Telefonica I+D. FIWARE Data/Context Chapter and Smart Cities Architect, @ggsergio (twitter)
  2. 2. Agenda FIWARE overview Open Data Introduction to Context Management Smart City examples Open & Agile Smart Cities 1
  3. 3. Cities are where daily life and businesses actually happen … 2 Smart Cities are not simply about more efficient municipality services but transforming Cities into ICT platform enabling innovation local economy growth and well-being
  4. 4. But … How can the Smart Cities ultimately enable local economy growth and well-being? 3 ecosystem open globalsustainable Build an
  5. 5. Ecosystem and platform: two tied concepts 4 Open Standard Platform ecosystem open globalsustainable
  6. 6. A successful ecosystem is that one which creates the right incentives to all stakeholders 5 Entrepreneurs, Developers • Develop once for a large market • Easily meet potential customers • Marketing, promotion • Ability to test with real data and end users • Simple yet powerful APIs that accelerate product development The Smart City • Connect to entrepreneurs • Put their data at work • Bring new innovative services to end users • Be more efficient • Social Reputation Platform technology Providers • Ability to “coopete” • Connect to entrepreneurs: jointly exploit the opportunities ecosystem open globalsustainable
  7. 7. Sonny Abesamis 6 An open smat city Kai SchreiberJason Kuffer Open Standards and Specifications to avoid vendor lock-in Modular and non- intrusive platform to build a smart city Enablement of solutions interoperability in the city and portability among cities
  8. 8. Entertainment Industry Tourism Mobility Agriculture 7 Software Platform Innovation Experimentation Nodes deployment and management Startups funding Global footprint Strategy: Build an ecosystem that will work as catalyst for capturing the opportunities, engaging data providers and entrepreneurs The FIWARE Public Private Partnership eHealth Smart Grids Public Administration Smart Cities Goal: capture opportunities derived from the new wave of digitalization of life and businesses that is coming
  9. 9. FIWARE PPP: main figures 8 … Had a single Company made an investment like this alone? 24 9 countries partners 122 = 85 + 37 M€ FIWARE budget (phases 1-2 + phase 3) 99 = 76 + 23 M€ FIWARE funding (phases 1-2 + phase 3) 80 + 20 M€ for startups/SMEs 1300 Startups/SMEs 21 Innovation Hubs 6,5 M€ marketing 450 K€ sponsorships next 2 years 16+ Cities 95/66 M€ budget/funding of Vertical Use Cases 18 = 17 + 1 FIWARE Lab nodes
  10. 10. FIWARE major differential features 9 • Federation of infrastructures (private/public regions) • Automated GE deployment Cloud • Complete Context Management Platform • Integration of Data and Media Content Data/Media Context Mgmt • Easy plug&play of devices using multiple protocols • Automated Measurements/Action Context updates IoT Services Enablement • Visualization of data (operation dashboards) • Publication of data sets/services Data/Services Delivery • Easy incorporation of advanced 3D and AR features • Visual representation of context information Advanced Web UI • Security Monitoring • Built-in Identity/Access/Privacy Management Security • Advanced networking (SDN) and middleware • Interface to robots I2ND
  11. 11. Reference Architecture 10 Presentation Layer & APIs Content/Data Storage, Management, Processing & Analysis IoT Backend Acquisition/Actuation Layer
  12. 12. Data • Data is the fuel that enables, inspires and boosts entrepreneurs, remarkably in Smart Cities. Lab • FIWARE Lab offers the environment where all the stakeholders meet together around innovation. FIWARE • FIWARE provides the technology to build the entrepreneurs and developers ecosystem. Ingredients to connect an Smart City 11
  13. 13. Open Data/Content approaches 12 Datasets Existing Datasets (census, geographical, tourism,...) Historic Data (from sensors, events...) Real Time Vertical Systems (mobility, events...) Internet of Things (sensors, Smart meters...) Media Video streams (traffic, surveillance..) Audio (microphones), speaches... Applications NGSICKAN WEBRTC KURENTO
  14. 14. Open Data/Content approaches 13 Datasets Existing Datasets (census, geographical, tourism,...) Historic Data (from sensors, events...) Real Time Vertical Systems (mobility, events...) Internet of Things (sensors, Smart meters...) Media Video streams (traffic, surveillance..) Audio (microphones), speaches... Applications CKAN
  15. 15. Open Data Platform De facto standard platform for open data in Europe and beyond. Search & Discover Data: •Search by keywords •Browse by facets •Explore data with previews & visualization •REST/JSON APIs to access data and metadata Data Management for publishers •Easy store & update of metadata and data Plenty of extensions: harvesting, geographical information, data visualization…. 14
  16. 16. Publishing and Managing Data 15
  17. 17. Search and discovery 16
  18. 18. Metadata 17
  19. 19. Open Data/Content approaches 18 Datasets Existing Datasets (census, geographical, tourism,...) Historic Data (from sensors, events...) Real Time Vertical Systems (mobility, events...) Internet of Things (sensors, Smart meters...) Media Video streams (traffic, surveillance..) Audio (microphones), speaches... Applications NGSI
  20. 20. Being “Smart” requires first being “Aware”  Implementing a Smart City requires gathering and managing context information describing the current and historic “state” of the city  Context information refers to the values of attributes characterizing entities relevant to city services, governance and third-party apps 19 Bus • Location • No. passengers • Driver • Licence plate Citizen • Name-Surname • Birthday • Preferences • Location • ToDo list Shop • Location • Business name • Franchise • offerings Context Information City Governance System City Services Third-party Apps
  21. 21. Context Management in FIWARE  The FIWARE Context Broker GE implements the OMA NGSI-9/10 API: a simple yet powerful standard API for managing Context information  The FIWARE NGSI API is Restful: any web/backend programmer gets quickly used to it 20 Application/Service Bus • Location • No. passengers • Driver • Licence plate Citizen • Name-Surname • Birthday • Preferences • Location • ToDo list Shop • Location • Business name • Franchise • offerings Context Broker NGSI API
  22. 22. FIWARE NGSI: Context from different sources  Context information may come from many sources: • Existing systems • Users, through mobile apps • Sensor networks  Source of info related to a given entity may vary over time 21 Place = “X”, temperature = 30º What’s the current temperature in place “X”? NGSI API A sensor in a pedestrian street The Public Bus Transport Management systemA person from his smartphone It’s too hot!
  23. 23. FIWARE NGSI: Basic interaction  Context Producers publish context information by invoking the updateContext operation on a Context Broker.  Context Consumers can retrieve context information by invoking the queryContext operation on a Context Broker 22 Bus = “X”, location = (x, y) updateContext Context Broker Context Producer Context Consumer queryContext
  24. 24. FIWARE NGSI: Subscription to notifications  Context Consumers can be subscribed to reception of context information complying with certain conditions, using the subscribeContext operation a ContextBroker exports. Such subscriptions may have a duration.  The Context Broker notifies updates on context information to subscribed Context Consumers by invoking the notifyContext operation they export 23 Bus = “X”, next_stop = “A”, arrived = “Yes” updateContext (context_info) Context Broker Context Producer Context Consumer (consumer1) notifyContext (id, context_info) Id = subscribeContext (consumer1, condition, duration )
  25. 25. FIWARE NGSI: Context Providers  Context Providers can be registered to the Context Broker as “holders” of certain context information.  A Context Broker will invoke the queryContext or updateContext operations exported by Context Providers whenever they are queried for, or asked to update, context information they hold 24 Bus = “X”, location = (x, y) queryContext / updateContext Context Broker Context Provider (provider-x) Context Consumer queryContext / updateContext registerContext (provider-x, registration_data, duration, id) )
  26. 26. Integration with existing systems  Context adapters will be developed to interface with existing systems (e.g., municipal services management systems in a smart city) acting as Context Providers, Context Producers, or both  Some attributes from a given entity may be linked to a Context Provider while other attributes may be linked to Context Producers 25 queryContext (e1, attr1, attr2) Context Provider queryContext (e1, attr1) Context Consumer updateContext (e1, attr2) Application Context Broker System B (e.g. Transport system) System A (e.g. GIS, POIs)
  27. 27. FIWARE NGSI: “The SNMP for IoT”  Capturing data from, or Acting upon, IoT devices becomes as easy as to read/change the value of attributes linked to context entities using a Context Broker 26 Context Broker NGSI APINGSI API GET <Oauth token> /V1/contextEntities/lamp1/attributes/presenceSensor PUT <Oauth token> /V1/contextEntities/lamp1/attributes/status “light on” Setting up the value of attribute “status” to “light on” triggers execution of a function in the IoT device that switches the lamp on Issuing a get operation on the “presenceSensor” attribute enables the application to get info about presence of people near the lamp
  28. 28. Integration with sensor networks  The backend IoT Device Management GE enables creation and configuration of NGSI IoT Agents that connect to sensor networks  Each NGSI IoT Agent can behave as Context Consumers or Context Providers, or both 27 FIWARE Context Broker IoT Agent-1 IoT Agent-2 IoT Agent-n IoT Agent Manager create/monitor FIWARE Backend IoT Device Management OMA NGSI API (northbound interface) (southbound interfaces) MQTTETSI M2M IETF CoAP
  29. 29. NGSI resource visualization
  30. 30. NGSI resources visualization
  31. 31. Open data publication Once context information is gathered, a lot of useful complementary FIWARE enablers can be used 30 Context Broker Advanced Web-based UI (AR, 3D) Data/Apps visualization Big Data AnalysisComplex Event Processing Multimedia processing
  32. 32. Target Smart City platform  Smart city platform as a Data/Knowledge Hub  Non-intrusive, open to third parties CKAN Big Data Context Broker Accounting&Payment&Billing IDM&Auth Short-term historic data BigData Processing Data Quering/Action, Publish/Subscr Open Data publishing Real-time processing BI ETL RULES DEFINITION TOOL OPERATIONA L DASHBOARD KPI GOVERNANCE OPEN DATA PORTALS Service orchestrator Context Adapters CEP 31 IoT Backend Device Management measures / commands IoT Broker & Config Management (from sensors to things) IoT/Sens or Open Dataactuators Media streams Real Time Media Stream Processing City Services GIS Inventory Specific Enablers Generic Enablers
  33. 33. Cities in FIWARE Lab 32 City (Country) IoT Open Data NGSI Prototype or details Amsterdam (NL) x x CitySDK-LD Barcelona (Spain) X API implementation to municipal portal Espoo (Finland) x Energy consumption dashboard Helsinki (Finland) x Participation dashboard (CitySDK-Open311) Las Palmas (Spain) x x Port management dashboard Lisbon (Portugal) x Mobility and social networks Lleida (Spain) x x Public transport and accessibility Logroño (Spain) x x Smart watering, City App Malaga (Spain) x x x Citizen as a sensor Rome (Italy) x Data dashboard Santander (Spain) x x Big Data / Open Data publication of IoT Sevilla (Spain) x x x Fountains water management, Crowds detection Torino (Italy) x Security & participation Trento (Italy) x SmartCampus” Open Service platform Valencia (Spain) x x Smart Taxi Vigo (Spain) x x City dashboard
  34. 34. IoT infrastructure from Smart Santander Description: Integration of the IoT SmartSantander facility on top of the FIWARE platform • Devices registration • Observations generation • Definition of new dictionaries to support sensor measurements as well as citizens observations (mainly social related events). • Guidelines to access SmartSantander data (both historical and live data) by using Generic Enablers Used GEs: Backend Device Management, Cosmos, IdM, Context Broker, CKAN Partners: University of Cantabria BackendDevice Management [IDAS] SmartSantander Core Network Service StorerResource Configurator Device Gateway (SensorML, Lightweight SensorML) ContextBroker [ORION] Big Data [COSMOS] IdM [KeyRock] PEP-Proxy [Wilma] FIWARE Architecture Token Request Context Request Historical Request
  35. 35. Malaga Citizen as a Sensor Description: The Malaga CitySense project aims to innovate in data generation for Smart City through the citizen cooperation using use of their Smartphone and his sensors. App that generates data (temperature, light, humidity, pressure, etc.), shows relevant open data from the city, detects iBeacons deployed to track tourism visits. Web application shows data and statistics. Used GEs: Context Broker, Wirecloud, CKAN Partners: Soidemer/TopDigital
  36. 36. Helsinki Open311 (issue reporting) Description: The main objective of the pilot was to prepare a working prototype software to a) integrate Helsinki Open311 issue reporting API to FIWARE platform b) store Helsinki issue reports (based on Open311 standard) to FIWARE Orion Context Broker c) display Helsinki issue reporting API data on FIWARE Wirecloud environment. For this purpose a dedicated Orion Context Broker instance was setup using FIWARE Cloud environment. In addition to this Helsinki Service registry API was used to receive additional information to be displayed in mashup environment. Used GEs: Context Broker, Wirecloud Partners: Forum Virium
  37. 37. Streets video based crowds detection Description: This system captures video stream from city video surveillance cameras and uses complex computational vision algorithms for detecting when a crowd of people is being formed, that is a group of people moving very slowly or not moving at all. The system detects different levels of crowdness and published them into the FIWARE Context Broker (Orion), so that applications can subscribe to them and execute specific actions based on thresholds or other mechanism. A Wirecloud widget allows to define areas for cloud detection in the canvas. Used GEs: Kurento, Orion, Wirecloud Partners: URJC,Naevatec, UPM Kurento Media Server FI-LAB Orion Context Broker Applica on Server h" p://& Video Stream Video Stream Context Events Media Events WWW traffic City streets WWW Browser Alarms Other applica ons
  38. 38. 37 Telefonica, Orange, Engineering & ATOS to build and support standards for smart cities through FIWARE Open Source Community
  39. 39. 31 cities all over Europe & Brazil joins Open and Agile Smart Cities initiative 38
  40. 40. 39 Open & Agile Smart cities initiative Implementation Driven (FIWARE Lab & Acceleration, Organicity, Code4Europe) Common APIs (NGSI) City Models (CitySDK, etc) Open Data Platform (CKAN) Helsinki, Espoo, Vantaa, Tampere, Oulu and Turku in Finland; Copenhagen, Aarhus and Aalborg in Denmark; Brussels, Ghent and Antwerp in Belgium; Porto, Lisbon, Fundão, Palmela, Penela and Águeda in Portugal; Milan, Palermo and Lecce in Italy; Valencia, Santander, Málaga and Sevilla in Spain; and Olinda (Recife), Anapólis (Goiás), Porto Alegre (Rio Grande do Sul), Vitória (Espírito Santo), Colinas de Tocantins (Tocantins) and Taquaritinga (São Paulo) in Brazil
  41. 41. Follow @Fiware on Twitter ! Check latest videos at Join us! 40
  42. 42. Why FIWARE Driver What is needed? What does FIWARE bring? Technology Open, driven by implementation, specs (open source reference implemenation) Sustainable investment over time Open specifications backed by open source reference implementations (see [1], [2]) 100+ M€ of investment (2011-2016) Experimental environment Ability to experiment with real data coming from cities (not just open historic datasets but real-time dat). Free Cloud capacity enabling entrepreneurs to test and host a permanent showcase of their applications. 15 cities (7 in Spain) already working on setting up a connection to FIWARE Lab [3] 3000+ cores, 16Tb RAM and 750+ Tb HD will be the free computing capacity provided by the FIWARE Lab Cloud across 16 nodes distributed in Europe Incentives for creating the ecosystem Engagement of technology providers, entrepreneurs, data providers, customers Funding for first entrepreneurs joining the ecosystem. Invesment in promotion and dissemination activities 52 partners, 13 countries (just FIWARE) 100 M€ devoted to fund entrepreneurs in 2014- 2016. Additional opportunities in Horizon 2020. 6,2+ M€ devoted to dissemination Global footprint Helping entrepreneurs and technology providers to create opportunities not just in Europe but other regions (Latam, Asia and, why not, USA) FIWARE Lab nodes in Mexico and Brazil. Conversations between EC and public authorities in Mexico and Brazil to explore collaboration opportunities [1] – [2] – [3] – 41
  43. 43. FIWARE = advanced OpenStack-based Cloud capabilities + library of APIs that ease development of applications 42 Sustainability ensured Driven by implementation
  44. 44. The FIWARE Accelerator Program 43
  45. 45. How can standard Smart City data models easing common solutions be defined? The problem  Smart City apps can be ported from one Smart City to another once their platforms provide the same set of APIs, that’s why FIWARE brings a rather high value  Without standard data models, Smart City apps would need to come with adapters that transform data made available by the city so that it complies with the data model handled by the app but that has proven to be easy with OMA NGSI (overall if NGSI is at both ends)  Creation of standard Smart City data models would allow to avoid performing this kind of adaptation and make portability of Smart City apps across Smart City platforms a pretty straightforward task  How creation of these standard Smart City data models can be fostered? 44
  46. 46. How can standard Smart City data models easing common solutions be defined? The solution  A “design by committee” approach would not be the best approach: • Such kind of approach has proven to be wrong in many other standardization efforts in the past • Who grants that the defined model is suitable for what apps need and developers want to have?  we need a “driven by implementation” approach: • Identify real applications that solves a real problem and cities would like to see running in their cities • Check what data model they have been designed to work with and take it as input • Carry out a “data curation” process targeted to extract commonalities and polish the data model  You will end with a set of standard data models and soon a portfolio of killer Smart City apps working! 45
  47. 47. How can standard Smart City data models easing common solutions be defined? FIWARE  The FIWARE Accelerator program enables to identify killer Smart City apps developed by entrepreneurs • These applications can as basis for definition of a standard set of Smart City data models • FIWARE resources would be devoted to help coordinating the data curation process • Involvement in this process becomes also an incentive for the entrepreneurs to join the program (“I want to influence the standard so that my app can easily align with it”, “I want to provide one of the first example applications”) • There are 80 M€ for entrepreneurs that can be put at work!  Cities connected to FIWARE Lab will play a key role: • Their data models will be contrasted/analyzed against those coming from the apps and other cities • They would get involved in the data curation process 46
  48. 48. FIWARE Generic Enablers (GEs)  A FIWARE Generic Enabler (GE):  set of general-purpose platform functions available through APIs  Building with other GEs a FIWARE Reference Architecture  FIWARE GE Specifications are open (public and royalty-free)  FIWARE GE implementation (FIWARE GEi):  Platform product that implements a given GE Open Spec  There might be multiple compliant GEis of each GE Open Spec  One open source reference implementation of each FIWARE GE (FIWARE GEri): • Well-known open source license • Publicly available Technical Roadmap updated in every release  Available FIWARE GEis, GEris and incubated enablers are published on the FIWARE Catalogue 47
  49. 49. FIWARE Instances  Future Internet Applications run on top of “FIWARE Instances” that are built by “FIWARE Instance Providers” upon: • selection of FIWARE GEis (products) from the FIWARE Catalogue. • assembly of selected FIWARE GEis with added-value components. FIWARE Catalogue FIWARE GE implementations FIWARE Instance 48 Future Internet App assemble… added-value components
  50. 50. Domain-specific platforms = FIWARE + specific enablers FIWARE 49 GEs SMART City Apps SMART Factory Apps SMART Agrifood Apps Domain specific enablers
  51. 51. FIWARE Ops: suite of tools for FIWARE providers Deployment Deployment of basic Cloud Hosting GEs and Monitoring Adapters in a FIWARE node Federation Management Federate a new FIWARE node within a given FIWARE instance (e.g., the FIWARE Lab) Connectivity Management Manage connectivity of services across FIWARE nodes of a FIWARE instance Service Offer Management Registration and deployment of additional Generic Enablers, Specific Enablers and complementary Future Internet Facilities
  52. 52. Building the FIWARE ecosystem: the vision FIWARE Catalogue creation of dedicated GE instances Use of Global shared GE instance FIWARE provider A 51 Applications FIWARE provider B Applications
  53. 53.  ff  4,2 M€ promotion campaign • Campus Party events • Startup Weekend events • Chambers of Commerce • 870 K€ in prizes  100 M€ of funding devoted to entrepreneurs in phase 3 of the FIWARE program • 12 nodes in Europe providing up to 3000+ cores, 16TB+ Ram, 750TB+ HD • Creation of nodes in Mexico (1000+ cores) and Brazil • Level 1 and Level 2 support for the nodes • Showcases for developers, infrastructures, smart businesses FIWARE Lab: A working instance of FIWARE working as “the meeting point” where innovation takes place