Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

FIWARE Global Summit - FogFlow, a new GE for IoT Edge Computing

204 views

Published on

Presentation by Bin Cheng
Senior Researcher, NEC Labs Europe

FIWARE Global Summit
8-9 May, 2018
Porto, Portugal

Published in: Technology
  • Be the first to comment

FIWARE Global Summit - FogFlow, a new GE for IoT Edge Computing

  1. 1. FIWARE at the edge: FogFlow, a new GE for IoT edge computing Bin Cheng (bin.cheng@neclab.eu), Ernö Kovacs (ernoe.kovacs@neclab.eu) NEC Labs Europe
  2. 2. Agenda § Overview of FogFlow • motivation, value proposition, major design, key feature § How to deploy FogFlow • Set up your own FogFlow system § How to use the programming models in FogFlow • Service topology: launching dynamic data flows on demand, development process and use case (anomaly detection) • Fog function: serverless edge computing, development process and use case (smart parking) § Integration with other systems • AWS IoT, Orion Context Broker § References 1
  3. 3. 2 Part I: Overview of FogFlow
  4. 4. Motivation of FogFlow 3 Lower Complexity Lower Cost Higher Efficiency Smart Cities Public Safety Smart Factories IoT services difficult to program Infrastructure difficult to manage Edges IoT devices not able to be smart After using NEC FogFlow Edge Computing Framework Lack of model High complexity from heterogeneity and dynamics No fast, closed control loop Development + Operation Development + Operation Faster response time, better scalability
  5. 5. Value Proposition of FogFlow 4 IoT Service providers Infrastructure Providers Device Providers Edge programming model Fast control loop Optimized orchestration Low learning cost Low OPEX Fast time-to-market Improved QoS
  6. 6. FogFlow: Context-driven Cloud-Edge Orchestration § FogFlow is a cloud-edge orchestrator to orchestrate dynamic NGSI-based data processing flows on-demand between producers and consumers for providing timely results to make fast actions, based on context (system context and data context) Producers (sensors) Consumers (actuators) cloud edge edge edge raw context information timely results fast actions FogFlow dynamic processing flows Data context System context
  7. 7. How It Works 6 Template of your IoT Service Execution Plan Deployment Plan cloud edge1 edge2 Task generation Task specification Task deployment Cloud-Edge Programming model Cloud-Edge Orchestration Graphical editor Automation & optimizations Cloud-Edge Environment IoT Devices
  8. 8. Key Features (1): Agile Edge Programming 7 Reusable building blocks Compositing them with declarative hints Submit Select Your service is ready in minutes Programming Cloud and Edges easily and fast
  9. 9. Key Features (1): Automated and Optimized Orchestration for Cloud-Edge 8 Connected Device Backend Cloud(s) Other Edges Sensing Notify Orchestrating Collaborative deployment Closed control loop Dynamic processing flows Reacting Nearby Edge Collaborative deployment Orchestrating and managing dynamic processing flows over cloud and edges in an efficient and optimized manner
  10. 10. 9 Part I: Deployment of FogFlow
  11. 11. System Architecture of FogFlow 10
  12. 12. System Components in FogFlow 11
  13. 13. Set Up The Cloud Part: § Step 1: download the following compressed file • wget https://github.com/smartfog/fogflow/raw/master/deployment/core/fogflow-core.tar.gz § Step 2: extract deployment scripts from the downloaded file • tar xvf fogflow-core.tar.gz § Step3: change the configuration files for each fogflow components, including • discovery, broker, master, worker, designer § Step4: use docker-compose to launch all components • docker-compose up or docker-compose start 12
  14. 14. How configuration files should be changed accordingly? discovery-config.json broker-config.json worker-config.json master-config.json designer-config.json Nothing to change "my_ip": "155.54.239.141", "host": "155.54.239.141", "my_ip": "155.54.239.141", "discoveryURL": "http://155.54.239.141:8071/ngsi9", "brokerURL": "http://155.54.239.141:8070/ngsi10", "agentIP": "155.54.239.141", HOST_IP = 155.54.239.141
  15. 15. Set Up The Edge Part: 14 § Step 1: download the following compressed file • wget https://github.com/smartfog/fogflow/raw/master/deployment/edge/fogflow-edge.tar.gz § Step 2: extract deployment scripts from the downloaded file • tar xvf fogflow-edge.tar.gz § Step3: change the configuration files for each fogflow components, including • broker, worker § Step4: use docker-compose to launch all components • docker-compose up or docker-compose start
  16. 16. How configuration files should be changed accordingly? 15 broker-config.json worker-config.json "host": "155.54.239.141", "discoveryURL": "http://155.54.239.141:8071/ngsi9", "my_ip": "155.54.239.141", "message_bus": "amqp://admin:mypass@155.54.239.141:5672/", "iot_discovery_url": "http://155.54.239.141:8071/ngsi9", HOST_IP = 155.54.239.141
  17. 17. Verify Your Deployment via FogFlow Dashboard § Check if the FogFlow dashboard can be accessed from your browser 16 HOST_IP = 155.54.239.141 http://HOST_IP:8080/
  18. 18. Other possible changes to the configuration files § Port numbers used by each component • Must pay attention to the dependency between different components § Geo-location of each component 17 "physical_location":{ "longitude": 139.463377, "latitude": 36.509163, "section": "01", "district": "02", "city": "Tokyo" },
  19. 19. 18 Part II: Programming Models in FogFlow
  20. 20. Two Types of Programming Models in FogFlow § Service topology: on-demand data processing flows • Triggered by a requirement on demand • Output-driven § Fog function: serverless edge computing, function triggered by context availability • Triggered by context availability automatically • Input-driven 19 requirement Fog function availability of input data Linked automatically
  21. 21. Orchestration of Service Topology 20 Service Topology Execution Plan Deployment Plan cloud edge1edge2 q Expected output q Scope q scheduler locality aware deployment dynamic execution graph
  22. 22. Context-driven Cloud-Edge Orchestration: granularity, scope 21 Data Context and availability (metadata, availability) System Context (locality, mobility, capacity, security, …) Programming model with graphical editor scope granularity
  23. 23. Development Process 22
  24. 24. Use Case Example 23
  25. 25. Designing a IoT Service 24
  26. 26. Developing Dockerized Operators 25 Templates for Python/Javascript are available in github fogflow/application/template
  27. 27. Defining Your Service Topology 26
  28. 28. Triggering Your IoT Service 27 Service Topology Execution Plan Deployment Plan cloud edge1edge2 q Expected output q Scope q scheduler locality aware deployment dynamic execution graph Orchestration requirement
  29. 29. Fog Function in FogFlow: serverless edge computing § Fog function: just do one thing for a single functionality • Triggered by the availability(metadata) of its inputs, light-weight 28 fog function input function instances trigger ff2 Twin ATwin BTwin C ff1 Digital Twins Fog Function Instances A network of twins, linked via fog functions
  30. 30. How Fog Function Works in FogFlow 29 Input Annotation Docker image Output Annotation IoT Discovery Broker(s) FogFlow Orchestrator Function instance(s) FogFlow Worker(s) (cloud or edges) IoT Device(s) fog function Twins 1) Submit Fog function 2)SUBSCRIBE, NGSI9 3) UPDATE, NGSI10 4) UPDATE, NGSI9 5)NOTIFY, NGSI9 5) Create and deploy fog function instances 6) Processing data and create/update digital twin
  31. 31. Editor to Annotate a Fog Function in FogFlow 30 Python or Javascript Code or annotation Implementation image docker images
  32. 32. Register a basic operator 31
  33. 33. Implement an operator § template is available at github • https://github.com/smartfog/fogflow/tree/master/application/fogfunction/javascript 32 exports.handler = function(contextEntity, publish, query, subscribe) Example of how to publish the generate result Example of how to query something Example of how to subscribe something
  34. 34. Use Case: Smart Parking 33 Connected car private site Public site Real-time estimation Prediction Arrival time Recommender Utilizing NGSI API to access digital twins Digital twins: Public parking sites, private parking sites, connected cars
  35. 35. 34 Part III: Integration with Other FIWARE GEs
  36. 36. Integration with Other GEs 35 Distributed Context Management System OrionProcessing tasks FogFlow subscription APPS Other GE(s) sensors actuators FogFlow Dashboard notify e.g. Cygnus non-NGSI devices Adapter(s) (e.g., openMTC, IoT Agent)
  37. 37. Reference and Acknowledgement § The code repository of FogFlow is available at github, https://github.com/smartfog/fogflow § Online tutorial is available at http://fogflow.readthedocs.io/ § This work has been partially funded by the European Union’s Horizon 2020 research and innovation program within the CPaaS.io project under Grant Agreement No. 723076 36
  38. 38. Thank you! http://fiware.org Follow @FIWARE on Twitter

×