Uploaded byFIWARE
FIWARE Global Summit - FogFlow, a new GE for IoT Edge Computing
FogFlow is a cloud-edge orchestration framework designed for IoT edge computing, enabling dynamic data processing flows and serverless edge computing. It offers simplified setup and programming models, including service topology for on-demand data and fog functions triggered by context availability. The document also details deployment steps, integration with other systems, and use cases such as smart parking.
More Related Content
![David Henthorn [Rose-Hulman Institute of Technology] | Illuminating the Dark ...](https://cdn.slidesharecdn.com/ss_thumbnails/influxdaysemeahenthorn-210504160049-thumbnail.jpg?width=640&height=640&fit=bounds)
![Phil Day [Configured Things] | Policy-Driven Real-Time Data Filtering from Io...](https://cdn.slidesharecdn.com/ss_thumbnails/influxdays2021policydrivenfilteringwithfluxphilday1-210426175430-thumbnail.jpg?width=640&height=640&fit=bounds)
Similar to FIWARE Global Summit - FogFlow, a new GE for IoT Edge Computing
FIWARE Global Summit - FogFlow, a new GE for IoT Edge Computing
- 1. FIWARE at theedge: FogFlow, a new GE for IoT edge computing Bin Cheng (bin.cheng@neclab.eu), Ernö Kovacs (ernoe.kovacs@neclab.eu) NEC Labs Europe
- 2. Agenda § Overview ofFogFlow • motivation, value proposition, major design, key feature § How to deploy FogFlow • Set up your own FogFlow system § How to use the programming models in FogFlow • Service topology: launching dynamic data flows on demand, development process and use case (anomaly detection) • Fog function: serverless edge computing, development process and use case (smart parking) § Integration with other systems • AWS IoT, Orion Context Broker § References 1
- 3. 2 Part I: Overviewof FogFlow
- 4. Motivation of FogFlow 3 LowerComplexity Lower Cost Higher Efficiency Smart Cities Public Safety Smart Factories IoT services difficult to program Infrastructure difficult to manage Edges IoT devices not able to be smart After using NEC FogFlow Edge Computing Framework Lack of model High complexity from heterogeneity and dynamics No fast, closed control loop Development + Operation Development + Operation Faster response time, better scalability
- 5. Value Proposition ofFogFlow 4 IoT Service providers Infrastructure Providers Device Providers Edge programming model Fast control loop Optimized orchestration Low learning cost Low OPEX Fast time-to-market Improved QoS
- 6. FogFlow: Context-driven Cloud-EdgeOrchestration § FogFlow is a cloud-edge orchestrator to orchestrate dynamic NGSI-based data processing flows on-demand between producers and consumers for providing timely results to make fast actions, based on context (system context and data context) Producers (sensors) Consumers (actuators) cloud edge edge edge raw context information timely results fast actions FogFlow dynamic processing flows Data context System context
- 7. How It Works 6 Templateof your IoT Service Execution Plan Deployment Plan cloud edge1 edge2 Task generation Task specification Task deployment Cloud-Edge Programming model Cloud-Edge Orchestration Graphical editor Automation & optimizations Cloud-Edge Environment IoT Devices
- 8. Key Features (1):Agile Edge Programming 7 Reusable building blocks Compositing them with declarative hints Submit Select Your service is ready in minutes Programming Cloud and Edges easily and fast
- 9. Key Features (1):Automated and Optimized Orchestration for Cloud-Edge 8 Connected Device Backend Cloud(s) Other Edges Sensing Notify Orchestrating Collaborative deployment Closed control loop Dynamic processing flows Reacting Nearby Edge Collaborative deployment Orchestrating and managing dynamic processing flows over cloud and edges in an efficient and optimized manner
- 10. 9 Part I: Deploymentof FogFlow
- 11. System Architecture ofFogFlow 10
- 12. System Components inFogFlow 11
- 13. Set Up TheCloud Part: § Step 1: download the following compressed file • wget https://github.com/smartfog/fogflow/raw/master/deployment/core/fogflow-core.tar.gz § Step 2: extract deployment scripts from the downloaded file • tar xvf fogflow-core.tar.gz § Step3: change the configuration files for each fogflow components, including • discovery, broker, master, worker, designer § Step4: use docker-compose to launch all components • docker-compose up or docker-compose start 12
- 14. How configuration filesshould be changed accordingly? discovery-config.json broker-config.json worker-config.json master-config.json designer-config.json Nothing to change "my_ip": "155.54.239.141", "host": "155.54.239.141", "my_ip": "155.54.239.141", "discoveryURL": "http://155.54.239.141:8071/ngsi9", "brokerURL": "http://155.54.239.141:8070/ngsi10", "agentIP": "155.54.239.141", HOST_IP = 155.54.239.141
- 15. Set Up TheEdge Part: 14 § Step 1: download the following compressed file • wget https://github.com/smartfog/fogflow/raw/master/deployment/edge/fogflow-edge.tar.gz § Step 2: extract deployment scripts from the downloaded file • tar xvf fogflow-edge.tar.gz § Step3: change the configuration files for each fogflow components, including • broker, worker § Step4: use docker-compose to launch all components • docker-compose up or docker-compose start
- 16. How configuration filesshould be changed accordingly? 15 broker-config.json worker-config.json "host": "155.54.239.141", "discoveryURL": "http://155.54.239.141:8071/ngsi9", "my_ip": "155.54.239.141", "message_bus": "amqp://admin:mypass@155.54.239.141:5672/", "iot_discovery_url": "http://155.54.239.141:8071/ngsi9", HOST_IP = 155.54.239.141
- 17. Verify Your Deploymentvia FogFlow Dashboard § Check if the FogFlow dashboard can be accessed from your browser 16 HOST_IP = 155.54.239.141 http://HOST_IP:8080/
- 18. Other possible changesto the configuration files § Port numbers used by each component • Must pay attention to the dependency between different components § Geo-location of each component 17 "physical_location":{ "longitude": 139.463377, "latitude": 36.509163, "section": "01", "district": "02", "city": "Tokyo" },
- 19. 18 Part II: ProgrammingModels in FogFlow
- 20. Two Types ofProgramming Models in FogFlow § Service topology: on-demand data processing flows • Triggered by a requirement on demand • Output-driven § Fog function: serverless edge computing, function triggered by context availability • Triggered by context availability automatically • Input-driven 19 requirement Fog function availability of input data Linked automatically
- 21. Orchestration of ServiceTopology 20 Service Topology Execution Plan Deployment Plan cloud edge1edge2 q Expected output q Scope q scheduler locality aware deployment dynamic execution graph
- 22. Context-driven Cloud-Edge Orchestration:granularity, scope 21 Data Context and availability (metadata, availability) System Context (locality, mobility, capacity, security, …) Programming model with graphical editor scope granularity
- 23.
- 24.
- 25. Designing a IoTService 24
- 26. Developing Dockerized Operators 25 Templatesfor Python/Javascript are available in github fogflow/application/template
- 27. Defining Your ServiceTopology 26
- 28. Triggering Your IoTService 27 Service Topology Execution Plan Deployment Plan cloud edge1edge2 q Expected output q Scope q scheduler locality aware deployment dynamic execution graph Orchestration requirement
- 29. Fog Function inFogFlow: serverless edge computing § Fog function: just do one thing for a single functionality • Triggered by the availability(metadata) of its inputs, light-weight 28 fog function input function instances trigger ff2 Twin ATwin BTwin C ff1 Digital Twins Fog Function Instances A network of twins, linked via fog functions
- 30. How Fog FunctionWorks in FogFlow 29 Input Annotation Docker image Output Annotation IoT Discovery Broker(s) FogFlow Orchestrator Function instance(s) FogFlow Worker(s) (cloud or edges) IoT Device(s) fog function Twins 1) Submit Fog function 2)SUBSCRIBE, NGSI9 3) UPDATE, NGSI10 4) UPDATE, NGSI9 5)NOTIFY, NGSI9 5) Create and deploy fog function instances 6) Processing data and create/update digital twin
- 31. Editor to Annotatea Fog Function in FogFlow 30 Python or Javascript Code or annotation Implementation image docker images
- 32. Register a basicoperator 31
- 33. Implement an operator §template is available at github • https://github.com/smartfog/fogflow/tree/master/application/fogfunction/javascript 32 exports.handler = function(contextEntity, publish, query, subscribe) Example of how to publish the generate result Example of how to query something Example of how to subscribe something
- 34. Use Case: SmartParking 33 Connected car private site Public site Real-time estimation Prediction Arrival time Recommender Utilizing NGSI API to access digital twins Digital twins: Public parking sites, private parking sites, connected cars
- 35. 34 Part III: Integrationwith Other FIWARE GEs
- 36. Integration with OtherGEs 35 Distributed Context Management System OrionProcessing tasks FogFlow subscription APPS Other GE(s) sensors actuators FogFlow Dashboard notify e.g. Cygnus non-NGSI devices Adapter(s) (e.g., openMTC, IoT Agent)
- 37. Reference and Acknowledgement §The code repository of FogFlow is available at github, https://github.com/smartfog/fogflow § Online tutorial is available at http://fogflow.readthedocs.io/ § This work has been partially funded by the European Union’s Horizon 2020 research and innovation program within the CPaaS.io project under Grant Agreement No. 723076 36
- 38.