For the full video of this presentation, please visit: https://www.edge-ai-vision.com/2020/11/federated-edge-computing-system-architectures-a-presentation-from-intel/ For more information about edge AI and computer vision, please visit: https://www.edge-ai-vision.com Vaidyanathan Krishnamoorthy, Edge Inference Solutions Architect at Intel, presents the “Federated Edge Computing System Architectures” tutorial at the September 2020 Embedded Vision Summit. With ever-increasing amounts of video and other sensor data, and growing requirements for privacy and low latency, inferencing at the edge is increasingly attractive. But there are many ways to allocate and coordinate computing resources for edge inferencing. For example, to achieve scale and fault tolerance, design principles from cloud computing can be applied to create compute clusters at the edge that are managed by the cloud, an approach called “federated computing.” In this talk, Krishnamoorthy explores a range of edge computing system architectures, with a focus on federated computing. He illustrates how these system architectures utilize Intel CPUs and accelerators to address real-world use cases in retail and industrial applications.

1. Federated Edge Computing
System Architectures
Vaidy Krishnamoorthy, Intel Corporation
September 2020

2. Notice and Disclaimers
2
INFORMATION IN THIS DOCUMENT IS PROVIDED IN CONNECTION WITH INTEL PRODUCTS. NO LICENSE, EXPRESS OR IMPLIED, BY
ESTOPPEL OR OTHERWISE, TO ANY INTELLECTUAL PROPERTY RIGHTS IS GRANTED BY THIS DOCUMENT. EXCEPT AS PROVIDED IN
INTEL'S TERMS AND CONDITIONS OF SALE FOR SUCH PRODUCTS, INTEL ASSUMES NO LIABILITY WHATSOEVER AND INTEL
DISCLAIMS ANY EXPRESS OR IMPLIED WARRANTY, RELATING TO SALE AND/OR USE OF INTEL PRODUCTS INCLUDING LIABILITY
OR WARRANTIES RELATING TO FITNESS FOR A PARTICULAR PURPOSE, MERCHANTABILITY, OR INFRINGEMENT OF ANY PATENT,
COPYRIGHT OR OTHER INTELLECTUAL PROPERTY RIGHT.
Intel technologies’ features and benefits depend on system configuration and may require enabled hardware, software or service
activation. Performance varies depending on system configuration. No computer system can be absolutely secure. Check with
your system manufacturer or retailer or learn more at www.intel.com.
This document contains information on products, services and/or processes in development. All information provided here is
subject to change without notice. Contact your Intel representative to obtain the latest forecast, schedule, specifications and
roadmaps.
© Intel Corporation. Intel, the Intel logo, and other Intel marks are trademarks of Intel Corporation or its subsidiaries. Other
names and brands may be claimed as the property of others.

3. Agenda
3
• Need for Edge Inferencing
• Federated Edge Inferencing Architectures
• Home / Small Retail
• Large Retail
• Edge Cloud Orchestration with Kubernetes
• Edge with AI Accelerators
• Edge with AI Appliances
• Industrial Edge (Private 5G)
• Telco Edge (future)
• Intel Products addressing Edge Inferencing

4. © 2020 Intel Corporation
Need for Edge Inferencing
4

5. 5
The Edge and its Challenges
DEVICES /
THINGS
CORE
NETWORK
CLOUD
DATA CENTER
NETWORK EDGE,
REGIONAL CLOUD
ON PREM
EDGE
▪ L A T E N C Y
▪ R E A L - T I M E
▪ B A N D W I D T H
▪ S E C U R I T Y
▪ C O N N E C T I V I T Y
EDGE
E D G E
R E Q U I R E M E N T S

6. © 2020 Intel Corporation
Federated Edge Inferencing Architectures
6

7. Residential and Small Retail
7
• IP Cameras connected to Cloud directly
• Limited Cameras
• Video as a service Business Model
possible
• Use-cases:
• Person Detection
• People Counting
• Challenges:
• Latency Critical usages
• Device Manageability

8. Large Retail
8
• IP Cameras and other sensors connected to Edge AI
servers
• Use-cases:
• Person-of-Interest
• People Counting
• Demographics
• Social distancing monitoring
• Shelf Monitoring
• Scale Edge AI server based on workload
• High Availability and Reliability using Cloud-native stacks
• Ability to federate workloads in Edge AI servers using
Cloud stacks
• Work even with intermittent connection interruption with
the cloud
• Possibility to add AI Cameras

9. Edge Cloud Orchestration
9
• Kubernetes provides Cloud
Orchestration and High Availability
• Edge Cloud scaling based on workload
• Cloud used for:
• Orchestration of Edge servers and
analytics
• Retraining to improve model
accuracies
• Long-term Storage
• Manageability and Software
Upgrades
kube
controller
manager
cloud
controller
manager
kube
scheduler
etcd
kube
apriserver
kubelet kubelet kubelet
kube
proxy
kube
proxy
kube
proxy
cloud
connector

10. Edge with Server+ AI Accelerator K8s Cluster
10
OS (Linux)
Container
Runtime
Kubelet
Pod...
Storage
K 8 s W o r ke r N o d e
PodPod
O S ( L i n u x )
Container
Runtime
CSP
Runtime
API Server
Replication
Controller
etcd
Storage
K 8 s C o n t ro l l e r

11. Edge with AI Appliance K8s Cluster
11
O S ( L i n u x )
Container
Runtime
CSP
Runtime
API Server
Replication
Controller
etcd
Storage
K 8 s C o n t ro l l e r
AI Appliance
AI Appliance
AI Appliance
AI Appliance
OS (Linux)
Container
Runtime
Kubelet
Pod...
Storage
K 8 s W o r ke r N o d e
PodPod

12. 5G Private Network Cloud for Industrial
12
• Edge cloud supporting 5G RAN
• Autonomous Systems needing low latency
• Industrial use-cases
• Fault Detection
• Robot Monitoring
• Emergency stop systems
• Private / Hybrid Cloud implementation
• Scale Edge Cloud based on Workload
• Orchestration using Cloud native
frameworks

13. Telco Cloud (Future)
EDGE DISTRIBUTION IS UNDERWAY
EDGE
CLIENT
TELCO
CLOUD
FUTURE EDGE
EDGE
CLIENT
UP UNTIL NOW
EDGE
CLIENT
NEXT STEP EVOLUTION
LATENCY BW, SECURITY
PRIVACY
LOWER
TCO
Decentralized,
Stateful,
Discoverable,
Collaborative
Edge
VALUE
DRIVERS
TO WA R D S … A M O R E DY N A M I C E D G E

14. © 2020 Intel Corporation
Intel Products addressing
Edge Inferencing
14

15. Intel Products
15
SOLUTIONS
Data Center
DL Inference
(Goya)
Real-Time &
Multi-Use DL
Inference
Data Center
DL Training
(Gaudi)
Edge
DL Inference
Workload Breadth AI Specific
GPU FPGACPU
AI, HPC, Media
& Graphics
Multi-Purpose
AI Foundation
DEDICATED
SOFTWARE
PROCESS
STORE MOVE
INTEL 3D NAND SSD
Silicon
Photonics Ethernet

16. © 2020 Intel Corporation
Recap
16

17. Recap
17
• AI Inferencing drives Edge Deployments
• Cloud-Native frameworks @ the Edge
• Intel – an Active Contributor to Cloud-Native Frameworks
• Different Intel Products scaling to needs @ the Edge

18. Resources
18
1) Kubernetes - https://www.Kubernetes.io
2) Kube Edge - https://www.kubeedge.io
3) Intel Movidius Products -
https://www.intel.com/content/www/us/en/products/processors/movidius-vpu.html
4) Intel OpenVINO - https://docs.openvinotoolkit.org/