Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

5G Cellular D2D RDMA Clusters


Published on

5G cellular communications opens an opportunity for a wider realm in networking, one which can encompass the Internet (including IOT). With D2D, Device-to-Device communications, devices can cluster themselves into high power computing platforms.
We show the practicality of mapping RDMA over wireless. Think wireless InfiniBand or wireless OmniPath.
The ensuing number of cluster can be exponentially larger than the number of devices. Clusters can run side-by-side as multi instance software.
See how protocol-free networking becomes realistic as D2D clusters facilitate networking code becoming loadable, user-space software.

Published in: Technology
  • Login to see the comments

5G Cellular D2D RDMA Clusters

  1. 1. Mobile D2D RDMA CAAP, Cluster As Application Platform • Opens new realm, beyond Internet • Run distributed apps on wireless clusters not PAAS (Platform As A Service), but native, CAAP • User-programmable, protocol-free networking Yitzhak Bar Geva, Prof. Jon Crowcroft Contact:
  2. 2. Easy to Grasp Model 1.Take a High Performance RDMA Computing center. 2.Replace all of the wire links with wireless. 3.Replace the computing nodes with 5G mobile devices. 4.All of the wireless links are now D2D. 5.Voilà. Autonomous Mobile Distributed Computing Platform
  3. 3. The key thing here is the Platform
  4. 4. Platform Evolution
  5. 5. D2D Clusters
  6. 6. Pooled Mobile Device Clusters Vastly wider Application Space Order of magnitude more clusters than devices
  7. 7. Pooled Device Clusters become Powerful Application Platforms App App App App App App
  8. 8. Powerful over and beyond sheer horsepower • Write, load and run distributed apps in popular languages • Opens the RDMA cluster model to the world • Devices pool their cores and storage, can belong to multiple clusters and the Internet • New flexible message-passing network schemes by programming RDMA verbs (or newer, MPI optimized PSM).
  9. 9. • Multi-Instance. Membership in a cluster is seen as a single instance in the device • Multi-Cluster Device may be member of multiple clusters simultaneously • Multi-App Cluster itself may be multi-instance, multi-process RDMA Model Flexibility
  10. 10. A cooperative distributed programming platform in an era of friendly concurrent programming languages Networking and clouds no longer externally supplied services Power grows over time Freely configurable, dynamically reconfigurable IOT-type pooled-device clouds, on a common platform
  11. 11. Copyright 2012 IOL15 Remote Direct Memory Access Remote – data transfers between nodes in a network Direct – no Operating System Kernel involvement in transfers – everything about a transfer offloaded onto Interface Card Memory – transfers between user space application virtual memory – no extra copying or buffering Access – send, receive, read, write, atomic operations
  12. 12. Copyright 2012 IOL16 RDMA Benefits High throughput Low latency High messaging rate Low CPU utilization Low memory bus contention Message boundaries preserved Asynchronous operation
  13. 13. Copyright 2012 IOL17 RDMA Technologies InfiniBand – (41.8% of top 500 supercomputers) – SDR 4x – 8 Gbps – DDR 4x – 16 Gbps – QDR 4x – 32 Gbps – FDR 4x – 54 Gbps iWarp – internet Wide Area RDMA Protocol – 10 Gbps RoCE – RDMA over Converged Ethernet – 10 Gbps – 40 Gbps
  14. 14. Copyright 2012 IOL18 RDMA Architecture Layering User Application OFA Verbs API Physical Data Link Network Transport IWARP “RNIC” RoCE “NIC” InfiniBand “HCA” RDMAP DDP MPA TCP IP IB Transport API IB Transport IB Network Ethernet MAC & LLC IB Link Ethernet PHY IB PHY OSI Layers CA
  15. 15. Kernel Bypass
  16. 16. RDMA User Space programmable
  17. 17. If only we had an InfiniBand wireless chip…
  18. 18. RDMA Over Wireless Clusters Wireless Channel Adapter Mobile Clusters
  19. 19. RDMA Mapped Over Wireless Clusters
  20. 20. Becomes Wireless HPC where these are mobile devices
  21. 21. InfiniBand
  22. 22. InfiniBand
  23. 23. RoCE useful in our humble opinion only in the interim. InfiniBand over wireless is the prime choice
  24. 24. ROCE
  25. 25. RoCE
  26. 26. Soft-RoCE
  27. 27. iWARP
  28. 28. iWARP - Internet Wide Area RDMA Protocol
  29. 29. Copyright 2012 IOL35 Software RDMA Drivers Softiwarp – – open source kernel module that implements iWARP protocols on top of ordinary kernel TCP sockets – interoperates with hardware iWARP at other end of wire Soft RoCE – – open source IB transport and network layers in software over ordinary Ethernet – interoperates with hardware RoCE at other end of wire
  30. 30. Means that… Clusters made from geographically disparate nodes Run smoothly over the Internet (TCP/IP) Mobile/Wireless clusters can cluster-up together with enormously powerful HPC clusters World wide multi-million node clusters, each its own Internet or cloud Clusters can be IP addressable entities, making a greatly expanded Internet, yet the vast D2D space may be distinct from the Internet.
  31. 31. For Linux and Windows. Includes kernel-drivers, channel-oriented RDMA and send/receive ops, kernel bypass, kernel and user-level API, MPI, sockets data exchange(RDS, SDP), NAS and SAN storage (iSER, NFS-RDMA, SRP) and File Systems/DB systems. RDMA network and fabric technologies for: legacy 10 GB Ethernet, iWARP for Ethernet, RoCE, and 10/20/40 Gigabit InfiniBand. OFED - OpenFabrics Enterprise Distribution
  32. 32. Legacy services through Upper Level Protocol kernel modules.
  33. 33. Programmability with RDMA verbs (or newer PSM) Any agreed upon networking code loaded on all the nodes becomes a so-called, protocol. Next are programmable RDMA verbs.
  34. 34. Verbs
  35. 35. Java 7 VM over verbs
  36. 36. Custom Networking Schemes Network management, switching and routing, programmed verbs or PSM all provided by the devices as user-space loadable software. Freedom from rigid protocols is the logical next step.
  37. 37. and began writing their own protocols
  38. 38. Protocols, essential where needed
  39. 39. Imagine if Software had to be Protocol Conformant but a hindrance where not needed
  40. 40. Network protocols are like software before compilers Painstaking, Meticulous Only one language Ran on only one type of machine
  41. 41. Talk about protocols Loadable networking code is application code Want to switch or update the so-called, eh, protocol? Just
  42. 42. SDNLike an to run different network schemes pushing code instead of data onto all the nodes
  43. 43. Escape Protocol Gravity
  44. 44. If programming RDMA verbs is so then why is hardly anyone doing it? A. Because it ain’t easy yet. That it’s programmable is remarkable in itself, but it's still like assembly coding. We need a compiler (on the checklist)
  45. 45. RDMA transfers between GPUs
  46. 46. Virtualized RDMA
  47. 47. Some Use Cases Realtime statistical data mining or data mashing generating visual outputs in a close region, like say, realtime crowd behavior patterns in shopping malls. Apps which today have to be manually installed, could be seeded and run. No need for web sites, uploads to the Internet, fixed IP addresses, run as local GUI apps. Cooperative coding in realtime. Developers/users hacking together on the same running app platform. Vastly more flexible Internet of Things
  48. 48. Car clusters
  49. 49. Such a Simple Conceptual Model must be easy to implement
  50. 50. Sorry, not quite There are Formidable Challenges, But…
  51. 51. Nothing we don’t know how to do
  52. 52. • On-the-fly clustering of mobile devices • Wireless InfiniBand Fabric Manager • Wireless InfiniBand chips. SoftROCE over Wi-Fi for the interim • Android virtualized RDMA or container D2D apps • Networking Meta Language or DSL over RDMA verbs Challenges we can meet Partial list
  53. 53. Partial list of challenges, cont’d • Lacking external switching devices, switching and routing have to be done by the devices themselves. • Seeding - bootstrapping the app/cluster • Isolation between overlapping app/clusters • Mixing and merging overlapping app/clusters • Member registration, volatile resource allocation manager • etc., etc….