1) The document introduces ExpEther and Wireless ExpEther, which extend PCI Express over Ethernet and provide reliable low-latency wireless connections, respectively. 2) ExpEther allows PCIe devices to be disaggregated over Ethernet networks while maintaining compatibility with existing software. Wireless ExpEther aggregates multiple wireless links to provide a virtual reliable connection with latency under 1ms. 3) NEC offers these technologies as IP cores and evaluation modules to enable wireless solutions for applications that require latency under 10ms, such as industrial robots, AGVs, and machine tools.

1. Introduction of
ExpEther and Wireless ExpEther
Jul. 17, 2019
NEC Digital Platform Division
OpenPOWER/OpenCAPI Meetup@Tokyo

2. 2 © NEC Corporation 2019 OpenPOWER/OpenCAPI
Introduction
▌鈴木 順 (すずき じゅん)
▌2005年NEC入社。デジタルプラットフォーム事業部兼データサイエンス
研究所所属。博士(情報理工学)。
▌これまでインターコネクト，アクセラレータミドルウェア，AIプラット
フォームの研究開発，製品化，事業企画，営業，社外アライアンス等に従
事。

3. 3 © NEC Corporation 2019 OpenPOWER/OpenCAPI
Outline
▌Background
▌ExpEther
Extension of PCI Express over Ethernet
▌Wireless ExpEther
Reliable Low-Latency Wireless Connection
▌Conclusions

4. ExpEther
PCIe over Ethernet

5. 5 © NEC Corporation 2019 OpenPOWER/OpenCAPI
PCI Express® is everywhere

6. 6 © NEC Corporation 2019 OpenPOWER/OpenCAPI
Processing and storage devices are emerging as I/O devices

7. 7 © NEC Corporation 2019 OpenPOWER/OpenCAPI
PCIe® is limited to internal I/O interconnection

8. 8 © NEC Corporation 2019 OpenPOWER/OpenCAPI
Resulting inefficient resource use

9. 9 © NEC Corporation 2019 OpenPOWER/OpenCAPI
ExpEther (Express Ether)

10. 10 © NEC Corporation 2019 OpenPOWER/OpenCAPI
ExpEther Features

11. 11 © NEC Corporation 2019 OpenPOWER/OpenCAPI
ExpEther I/O Disaggregated System

12. 12 © NEC Corporation 2019 OpenPOWER/OpenCAPI
Case: Large-Scale Single Computer

13. 13 © NEC Corporation 2019 OpenPOWER/OpenCAPI
Case: Resource Pool System

14. 14 © NEC Corporation 2019 OpenPOWER/OpenCAPI
ExpEther Architecture

15. 15 © NEC Corporation 2019 OpenPOWER/OpenCAPI
ExpEther Protocol Stack

16. 16 © NEC Corporation 2019 OpenPOWER/OpenCAPI
Conventional software is used as is

17. 17 © NEC Corporation 2019 OpenPOWER/OpenCAPI
ExpEther’s Reliable Ethernet

18. 18 © NEC Corporation 2019 OpenPOWER/OpenCAPI
Allocation of I/O devices to hosts

19. 19 © NEC Corporation 2019 OpenPOWER/OpenCAPI
Group ID is flexibly set by management software

20. Wireless ExpEther
Reliable Low-Latency Wireless Connection

21. 21 © NEC Corporation 2019 OpenPOWER/OpenCAPI
Wireless ExpEther Vision
Accelerate integration of IT and OT systems
through reliable and low-latency wireless connectivity
Accelerate integration of IT and OT systems
through reliable and low-latency wireless connectivity
Mobility
Retail/Consumer
Manufacturing
Healthcare
City Surveillance
Social Infrastructure
IT/OA
Future Wireless Systems
Reliable, Low-Latency (<1ms)
and Low-Jitter
Real-time Sensing
Conventional Wireless Systems
Unstable,
High-Latency(>100ms) Production Automation

22. 22 © NEC Corporation 2019 OpenPOWER/OpenCAPI
GPIO
Field Bus
Sync Clock
What is Wireless ExpEther?
NEC’s original technology which seamlessly converts conventional
wired network into reliable low-latency wireless network
NEC’s original technology which seamlessly converts conventional
wired network into reliable low-latency wireless network
▌Combination of original encoding and virtualization that aggregates
multiple wireless links into single wireless connection
▌Conventional Ethernet/GPIO connections seamlessly become
wireless just by connecting Wireless ExpEther adaptors
GPIO
Field Bus
Sync Clock
…
…
Link virtualization method aggregates multiple
wireless links to provide reliable connection
Redundant coding method enables end-to-end
low-latency connection without retransmission
Data communication on existing
system can be wireless as is

23. 23 © NEC Corporation 2019 OpenPOWER/OpenCAPI
Core Technology#1: Redundant Coding
Packets are regenerated at receiver without retransmission
even some packets are lost during transmission
Packets are regenerated at receiver without retransmission
even some packets are lost during transmission
▌Packet retransmission or carrier sense time increases transmission latency
▌Our method regenerates all source packets at receiver if encoded packets
enough to regenerate source packets are received from underlying links
c b ade
Source
Packets
Coding unit
Encoder
Wireless ExpEther
If receiver receives
enough encoded packets
(>3 in this case),
it can decode source
packets.
Encoded Packet:
Random combination of multiple source packets
Sender
loss
loss
Decoder
c b a
Receiver
Sender aggregate multiple
source packets, and
generates encoded packets.

24. 24 © NEC Corporation 2019 OpenPOWER/OpenCAPI
Core Technology#2 : Virtual Reliable Link
Wireless ExpEther provides virtual reliable link by combining
multiple wireless links.
Wireless ExpEther provides virtual reliable link by combining
multiple wireless links.
▌Encoded packets are transmitted through a virtual link which consists
of multiple physical wireless links
PLC
Wireless EE
Engine
Sensor
Wireless EE
Engine
Pendant
Wireless EE
Engine
Provides stable communication
with multiple unstable wireless links
×
Wireless Link A
Wireless Link B
Utilizes multiple available links
to achieve reliable communication

25. 25 © NEC Corporation 2019 OpenPOWER/OpenCAPI
Aggregation of Links of Different Wireless Standards
Wireless ExpEther simultaneously provides reliability and low-
latency by combining links of different wireless standards
Wireless ExpEther simultaneously provides reliability and low-
latency by combining links of different wireless standards
Performance
Reliability
WiFi
(2.4G/5GHz)
Throughput: >10Mbps
Latency: >10ms
Interference: High
60GHz
Throughput: >1Gbps
Latency: >100us
Interference: Middle
DECT(1.9GHz)
Throughput: >100Kbps
Latency: >10ms
Interference: Low
Throughput: >1Gbps
Latency: <1ms
Interference: Low
Wireless ExpEther
Multi-Link Aggregation
1.9/2.4/5.0/60GHz

26. 26 © NEC Corporation 2019 OpenPOWER/OpenCAPI
Performance
5GHz Ch:W53 5GHz Ch:W56 W53&W56 Multi Link
#ofReceivedPackets
Transmission Latency[ms]
wEE wEE
W53
W56
W53 or W56
Configuration
Application:Ping (interval:4ms)
～0.5[ms]
74%
～0.5[ms]
86%
～0.5[ms]
98%
#ofReceivedPackets
Transmission Latency[ms]
#ofReceivedPackets
Transmission Latency[ms]
Conventional WiFi Wireless ExpEther

27. 27 © NEC Corporation 2019 OpenPOWER/OpenCAPI
Wireless ExpEther Solution
Wireless ExpEther enables wireless solution which requires
network latency to be less than 10ms
Wireless ExpEther enables wireless solution which requires
network latency to be less than 10ms
100[ms] 10[ms] 1[ms]1000[ms]
Laptop
Tablet
Conventional Wireless FY19～ FY20～
Pendant
Robot
PLC・CNC
Latency must be less than
10ms
AGV
Latency must be less than
1ms
Panel I/OSensor
Wireless ExpEther
Allowable
MAX Lat

28. 28 © NEC Corporation 2019 OpenPOWER/OpenCAPI
Wireless ExpEther Product（Planning）
To meet individual demands from different applications, NEC
offers Wireless ExpEther as IP core
To meet individual demands from different applications, NEC
offers Wireless ExpEther as IP core
▌Wireless ExpEther IP Core
 ROM data which can be implemented into FPGA
 We also offer custom engineering and application-specific design to apply it to customer
products
▌Evaluation Module / Unit
 Evaluation module is available for customers who want to consider adopting IP core
Wireless ExpEther IP Core Evaluation Module/Unit
65mm
60mm

29. 29 © NEC Corporation 2019 OpenPOWER/OpenCAPI
MAC-IP
MAC-IP
IP Core Architecture
Off-the-shelf wireless TX/RX modules can be used
for RF front ends of Wireless ExpEther IP Core
Off-the-shelf wireless TX/RX modules can be used
for RF front ends of Wireless ExpEther IP Core
▌Supports WiFi 2.4GHz, WiFi 5.0GHz, and DECT(1.9GHz) by default
▌IP can be implemented to low-end FPGA (Cyclone V).
Reliable Wireless
Transport
IP-Ethernet
Bridge
Encap/Decap
Routing
Session Mng
Virtual Wire
Encode/Decode
Multi-Link Virt
Rate Control
・
・
・
Wireless
LAN ch1
Wireless
LAN ch2
Ether I/F
GPIO I/F
Wireless ExpEther IP Core
MAC-IP

30. 30 © NEC Corporation 2019 OpenPOWER/OpenCAPI
Evaluation Module / Unit
Evaluation module is offered to customers for PoC before it
decides adoption of Wireless ExpEther IP core
Evaluation module is offered to customers for PoC before it
decides adoption of Wireless ExpEther IP core
▌PoC systems are built easily to demonstrate effectiveness of IP Core in
customer environments
▌Contains Cyclone V FPGA, 2 WiFi modules, and DECT(1.9GHz) module
▌Support Ethernet (RJ45) and GPIO (8pin) interfaces
Wireless ExpEther
Module / Unit System Examples
Robot
Wireless
ExpEther
Machine Tools
PLC
Pendant
Module
Unit
65mm
55mm

31. 31 © NEC Corporation 2019 OpenPOWER/OpenCAPI
Application Examples
Wireless ExpEther can be applied to various industrial systemsWireless ExpEther can be applied to various industrial systems
Wireless Teach PendantWireless Teach Pendant
Wireless Sensor Unit for MonitoringWireless Sensor Unit for Monitoring
Wireless Mobile RobotWireless Mobile Robot
Reconfigurable
manufacturing
cells
Flexible sensor
implementation
Efficiency
improved
for operators
And Yours

32. 32 © NEC Corporation 2019 OpenPOWER/OpenCAPI
Conclusions
Your interconnect is accommodated in our reliable and low-
latency wired and wireless solution
Your interconnect is accommodated in our reliable and low-
latency wired and wireless solution
▌ExpEther
PCIe extension over Ethernet
No Modification to OS, device drivers, Ethernet switches and I/O devices
Flexible I/O assignments among hosts
Reliable short latency TLP transports over Ethernet
▌Wireless ExpEther
Converts wired interconnect to reliable low-latency wireless one
Combines multiple wireless links as single virtual wireless connection
Communication among multiple nodes are available