Learn about the upcoming NVMe Plugfest in this webinar and hear about the latest test updates for NVMe and NVMe-oF products and how to get your SSD, NVMe-oF, and NVMe-MI products on the NVMe Integrators List. Products are tested for conformance to the NVMe standard. Check out our website for more info www.iol.unh.edu

1. NVMe™ Plugfest #12 Webinar
Presented by Tim Sheehan & Kerry Munson
www.iol.unh.edu
© 2019 University of New Hampshire InterOperability Laboratory

2. 2
The UNH InterOperability Laboratory
• The InterOperability Laboratory (UNH-IOL) is located in Durham,
New Hampshire, USA. We are a non-profit, neutral, third-party test
laboratory dedicated to testing data networking technologies
through industry collaboration
• Our work force at the UNH-IOL consists of 100 undergrad and
graduate students and 20 full time staff
• The Datacenter Industry group includes: NVMe, NVMe-oF, Fibre
Channel, Ethernet Switching Protocols, and Open Networking
testing services supported by 3 full time staff and 16 undergrad
students
• NVMe Org has collaborated with UNH-IOL to manage the
Integrator’s List based Test Program on behalf of NVMe
Organization since 2012

3. Plugfest events held at the UNH-IOL in Durham, NH:
– NVMe-oF Novmeber 4 - 8, 2019
– NVMe-PCIe Novmeber 11 - 14, 2019
Testing will include conformance and interoperability
#12 test plans
Primary purpose is to achieve NVMe.orgs Integrator’s
List status for participating company products
NVMe-oF Other:
– NVMe-oF Large Builds if time permits
NVMe Plugfest #12
3

4. ● Integrators List
● IOL INTERACT™ Information
● Test Plan #12 updates
● Futures
● Logistics
● Q&A
Agenda
4

5. Products Listed
NVMe
Integrators List
Products
NVMe-MI
Integrators List
Products
NVMe-oF
Integrators List
Products
11 60193
Products Listed on IL as of October 8, 2019
5

6. NVMe Integrator’s List will accept:
• NVMe/PCIe drives, host platforms and switches
NVMe-MI Integrator’s List will accept:
• NVMe-MI drives
NVMe-oF Integrator’s List will accept:
• NVMe/RoCE initiators, targets, and switches
• NVMe/TCP initiators, targets, and switches
• NVMe/FC initiators, targets, and switches
NVMe Integrator’s List Products
6

7. Requirements for NVMe/PCIe SSDs:
1. Pass NVMe Conformance Tests with PC and
LeCroy Edition of the v12.0 IOL INTERACT
test tool
2. Pass NVMe Interoperability: I/O Tests with
5/6 NVMe Host Platforms, 2 boot systems,
and Hot-plug for U.2 form factors only.
Requirements for NVMe/PCIe Host Platforms and PCIe
Switches:
1. Pass NVMe Interop Tests with 5 NVMe SSDs
Requirements for NVMe-MI SSDs:
1. Pass NVMe-MI Conformance Tests with
Teledyne LeCroy test tool
NVMe Integrator’s List Requirements
#12
7

8. Requirements for Initiators/Hosts:
1. Pass NVMe-oF Interop Tests with 2 NVMe-oF Subsystem
Interop Partners
Requirements for Targets/NVMe-oF Subsystems:
1. Pass NVMe-oF Interop Tests with 2 NVMe-oF Host Interop
Partners
2. Pass NVMe Protocol Conformance tests
3. Run NVMe-oF Protocol Conformance tests (M)
Two Partners could be same HW with different OS/Driver.
NVMe-oF Integrator’s List
Requirements #12
8

9. ● Integrator’s List
● IOL INTERACT™ Information
● Test Plan #12 updates
● Futures
● Logistics
● Q&A
Agenda
9

10. • Step 1 : Obtain an IOL membership in either the NVMe
or NVMe-oF test groups
• Step 2: Fill out the IOL Licensing Agreement
• Step 3: Obtain access to the box utility to download
software
• Step 4: Obtain license from IOL staff
• Step 5: Follow the IOL-INTERACT ™ installation
documentation
• There is a demo version available
Bug Tracking Improvements
How do I get IOL-INTERACT ™ PC-
Edition?
10

11. • When a support issue is reported, reporter will receive
an issue number, and estimated fix date based on next
release schedule
• Issues tracking internally using Jira and Git/BitBucket
for tracking and source control.
• The release notes will show what issues (with their
issue numbers) have been addressed in that release
• Enable user to make a connection to issue reported >
issue fixed
• 2 Major releases and monthly Minor IOL INTERACT™
releases
• Issues can be reported to the MyIOL:
https://www.iol.unh.edu/my/login
Bug Tracking ImprovementsSupport Issue Tracking Process
11

12. Conformance ToolsNVMe/ NVMe-MI/ NVMe-oF Test Plans
NVMe Interop v12.0
NVMe-MI Specification Conformance v12.0
NVMe Base Specification Conformance v12.0
NVMe-oF Interop v12.0
NVMe-oF Specification Conformance v12.0
NVMe/PCIe SSDs and Hosts
and NVMe-MI SSDs
NVMe-oF Subsystems and Hosts
(ROCE, TCP, FC)
12

13. ● Integrator’s List
● IOL INTERACT™ Information
● Test Plan #12 updates
● Futures
● Logistics
● Q&A
Agenda
13

14. Conformance ToolsNVMe/PCIe Conformance Tools /
Plans
14
NVMe Base Specification
Conformance v12.0
(unique low-level PCIe test cases)
IOL INTERACT™ PC Edition v12.0... … for NVMe/PCIe SSDs
IOL INTERACT™ LeCroy Edition v12.0 ...
NVMe Base Specification
Conformance v12.0
...Performs this required test plan...
...Performs this required test plan...
Download and
run tools today!

15. NVMe Conformance Test Plan #12
Updates
15
NVMe
Conformance
TP #11 65+
Changes
NVMe
Conformance
TP #12

16. • 28 new FYI tests across the entire test plan #12
– NVMe Sanitize command support tests
– Additional NVMe format command tests
– IO Determinism tests
– NVMe Verify command support tests
– Reservations tests
• 14 tests went from MàFYI, Reservations
• Group 7 Reservations
– Added to NVMe-oF testing, as OF-FYI tests
– All dual-port tests are FYI
• 42 additional existing tests now supported for NVMe-oF
NVMe Conformance Test Plan #12
Updates
16

17. Conformance ToolsNVMe-oF Conformance Tools / Plans
17
NVMe-oF Specification
Conformance v12.0
...for NVMe-oF Subsystems and HostsIOL INTERACT™ PC Edition...
NVMe Base Specification
Conformance v12.0
...Performs these required test plans...
Download and
run tool today!

18. NVMe-oF Conformance Test Plan #12
Updates
18
NVMe-oF
Conformance
TP #11 11
Changes
NVMe-oF
Conformance
TP #12

19. • 1 new FYI test, Keep Alive Timeout
• All tests are still FYI
• All existing tests have modifications
• Common Theme #1 : Make sure disconnect NVMe-oF
controllers from the testing Station at the beginning and end
of each test.
• Common Theme #2 : Test station and DUT should be on the
same fabric network and configured correctly
NVMe-oF Conformance Test Plan #12
Updates
19

20. Conformance ToolsNVMe-MI Conformance Tools / Plans
20
Teledyne LeCroy T34 ...
NVMe-MI Specification
Conformance v12.0
… for NVMe-MI SSDs...Performs this required test plan...

21. NVMe-MI Conformance Test Plan #12
Updates
21
NVMe-MI
Conformance
TP #11 2
Changes
NVMe-MI
Conformance
TP #12

22. • Test 7.3 now reflects proper use of the CESF bit, per
TP2008
• Updated the NVMe Integrators List requirements to match
the NVMe Integrators List Policy Document
NVMe-MI Conformance Test Plan #12
Updates
22

23. NVMe/PCIe Interop Systems
https://www.iol.unh.edu/testing/storage/nvme/equipment 23
MAKE MODEL BIOS VERSION OPERATING SYSTEM DRIVER VERSION CPU MODEL PURPOSE
ASRock ASRock Z97 Extreme 6 P2.3 Ubuntu Linux Inbox Microsoft NVMe
Driver
Intel i5-4690K @3.50
GHz
Boot test
ASUSTeK
Computer Inc.
ASUS ROG STRIX
B350-F
0809 Windows 10 Inbox Microsoft NVMe
Driver
AMD Ryzen 3 1300X @
3.50 GHz
I/O test
Dell EMC Dell PowerEdge R740xd 1.3.7 Windows Server
2016
Inbox Microsoft NVMe
Driver
Intel Xeon Bronze 3104
CPU @ 1.70GHz
Hotplug test
Dell, Inc. Dell PowerEdge R720 2.2.2 Windows Server
2012 R2
Inbox Microsoft NVMe
Driver
Intel Xeon CPU E5-2640
@ 2.5GHz
I/O test
Dell, Inc. Dell PowerEdge R710 6.4.0 Ubuntu 14.04 NVMe Driver 1.0 Intel Xeon CPU X5570
@ 2.93 Ghz
I/O test
GIGABYTE Gigabyte H170 Gaming-
3
F5 Ubuntu 16.04,
Kernel 4.4.0
Linux Kernel 4.4.0
NVMe Driver
Intel i3-6100 @ 3.70
GHz
Boot test
Intel
Corporation
Intel Server #1 S2600WT
(Wildcat Pass)
2.7 Windows 10, Build
10240
Inbox Microsoft NVMe
Driver
Intel Xeon CPU E5-
2678W v4 @ 3.00 GHz
I/O test
Intel
Corporation
Intel Server #2 S2600WT
(Wildcat Pass)
2.7 CentOS 6.7, Kernel
2.6.32
Linux Kernel 2.6.32
NVMe Driver
Intel Xeon CPU E5-
2678W v4 @ 3.00 GHz
I/O test

24. NVMe Interoperability Test Plan #12
Updates
24
NVMe
Interoperability
TP #11 3
Changes
NVMe
Interoperability
TP #12

25. • Test 1.4 clarified expectations for multi-port devices
• Test 1.5 clarified expectations for multi-port devices
• Test 1.10 procedure steps clarified
NVMe Interoperability Test Plan #12
Updates
25

26. Pass NVMe-oF Interop Tests with
2 partners in a multi switch setup.
NVMe-oF Interop Test Suite
Document
NVMe-oF Interop Tests
NVMe-oF Interop Tests
Summary:
1. Discover Target
2. Send Traffic
3. Link Pull
4. Power Up
5. Keep Alive Timeout
Server
RNIC/HBA
NVMe-oF
Target
DUT
Switch
Switch
26

27. NVMe-oF Interoperability Test Plan #12
Updates
27
NVMe-oF
Interoperability
TP #11 1
Changes
NVMe-oF
Interoperability
TP #12

28. • Tests 1.1-1.4 are now MANDATORY
• Test 1.5 (Keep Alive Timeout) is still FYI
NVMe-oF Interoperability Test Plan #12
Updates
28

29. ● Integrator’s List
● IOL INTERACT™ Information
● Test Plan #12 updates
● Futures
● Logistics
● Q&A
Agenda
29

30. • Hsinchu, Taiwan
• January 13 – 16, 2020
• Register Here by October 18th! Event dependent on participation
https://www.iol.unh.edu/event/2020/01/nvme-plugfest-12-taiwan
• Details to be determined after registration closes
Goals + MindsetNVMe Plugfest #12 Asia
30

31. ICC Calls
Interop and Compliance Committee (ICC)
Chair: Ryan Holmqvist, Microsemi
• Meets every other Tuesday 2PM Eastern /
11AM Pacific
• Announcements/Meeting invites made on ICC
reflector
ICC Calls
31

32. • Integrator’s List
• IOL INTERACT™ Software Update
• Test Plan #12 updates
• Futures
• Logistics
• Q&A
Agenda
32

33. • Test companies attend the event to support vendor devices.
• Devices will have test slots with test equipment companies for informative
purposes.
• Test support companies include:
Conformance ToolsPartners and Support
33

34. Download the latest tools and run them!
● Tools used at the plugfest are available now
● Use them to pre-qualify your products
● Link to licensing agreement
–https://www.iol.unh.edu/solutions/test-tools/interact
● Link to UNH Box PC Edition IOL INTERACT™ Software
–https://unh.box.com/s/cktgos25cjfk6alclromuvqfn1lgebgd
● Link to UNH Box LeCroy Edition IOL INTERACT™ Software
–https://unh.app.box.com/folder/75601888099
Be Prepared!
34

35. Bring 2+ Samples of each product
• Conformance tests will only require 1 sample
• Interop tests require that 2 samples be used
simultaneously
• Bring any necessary PCs, tools, cables that will be necessary to re-program
or re-flash your device
• If you have a larger product (e.g. large PCIe dev board), bring your own PCIe
flex or riser cables
• Adapters available from serialcables.com and teledynelecroy.com
Be Prepared - PCIe!
35

36. • Bring host system for products to run in
• Bring required cabling for your product
(including break out cables & management
cables)
• Bring any necessary PCs, tools, cables that
will be necessary to re-program or re-flash
your device
• Short Range, multimode transceivers
• Power cables (if you have unique
requirements please let us know in advance)
Be Prepared - Fabrics!
36

37. Available!
● Wired and wireless internet provided
● Power cords, mice, keyboards, monitors
● Conference rooms available upon request
● Shipping Supplies – tape, labels, etc.
37

38. 38

39. Travel
Visa Invitation Letter Contact:
kerry.munson@iol.unh.edu
Shipping Address:
UNH-IOL NVMe Plugfest
Attn: Kerry Munson
21 Madbury Rd Suite 100
Durham, NH 03824
📞: 603-862-0090
All equipment must arrive by November 1st
for NVMe-oF Plugfest and November 8th for
NVMe/PCIe Plugfest.
Airports:
MHT - Manchester Boston Regional Airport
( 🕐1 hr)
BOS - Boston Logan International Airport
( 🕒1hr 30mins)
Hotels, Travel, Parking Info:
https://www.iol.unh.edu/about/visit
Train: 🚄
Amtrak - Downeaster
Boston North Station > Durham-UNH
http://www.amtrakdowneaster.com/
39

40. Parking
Parking code will be provided
to registrants
UNH Campus Map:
https://www.iol.unh.edu/sites/
default/files/images/direction
s/iol-campus-map.pdf
40

41. Logistics
Ø NVMe-oF Plugfest runs Monday – Thursday
Ø NVMe PCIe Plugfest runs Monday - Thursday
Ø Doors open at 8AM, testing until 6PM
Ø Kickoff meeting Mondays @ 11AM
Ø Light breakfast and lunch provided each day
Ø Device schedule will be made after registration closes
41

42. Registration
NVMe-oF Plugfest - https://www.iol.unh.edu/event/2019/11/nvme-plugfest-12
NVMe PCIe Plugfest - https://www.iol.unh.edu/event/2019/11/nvme-pcie-plugfest-12
Registration will close October 18, 2019 (Fabrics), October 25, 2019 (PCIe)
Registration Limits:
● At registration 2 unique products may be registered.
● All other products will be waitlisted and accepted on a space-available basis.
● At registration you can register 4 engineers.
● Additional engineers will incur an additional fee.
42

43. Contact Information
43
Kerry Munson
Operations Manager,
Datacenter Technologies
Kerry.Munson@iol.unh.edu
Tim Sheehan
Manager, Datacenter
Technologies
Tim.Sheehan@iol.unh.edu

44. Questions?
44
Please type any questions you have into the
“questions” window
Twitter @UNH_IOL | Facebook @UNHIOL
LinkedIn @UNH InterOperability Lab