InfiniBand Management on 
localhost
Vikentsi Lapa, nop@tut.by
Software Automation Engineer 
   
InfiniBand Management
Subnet Manager
HCA
Switch
IB Initialization
Local Identificator = LID
Port Port Subnet Agent
Hos...
   
IB stack (ib_umad.ko)
Applications (OpenSM, Diagnostic Utilities)
MAD Libraries (libibumad,libibmad)
LD_PRELOAD=libuma...
   
How to use ibsim
● Install software from
– OFED (rpms)
– https://github.com/jgunthorpe/ibsim/
● Run ibsim
– ibsim ­s(t...
   
Run applications
LD_PRELOAD=./umad2sim/libumad2sim.so ibnetdiscover
● Subnet Manager 
– opensm
● Diagnostic Utilities ...
   
ibsim configuration file
Switch  8 "Switch1"
[1]     "Hca1"[1]
[2]     "Hca2"[2]
[3]     "Switch2"[3]
[4]     "Switch2...
   
Conclusion
● Intro to IB management
– start point if you meet IB fabrics
● ibsim application
– features, how to start,...
   
Questions?
   
Example
# ibsim ­s /usr/share/doc/ibsim­0.5mlnx1/net­examples/net
parsing: /usr/share/doc/ibsim­0.5mlnx1/net­examples/...
   
Example
sim> dump
# Net status ­ Sun May 19 12:05:04 2013
Switch 8 "Switch1"      nodeguid 200000 sysimgguid 200000
# ...
Upcoming SlideShare
Loading in …5
×

InfiniBand Management on localhost

435 views
356 views

Published on

Викентий Лапа «InfiniBand Management on localhost»

Доклад на майской линуксовке MLUG 2013

Published in: Technology, Business
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
435
On SlideShare
0
From Embeds
0
Number of Embeds
4
Actions
Shares
0
Downloads
0
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

InfiniBand Management on localhost

  1. 1.     InfiniBand Management on  localhost Vikentsi Lapa, nop@tut.by Software Automation Engineer 
  2. 2.     InfiniBand Management Subnet Manager HCA Switch IB Initialization Local Identificator = LID Port Port Subnet Agent Host Host Host Host Switch Host MAD
  3. 3.     IB stack (ib_umad.ko) Applications (OpenSM, Diagnostic Utilities) MAD Libraries (libibumad,libibmad) LD_PRELOAD=libumad2sim.so <­­> ibsim InfiniBand management simulator
  4. 4.     How to use ibsim ● Install software from – OFED (rpms) – https://github.com/jgunthorpe/ibsim/ ● Run ibsim – ibsim ­s(tart) <network_config_file>  – ­r(emote_mode) ­l(isten_to_port) port ● Manage from console or batch commands
  5. 5.     Run applications LD_PRELOAD=./umad2sim/libumad2sim.so ibnetdiscover ● Subnet Manager  – opensm ● Diagnostic Utilities  – ibstat – ibdiagnet – ibstatus ● Counters – perfquery
  6. 6.     ibsim configuration file Switch  8 "Switch1" [1]     "Hca1"[1] [2]     "Hca2"[2] [3]     "Switch2"[3] [4]     "Switch2"[4] Hca     2 "Hca1" [1]     "Switch1"[1]  Switch1 8 ports HCA1 2 ports  Switch2 8 ports HCA2
  7. 7.     Conclusion ● Intro to IB management – start point if you meet IB fabrics ● ibsim application – features, how to start, configuration file ● ibsim was used for lab practice – no equipment, only VM
  8. 8.     Questions?
  9. 9.     Example # ibsim ­s /usr/share/doc/ibsim­0.5mlnx1/net­examples/net parsing: /usr/share/doc/ibsim­0.5mlnx1/net­examples/net /usr/share/doc/ibsim­0.5mlnx1/net­examples/net: parsed 39 lines ######################## Network simulator ready. MaxNetNodes    = 2048 MaxNetSwitches = 256 MaxNetPorts    = 13312 MaxLinearCap   = 30720 MaxMcastCap    = 1024 sim>
  10. 10.     Example sim> dump # Net status ­ Sun May 19 12:05:04 2013 Switch 8 "Switch1"      nodeguid 200000 sysimgguid 200000 #       linearcap 30720 FDBtop 0 portchange 1 200000  [0]     "Sma Port"[0]    lid 0 lmc 0 smlid 0  4x  2.5G Active/LinkUp 200000  [1]     "Hca1"[1]         4x  2.5G Init/LinkUp 200000  [2]     "Hca2"[2]         4x  2.5G Init/LinkUp 200000  [3]     "Switch2"[3]      4x  2.5G Init/LinkUp 200000  [4]     "Switch2"[4]      4x  2.5G Init/LinkUp 200000  [5]                       4x  2.5G Down/Polling 200000  [6]                       4x  2.5G Down/Polling 200000  [7]                       4x  2.5G Down/Polling 200000  [8]                       4x  2.5G Down/Polling

×