Reading Circle

1. Loom:FlexibleandEfficientNICPacket
Scheduling
ReadingCircle
M2TakuyaFukuoka
November27,2020
1

2. Aboutthispaper
PublishedinNSDI'19.
PresentLoom,anewNICdesignthatmovesallper-flow
schedulingdecisionsoutoftheOSandintotheNIC
Sourcecodeandofficialslideisavailable
Iborrowedmanyfiguresandsentencesfromtheofficialslide
inthispresentation
2

3. Datacentersandcloud
Datacentershavemanycompetingtenants,whichshouldbe
isolated
WithVMsandcontainers,itiscurrentlypossibletoensurethat
tenantsfairlyshareCPUandmemory
Ontheotherhand,networkisolationisproblematicespecially
wheneachtenantrunavarietyofapplicationswithdifferent
performanceneeds
Latency-sensitiveapplicationsuchasmemcached
Throughput-sensitiveapplicationssuchasHadoopandSpark
3

4. Thegoal
Networkoperatorsmustspecifyandenforceanetworkisolation
policy,whichrequiresapporopriatepacketsscheduling
4

5. Coremotivationofthiswork
TheOSisresponsiblefordynamicallyenforcingthesepolicies
However,itcannotfullyachievethegoalbecausetoday'sNICis
notsoflexible
Proposeloom,whichmovesallper-flowschedulingdecisionsout
oftheOSandintotheNIC
5

6. Outlines
Motivation
Loom'sDesign
.Anewnetworkpolicyabstraction:restricteddirectedacyclic
graphs(DAGs)
.Aprogrammablehierarchicalpacketscheduler
.AnewexpressiveandefficientOS/NICinterface
Loom'sImplementation
RelatedWork
Evaluation
6

7. Whyisapplyingthepoliciesdifficult?
Inschedulingpackets,youchoosethenumberofqueuesinthe
NICasaninterfacebetweentheOSandtheNIC
Whenmultiplequeuesareutilized,thepacketsareinflexibly
scheduledinaround-robbinway
Youencounterrespectiveproblemsregardlessofyouchoose
singlequeue(SQ)ormultiplequeues(MQ)
7

8. ProblemsofSingleQueue(SQ)
Cannotachieveline-rate(100Gbps)duetothecontention
Onlyachievearound36-40Gbpsevenwithbatching
HighCPUutilization
(Butitcanenforcethepolicies)
8

9. ProblemsofMultipleQueues(MQ)
Itcannotenforcepolicies
Everycoreindependentlyenforcespoliciesonlocaltraffic
Highlatencybytheinflexibleroundrobbinschedulingofthe
queuesforlatency-sensitiveapplications
9

10. Loom,anewNICdesign
HowtocopewiththeinflexibleNIC?
Solution:moveallper-flowschedulingdecisionsoutoftheOSand
intotheNIC!
Loomdesignchoice
Howshouldpoliciesbespecified?
HowdowebuildaNICthatcanenforceLoomʼsnew
abstraction?
10

11. Outlines
Motivation
Loom'sDesign
.Anewnetworkpolicyabstraction:restrictedDAGs
.Aprogrammablehierarchicalpacketscheduler
.AnewexpressiveandefficientOS/NICinterface
Loom'sImplementation
RelatedWork
Evaluation
11

12. DAGpolicyabstraction(1)
NetworkpolicyinLoomcanberepresentedwithschedulingand
shapingnodes
Schedulingnodesdeterminetherelativeorderingofdifferent
packets(WeightedFairQueuing,Prioriyscheduling)
Shapingnodesdeterminethetimingofpackets(rate-limits)
LoomadoptDAGpolicyabstractionofthesekindsofnodes,not
treeabstraction
Treeabstractionisnotabletorepresentcomplexhierarchy
12

13. DAGpolicyabstraction(2)
NotalltheDAGsarepermitted
DAGswhichformsatreeifallshapingnodesareremovedisonly
allowed
Thisensuresthatschedulingnodesclosertotherootdonot
reorderpacketsthatwerealreadyorderedbyachildnode.
13

14. Outlines
Motivation
Loom'sDesign
.Anewnetworkpolicyabstraction:restricteddirectedacyclic
graphs(DAGs)
.Aprogrammablehierarchicalpacketscheduler
.AnewexpressiveandefficientOS/NICinterface
Loom'sImplementation
RelatedWork
Evaluation
14

15. APIFOtree
Manyschedulingalgorithmscanbeimplementedbyatreeof
priotiryqueues(push-infirst-outqueues,PIFOs) 1
Differentschedulingalgorithmsareimplementedbychanginghow
therankiscomputed
Strictpriorityschedulingcomputesapriority
WeightFairQueuingcomputesavirtualtimeforthepacket
Rate-limitingcomputesatransmittime
andsoforth...
1 Sivaraman,A.,Subramanian,S.,Alizadeh,M.,Chole,S.,Chuang,S.T.,Agrawal,A.,Balakrishnan,H.,Edsall,T.,Katti,S.,
&McKeown,N.(2016).Programmablepacketschedulingatlinerate.SIGCOMM2016-Proceedingsofthe2016ACM
ConferenceonSpecialInterestGrouponDataCommunication,44–57.
15

16. Anewschedulinghierarchy
However,priorPIFO-basedschedulercannotenforcethepolicies
representedasDAGs
Itisonlyapplicabletoatreeofnodesofpolicies
Iamgoingtoexplainthereasonusingexamples
16

17. PriorPIFO-basedscheduler(1)
Therearequeuesinthehierarchy,theRootqueue,andQ1.
Rate-limitingandsomescheduinginthefigureiscompiled
intoQ1. 17

18. PriorPIFO-basedscheduler(1)
D1isenqueuedtotheQ1Scheduledqueue.
ApointertoQ1isalsoinsertedintotheRootqueue.
18

19. PriorPIFO-basedscheduler(1)
ThepointerisdequeuedfromtheRoot,andthepacketatthehead
oftheschedulingqueueatQ1willbereturned.
19

20. PriorPIFO-basedscheduler(2)
Therearequeuesinthehierarchy,theRootqueue,andQ1.
20

21. PriorPIFO-basedscheduler(2)
D2,whichhasatransmittimeofT_D2asaresultofrate-limitting,
isinsertedbothQ1SchedulingandShapingqueue
Butitisnotenqueuedhigherupthetreeduetotherate-limiting
21

22. PriorPIFO-basedscheduler(2)
AttheT_D2,T_D2isdequeuedfromShapingqueue,andapointer
toQ1isinsertedintotheRootqueue.
22

23. PriorPIFO-basedscheduler(2)
ThepointerisdequeuedfromtheRoot,andthepacketatthehead
oftheschedulingqueueatQ1willbereturned.
23

24. PriorPIFO-basedscheduler(3)
Usingthisexample,Iwillshowproblemsofenforcementof
policies
BothD1andD2willbeinserted
24

25. PriorPIFO-basedscheduler(3)
D1isenqueuedtotheScheduledqueue.
ApointertoQ1isalsoinsertedintotheRootqueue.
25

26. PriorPIFO-basedscheduler(3)
AssumeD2israte-limitedandhashigherrankthanD1
D2isinsertedbothQ1SchedulingandShapingqueue
ItisorderedinQ1aheadofD1consideringthehigherrank
26

27. PriorPIFO-basedscheduler(3)
ApointerisdequeuedfromtheRoot,andinthistime,thepacketat
theheadoftheschedulingqueueatQ1willbereturned.
ThismeansD2,notD1isdequeuedfromQ1
Thisisnotwhatwehaveexpected.Wewanttorate-limitD2,
notD1
27

28. PriorPIFO-basedscheduler(3)
D1isenqueuedtotheQ1Scheduledqueue.
ApointertoQ1isalsoinsertedintotheRootqueue.
28

29. PriorPIFO-basedscheduler(3)
ThepointerisdequeuedfromtheRoot,andthepacketatthehead
oftheschedulingqueueatQ1willbereturned.
29

30. Howtomodifythescheduler?
Usesaglobalshapingqueuetoimplementrate-limitinginsteadof
per-nodeshapingqueues
Alltrafficisfirstonlyplacedinschedulingqueues
Ifapacketisdequeuedbeforeitsshapingtime,itisplacedin
aglobalshapingqueue
Aftershaping,thepacketisplacedbackinschedulingqueues
30

31. LoomPIFO-basedscheduler
D1isalreadyinsertedintotheQ1SchedulingqueueandRoot
queue.
31

32. LoomPIFO-basedscheduler
Thedifferenceis
TheT_D2isnotinsertedintoQ1Shapingqueue
ApointertoQ1isinsertedtoRootqueuethoughD2israte-
limited
32

33. LoomPIFO-basedscheduler
AdequeueoprationtoRootisconducted
ApointerisdequeuedfromtheRoot,andinthistime,thepacketat
theheadoftheschedulingqueueatQ1willbereturned.
Butthedifferenceis
Itisre-enqueuedinaglobalshapingqueueuntilitsreaches
itstransmissiontimeofT_D2. 33

34. LoomPIFO-basedscheduler
ThepointerisdequeuedfromtheRoot,andthepacketatthehead
oftheschedulingqueueatQ1willbereturned.
34

35. LoomPIFO-basedscheduler
AttheT_D2,theD2isre-insertedintothehierarcy.
35

36. Trade-offofthenewscheduling
Itcanincreasethenumberofenqueueanddequeueoperations
performedperpacket
Intheworstcase,apacketinLoommaybeenqueuedand
dequeuedtwotimesfromthePIFOhierarchy.
TheauthorsinsistthatitdoesnotpreventLoomfromoperatingat
line-rate(100Gbps)
ANICwithfrequencyof1GHzof100Gbpscanschedulea
packetevery2ns.
Achievingline-rateonlyneedstosendapacketevery5.12ns
Otheroptimizationsareexplainedaboutthat
36

37. Outlines
Motivation
Loom'sDesign
.Anewnetworkpolicyabstraction:restricteddirectedacyclic
graphs(DAGs)
.Aprogrammablehierarchicalpacketscheduler
.AnewexpressiveandefficientOS/NICinterface
Loom'sImplementation
RelatedWork
Evaluation
37

38. OS/NICInterface
MinimizesthetotalnumberofPCIewritesneededbytheOS
TheOSusesbatcheddoorbellstonotifytheNICofnew
segments.
Schedulingandmetadataupdatesarepassedin-linewith
packetdescriptorstoavoidgeneratingadditionalPCIewrites.
Inthispresentation,Idon'texplainmuchaboutit.
38

39. Outlines
Motivation
Loom'sDesign
.Anewnetworkpolicyabstraction:restricteddirectedacyclic
graphs(DAGs)
.Aprogrammablehierarchicalpacketscheduler
.AnewexpressiveandefficientOS/NICinterface
Loom'sImplementation
RelatedWork
Evaluation
39

40. Loom'sImplementation
ThepolicyDAGcompiler
ExistingcompilerforschedulingtreeswritteninDomino 1
SoftwareprototypeLoomNIConBerkeleyExtensibleSoftware
Switch(BESS,formallySoftNIC) 2
1 https://github.com/packet-transactions/domino-compiler
2 http://span.cs.berkeley.edu/bess.html
40

41. Outlines
Motivation
Loom'sDesign
.Anewnetworkpolicyabstraction:restricteddirectedacyclic
graphs(DAGs)
.Aprogrammablehierarchicalpacketscheduler
.AnewexpressiveandefficientOS/NICinterface
Loom'sImplementation
RelatedWork
Evaluation
41

42. RelatedWork
SENIC 1
ANICdesignthatcannativelysupportratelimiters
Aqueue-per-corewithunbatcheddoorbells
Requiresynchronizationacrossmultiplecores
Eiffel 2
Improvestheefficiencyofsoftwarepacketscheduling
EiffelandLoomarecomplementary.
1 Radhakrishnan,S.,Geng,Y.,Jeyakumar,V.,Kabbani,A.,Porter,G.,&Vahdat,A.(2014).SENIC:ScalableNICforend-
hostratelimiting.Proceedingsofthe11thUSENIXSymposiumonNetworkedSystemsDesignandImplementation,NSDI
2014,120,475–488.
2 Saeed,A.,Zhao,Y.,Dukkipati,N.,Ammar,M.,Zegura,E.,Harras,K.,&Vahdat,A.(2019).Eiffel:Efficientandflexible
softwarepacketscheduling.Proceedingsofthe16thUSENIXSymposiumonNetworkedSystemsDesignand
Implementation,NSDI2019,17–31.
42

43. Outlines
Motivation
Loom'sDesign
.Anewnetworkpolicyabstraction:restricteddirectedacyclic
graphs(DAGs)
.Aprogrammablehierarchicalpacketscheduler
.AnewexpressiveandefficientOS/NICinterface
Loom'sImplementation
RelatedWork
Evaluation
43

44. Evaluation
SendingdatabetweentwoserversonCloudLab 1
Twodifferentexperimentconfigurations
IntelX52010GbENICs
40GbpsMellanoxConnectX-3NICs
ThesoftwareNICusesonecoreforpackettransmissionandone
coreforreception
1 https://www.cloudlab.us/
44

45. Policyenforcement(1)
Therearefiveactivetenants
Every2sanewtenantstartsorstops
45

46. Policyenforcement(2)
SQisnotabletofullydrivethe40Gbpsline-rateandMQisnot
ableenforcethispolicy
Loomachievebothline-rateandenforcementofpolicy
46

47. Policyenforcement(3)
90thpercentilelatencyofT0with10GbENICs
Loomachieveslowlatencyevenwhenthenumberofflows
increases
47

48. OS/NICinterface
ComparethenumberofPCIewritespersecondgeneratedby
Loom(batcheddoorbell)andSENIC(unbatcheddoorbells).
Inthebestcasescenario(leftfigure),batcheddoorbellswasable
toreducethenumberofwritesbyupto19.2%.
Intheworstcasesenario(rightfigure),thenumberofwritesof
barcheddoorbellbecameupto2.3%.
Existingapproachwillhavedifficultydrivingincreasingline-rates
48

49. DCapplications
Evaluateisolationwhenusinghierarchicalpolicy
TenantsT1(memcachedandSpark)andT2(Spark)areallocated
equalsharesofbandwidth
Withloom,twoSparkapplicationsachievealmostthesame
bandwidth,andthelatencyofmemcachedisnotimpactedby
Spark
49

50. Conclusion
ProposeLoom,anewNICdesignthatmovesallper-flow
schedulingdecisionsoutoftheOSandintotheNIC
Loomcontributes
(1)AnewpolicyDAGabstraction
(2)Anewflexibleprogrammableschedulinghierarchy
(3)AnewexpressiveandefficientOS/NICinterface
LoomcancompletelyoffloadallpacketschedulingtotheNICwith
lowCPUoverheadwhilestilldrivingincreasingline-rates
(100Gbps)
Itleadstoreductionsinlatency,increasesinthroughput,and
improvementsinfairnessforcompetingtenantsandapplications
50

51. Mythroughts
IlikethispaperbecauseitclearlyexplainsthemotivationasNSDIpapersusually
do
ThetechnicalcontributionisextendingthePIFOincrementallyandapplyingthe
optimizationtocancelthetrade-offs
Butthereseembemoretrade-offsthatauthorsdonotmention
TheyinsistthatLoom'sdesigncanachieve100Gbpsbutisitreallythecase?
Thereisnoevaluationof100GbpsNIC,only10Gbpsand40Gbps.They
justgiveexplanationusingplentifulsentencesfor100Gbps.
Partlyduetothelackofmyspeciality,Icannotunderstandwhytheycannot
doexperimentsfor100Gbps
HowcanBESSbypasstheoriginalNICqueuesbecauseitisjustthesoftware-
basedimplementation?Theremustbesomewaystodirectlyconnectthewire.
51