Igor Gashinsky Principal Architect Yahoo! ONS2015: http://bit.ly/ons2015sd ONS Inspire! Webinars: http://bit.ly/oiw-sd Watch the talk (video) on ONS Content Archives: http://bit.ly/ons-archives-sd

3. SDN in Warehouse Scale Datacenters
v2.0
Igor Gashinsky
igor@yahoo-inc.com
Principal Architect
Yahoo!
April 17th, 2012

4. Some Terminology

5. - 3 -
Control Plane
Forwarding Plane
Other components
CPU
Management Plane
Networking
today
Servers
today
HW management
Plane
Generic PC
OS (linux, freebsd,
windows, solaris)
Applications
(apache, IIS, etc)
X86 instruction set
Openflow
SDN
What is SDN and OpenFlow

6. - 4 -
What stayed the same?

7. - 5 -
Datacenter Virtualization

8. - 6 -
Why SDN for Virtualization?
• Requirements:
• 20k servers per cluster = 400k VM's
• Full any to any communication
• Place any VM anywhere, anytime
• VM migration (sub-second)
• Guaranteed Consistency Model
• Problem:
• How do you keep 20k devices in sync w/ 400k+ entities each?
• Solutions:
• Current hardware can't keep up with FIB requirements
• Current routing protocols don't do this well
l Lack of a consistency model
• Flood and (s)pray doesn't work very well!
• Program the vSwitch from a central, distributed
database!

9. How has the market & vision
evolved?

10. - 8 -
Predictions from 6 months ago:
Control Plane
Forwarding Plane
Management Plane
Forwarding Plane
Management Plane
Control Plane
Then SDN

11. - 9 -
What we see now
Control Plane
Forwarding Plane
Management Plane
Forwarding Plane
Management Plane
Control
Plane
Then SDN
Unix +
APIs

12. Why is that important?

13. - 11 -
Configuration & Deployment Automation
Manual Magic
Network
Today
Network
Tomorrow
Servers
Today

14. Self-Healing Fabrics
(and a pony!)

15. - 13 -
Blast from the past (Y! Presentation to HSSG in 2007)
L3 Switch L3 Switch
L3 Switch <10GE> L3 Switch
Host <GE> Switch
L3
Switch
N x
H
L3
Switch
Switch Switch Switch
H H H H H H H H
L3 Switch L3 Switch L3 Switch L3 SwitchL3 Switch L3 Switch
L3
Switch
N x
H
L3
Switch
Switch Switch Switch
H H H H H H H H
L3
Switch
N x
H
L3
Switch
Switch Switch Switch
H H H H H H H H
L3
Switch
N x
H
L3
Switch
Switch Switch Switch
H H H H H H H H
L3
Switch
N x
L3
Switch
Switch Switch Switch
H H H H H H H H
L3
Switch
N x
H
L3
Switch
Switch Switch Switch
H H H H H H H H
L3
Switch
N x
H
L3
Switch
Switch Switch Switch
H H H H H H H H
L3
Switch
N x
H
L3
Switch
Switch Switch Switch
H H H H H H H H
L3 Switch <GE> Switch
** 8 way ECMP w/ 2x10GE LAGs**
** Way too many paths **
** Way too many cables **

16. - 14 -
Today's Topologies
l CLOS-like
l 20k server cluster ~= 16k internal links
l This means upto 1024 distinct links between a pair of
hosts
l How do you troubleshoot this (for packetloss, etc)?
l # of links to test = 1024/2 = 512
l 30 seconds/test
l 256 man-minutes for most-basic troubleshooting!
l Is that acceptable?
l Really? :)

17. - 15 -
Enter SDN
l Local Testing Agent
l Looks at interface counters (errors, etc)
l Performs interface/RIB/FIB health checking
l Local Repair Agent
l Perform local repairs (ie FIB consistency check)
l If certain conditions are met, automatically remove failed link(s)
l If unsure of a safe action, ask the controller
l Global Controller
l Has full visibility of the entire network
l Can initiate repair/fixup actions of it's own

18. - 16 -
Where's my pony?

19. - 17 -
Thank you!
Questions?
igor@yahoo-inc.com