SlideShare a Scribd company logo
1 of 22
Download to read offline
Back to Rings but not Tokens:
2015/11/17
Marat Zhanikeev
maratishe@gmail.com
IN研@熊本
PDF: bit.do/151117
Physical and Logical Designs for Distributed Filesystems
intended for Bulk Transfer over E2E Emulated Cut-Through Circuits
The Big Picture
1. increase capacity and flexibility at the same number of ports
2. do it at a relatively low cost
3. stay at hardware level -- namely the cut-through mode
• ultimate goal: a better storage grid for BigData 14
Super-duper
32-port
Software Switch
(ClickOS, SDN, NFV,...)
Simple
4-port
Simple
4-port
Simple
4-port
Simple
4-port
Simple
4-port
Simple
4-port
Simple
4-port
Simple
4-port
vs
>cost
capacity<<<
14 M.Zhanikeev "Streaming Algorithms for Big Data Processing on Multicore" Big Data: Algorithms..., CRC (2015)
M.Zhanikeev -- maratishe@gmail.com -- ...Rings but not Tokens: ...Distributed Filesystems ...over E2E Emulated Cut-Through Circuits -- bit.do/151117 2/22
2/22
Model of Per-Packet Overhead
C: Cut Through
Check,
etc. Q: Queue
D: Drop
QoS
classes
M.Zhanikeev -- maratishe@gmail.com -- ...Rings but not Tokens: ...Distributed Filesystems ...over E2E Emulated Cut-Through Circuits -- bit.do/151117 3/22
3/22
Circuits at a Scheduling Problem
Line=
outgoing
port
Overhead =
contention
No. of flows
Line=
outgoing
port
Overhead
Scheduling
Traditional
Circuits
M.Zhanikeev -- maratishe@gmail.com -- ...Rings but not Tokens: ...Distributed Filesystems ...over E2E Emulated Cut-Through Circuits -- bit.do/151117 4/22
4/22
The Tall Gate Model
• sensing is close to the wireless opportunistic/cognitive tech
• the target: make even long-haul e2e circuits possible (DC-DC)
Tall Gates
Bulks
Send
Highway
Sources
Destination
06 M.Zhanikeev "A City Traffic Model for Optical Circuit Switching in Data Centers" IEICE・OCS研 (2014)
M.Zhanikeev -- maratishe@gmail.com -- ...Rings but not Tokens: ...Distributed Filesystems ...over E2E Emulated Cut-Through Circuits -- bit.do/151117 5/22
5/22
Optical Circuits = OCS (no OBS/OPS)
E-O
ingress
O-E
egress
O-E-O
CORE
O-O
NOC
Management over Ethernet
Contention
Resolution
Bulk
E-O
ingress
O-E
egress
NOC
Management over Ethernet
Contention
Resolution, TE
O-O
CORE
Bulk
• OBS today is considered the best
technology
• however, when contention
management is moved to NOC,
OCS is feasible
• Tall Gate model is applicable
M.Zhanikeev -- maratishe@gmail.com -- ...Rings but not Tokens: ...Distributed Filesystems ...over E2E Emulated Cut-Through Circuits -- bit.do/151117 6/22
6/22
Circuits: Feature Comparison
• Tall Gate is better than Traditional Scheduler
• already discussed in DC networking 1112
Interference Overhead Isolation
Do Nothing HIGH ZERO NO
Network Virtualization HIGH HIGH NO
(store-and -forward)
Traditional Scheduler LOW HIGH YES (cut -through)
P2Px1N (1 network) HIGH VERY HIGH YES
P2Px2N (2 networks) ZERO VERY HIGH YES (cut -through)
Tall Gate (sensing) VERY LOW HIGH YES (cut -through)
11 "Cut-Through and Store-and-Forward Ethernet Switching for Low-Latency Environments" Cisco White Paper (2014)
12 G.Wang+5 "c-Through: Part-time Optics in Data Centers" ACM SIGCOMM (2010)
M.Zhanikeev -- maratishe@gmail.com -- ...Rings but not Tokens: ...Distributed Filesystems ...over E2E Emulated Cut-Through Circuits -- bit.do/151117 7/22
7/22
Circuits: Performance under Hotspots =bigdata
0 2 4 6 8
Ordered list
0
0.45
0.9
1.35
1.8
2.25
log(duration)
Do Nothing Network VirtualizationTraditional Scheduler
P2Px1N P2Px2N Tall Gate
0 2 4 6 8
Ordered list
0
0.8
1.6
log(duration)
0 2 4 6 8
Ordered list
0.6
1.2
1.8
2.4
log(duration)
0 2 4 6 8
Ordered list
1.5
1.8
2.1
2.4
log(duration)
0 2 4 6 8
Ordered list
1.65
1.95
2.25
2.55
log(duration)
0 2 4 6 8
Ordered list
1.2
1.6
2
2.4
log(duration)
Size: 10M..100M Size: 100M..500M
Size: 500M..1G
Size: 10G..100G
Size: 1G..10G
Size: 10G..50G
2.4
• hotspot traffic: few very
large flows
• group by size range
• compare transfer time
across the models
M.Zhanikeev -- maratishe@gmail.com -- ...Rings but not Tokens: ...Distributed Filesystems ...over E2E Emulated Cut-Through Circuits -- bit.do/151117 8/22
8/22
Today: The Basic Idea
M.Zhanikeev -- maratishe@gmail.com -- ...Rings but not Tokens: ...Distributed Filesystems ...over E2E Emulated Cut-Through Circuits -- bit.do/151117 9/22
9/22
The Basic Idea: Logical Circuits
Physical
Topological
A switch 4 ports 8 ports …
Legend
Port
Switch
Node
Logical link
(domain)
Physical link
…
…
<2,2,2>
<3,1,2>
<1,4,4>
<3,5,3>
• take standard
switches (cut-through OK
on most)
• design isolated
rings within the
available port
• ⟨a, b, c⟩ means: a
ports/nodes on inner
ring, b outer rings, c
largest hub/port
• implementation:
VLANs on Ethernet,
MEMS in optical,
etc.
M.Zhanikeev -- maratishe@gmail.com -- ...Rings but not Tokens: ...Distributed Filesystems ...over E2E Emulated Cut-Through Circuits -- bit.do/151117 10/22
10/22
Deep/Dark Theory
M.Zhanikeev -- maratishe@gmail.com -- ...Rings but not Tokens: ...Distributed Filesystems ...over E2E Emulated Cut-Through Circuits -- bit.do/151117 11/22
11/22
Deep/Dark Theory (1)
<x1,y1,z1>
<x2,y2,z2> <x2,y2,z2> …
<xn,yn,zn>
…
<x2,y2,z2>
… … …
1..m (different for each level)
n
…
k Processing nodes
Storage nodes
Connections
to top level
…
M.Zhanikeev -- maratishe@gmail.com -- ...Rings but not Tokens: ...Distributed Filesystems ...over E2E Emulated Cut-Through Circuits -- bit.do/151117 12/22
12/22
Deep/Dark Theory (2)
• can be used to analyze connectivity using adjacency matrices
N11 N12
N21
Nn1 Nnq
N1p…
N2m
… …
… …
…
… Processors
Storage grid
row a
row b
a size
a size
a x a matrix
of peer mesh
a size b size
a x b matrix
between levels
M.Zhanikeev -- maratishe@gmail.com -- ...Rings but not Tokens: ...Distributed Filesystems ...over E2E Emulated Cut-Through Circuits -- bit.do/151117 13/22
13/22
The Much Simpler Practice
M.Zhanikeev -- maratishe@gmail.com -- ...Rings but not Tokens: ...Distributed Filesystems ...over E2E Emulated Cut-Through Circuits -- bit.do/151117 14/22
14/22
The Much Simpler Practice (1)
• in concept, close to harness braiding (photo)
• similar to a Google paper on the design of its in-rack and
rack-rack crosses
• immediately obvious: at least 2 distinct routes between
any 2 nodes
4 ports
<1,4,4>
M.Zhanikeev -- maratishe@gmail.com -- ...Rings but not Tokens: ...Distributed Filesystems ...over E2E Emulated Cut-Through Circuits -- bit.do/151117 15/22
15/22
The Much Simpler Practice (2)
• with ⟨3, 5, 3⟩ tuple, much more flexibility
• scientifically interesting: where to connect extra ports (above
minimum connectivity) -- 2 or 3 switches away?
◦ ... and how does this affect overall connectivity?
8 ports
<3,5,3>
?
?
M.Zhanikeev -- maratishe@gmail.com -- ...Rings but not Tokens: ...Distributed Filesystems ...over E2E Emulated Cut-Through Circuits -- bit.do/151117 16/22
16/22
Switch Robotics?
• some book libraries have already been roboticized
• why not return to a telephone switchboard, but in a
robotic version?
• goal: dynamic management of connectivity using
robotically migrating ports
• mid-goal: switches with wobbling physical ports
+
M.Zhanikeev -- maratishe@gmail.com -- ...Rings but not Tokens: ...Distributed Filesystems ...over E2E Emulated Cut-Through Circuits -- bit.do/151117 17/22
17/22
That’s all, thank you ...
M.Zhanikeev -- maratishe@gmail.com -- ...Rings but not Tokens: ...Distributed Filesystems ...over E2E Emulated Cut-Through Circuits -- bit.do/151117 18/22
18/22
Application: BigData Replay
• Hadoop/MapReduce has failed 02 -- can only support (barely) 15k concurrent
users 01 (among many other problems)
• BigData Replay on massively multicore 14 is a valid alternative
Name Node
Storage Node (shard)
file A
file B
file C
…
Hadoop Space
Manager
Hadoop Job
(your code)
Hadoop Job
(your code)
Hadoop Job
(your code)
MapReduce
job (your code)
manymany
Name
Server(s)
Client Machine
Hadoop Client
Your
Code
You
Start Use
Deploy
FindRead/parse
many
Storage Node
(shard)
Time-Aware
Sub-Store(s)
Manager
Client Machine
Client
Your
Sketcher
You
Start Use
Schedule
Multicore
Replay
Replay Node
many
02 A.Rowstron+4 "Nobody ever got fired for using Hadoop on a cluster" 1st Hot Topics in Cloud Data Processing (2012)
01 K.Shvachko "HDFS Scalability: the Limits to Growth" the Magazine of USENIX, vol.35, no.2 (2012)
14 M.Zhanikeev "Streaming Algorithms for Big Data Processing on Multicore" Big Data: Algorithms..., CRC (2015)
M.Zhanikeev -- maratishe@gmail.com -- ...Rings but not Tokens: ...Distributed Filesystems ...over E2E Emulated Cut-Through Circuits -- bit.do/151117 19/22
19/22
Bigdata Replay on Massively Multicore
….
Time
Now
(buffer head)
Manager
Job
Job
Buffer
tail
pos
pos
Controller
Kill
2 Report
Manage
in realtime
One Replay Batch
One
Buffer
One
Buffer
One
BufferJobs
Jobs
Jobs
Replay at
a scale
1
• massively multicore ̸=
manycore 08
• 100+ cores on
conventional hardware --
standard RAM, shmap, etc.
• target: 100k jobs,
using Multiple
Replay nodes
08 M.Zhanikeev "...Massively Multicore, Heterogeneous Jobs with Hotspots, and Data Streaming" SWoPP (2015)
M.Zhanikeev -- maratishe@gmail.com -- ...Rings but not Tokens: ...Distributed Filesystems ...over E2E Emulated Cut-Through Circuits -- bit.do/151117 20/22
20/22
NextGen NOC
• circuits are really really really valuable when they considerably reduce bulk
transfer time
• future NOCs will develop, advertise, and sell their ability to provide
circuits
• part of NGN and autonomy network management -- see recent IETF/
MRTG meeting
NOC
10
M.Zhanikeev "The Next Generation of Networks is all about Hotspot Distributions and Cut-Through Circuits" IEICE・CQ研
(2015)
M.Zhanikeev -- maratishe@gmail.com -- ...Rings but not Tokens: ...Distributed Filesystems ...over E2E Emulated Cut-Through Circuits -- bit.do/151117 21/22
21/22
Hotspot Distribution
• a hotspot distribution consists of normal, popular and hot/flash sets
• describe a wide range of natural processes, traffic in particular
0 10 20 30 40 50
List of traffic sources
0
0.4
0.8
1.2
1.6
2
2.4
2.8
log(trafficvolume)
0 10 20 30 40 50
List of traffic sources
0
0.4
0.8
1.2
1.6
2
2.4
2.8
log(trafficvolume)
Magnitude=2 Magnitude=10
Hotspots
Normal
Hotspot
under
a Flash
event
M.Zhanikeev -- maratishe@gmail.com -- ...Rings but not Tokens: ...Distributed Filesystems ...over E2E Emulated Cut-Through Circuits -- bit.do/151117 22/22
22/22

More Related Content

What's hot

Advanced Components on Top of L4Re
Advanced Components on Top of L4ReAdvanced Components on Top of L4Re
Advanced Components on Top of L4ReVasily Sartakov
 
xCORE architecture flyer
xCORE architecture flyerxCORE architecture flyer
xCORE architecture flyerXMOS
 
Data Center Networks:Virtual Bridging
Data Center Networks:Virtual BridgingData Center Networks:Virtual Bridging
Data Center Networks:Virtual Bridgingrjain51
 
Software Defined Networking: A Concept and Related Issues
Software Defined Networking: A Concept and Related IssuesSoftware Defined Networking: A Concept and Related Issues
Software Defined Networking: A Concept and Related IssuesEswar Publications
 
IPC in Microkernel Systems, Capabilities
IPC in Microkernel Systems, CapabilitiesIPC in Microkernel Systems, Capabilities
IPC in Microkernel Systems, CapabilitiesMartin Děcký
 
Microkernel-based operating system development
Microkernel-based operating system developmentMicrokernel-based operating system development
Microkernel-based operating system developmentSenko Rašić
 
SDN interfaces and performance analysis of SDN components
SDN interfaces and performance analysis of SDN componentsSDN interfaces and performance analysis of SDN components
SDN interfaces and performance analysis of SDN componentsSteffen Gebert
 
Networking
NetworkingNetworking
NetworkingSNancy
 
SDN: an introduction
SDN: an introductionSDN: an introduction
SDN: an introductionLuca Profico
 
Analytical Modeling of End-to-End Delay in OpenFlow Based Networks
Analytical Modeling of End-to-End Delay in OpenFlow Based NetworksAnalytical Modeling of End-to-End Delay in OpenFlow Based Networks
Analytical Modeling of End-to-End Delay in OpenFlow Based NetworksAzeem Iqbal
 

What's hot (18)

Memory, IPC and L4Re
Memory, IPC and L4ReMemory, IPC and L4Re
Memory, IPC and L4Re
 
Advanced Components on Top of L4Re
Advanced Components on Top of L4ReAdvanced Components on Top of L4Re
Advanced Components on Top of L4Re
 
Introduction to socket programming nbv
Introduction to socket programming nbvIntroduction to socket programming nbv
Introduction to socket programming nbv
 
xCORE architecture flyer
xCORE architecture flyerxCORE architecture flyer
xCORE architecture flyer
 
Data Center Networks:Virtual Bridging
Data Center Networks:Virtual BridgingData Center Networks:Virtual Bridging
Data Center Networks:Virtual Bridging
 
Software Defined Networking: A Concept and Related Issues
Software Defined Networking: A Concept and Related IssuesSoftware Defined Networking: A Concept and Related Issues
Software Defined Networking: A Concept and Related Issues
 
Microkernel design
Microkernel designMicrokernel design
Microkernel design
 
Df35592595
Df35592595Df35592595
Df35592595
 
Microkernel Evolution
Microkernel EvolutionMicrokernel Evolution
Microkernel Evolution
 
IPC in Microkernel Systems, Capabilities
IPC in Microkernel Systems, CapabilitiesIPC in Microkernel Systems, Capabilities
IPC in Microkernel Systems, Capabilities
 
Microkernel-based operating system development
Microkernel-based operating system developmentMicrokernel-based operating system development
Microkernel-based operating system development
 
SDN interfaces and performance analysis of SDN components
SDN interfaces and performance analysis of SDN componentsSDN interfaces and performance analysis of SDN components
SDN interfaces and performance analysis of SDN components
 
Networking
NetworkingNetworking
Networking
 
42 46
42 4642 46
42 46
 
SDN: an introduction
SDN: an introductionSDN: an introduction
SDN: an introduction
 
Bglrsession4
Bglrsession4Bglrsession4
Bglrsession4
 
Analytical Modeling of End-to-End Delay in OpenFlow Based Networks
Analytical Modeling of End-to-End Delay in OpenFlow Based NetworksAnalytical Modeling of End-to-End Delay in OpenFlow Based Networks
Analytical Modeling of End-to-End Delay in OpenFlow Based Networks
 
Sockets
SocketsSockets
Sockets
 

Similar to Back to Rings but not Tokens: Physical and Logical Designs for Distributed Filesystems intended for Bulk Transfer over E2E Emulated Cut-Through Circuits

What is SDN and how to approach it with Python
What is SDN and how to approach it with PythonWhat is SDN and how to approach it with Python
What is SDN and how to approach it with PythonJustin Park
 
Resilient Network Design Concepts Educat
Resilient Network Design Concepts EducatResilient Network Design Concepts Educat
Resilient Network Design Concepts EducatSamGrandprix
 
Programmable Exascale Supercomputer
Programmable Exascale SupercomputerProgrammable Exascale Supercomputer
Programmable Exascale SupercomputerSagar Dolas
 
Topic02-Architecture.pptx
Topic02-Architecture.pptxTopic02-Architecture.pptx
Topic02-Architecture.pptxImXaib
 
Naveen nimmu sdn future of networking
Naveen nimmu sdn   future of networkingNaveen nimmu sdn   future of networking
Naveen nimmu sdn future of networkingOpenSourceIndia
 
Ethcon seoul 2019 presentation final
Ethcon seoul 2019 presentation finalEthcon seoul 2019 presentation final
Ethcon seoul 2019 presentation finalHeung-No Lee
 
数据中心网络研究:机遇与挑战
数据中心网络研究:机遇与挑战数据中心网络研究:机遇与挑战
数据中心网络研究:机遇与挑战Weiwei Fang
 
1. Networking Fundamentals.pptx
1. Networking Fundamentals.pptx1. Networking Fundamentals.pptx
1. Networking Fundamentals.pptxMiguel Prado
 
A Software Design and Algorithms for Multicore Capture in Data Center Forensics
A Software Design and Algorithms for Multicore Capture in Data Center ForensicsA Software Design and Algorithms for Multicore Capture in Data Center Forensics
A Software Design and Algorithms for Multicore Capture in Data Center ForensicsTokyo University of Science
 
Can We Emulate Local Circuit Switching in Cloud Storage?
Can We Emulate Local Circuit Switching in Cloud Storage?Can We Emulate Local Circuit Switching in Cloud Storage?
Can We Emulate Local Circuit Switching in Cloud Storage?Tokyo University of Science
 
Smaller and Easier: Machine Learning on Embedded Things
Smaller and Easier: Machine Learning on Embedded ThingsSmaller and Easier: Machine Learning on Embedded Things
Smaller and Easier: Machine Learning on Embedded ThingsNUS-ISS
 
Lecture12 ie321 dr_atifshahzad - networks
Lecture12 ie321 dr_atifshahzad - networksLecture12 ie321 dr_atifshahzad - networks
Lecture12 ie321 dr_atifshahzad - networksAtif Shahzad
 
Container Attached Storage (CAS) with OpenEBS - Berlin Kubernetes Meetup - Ma...
Container Attached Storage (CAS) with OpenEBS - Berlin Kubernetes Meetup - Ma...Container Attached Storage (CAS) with OpenEBS - Berlin Kubernetes Meetup - Ma...
Container Attached Storage (CAS) with OpenEBS - Berlin Kubernetes Meetup - Ma...OpenEBS
 
MPLS in DC and inter-DC networks: the unified forwarding mechanism for networ...
MPLS in DC and inter-DC networks: the unified forwarding mechanism for networ...MPLS in DC and inter-DC networks: the unified forwarding mechanism for networ...
MPLS in DC and inter-DC networks: the unified forwarding mechanism for networ...Dmitry Afanasiev
 
Disaggregated Networking - The Drivers, the Software & The High Availability
Disaggregated Networking - The Drivers, the Software & The High AvailabilityDisaggregated Networking - The Drivers, the Software & The High Availability
Disaggregated Networking - The Drivers, the Software & The High AvailabilityOpen Networking Summit
 

Similar to Back to Rings but not Tokens: Physical and Logical Designs for Distributed Filesystems intended for Bulk Transfer over E2E Emulated Cut-Through Circuits (20)

What is SDN and how to approach it with Python
What is SDN and how to approach it with PythonWhat is SDN and how to approach it with Python
What is SDN and how to approach it with Python
 
Exascale Capabl
Exascale CapablExascale Capabl
Exascale Capabl
 
Resilient Network Design Concepts Educat
Resilient Network Design Concepts EducatResilient Network Design Concepts Educat
Resilient Network Design Concepts Educat
 
Programmable Exascale Supercomputer
Programmable Exascale SupercomputerProgrammable Exascale Supercomputer
Programmable Exascale Supercomputer
 
Again music
Again musicAgain music
Again music
 
Topic02-Architecture.pptx
Topic02-Architecture.pptxTopic02-Architecture.pptx
Topic02-Architecture.pptx
 
Naveen nimmu sdn future of networking
Naveen nimmu sdn   future of networkingNaveen nimmu sdn   future of networking
Naveen nimmu sdn future of networking
 
IP NETWORKS
IP NETWORKSIP NETWORKS
IP NETWORKS
 
Ethcon seoul 2019 presentation final
Ethcon seoul 2019 presentation finalEthcon seoul 2019 presentation final
Ethcon seoul 2019 presentation final
 
数据中心网络研究:机遇与挑战
数据中心网络研究:机遇与挑战数据中心网络研究:机遇与挑战
数据中心网络研究:机遇与挑战
 
SDN approach.pptx
SDN approach.pptxSDN approach.pptx
SDN approach.pptx
 
10 sdn-vir-6up
10 sdn-vir-6up10 sdn-vir-6up
10 sdn-vir-6up
 
1. Networking Fundamentals.pptx
1. Networking Fundamentals.pptx1. Networking Fundamentals.pptx
1. Networking Fundamentals.pptx
 
A Software Design and Algorithms for Multicore Capture in Data Center Forensics
A Software Design and Algorithms for Multicore Capture in Data Center ForensicsA Software Design and Algorithms for Multicore Capture in Data Center Forensics
A Software Design and Algorithms for Multicore Capture in Data Center Forensics
 
Can We Emulate Local Circuit Switching in Cloud Storage?
Can We Emulate Local Circuit Switching in Cloud Storage?Can We Emulate Local Circuit Switching in Cloud Storage?
Can We Emulate Local Circuit Switching in Cloud Storage?
 
Smaller and Easier: Machine Learning on Embedded Things
Smaller and Easier: Machine Learning on Embedded ThingsSmaller and Easier: Machine Learning on Embedded Things
Smaller and Easier: Machine Learning on Embedded Things
 
Lecture12 ie321 dr_atifshahzad - networks
Lecture12 ie321 dr_atifshahzad - networksLecture12 ie321 dr_atifshahzad - networks
Lecture12 ie321 dr_atifshahzad - networks
 
Container Attached Storage (CAS) with OpenEBS - Berlin Kubernetes Meetup - Ma...
Container Attached Storage (CAS) with OpenEBS - Berlin Kubernetes Meetup - Ma...Container Attached Storage (CAS) with OpenEBS - Berlin Kubernetes Meetup - Ma...
Container Attached Storage (CAS) with OpenEBS - Berlin Kubernetes Meetup - Ma...
 
MPLS in DC and inter-DC networks: the unified forwarding mechanism for networ...
MPLS in DC and inter-DC networks: the unified forwarding mechanism for networ...MPLS in DC and inter-DC networks: the unified forwarding mechanism for networ...
MPLS in DC and inter-DC networks: the unified forwarding mechanism for networ...
 
Disaggregated Networking - The Drivers, the Software & The High Availability
Disaggregated Networking - The Drivers, the Software & The High AvailabilityDisaggregated Networking - The Drivers, the Software & The High Availability
Disaggregated Networking - The Drivers, the Software & The High Availability
 

More from Tokyo University of Science

A Method for Cloud-Assisted Secure Wireless Grouping of Client Devices at Net...
A Method for Cloud-Assisted Secure Wireless Grouping of Client Devices at Net...A Method for Cloud-Assisted Secure Wireless Grouping of Client Devices at Net...
A Method for Cloud-Assisted Secure Wireless Grouping of Client Devices at Net...Tokyo University of Science
 
Ultrasound Relative Positioning for IoT Devices in Dense Wireless Spaces
Ultrasound Relative Positioning for IoT Devices in Dense Wireless SpacesUltrasound Relative Positioning for IoT Devices in Dense Wireless Spaces
Ultrasound Relative Positioning for IoT Devices in Dense Wireless SpacesTokyo University of Science
 
Towards a Packet Traffic Genome Project as a Method for Realtime Sub-Flow Tra...
Towards a Packet Traffic Genome Project as a Method for Realtime Sub-Flow Tra...Towards a Packet Traffic Genome Project as a Method for Realtime Sub-Flow Tra...
Towards a Packet Traffic Genome Project as a Method for Realtime Sub-Flow Tra...Tokyo University of Science
 
What if We Atomize Student Data and Apps and Put Them on Docker Containers?
What if We Atomize Student Data and Apps and Put Them on Docker Containers?What if We Atomize Student Data and Apps and Put Them on Docker Containers?
What if We Atomize Student Data and Apps and Put Them on Docker Containers?Tokyo University of Science
 
Large-Scale Crowdsourcing by Vehicular Data Packets in a Sparse Roadside Infr...
Large-Scale Crowdsourcing by Vehicular Data Packets in a Sparse Roadside Infr...Large-Scale Crowdsourcing by Vehicular Data Packets in a Sparse Roadside Infr...
Large-Scale Crowdsourcing by Vehicular Data Packets in a Sparse Roadside Infr...Tokyo University of Science
 
On Performance Under Hotspots in Hadoop versus Bigdata Replay Platforms
On Performance Under Hotspots in Hadoop versus Bigdata Replay PlatformsOn Performance Under Hotspots in Hadoop versus Bigdata Replay Platforms
On Performance Under Hotspots in Hadoop versus Bigdata Replay PlatformsTokyo University of Science
 
Taking the Step from Software to Product Development \\ when teaching PBL at ...
Taking the Step from Software to Product Development \\ when teaching PBL at ...Taking the Step from Software to Product Development \\ when teaching PBL at ...
Taking the Step from Software to Product Development \\ when teaching PBL at ...Tokyo University of Science
 
Design and Implementation of a 3-Party Cloud-Backed Handshake for Secure Grou...
Design and Implementation of a 3-Party Cloud-Backed Handshake for Secure Grou...Design and Implementation of a 3-Party Cloud-Backed Handshake for Secure Grou...
Design and Implementation of a 3-Party Cloud-Backed Handshake for Secure Grou...Tokyo University of Science
 
The Switchboard Optimization Problem and Heuristics for Cut-Through Networking
The Switchboard Optimization Problem and Heuristics for Cut-Through NetworkingThe Switchboard Optimization Problem and Heuristics for Cut-Through Networking
The Switchboard Optimization Problem and Heuristics for Cut-Through NetworkingTokyo University of Science
 
The Switchboard Traffic Engineering Problem for Mixed Contention/Cut-Through ...
The Switchboard Traffic Engineering Problem for Mixed Contention/Cut-Through ...The Switchboard Traffic Engineering Problem for Mixed Contention/Cut-Through ...
The Switchboard Traffic Engineering Problem for Mixed Contention/Cut-Through ...Tokyo University of Science
 
Bulk-n-Pick Method for One-to-Many Data Transfer in Dense Wireless Spaces
Bulk-n-Pick Method for One-to-Many Data Transfer in Dense Wireless SpacesBulk-n-Pick Method for One-to-Many Data Transfer in Dense Wireless Spaces
Bulk-n-Pick Method for One-to-Many Data Transfer in Dense Wireless SpacesTokyo University of Science
 
Fog Cloud Caching at Network Edge via Local Hardware Awareness Spaces
Fog Cloud Caching at Network Edge via Local Hardware Awareness SpacesFog Cloud Caching at Network Edge via Local Hardware Awareness Spaces
Fog Cloud Caching at Network Edge via Local Hardware Awareness SpacesTokyo University of Science
 
On a Hybrid Packets-and-Circuits Switching Logic
On a Hybrid Packets-and-Circuits Switching LogicOn a Hybrid Packets-and-Circuits Switching Logic
On a Hybrid Packets-and-Circuits Switching LogicTokyo University of Science
 
Image-Related Uses for Roadside Infrastructure \\ based on Wireless Beacons
Image-Related Uses for Roadside Infrastructure \\ based on Wireless BeaconsImage-Related Uses for Roadside Infrastructure \\ based on Wireless Beacons
Image-Related Uses for Roadside Infrastructure \\ based on Wireless BeaconsTokyo University of Science
 
Complexity Resolution Control for Context Based on Metromaps
Complexity Resolution Control for Context Based on MetromapsComplexity Resolution Control for Context Based on Metromaps
Complexity Resolution Control for Context Based on MetromapsTokyo University of Science
 
The Declarative-Coordinated Model for Self-Optimization of Service Networks
The Declarative-Coordinated Model for Self-Optimization of Service NetworksThe Declarative-Coordinated Model for Self-Optimization of Service Networks
The Declarative-Coordinated Model for Self-Optimization of Service NetworksTokyo University of Science
 
3-Way Scripts as a Practical Platform for Secure Distributed Code in Clouds
3-Way Scripts as a Practical Platform for Secure Distributed Code in Clouds3-Way Scripts as a Practical Platform for Secure Distributed Code in Clouds
3-Way Scripts as a Practical Platform for Secure Distributed Code in CloudsTokyo University of Science
 
3-Way Scripts as a Base Unit for Flexible Scale-Out Code
3-Way Scripts as a Base Unit for Flexible Scale-Out Code3-Way Scripts as a Base Unit for Flexible Scale-Out Code
3-Way Scripts as a Base Unit for Flexible Scale-Out CodeTokyo University of Science
 
Towards Social Robotics on Smartphones with Simple XYZV Sensor Feedback
Towards Social Robotics on Smartphones with Simple XYZV Sensor FeedbackTowards Social Robotics on Smartphones with Simple XYZV Sensor Feedback
Towards Social Robotics on Smartphones with Simple XYZV Sensor FeedbackTokyo University of Science
 
Browser Visualization using PNGs Generated by HTML5 Workers on Multicore
Browser Visualization using PNGs Generated by HTML5 Workers on MulticoreBrowser Visualization using PNGs Generated by HTML5 Workers on Multicore
Browser Visualization using PNGs Generated by HTML5 Workers on MulticoreTokyo University of Science
 

More from Tokyo University of Science (20)

A Method for Cloud-Assisted Secure Wireless Grouping of Client Devices at Net...
A Method for Cloud-Assisted Secure Wireless Grouping of Client Devices at Net...A Method for Cloud-Assisted Secure Wireless Grouping of Client Devices at Net...
A Method for Cloud-Assisted Secure Wireless Grouping of Client Devices at Net...
 
Ultrasound Relative Positioning for IoT Devices in Dense Wireless Spaces
Ultrasound Relative Positioning for IoT Devices in Dense Wireless SpacesUltrasound Relative Positioning for IoT Devices in Dense Wireless Spaces
Ultrasound Relative Positioning for IoT Devices in Dense Wireless Spaces
 
Towards a Packet Traffic Genome Project as a Method for Realtime Sub-Flow Tra...
Towards a Packet Traffic Genome Project as a Method for Realtime Sub-Flow Tra...Towards a Packet Traffic Genome Project as a Method for Realtime Sub-Flow Tra...
Towards a Packet Traffic Genome Project as a Method for Realtime Sub-Flow Tra...
 
What if We Atomize Student Data and Apps and Put Them on Docker Containers?
What if We Atomize Student Data and Apps and Put Them on Docker Containers?What if We Atomize Student Data and Apps and Put Them on Docker Containers?
What if We Atomize Student Data and Apps and Put Them on Docker Containers?
 
Large-Scale Crowdsourcing by Vehicular Data Packets in a Sparse Roadside Infr...
Large-Scale Crowdsourcing by Vehicular Data Packets in a Sparse Roadside Infr...Large-Scale Crowdsourcing by Vehicular Data Packets in a Sparse Roadside Infr...
Large-Scale Crowdsourcing by Vehicular Data Packets in a Sparse Roadside Infr...
 
On Performance Under Hotspots in Hadoop versus Bigdata Replay Platforms
On Performance Under Hotspots in Hadoop versus Bigdata Replay PlatformsOn Performance Under Hotspots in Hadoop versus Bigdata Replay Platforms
On Performance Under Hotspots in Hadoop versus Bigdata Replay Platforms
 
Taking the Step from Software to Product Development \\ when teaching PBL at ...
Taking the Step from Software to Product Development \\ when teaching PBL at ...Taking the Step from Software to Product Development \\ when teaching PBL at ...
Taking the Step from Software to Product Development \\ when teaching PBL at ...
 
Design and Implementation of a 3-Party Cloud-Backed Handshake for Secure Grou...
Design and Implementation of a 3-Party Cloud-Backed Handshake for Secure Grou...Design and Implementation of a 3-Party Cloud-Backed Handshake for Secure Grou...
Design and Implementation of a 3-Party Cloud-Backed Handshake for Secure Grou...
 
The Switchboard Optimization Problem and Heuristics for Cut-Through Networking
The Switchboard Optimization Problem and Heuristics for Cut-Through NetworkingThe Switchboard Optimization Problem and Heuristics for Cut-Through Networking
The Switchboard Optimization Problem and Heuristics for Cut-Through Networking
 
The Switchboard Traffic Engineering Problem for Mixed Contention/Cut-Through ...
The Switchboard Traffic Engineering Problem for Mixed Contention/Cut-Through ...The Switchboard Traffic Engineering Problem for Mixed Contention/Cut-Through ...
The Switchboard Traffic Engineering Problem for Mixed Contention/Cut-Through ...
 
Bulk-n-Pick Method for One-to-Many Data Transfer in Dense Wireless Spaces
Bulk-n-Pick Method for One-to-Many Data Transfer in Dense Wireless SpacesBulk-n-Pick Method for One-to-Many Data Transfer in Dense Wireless Spaces
Bulk-n-Pick Method for One-to-Many Data Transfer in Dense Wireless Spaces
 
Fog Cloud Caching at Network Edge via Local Hardware Awareness Spaces
Fog Cloud Caching at Network Edge via Local Hardware Awareness SpacesFog Cloud Caching at Network Edge via Local Hardware Awareness Spaces
Fog Cloud Caching at Network Edge via Local Hardware Awareness Spaces
 
On a Hybrid Packets-and-Circuits Switching Logic
On a Hybrid Packets-and-Circuits Switching LogicOn a Hybrid Packets-and-Circuits Switching Logic
On a Hybrid Packets-and-Circuits Switching Logic
 
Image-Related Uses for Roadside Infrastructure \\ based on Wireless Beacons
Image-Related Uses for Roadside Infrastructure \\ based on Wireless BeaconsImage-Related Uses for Roadside Infrastructure \\ based on Wireless Beacons
Image-Related Uses for Roadside Infrastructure \\ based on Wireless Beacons
 
Complexity Resolution Control for Context Based on Metromaps
Complexity Resolution Control for Context Based on MetromapsComplexity Resolution Control for Context Based on Metromaps
Complexity Resolution Control for Context Based on Metromaps
 
The Declarative-Coordinated Model for Self-Optimization of Service Networks
The Declarative-Coordinated Model for Self-Optimization of Service NetworksThe Declarative-Coordinated Model for Self-Optimization of Service Networks
The Declarative-Coordinated Model for Self-Optimization of Service Networks
 
3-Way Scripts as a Practical Platform for Secure Distributed Code in Clouds
3-Way Scripts as a Practical Platform for Secure Distributed Code in Clouds3-Way Scripts as a Practical Platform for Secure Distributed Code in Clouds
3-Way Scripts as a Practical Platform for Secure Distributed Code in Clouds
 
3-Way Scripts as a Base Unit for Flexible Scale-Out Code
3-Way Scripts as a Base Unit for Flexible Scale-Out Code3-Way Scripts as a Base Unit for Flexible Scale-Out Code
3-Way Scripts as a Base Unit for Flexible Scale-Out Code
 
Towards Social Robotics on Smartphones with Simple XYZV Sensor Feedback
Towards Social Robotics on Smartphones with Simple XYZV Sensor FeedbackTowards Social Robotics on Smartphones with Simple XYZV Sensor Feedback
Towards Social Robotics on Smartphones with Simple XYZV Sensor Feedback
 
Browser Visualization using PNGs Generated by HTML5 Workers on Multicore
Browser Visualization using PNGs Generated by HTML5 Workers on MulticoreBrowser Visualization using PNGs Generated by HTML5 Workers on Multicore
Browser Visualization using PNGs Generated by HTML5 Workers on Multicore
 

Recently uploaded

Empowering Africa's Next Generation: The AI Leadership Blueprint
Empowering Africa's Next Generation: The AI Leadership BlueprintEmpowering Africa's Next Generation: The AI Leadership Blueprint
Empowering Africa's Next Generation: The AI Leadership BlueprintMahmoud Rabie
 
Computer 10: Lesson 10 - Online Crimes and Hazards
Computer 10: Lesson 10 - Online Crimes and HazardsComputer 10: Lesson 10 - Online Crimes and Hazards
Computer 10: Lesson 10 - Online Crimes and HazardsSeth Reyes
 
UiPath Community: AI for UiPath Automation Developers
UiPath Community: AI for UiPath Automation DevelopersUiPath Community: AI for UiPath Automation Developers
UiPath Community: AI for UiPath Automation DevelopersUiPathCommunity
 
9 Steps For Building Winning Founding Team
9 Steps For Building Winning Founding Team9 Steps For Building Winning Founding Team
9 Steps For Building Winning Founding TeamAdam Moalla
 
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostKubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostMatt Ray
 
UiPath Platform: The Backend Engine Powering Your Automation - Session 1
UiPath Platform: The Backend Engine Powering Your Automation - Session 1UiPath Platform: The Backend Engine Powering Your Automation - Session 1
UiPath Platform: The Backend Engine Powering Your Automation - Session 1DianaGray10
 
Machine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdfMachine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdfAijun Zhang
 
UiPath Studio Web workshop series - Day 6
UiPath Studio Web workshop series - Day 6UiPath Studio Web workshop series - Day 6
UiPath Studio Web workshop series - Day 6DianaGray10
 
UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8DianaGray10
 
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...Aggregage
 
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPAAnypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPAshyamraj55
 
Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024SkyPlanner
 
VoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBXVoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBXTarek Kalaji
 
20230202 - Introduction to tis-py
20230202 - Introduction to tis-py20230202 - Introduction to tis-py
20230202 - Introduction to tis-pyJamie (Taka) Wang
 
Nanopower In Semiconductor Industry.pdf
Nanopower  In Semiconductor Industry.pdfNanopower  In Semiconductor Industry.pdf
Nanopower In Semiconductor Industry.pdfPedro Manuel
 
NIST Cybersecurity Framework (CSF) 2.0 Workshop
NIST Cybersecurity Framework (CSF) 2.0 WorkshopNIST Cybersecurity Framework (CSF) 2.0 Workshop
NIST Cybersecurity Framework (CSF) 2.0 WorkshopBachir Benyammi
 
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdfUiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdfDianaGray10
 
Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024D Cloud Solutions
 
COMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a WebsiteCOMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a Websitedgelyza
 

Recently uploaded (20)

Empowering Africa's Next Generation: The AI Leadership Blueprint
Empowering Africa's Next Generation: The AI Leadership BlueprintEmpowering Africa's Next Generation: The AI Leadership Blueprint
Empowering Africa's Next Generation: The AI Leadership Blueprint
 
Computer 10: Lesson 10 - Online Crimes and Hazards
Computer 10: Lesson 10 - Online Crimes and HazardsComputer 10: Lesson 10 - Online Crimes and Hazards
Computer 10: Lesson 10 - Online Crimes and Hazards
 
UiPath Community: AI for UiPath Automation Developers
UiPath Community: AI for UiPath Automation DevelopersUiPath Community: AI for UiPath Automation Developers
UiPath Community: AI for UiPath Automation Developers
 
9 Steps For Building Winning Founding Team
9 Steps For Building Winning Founding Team9 Steps For Building Winning Founding Team
9 Steps For Building Winning Founding Team
 
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostKubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
 
UiPath Platform: The Backend Engine Powering Your Automation - Session 1
UiPath Platform: The Backend Engine Powering Your Automation - Session 1UiPath Platform: The Backend Engine Powering Your Automation - Session 1
UiPath Platform: The Backend Engine Powering Your Automation - Session 1
 
Machine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdfMachine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdf
 
UiPath Studio Web workshop series - Day 6
UiPath Studio Web workshop series - Day 6UiPath Studio Web workshop series - Day 6
UiPath Studio Web workshop series - Day 6
 
UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8
 
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
 
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPAAnypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPA
 
Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024
 
VoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBXVoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBX
 
20230202 - Introduction to tis-py
20230202 - Introduction to tis-py20230202 - Introduction to tis-py
20230202 - Introduction to tis-py
 
Nanopower In Semiconductor Industry.pdf
Nanopower  In Semiconductor Industry.pdfNanopower  In Semiconductor Industry.pdf
Nanopower In Semiconductor Industry.pdf
 
NIST Cybersecurity Framework (CSF) 2.0 Workshop
NIST Cybersecurity Framework (CSF) 2.0 WorkshopNIST Cybersecurity Framework (CSF) 2.0 Workshop
NIST Cybersecurity Framework (CSF) 2.0 Workshop
 
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdfUiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
 
Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024
 
COMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a WebsiteCOMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a Website
 
20230104 - machine vision
20230104 - machine vision20230104 - machine vision
20230104 - machine vision
 

Back to Rings but not Tokens: Physical and Logical Designs for Distributed Filesystems intended for Bulk Transfer over E2E Emulated Cut-Through Circuits

  • 1. Back to Rings but not Tokens: 2015/11/17 Marat Zhanikeev maratishe@gmail.com IN研@熊本 PDF: bit.do/151117 Physical and Logical Designs for Distributed Filesystems intended for Bulk Transfer over E2E Emulated Cut-Through Circuits
  • 2. The Big Picture 1. increase capacity and flexibility at the same number of ports 2. do it at a relatively low cost 3. stay at hardware level -- namely the cut-through mode • ultimate goal: a better storage grid for BigData 14 Super-duper 32-port Software Switch (ClickOS, SDN, NFV,...) Simple 4-port Simple 4-port Simple 4-port Simple 4-port Simple 4-port Simple 4-port Simple 4-port Simple 4-port vs >cost capacity<<< 14 M.Zhanikeev "Streaming Algorithms for Big Data Processing on Multicore" Big Data: Algorithms..., CRC (2015) M.Zhanikeev -- maratishe@gmail.com -- ...Rings but not Tokens: ...Distributed Filesystems ...over E2E Emulated Cut-Through Circuits -- bit.do/151117 2/22 2/22
  • 3. Model of Per-Packet Overhead C: Cut Through Check, etc. Q: Queue D: Drop QoS classes M.Zhanikeev -- maratishe@gmail.com -- ...Rings but not Tokens: ...Distributed Filesystems ...over E2E Emulated Cut-Through Circuits -- bit.do/151117 3/22 3/22
  • 4. Circuits at a Scheduling Problem Line= outgoing port Overhead = contention No. of flows Line= outgoing port Overhead Scheduling Traditional Circuits M.Zhanikeev -- maratishe@gmail.com -- ...Rings but not Tokens: ...Distributed Filesystems ...over E2E Emulated Cut-Through Circuits -- bit.do/151117 4/22 4/22
  • 5. The Tall Gate Model • sensing is close to the wireless opportunistic/cognitive tech • the target: make even long-haul e2e circuits possible (DC-DC) Tall Gates Bulks Send Highway Sources Destination 06 M.Zhanikeev "A City Traffic Model for Optical Circuit Switching in Data Centers" IEICE・OCS研 (2014) M.Zhanikeev -- maratishe@gmail.com -- ...Rings but not Tokens: ...Distributed Filesystems ...over E2E Emulated Cut-Through Circuits -- bit.do/151117 5/22 5/22
  • 6. Optical Circuits = OCS (no OBS/OPS) E-O ingress O-E egress O-E-O CORE O-O NOC Management over Ethernet Contention Resolution Bulk E-O ingress O-E egress NOC Management over Ethernet Contention Resolution, TE O-O CORE Bulk • OBS today is considered the best technology • however, when contention management is moved to NOC, OCS is feasible • Tall Gate model is applicable M.Zhanikeev -- maratishe@gmail.com -- ...Rings but not Tokens: ...Distributed Filesystems ...over E2E Emulated Cut-Through Circuits -- bit.do/151117 6/22 6/22
  • 7. Circuits: Feature Comparison • Tall Gate is better than Traditional Scheduler • already discussed in DC networking 1112 Interference Overhead Isolation Do Nothing HIGH ZERO NO Network Virtualization HIGH HIGH NO (store-and -forward) Traditional Scheduler LOW HIGH YES (cut -through) P2Px1N (1 network) HIGH VERY HIGH YES P2Px2N (2 networks) ZERO VERY HIGH YES (cut -through) Tall Gate (sensing) VERY LOW HIGH YES (cut -through) 11 "Cut-Through and Store-and-Forward Ethernet Switching for Low-Latency Environments" Cisco White Paper (2014) 12 G.Wang+5 "c-Through: Part-time Optics in Data Centers" ACM SIGCOMM (2010) M.Zhanikeev -- maratishe@gmail.com -- ...Rings but not Tokens: ...Distributed Filesystems ...over E2E Emulated Cut-Through Circuits -- bit.do/151117 7/22 7/22
  • 8. Circuits: Performance under Hotspots =bigdata 0 2 4 6 8 Ordered list 0 0.45 0.9 1.35 1.8 2.25 log(duration) Do Nothing Network VirtualizationTraditional Scheduler P2Px1N P2Px2N Tall Gate 0 2 4 6 8 Ordered list 0 0.8 1.6 log(duration) 0 2 4 6 8 Ordered list 0.6 1.2 1.8 2.4 log(duration) 0 2 4 6 8 Ordered list 1.5 1.8 2.1 2.4 log(duration) 0 2 4 6 8 Ordered list 1.65 1.95 2.25 2.55 log(duration) 0 2 4 6 8 Ordered list 1.2 1.6 2 2.4 log(duration) Size: 10M..100M Size: 100M..500M Size: 500M..1G Size: 10G..100G Size: 1G..10G Size: 10G..50G 2.4 • hotspot traffic: few very large flows • group by size range • compare transfer time across the models M.Zhanikeev -- maratishe@gmail.com -- ...Rings but not Tokens: ...Distributed Filesystems ...over E2E Emulated Cut-Through Circuits -- bit.do/151117 8/22 8/22
  • 9. Today: The Basic Idea M.Zhanikeev -- maratishe@gmail.com -- ...Rings but not Tokens: ...Distributed Filesystems ...over E2E Emulated Cut-Through Circuits -- bit.do/151117 9/22 9/22
  • 10. The Basic Idea: Logical Circuits Physical Topological A switch 4 ports 8 ports … Legend Port Switch Node Logical link (domain) Physical link … … <2,2,2> <3,1,2> <1,4,4> <3,5,3> • take standard switches (cut-through OK on most) • design isolated rings within the available port • ⟨a, b, c⟩ means: a ports/nodes on inner ring, b outer rings, c largest hub/port • implementation: VLANs on Ethernet, MEMS in optical, etc. M.Zhanikeev -- maratishe@gmail.com -- ...Rings but not Tokens: ...Distributed Filesystems ...over E2E Emulated Cut-Through Circuits -- bit.do/151117 10/22 10/22
  • 11. Deep/Dark Theory M.Zhanikeev -- maratishe@gmail.com -- ...Rings but not Tokens: ...Distributed Filesystems ...over E2E Emulated Cut-Through Circuits -- bit.do/151117 11/22 11/22
  • 12. Deep/Dark Theory (1) <x1,y1,z1> <x2,y2,z2> <x2,y2,z2> … <xn,yn,zn> … <x2,y2,z2> … … … 1..m (different for each level) n … k Processing nodes Storage nodes Connections to top level … M.Zhanikeev -- maratishe@gmail.com -- ...Rings but not Tokens: ...Distributed Filesystems ...over E2E Emulated Cut-Through Circuits -- bit.do/151117 12/22 12/22
  • 13. Deep/Dark Theory (2) • can be used to analyze connectivity using adjacency matrices N11 N12 N21 Nn1 Nnq N1p… N2m … … … … … … Processors Storage grid row a row b a size a size a x a matrix of peer mesh a size b size a x b matrix between levels M.Zhanikeev -- maratishe@gmail.com -- ...Rings but not Tokens: ...Distributed Filesystems ...over E2E Emulated Cut-Through Circuits -- bit.do/151117 13/22 13/22
  • 14. The Much Simpler Practice M.Zhanikeev -- maratishe@gmail.com -- ...Rings but not Tokens: ...Distributed Filesystems ...over E2E Emulated Cut-Through Circuits -- bit.do/151117 14/22 14/22
  • 15. The Much Simpler Practice (1) • in concept, close to harness braiding (photo) • similar to a Google paper on the design of its in-rack and rack-rack crosses • immediately obvious: at least 2 distinct routes between any 2 nodes 4 ports <1,4,4> M.Zhanikeev -- maratishe@gmail.com -- ...Rings but not Tokens: ...Distributed Filesystems ...over E2E Emulated Cut-Through Circuits -- bit.do/151117 15/22 15/22
  • 16. The Much Simpler Practice (2) • with ⟨3, 5, 3⟩ tuple, much more flexibility • scientifically interesting: where to connect extra ports (above minimum connectivity) -- 2 or 3 switches away? ◦ ... and how does this affect overall connectivity? 8 ports <3,5,3> ? ? M.Zhanikeev -- maratishe@gmail.com -- ...Rings but not Tokens: ...Distributed Filesystems ...over E2E Emulated Cut-Through Circuits -- bit.do/151117 16/22 16/22
  • 17. Switch Robotics? • some book libraries have already been roboticized • why not return to a telephone switchboard, but in a robotic version? • goal: dynamic management of connectivity using robotically migrating ports • mid-goal: switches with wobbling physical ports + M.Zhanikeev -- maratishe@gmail.com -- ...Rings but not Tokens: ...Distributed Filesystems ...over E2E Emulated Cut-Through Circuits -- bit.do/151117 17/22 17/22
  • 18. That’s all, thank you ... M.Zhanikeev -- maratishe@gmail.com -- ...Rings but not Tokens: ...Distributed Filesystems ...over E2E Emulated Cut-Through Circuits -- bit.do/151117 18/22 18/22
  • 19. Application: BigData Replay • Hadoop/MapReduce has failed 02 -- can only support (barely) 15k concurrent users 01 (among many other problems) • BigData Replay on massively multicore 14 is a valid alternative Name Node Storage Node (shard) file A file B file C … Hadoop Space Manager Hadoop Job (your code) Hadoop Job (your code) Hadoop Job (your code) MapReduce job (your code) manymany Name Server(s) Client Machine Hadoop Client Your Code You Start Use Deploy FindRead/parse many Storage Node (shard) Time-Aware Sub-Store(s) Manager Client Machine Client Your Sketcher You Start Use Schedule Multicore Replay Replay Node many 02 A.Rowstron+4 "Nobody ever got fired for using Hadoop on a cluster" 1st Hot Topics in Cloud Data Processing (2012) 01 K.Shvachko "HDFS Scalability: the Limits to Growth" the Magazine of USENIX, vol.35, no.2 (2012) 14 M.Zhanikeev "Streaming Algorithms for Big Data Processing on Multicore" Big Data: Algorithms..., CRC (2015) M.Zhanikeev -- maratishe@gmail.com -- ...Rings but not Tokens: ...Distributed Filesystems ...over E2E Emulated Cut-Through Circuits -- bit.do/151117 19/22 19/22
  • 20. Bigdata Replay on Massively Multicore …. Time Now (buffer head) Manager Job Job Buffer tail pos pos Controller Kill 2 Report Manage in realtime One Replay Batch One Buffer One Buffer One BufferJobs Jobs Jobs Replay at a scale 1 • massively multicore ̸= manycore 08 • 100+ cores on conventional hardware -- standard RAM, shmap, etc. • target: 100k jobs, using Multiple Replay nodes 08 M.Zhanikeev "...Massively Multicore, Heterogeneous Jobs with Hotspots, and Data Streaming" SWoPP (2015) M.Zhanikeev -- maratishe@gmail.com -- ...Rings but not Tokens: ...Distributed Filesystems ...over E2E Emulated Cut-Through Circuits -- bit.do/151117 20/22 20/22
  • 21. NextGen NOC • circuits are really really really valuable when they considerably reduce bulk transfer time • future NOCs will develop, advertise, and sell their ability to provide circuits • part of NGN and autonomy network management -- see recent IETF/ MRTG meeting NOC 10 M.Zhanikeev "The Next Generation of Networks is all about Hotspot Distributions and Cut-Through Circuits" IEICE・CQ研 (2015) M.Zhanikeev -- maratishe@gmail.com -- ...Rings but not Tokens: ...Distributed Filesystems ...over E2E Emulated Cut-Through Circuits -- bit.do/151117 21/22 21/22
  • 22. Hotspot Distribution • a hotspot distribution consists of normal, popular and hot/flash sets • describe a wide range of natural processes, traffic in particular 0 10 20 30 40 50 List of traffic sources 0 0.4 0.8 1.2 1.6 2 2.4 2.8 log(trafficvolume) 0 10 20 30 40 50 List of traffic sources 0 0.4 0.8 1.2 1.6 2 2.4 2.8 log(trafficvolume) Magnitude=2 Magnitude=10 Hotspots Normal Hotspot under a Flash event M.Zhanikeev -- maratishe@gmail.com -- ...Rings but not Tokens: ...Distributed Filesystems ...over E2E Emulated Cut-Through Circuits -- bit.do/151117 22/22 22/22