This document discusses how Midokura builds its virtualization software for networking using open source technologies. It explains that Midokura uses Zookeeper to provide consistency for tracking changes to the virtual network topology and state, and uses Cassandra for high write volumes to backup stateful connection tracking information like flow state and metrics. The document also describes how Midokura leverages distributed intelligence at the edge by pushing SDN intelligence to agents, and how it must optimize consistency, availability, and partition tolerance differently for different types of data.

1. Building Open Source
with Open Source
Susan Wu
Director of Technical Marketing
Midokura

2. About Me
§ Took the scenic drive to Open Source from proprietary software like Oracle, Citrix, Sun
§ Product marketing for container technologies like Solaris Zones, Docker
§ Plugins/Connectors for enterprise manager
§ Open Source community experience: Ubuntu, Docker, OpenStack, CloudStack, MidoNet

3. About Midokura
§ Global startup with about 50+ employees
§ Founders built distributed systems at Amazon and Google
§ Engineering in Barcelona, Tokyo, Tel Aviv
§ Sales and Marketing in San Francisco
§ Network Virtualization for OpenStack and Docker
§ Customers in web scale, enterprise, higher ed and service provider segments

4. Why Open Source
§ It’s actually a two-fold question…for Midokura
- Why build virtualization software for networking with open source technologies
- Why did the company open source the software
§ Today, we’re covering the first topic
- Why we build with open source
- And how we use open source technologies like Zookeeper and Cassandra

5. /* Copyright (c) 2015 Midokura SARL,All Rights Reserved. */
import Ember from 'ember';
import isValidIP4 from 'midonet-manager/utils/is-valid-ip4';
import {
copyOpt121Route,
createOpt121Route
} from 'midonet-manager/utils/dhcp-subnets';
import DHCPSubnetCommons from 'midonet-manager/mixins/dhcp-subnet; -commons’;
const {
get,
set,
setMeta,
Logger: {
debug
},
Component
} = Ember;
export default Component.extend(DHCPSubnetsCommons, {
selectedDevice: null,
actions: {
update() {
let model;
let networkId;
let {
dnsServerAddrs,
defaultGateway,
subnetPrefix,
subnetLength,
interfaceMTU,
opt121Routes
} = this.validateDHCPSubnet();
model = get(this, 'model');
networkId = get(this, 'selectedDevice').get('id');
setMeta(model, '_networkId', networkId);
setMeta(model, '_uuidBeforeSave', model.get('id'));

6. Open Source Leads in Innovation
§ Used to be imitation and commoditization to an expensive, proprietary solution
§ Now open source leads in innovation
§ Leveraging open source as building blocks speed up innovation
§ OpenStack - building an advanced Infrastructure as a service (IaaS) platform with open
source
§ Same with open source MidoNet

8. Cloud Networking

9. MidoNet
Topology

10. Intelligence at the Edge
MidoNet leverages a distributed architecture where the SDN intelligence is pushed at the edge
1. VM 1 sends a packet through the virtual network
2. MN Agent fetches the virtual topology/state
3. It simulates the packet through the virtual network
4. It installs a flow rule in the kernel at the ingress host
5. Tunnel packets to egress host

11. Data type drives technology used

12. Data type drives technology used
3. Host State 4. Flow State
Host State
Active
Inactive
Flow State
Flow IP Source
IP Destination
Port Source
Port Destination
Protocol
NAT
Router ID
Type of NAT (dst or src)
NAT Binding
Connection Tracking
+ Device ID

13. Can only optimize 2 out of 3

14. Zookeeper provides consistency
§ Suited for Consistency
§ Performance (in memory, r > w)
- In memory: limits dataset size (backed by disk)
§ Reliable delivery
- If a node sees an update, all will eventually
§ Total & Causal Order
- Data is delivered in the same order it is sent
- A message m is delivered only after all messages sent before m have been delivered
§ Not suited for high write volume
Consistency Availability Partition Tolerance Solution
Virtual Network Topology X X Zookeeper
Virtual Network State X X Zookeeper
Host State X X Zookeeper

15. Three Zookeeper use cases
§ Watch for Data Changes
§ Service Discovery
§ Coordination

16. Zookeeper tracks the changes

17. Zookeeper notifies subscribers
Updating changes to logical network configurations

18. Deploy in odd numbers for decisive vote
Leader

19. Service Discovery
Leader

20. Cassandra used for backup
§ Well suited for high write volume
§ Used in storing stateful connection tracking information like flow state and metrics
§ Chosen for availability
§ Although the agent can function in the case of Cassandra failure as flow state is stored in
memory, and so Cassandra is primarily used as backup
Consistency Availability Partition Tolerance Solution
Flow State X X Cassandra

21. Fault Tolerance
FAIL
w(x)ok
ok
ok

22. Fault Tolerance
x
x
x

23. Trading off Consistency
read_repair
x1
x
x

24. Trading off Consistency
?

25. Must Haves: Low Latency, High Throughput
for Network Address Translation
§ Trivial Schema Changes
- vs. schema changes in shards
§ Flexible (good on growth mode)

26. Putting it all together
§ Start with a packet
§ What’s the virtual
topology?
§ Go there (you and
all other similar
packets)

27. Ready to Try MidoNet?
https://www.midonet.org/#quickstart