Integrating OpenStack To Existing Infrastructure

Integrating OpenStack to Existing Infrastructure Cheng, Hui freedomhui@gmail.com Twitter:@freedomhui 1 2012-04-19

AgendaBackground● Who We Are● Infrastructure & Platform● ChallengesIntegration Challenges● Network Deployment● Security Consideration● Load Balancer● Swift EvaluationOur Contributions● Billing● Monitoring

Who Are We Sina.com • Largest infotainment web portal in China • Provides various on-line services, like news, Finance, video, email, blog hosting, etc. • Operates first PaaS cloud computing platformSina Weibo• twitter-like microblog service• over 300m users• huge influence on Chinas society We are building a reliable, scalable and secure infrastructure and platform to support our business.

Infrastructure & PlatformPhysical ServersTraditional OperationVirtualization Platform(IaaS)●VM Management System(VMMS) → Sina WebService(SWS)●VMMS is private solution developed in-house●SWS is based on OpenStackApplication Platform(PaaS)●Virtual Host → Sina App Engine(SAE)●SAE provides both Public and Private Service.

Sina App Engine• No. 1 Public PaaS Platform inChina launched in Nov 2009• PHP, Python, Java and RubySupport• Numbers160,000+ developers200,000+ apps on SAE800 million page views per day20+ Services• SAE Cloud Storage Service is replaced by Swift• Deploy SAE on OpenStack

ChallengesSAE meets the majority of business needs, but does not coverall, especially for web gamesCustomers require full stack of cloud computingWe Choose OpenStack as our IaaS solution

Why Choose OpenStack 100% Python & Open Source

OpenStack Deployment Rabbit MySQL dashboard schedule nova-api nova-compute nova-compute nova-network nova-network keystone glanceSina SSO Swift

Nova NetworkNetworking is the biggest challenges for IaaSNetwork Topology:• VLAN• FlatDHCP• FlatDHCP & Multihost

Network Topology --- VLANCapability:• Accessibility of VMs within one tenant• Isolation of VMs from different tenants• VM is able to access public network• VM can be accessible from public network• Isolation between virtual network and internal network Drawback: • Pre-allocate network for future projects • Traffic bottleneck in the NAT gateway 12

Network Topology(Flat)Capability:• Accessibility of all VMs in the fixed IP range• VM is able to access public network• VM can be accessible from public network• Full isolation between virtual network and internal networkDrawback:Tenant isolation lessensTraffic bottleneck in the NAT gateway 13

Network Topology(Flat & Multihost)Capability:• Accessibility of all VMs in the fixed IP range• VM is able to access public network• VM can be accessible from public networkBonus:• Totally distributed architecture avoid single-point failure.• Multiple gateway eliminates NAT bottleneck• High throughout between OS regionsDrawback:• Tenant isolation lessens• Need security facility(SWS-filter) to protect intranet If security problems were solved, this would be our best choice! 14

Security in OpenStackSecurity Group --- Layer 3 Filter Static filters --- Layer 2 FilterRole-based firewall MAC, IP, and ARP spoofing protection One security group is a Role  Not configurableIngress filtering  Defined in /etc/libvirt/nwfilter/*.xml Target is the instance Implemented by ebtables Source can be CIDR or another group  ebtables -t nat --listImplemented by iptables See details: iptables -t filter -n -L Whitelist mechanism(ACCEPT rules) 15

Security EnhancementSWS FilterPrevent Intranet Penetration• Intranet is the internal network outside of OpenStackEgress filtering• Target is internal network• Source is instances in OpenStackImplementation• Whitelist mechanism(ACCEPT rules)• On the top of nova-filter-top Forward ChainRational• SWS filter is managed by cloud manager• Only explicit authorized packets can reach Internal network C• Packet should be controlled within Compute Node 16

Security EnhancementSecurity Group VS SWS Filter 17

Load BalancerGoalsLoad Balance• Dispatch request DNS Acceleration Design• Support multiple routing algorithm• Health check Smart DNSAcceleration• Reality: narrow bandwidth between ISPs• Building fiber channels from ISPs to pivot Public Network• Given the same endpoint within user’s ISP Telecom Unicom Mobile Others ISPIPv4 Shortage• Reality: dozens of public IPs support hundreds of VMs High speed fiber channel• IPv4 has been exhausted• IPv6 is not realistic yet in China Pivot 18

Load BalancerLayer 7 Load BalancerConsideration:1. dispatch request by Host header2. nginx module 19

Load BalancerLayer 4 Load BalancerConsideration:1. dispatch request by TCP port2. lvs + haproxy 20

Swift Evaluation Extremely Durable and Highly Available Superior Scalability Linear Growth of Performance Symmetric Architecture No Single-failure Simple & Reliable 21

Swift Evaluation • 1 Zone = 1 Physical Server with 12x2T disk GET abc.png • Write/Read applies quorum protocol PUT abc.png Load Balancer Zone1 Zone2 Zone3 Zone4 Zone5 Proxy Server Proxy Server Proxy Server Proxy Server Proxy ServerObject Server Object Server Object Server Object Server Object ServerContainer Server Container Server Container Server Container Server Container ServerAccount Server Account Server Account Server Account Server Account Server 22

Swift Evaluation Swift packages Proxy Server Account Server Container Server Object Server Physical Deployment Storage Nodes OS installation sda sdb sdc sdd sdk raid 1 ……disk1 disk2 disk3 disk4 disk5 disk12 23

Swift EvaluationPerformance issueCPU utilization rate up to 100% even without requestTesting environment: Audit:Nodes: 5 x Dell R510 swift-account-auditor : 1.5mCPU: Intel® Xeon® E5360 swift-account-replicator: 9.5mMemory: 12GBReplica: 3 swift-container-auditor: 8.4m swift-container-replicator: 9.3mNo. of Objects: 150,000,000 swift-container-updater: 19.0mNo. of Accounts: 120,000No. of Containers: 160,000 swift-object-updater: 0.1 s swift-object-replicator: 10.5 hours swift-object-auditor: 48.3 hoursResult:Periodic scanning all partitions, calculating checksum and synchronization 24

RPC● Biling & Monitoring Database Client Compute Network RDBMS Dashboard Storage Monitoring Billing (Metering) 25 NoSQL

● Kanyun: Monitoring system Compute Worker Network RDBMS Dashboard Storage Worker Retrieve usage info API daemon Billing Aggregator Responds to client Calculates/stores request metrics http://github.com/lzyeval/kanyun 26 NoSQL

RPC● Dough:Billing system Database Client Compute Network RDBMS Dashboard Storage Collector Monitoring Farmer API daemon (Metering) Dispatch jobs Subscribe or Collector unsubscribe products / Check status / Query info Retrieve usage / http://github.com/lzyeval/dough 27 Create purchases

Q&A 28

Integrating OpenStack To Existing Infrastructure

by Hui Cheng on Apr 17, 2012

Accessibility

Categories

Upload Details

Usage Rights

1 Embed 3

Statistics

Integrating OpenStack To Existing Infrastructure Presentation Transcript