InMobi Private Cloud: Going the 
CloudPlatform Way 
Iliyas Shirol 
InMobi
#whoami 
• BigData/Cloud Evangelist at InMobi 
• Working in CloudStack Since 2012 
• CloudStack Community Gardener 
@iliyas_shirol
Agenda 
• About InMobi 
• Going Cloud Way 
• Architecture & Design 
• Questions
About InMobi
About InMobi
Going Cloud Way
Why Cloud ? 
• Development and Test Environment 
• Scalable Web Applications and Content Services 
• High Speed Server Resource Deployment 
• And More..
Why Dev / Test in the Cloud ? 
• Disconnected Tools, Servers, People 
• Slow Build and Release Cycles 
• Multiple Build and Test Systems Across Locations 
• Home-Built Systems are Brittle, Error-Prone, and Not Scalable 
• Difficult to Build, Test, and Recreate the Data 
• Difficult to Audit Processes and Predict Schedules 
• Time, Money Wasted on Broken Builds 
• Dedicated Build/Test Servers are not Fully Utilized
How Does a Dev/Test Cloud Look Like ? 
• Self-Service 
Developers can provision their own environments 
• Usage Measurement 
We worry about VM sprawl 
• Isolated Networks 
Must not let dev/test interfere with the real world 
• Commodity 
As cheap as practical 
• May also House Production Workloads
Early Assessments & Results 
• Budget 
• Integration with Public Cloud 
• Scaling 
• Reconfiguration 
• Legacy Hardware
Why Private Cloud ? 
• Leverage Existing Infrastructure 
• Customization and Control 
• Regulatory/Security Requirements 
• Cost
Why Citrix CloudPlatform ? 
• Scalable 
Grow-as-you-need, expand across borders 
• Open Source Community Support 
Active user and development support base 
• Advanced Networking 
VLAN integrated, SDN ready 
• Multi-Tenancy 
• Multi-Hypervisor Support
Why Citrix CloudPlatform ? 
• Easy to Deploy 
• Speed to Production 
• Documentation
Architecture & Design
The Design Phase 
• Bring Everyone in the Room 
• Know the Fine Print 
• Understand Workloads 
• Understand Bottlenecks
Redundancy and Resilience 
• CloudStack Management Server 
• MySQL Servers 
• Primary & Secondary Storage 
• Hypervisor Farm
Management Layout & Design
Compute 
• Dell PE R610’s & R720’s 
• 96 – 256 GB RAM 
• 40-80 CPU Cores 
• 2 Advanced + 1 Basic Zones 
• 1 Pod in Each Zone 
• 1 – 5 Clusters in Each Pod 
• XenServer 6.2
Network 
• Gigabytes 
• Management Uses 1 Bonded NIC 
• Public + Guest Uses 1 Bonded NIC 
• Storage Uses 1 Bonded NIC (10G) 
• Access & Storage Networks are Isolated 
• Dell 6248 
• Dell 8132
Primary Storage Design (NexentaStor) 
• ZFS Filesystem, Based on OpenSolaris and Debian 
• Linux Open Source Code Bases 
( Moving to illumos) 
• Software Only 
• Platform, Plugins and Adapters 
• Compelling Advantages over Proprietary Filesystems
What is NexentaStor ?
Advantages of NexentaStor Storage Appliance 
• Simple Installation and Management. 
• Optimized Base Configuration that Allows for: 
‣ Easily reproducible configuration on a massive scale 
‣ Generalized storage skill set required to maintain extensive infrastructure 
‣ Little or no need for modification to achieve high performance
Primary Storage – Hardware List 
Head Nodes – 2 Numbers 
2x Dell R720 with Onboard Intel 1Gbps NIC, Perc H710 
4x Intel® Xeon® Hex Core – 2.3Ghz or higher 
24x 16GB 1600Mhz Memory 
4x 300GB 2.5" 10K RPM SAS disks (Nexenta Syspool Configuration) 
4x 2port Dell (LSI) SAS HBA 
4x 10Gbps Intel NIC's low profile
Primary Storage – Hardware List 
JBODs – 2 Numbers 
PowerVault(TM) MD1220 Disk Storage Enclosure, 2U 
24 Disk Slots 
SSD Disks 
30 x 2TB SSD Drives
HeadNode-JBOD SAS Connectivity
HeadNode Network Connectivity
Provisioning Primary Storage 
• Expose a Volume Using NFS/iSCSI as per the Cluster 
Requirement 
• Add the Respective Volume as a ‘Primary Storage’ from the 
CloudPlatform Management Portal 
• CloudPlatform Provisioning with NexentaStor is Automated 
Using Enterprise Chef™ Cookbooks, using the Knife API Plug-ins
Secondary Storage Design 
• GlusterFS with One-to-One Replication 
• uCarp to Manage Floating IP 
• Hardware Specs 
• SuperMicro MBD-XS8DTE-F-O 
• 32GB Memory 
• 24x1TB SATA
GlusterFS Network Layout
Current Workload 
• Cassandra Cluster 
• Client Facing Applications 
• Database Servers (PostgreSQL, MySQL, Redis etc.) 
• And More …
What We Learnt ? 
• Preparation 
• Compatibility Testing 
• Stable Network 
• Pay Attention on HA 
• Staging Cluster 
• Schedule Backup Solution 
• Contribute Back to the Community
Steps Ahead 
• Migrate traditional data centers into Cloud 
• Evaluate and build a Hyper-V zone
Questions ?
Thank You! 
mailto: iliyas.shirol@inmobi.com

Citrix Synergy 2014: Going the CloudPlatform Way

  • 1.
    InMobi Private Cloud:Going the CloudPlatform Way Iliyas Shirol InMobi
  • 2.
    #whoami • BigData/CloudEvangelist at InMobi • Working in CloudStack Since 2012 • CloudStack Community Gardener @iliyas_shirol
  • 3.
    Agenda • AboutInMobi • Going Cloud Way • Architecture & Design • Questions
  • 4.
  • 5.
  • 6.
  • 7.
    Why Cloud ? • Development and Test Environment • Scalable Web Applications and Content Services • High Speed Server Resource Deployment • And More..
  • 8.
    Why Dev /Test in the Cloud ? • Disconnected Tools, Servers, People • Slow Build and Release Cycles • Multiple Build and Test Systems Across Locations • Home-Built Systems are Brittle, Error-Prone, and Not Scalable • Difficult to Build, Test, and Recreate the Data • Difficult to Audit Processes and Predict Schedules • Time, Money Wasted on Broken Builds • Dedicated Build/Test Servers are not Fully Utilized
  • 9.
    How Does aDev/Test Cloud Look Like ? • Self-Service Developers can provision their own environments • Usage Measurement We worry about VM sprawl • Isolated Networks Must not let dev/test interfere with the real world • Commodity As cheap as practical • May also House Production Workloads
  • 10.
    Early Assessments &Results • Budget • Integration with Public Cloud • Scaling • Reconfiguration • Legacy Hardware
  • 11.
    Why Private Cloud? • Leverage Existing Infrastructure • Customization and Control • Regulatory/Security Requirements • Cost
  • 12.
    Why Citrix CloudPlatform? • Scalable Grow-as-you-need, expand across borders • Open Source Community Support Active user and development support base • Advanced Networking VLAN integrated, SDN ready • Multi-Tenancy • Multi-Hypervisor Support
  • 13.
    Why Citrix CloudPlatform? • Easy to Deploy • Speed to Production • Documentation
  • 14.
  • 15.
    The Design Phase • Bring Everyone in the Room • Know the Fine Print • Understand Workloads • Understand Bottlenecks
  • 16.
    Redundancy and Resilience • CloudStack Management Server • MySQL Servers • Primary & Secondary Storage • Hypervisor Farm
  • 17.
  • 18.
    Compute • DellPE R610’s & R720’s • 96 – 256 GB RAM • 40-80 CPU Cores • 2 Advanced + 1 Basic Zones • 1 Pod in Each Zone • 1 – 5 Clusters in Each Pod • XenServer 6.2
  • 19.
    Network • Gigabytes • Management Uses 1 Bonded NIC • Public + Guest Uses 1 Bonded NIC • Storage Uses 1 Bonded NIC (10G) • Access & Storage Networks are Isolated • Dell 6248 • Dell 8132
  • 20.
    Primary Storage Design(NexentaStor) • ZFS Filesystem, Based on OpenSolaris and Debian • Linux Open Source Code Bases ( Moving to illumos) • Software Only • Platform, Plugins and Adapters • Compelling Advantages over Proprietary Filesystems
  • 21.
  • 22.
    Advantages of NexentaStorStorage Appliance • Simple Installation and Management. • Optimized Base Configuration that Allows for: ‣ Easily reproducible configuration on a massive scale ‣ Generalized storage skill set required to maintain extensive infrastructure ‣ Little or no need for modification to achieve high performance
  • 23.
    Primary Storage –Hardware List Head Nodes – 2 Numbers 2x Dell R720 with Onboard Intel 1Gbps NIC, Perc H710 4x Intel® Xeon® Hex Core – 2.3Ghz or higher 24x 16GB 1600Mhz Memory 4x 300GB 2.5" 10K RPM SAS disks (Nexenta Syspool Configuration) 4x 2port Dell (LSI) SAS HBA 4x 10Gbps Intel NIC's low profile
  • 24.
    Primary Storage –Hardware List JBODs – 2 Numbers PowerVault(TM) MD1220 Disk Storage Enclosure, 2U 24 Disk Slots SSD Disks 30 x 2TB SSD Drives
  • 25.
  • 26.
  • 27.
    Provisioning Primary Storage • Expose a Volume Using NFS/iSCSI as per the Cluster Requirement • Add the Respective Volume as a ‘Primary Storage’ from the CloudPlatform Management Portal • CloudPlatform Provisioning with NexentaStor is Automated Using Enterprise Chef™ Cookbooks, using the Knife API Plug-ins
  • 28.
    Secondary Storage Design • GlusterFS with One-to-One Replication • uCarp to Manage Floating IP • Hardware Specs • SuperMicro MBD-XS8DTE-F-O • 32GB Memory • 24x1TB SATA
  • 29.
  • 30.
    Current Workload •Cassandra Cluster • Client Facing Applications • Database Servers (PostgreSQL, MySQL, Redis etc.) • And More …
  • 31.
    What We Learnt? • Preparation • Compatibility Testing • Stable Network • Pay Attention on HA • Staging Cluster • Schedule Backup Solution • Contribute Back to the Community
  • 32.
    Steps Ahead •Migrate traditional data centers into Cloud • Evaluate and build a Hyper-V zone
  • 33.
  • 34.
    Thank You! mailto:iliyas.shirol@inmobi.com