Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Cisco: Cassandra adoption on Cisco UCS & OpenStack

2,758 views

Published on

n this talk we will address how we developed our Cassandra environments utilizing Cisco UCS Open Stack Platform with the DataStax Enterprise Edition software. In addition we are utilizing OpenSource CEPH storage in our Infrastructure to optimize the Performance and reduce the costs.

Published in: Technology
  • Be the first to comment

Cisco: Cassandra adoption on Cisco UCS & OpenStack

  1. 1. 1 TOMORROW starts here
  2. 2. Cassandra Adoption on Cisco UCS & OpenStack Nayeem Khaja, Program Manager, Cisco IT Bidhu Das, Database Domain Architect, Cisco IT
  3. 3. 3 •  About Cisco •  Why Cassandra at Cisco •  Use Cases •  Architecture & Implementation •  Key Callout •  Q & A Agenda
  4. 4. 4 About Cisco locations in countries offices employees 2000+ Applications 1500+ Databases (Prod & Non-Prod) HANA, Legacy EDW, Hadoop Supporting Mission Critical Environments 32 data centers and server rooms of data center space of UPS power to raised floors servers virtualized in new DCs, overall Virtualization goal =
  5. 5. 5 Cisco Products http://www.cisco.com/c/en/us/products/index.html
  6. 6. 6 Cisco UCS Models
  7. 7. 7 Why Cassandra at Cisco •  Active ~ Active i.e. distributed Architecture •  Linear scalability •  High availability with zero downtime •  Better throughput with Multi-DC Architecture •  Align with Industry Cloud Native Application strategy
  8. 8. 8 Use Cases Cloud and Operations! v  Cisco Commerce" v  Cisco Collaboration" v  Quote & Validate system" v  AutoTest scanning for Cisco Services Group" Mobility! v  Cisco Network Management" Security & Fraud 
 Detection! v  Cisco Security" v  Cisco Finance Fraud Analytics" Personalization and Recommendation! v  Cisco Video" IoT ! v  Cisco Supply Chain" v  Cisco Project Polaris" v  Cisco Webex" v  Cisco Voicemail Player" Cassandra
  9. 9. 9 Cisco Commerce Renewals Cloud Clients IaaS StorageCompute Networking Platform Cassandra (Database) Applications Notifications Pricing Search Quoting Data Loader Validation (drools) Cisco UCE Browser App AndroidPartner app IOS Upload Conversion Nginx (Web Server) Platform,build,TestAutomation (Puppet,Nagios,Jenkins Ordering Web Tomcat (Java appServer) Elastic Search (Search Engine, Log Mining) Rabbit MQ (Messaging) HAProxy (Load Balancer) Memcached (In Memory Cache) Logstash (Log Forwarder) Kibana (Log Visualizer) Quartz (Scheduler)
  10. 10. 10 Commerce Analytics & Reporting Transactional (Oracle)
  11. 11. 11 eStore : Database Provisioning Tool DB Category RDBMS BigData - Hadoop NoSQL RDBMS Capability Open Source High Transactional Immediate Consistency Cost & Support Vendor Supported – P1 Apps Community Support < P1 Apps Maria DB Multi-Master replication Large Database Support In-Memory Capability Better Security NoSQL Capability Open Source Distributed across Geo-location Active ~ Active Multi-Master replication Columnar Search Key-Value pair – In-Memory, small read/ small write, large objects Document Oriented Very high Data volume Graph Database – for complex, highly connected hierarchical data Schema less Architecture BigData Capability Huge Dataset with structures as well as unstructured Highly Scalable Master – Slave Architecture Batch Oriented
  12. 12. 12 Architecture & Implementation
  13. 13. 13 What is OpenStack ? OpenStack is an open source platform consisting of set of software tools , used for building and managing the cloud computing platform for both public & private clouds.
  14. 14. 14 OpenStack Components libvirt ovs KVM Haproxy (Load balancer) Swift (Object Storage) Glance (Imaging) Cinder (block storage) Nova (Compute) Neutron (Networking) Heat (Orchrastration) Cellometer (Telemetry) Keystone (Identity)
  15. 15. 15 OpenStack on Cisco UCS Red Hat OpenStack Infrastructure RHEL RHEL RHEL …… APP OS APP OS APP OS APP OS APP OS APP OS APP OS … ….. RHEL RHEL Red Hat OpenStack Infrastructure Ceph Cluster …. FI -A FI - B FEX -A FEX -B Compute Nodes Storage Nodes APP OS
  16. 16. 16 Cassandra on OpenStack Architecture Automati on Packs Process Orchestrator Prime Service Catalog Users c Openstack Mangement Nodes Network Fabric UCS B Series UCS C Series Storage Cluster Compute Cluster Nexus 5000 Nova Compute Nodes RADOS GW (Swift) OSD / MON OpenStack Infrastructure Cassandra Cluster Config : •  8 CPU & 64 GB Memory per Node •  Ceph storage •  OS : RHEL 6.4 •  Apache Cassandra 2.1 Replication Factor = 3 Consistency Level = Local Quorum Replication Strategy = NetworkTopologyStrategy Node 2 Node 3 Node 4 Node 5Node 6 Node 9 Node 8 Node 7 R A C K 3 R A C K 2 Co-ordinator Node Node 1 Client Driver
  17. 17. 17 Cassandra & Spark on Physical host & SSD 4 1 6 5 2 3 (Transactional Physical/SSD) 3 1 2 (ETL /Spark Physical/SSD) •  C220 M4 Servers •  256 GB Memory each •  8 SSD Drives 960GB each •  RHEL 6.5 OS 64bit •  JBOD Configuration •  Datastax 4.7
  18. 18. 18 Ephemeral storage is a storage solution where the storage is directly attached to compute (nova) layer i.e. local to the hypervisor. The data is not persistent on the disk which means the data is lost in case of terminating the VM, but not in case of VM shutdown. Hypervisor 1 Hypervisor 2 Hypervisor 3 Hypervisor 4 Ceph Hypervisor 1 Hypervisor 2 Hypervisor 3 Hypervisor 4 Hypervisor with local Storage (dedicated per hypervisor) Hypervisor with Ceph Storage (shared)
  19. 19. 19 Redhat Openstack Redhat Openstack Ephemeral solution for Cassandra C-Series Host C-Series Host C-Series Host Redhat Openstack Cass andra OS 4 35 6 1 2 Dedicated Cluster at DB level for each applications, but Multi- Tenant at OpenStack level with common storage pool per Hypervisor. Cluster 1 4 35 6 1 2 Cluster 2 Nova Nova Nova This configuration provides consistent/ better I/O throughput as compared to OpenStack w/Ceph (shared) storage. … … … Cass andra OS Cass andra OS Cass andra OS Cass andra OS Cass andra OS Cass andra OS Cass andra OS Cass andra OS…. Storage (Locally attached or External (connected thru FC)) Storage (Locally attached or External (connected thru FC)) Storage (Locally attached or External (connected thru FC))
  20. 20. 20 Platform Migration & Upgrade with zero downtime VTL/Data Domain Snapshot Snapshot 8 days retention 2 weeks retention . . . . . . Snapshot Snapshot Snapshot 30 days retention 2 weeks retention . . . . . . . . . Backup Type Frequency Retention Comments Cluster VTL Metadata Daily 2 weeks 30 days Full Daily 2 weeks 30 days Incremental 6 hrs 1 week 30 days Only in case of high critical applications as an exception. 5 7 1 6 2 3 9 4 8 5 1 9 2 3 4DC2- PROD Physical/SSD 7 8 6 DC1- PROD Openstack Shared Ceph Storage
  21. 21. 21 Monitoring Cassandra : •  Compaction status •  Nodetool tpstats for pending request or dropped mutations •  Nodetool cfstats/ cfhistograms for latency distribution •  Recent restarts, Dead Node •  Node Health check wrt Gossip, Thrift & Native transport •  Event driven / metrics driven Alerts System : •  CPU , Memory, IO status •  Load average
  22. 22. 22 Centralized Operations Dashboard 123.45.678.001 123.45.678.002 123.45.678.003 123.45.678.004 123.45.678.005 123.45.678.006 123.45.678.007 123.45.678.001 123.45.678.002 123.45.678.003 123.45.678.004 123.45.678.005 123.45.678.006 123.45.678.007 123.45.678.008 123.45.678.009 CCW Cassandra cass-prd-08:8900 cass-prd-09:8900 cass-prd-08.cisco.com cass-prd-09.cisco.com 8 9 123.45.678.008 123.45.678.009 P3 P3 P3 P3 P3 P3 P3 P3 P3 P2 P2 P2 P2 P2 P2 P2 P2 P2 CCW Cassandra CCW Cassandra CCW Cassandra CCW Cassandra CCW Cassandra CCW Cassandra CCW Cassandra CCW Cassandra CCW Cassandra 1 2 3 4 5 6 7 8 9
  23. 23. 23 Integrated Real time Database Dashboard xyz-prd-03.cisco.com xyz-prd-03.cisco.com xyz-prd-03 xyz-prd-03 abc-prd-01 cass-prd-07 cass-prd-07 123.45.678.007 cass-prd-07.cisco.com abc-prd-01.cisco.com XYZPRD.CISCO.COM_XYZPRD1 LISTENER_XYZPRD1_xyz-prd-03 ABCPRD
  24. 24. 24 Monitoring using DataStax Opscenter
  25. 25. 25 •  Do not use Nodetool command with more concurrency. •  Enable Incremental backup, only if required. •  Snapshot retention policy. •  Use Nodetool repair thru OpsCenter •  Compaction Strategy (Size tiered ~ Level Tiered) •  Disable replication at storage layer Lesson Learn & Key Call out
  26. 26. 26
  27. 27. 27 Thank you

×