- Ctrip uses Ceph for storage in their private cloud and VDesktop environments, storing over 10 petabytes of data across various use cases including databases, file sharing, and big data. - They implemented Ceph RGW with Swift API for object storage to enable continuous delivery of software packages across data centers with low latency access. - For data sync between data centers, Ctrip deployed a federated Ceph gateway solution instead of direct sync due to issues with stability, flexibility, and scalability of other approaches.

1. Ceph in Ctrip
Yong Luo / Jun Liu

2. • Status of Ctrip Cloud
• Ceph Scenarios：Continues Delivery
• Ceph RGW & COS
• Next Steps
• Q&A
Agenda

3. Ctrip Cloud Introduction
• Focus on IaaS / PaaS
• Base on OpenStack
• Icehouse
• Key Business Cases
• Ctrip Private Cloud
• Ctrip VDesktop

4. Ctrip Private Cloud
Shanghai Regions
• 3 Data Centers
• Production/IT/CC
• 8000+ VM
• 6000+ bare metals
Nantong Region
• QA/Dev Cloud
• UAT/FAT/LPT
• 100% on KVM
• 3000+ VM (fast growing)
• 400+ bare metals
3 Region ，Hypervisor 2000 + ，VM 12000+
Chengdu Region
• Onboard in 2016

5. Ctrip VDesktop
Agents: 15000+, VM:6000+ , Calls: 450k+/day, Peak: 850k+/day
Shanghai Call Center
• 1000+
Xingyang Call Center
• 400+
Hefei Call Center
• 500+
Nantong Call Center
• 3000+
Rugao Call Center
• 400+

6. Storage in Ctrip
Now
• Commercial
• SAN (HP/ HPS) 1+ PB, Database
• NAS (HW) 800+T, File Sharing
• Open Source
• GlusterFS 1+ PB, Database Backup
• FastDFS 1+ PB, 100000K Picture
• HDFS 10+ PB, Big Data
10X UP
Future
OTA Business

7. Why Ceph
http://www.mellanox.com/blog/2015/06/ceph-is-a-hot-storage-solution-but-why/

8. Why Ceph
http://www.mellanox.com/blog/2015/06/ceph-is-a-hot-storage-solution-but-why/

9. Scenarios : Continuous Delivery
• Requirements of Package Storage
• Massive package storage
• Package sync between IDCs with low latency
• Access cross platform
• Date access control
Package Repo
Package Repo
Package Repo
Package Repo
DEV FAT UAT Prod

10. Solution

11. RGW Profile
• Ceph Version: 0.94.2， H release
• Object Storage: RGW + Swift API
• SDK: Python/ Java/ C#/ Ruby
• OS: Centos 6.4
• Hardware：
• CPU(2 channels & 32 Core)、Mem 128GB、 disk（
12*3TB/SATA disk +2*256GB raid1 SSD）、
NIC(4*Gigabit LAN, bond 2 in 1 pair)

12. Deployment
RGW Deployment

13. RGW Architecture

14. Data Sync between IDCs
VS.

15. Federate Gateways
Drawbacks
•Unstable
•Inflexible
•Unexpandable

16. Our Solution

17. COS Architecture
Points of Focus
• Stable
• Flexible
• Expandable

18. COS Deployment

19. Next Steps
• Database on Ceph (Dev & QA Farm)
• Openstack / Docker Integrate with Ceph
• IT “Dropbox”

20. Q&A

21. We are hiring!
• Storage Development Engineer
• OpenStack Development Engineer
• Cloud DevOps Engineer
Yong Luo Jun Liu

22. Thanks