Narasimhan Sampath and Avinash Ramineni share how Choice Hotels International used Spark Streaming, Kafka, Spark, and Spark SQL to create an advanced analytics platform that enables business users to be self-reliant by accessing the data they need from a variety of sources to generate customer insights and property dashboards and enable data-driven decisions with minimal IT engagement. Narasimhan and Avinash highlight the architecture, lessons learned, and the challenges that were overcome on both the business and technology fronts. The analytics platform is designed as a framework to enable self-service data intake, data processing, and report/model generation by the business users. The data-driven framework consists of a distributed hybrid-cloud data ingestor for data intake and a Cloudera CDH cluster with Spark as the distributed compute engine. The solution is built in such a way that storage and compute have been decoupled and encourages the concept of BYOC (bring your own compute). The platform uses EC2 instances to run CDH and leverages Amazon S3 as a data warehouse storage layer (data lake), Spark as an ETL engine, and Spark SQL as a distributed query engine. Results (computations/derived tables) are exposed to the end users via Spark SQL and are discovered via Tableau. The platform supports both batch and streaming use cases and is built on the following technology stack: AWS (S3, EC2, SQS, SNS), Cloudera CDH (YARN, Navigator, Sentry), Spark, Kafka, Spark SQL, and Spark Streaming.

1. Strata Hadoop World | New York City | September 29th, 2016
Choice Hotels’ journey to better
understand its customers
through self-service analytics
Narasimhan Sampath & Avinash Ramineni

2. Agenda
• Who is Choice Hotels
• Platform Architecture
• Implementation
• Value Add
Strata Hadoop Word | New York City | September 29th, 2016Page 3

3. Who is Choice Hotels?
Page 4 Strata Hadoop Word | New York City | September 29th, 2016

4. Who is Choice Hotels?
United States & Caribbean
Hotels open 5,276
Hotels under development 606
Rooms open & under dev. 446,813
Canada
Hotels open 323
Hotels under development 45
Rooms open & under dev. 30,135
South America
Hotels open 64
Hotels under development 7
Rooms open & under dev. 9,737
Asia Pacific
Hotels open 315
Hotels under development 25
Rooms open & under dev. 23,289
Europe
Hotels open 402
Hotels under development 31
Rooms open & under dev. 50,388
Mexico
Hotels open 28
Hotels under development 4
Rooms open & under dev. 3,219
Central America
Hotels open 14
Hotels under development 0
Rooms open & under dev. 1,468
Middle East
Hotels open 1
Hotels under development 2
Rooms open & under dev. 564

5. How About a Technology Company?
Strata Hadoop World | New York City | September 29th, 2016Page 6

6. Evolution of Guest Experience
Page 7 Strata Hadoop Word | New York City | September 29th, 2016

7. Project Goals
Page 8 Strata Hadoop Word | New York City | September 29th, 2016
• Business Drivers
− Self Service Reporting and Analytics
− Requirements for near real-time analytics
− Simplify Governance, Compliance and Auditing
− Better support for new applications
• Technical Drivers
− Unable to handle volume, velocity, and veracity
− Retire Legacy Systems
− Difficult to find skillset (Informix 4GL)
− Simplify Technology Stack

8. Key Design Tenets
• Separation of Compute and Storage
• Independently scale compute and storage
• Data Democratization and Governance
• Bring your own Compute (BYOC)
• Lift and Shift between cloud provider(s)
and On-premise
• HA / DR
• Open Source Stack
Page 9

9. Separation of Compute and Storage
• Scale storage and compute independently (up or down)
• Shifts bottleneck from Disk IO to Network
• Centralized Data Storage
• Write once & read everywhere
• Data Democratization
• Easier Hardware upgrade paths
• Flexibile Architecture
Page 10
Storage
Servers

10. BYOC (Bring Your Own Cluster)
• Eliminates the need for very large clusters
• Easier to administer and maintain
• Reduces multi-tenancy issues
• Clusters can be upgraded independently
• Enables on-demand computing
• Lower costs
Page 11
Marketing
Cluster
Centralized
Storage
Personalization
Cluster
Main
Cluster

11. Platform Architecture
Page 12

12. Platform Architecture – Data Ingestion Layer
• DB Ingestor
• Stream Ingestor
− Kafka and Spark Streaming
• File Ingestor
• FTP / SFTP / Logs
• Ingestion using Service API
Page 13

13. Platform Architecture – Data Processing Layer
• Storage layer carved into logical buckets
• Landing, Raw, Derived and Delivery
• Schema stored with data (no guesswork)
• Platform Jobs for
• Converting text to Parquet
• Saving streaming data Parquet
• Derivatives
• Compaction
• Standardization
Page 14

14. Platform Architecture – Data Delivery Layer
• Data Delivery
• SQL - Spark Thrift Server / Impala
• Tableau, SQL IDE, Applications
• SparkR
• Self Service
• Derivatives
• Represented Via SQL on Delivery Layer
• Stored in Derived Storage Layer
• Metadata driven
• Derived Layer Generators
• Long running Spark Job
• Derivative Refresh
Page 15

15. Implementation
• CDH Cloud ready-ness
• Cloudera Director Limitations
• Multi-Availability zone, regions
• Spark Thrift Server
• Support
• Performance Tuning
• Concurrency, partition strategy
• Cache Tables
• Security
• Sentry Integration
• Kerberos Ticket Renewal
• Navigator Integration
Page 16

16. Implementation
• Rapidly Changing Technology
• Feature addition
• Documentation
• Bugs
• Jar hell
• Compression Codec for Parquet
• S3 Eventual Consistency
• Small files
• Performance Issues
• Compaction
Page 17

17. Implementation
• Partition Strategy
• Parquet Files
• Balancing parallelism and throughput
• Table Partitions
• Cluster sizing, optimization and tuning
• Integrating with Corporate infrastructure
• Deployment practices
• Monitoring and Alerting
• Information Security Policies
Page 18

18. Value Add
Enabling predictive analytics and real-time decisions
Integrated Scorecards – Daily /Weekly / Monthly Insights Near Real Time / Hourly / Daily Insights
Multivariate Testing, APT (Test vs. Control Analysis), and Text
Analytics
Testing for Both Hotel and Customer / Research For Guest
Insights
Personalized Display Ad Serving
Real-time Actions (Machine Learning) Across Guest Touch
Points
Hotel Lifecycle Data Real-time Alerts for Hotel Related Actions
Strata Hadoop World | New York City | September 29th, 2016Page 19

19. • One of the fastest growing big data companies
• Extensive experience in providing strategic and architectural consulting on Big
Data platforms and implementations
• Global delivery experience across multiple locations in US, Asia and Latin
America
• 100+ big data experts worldwide - US, Latin America and Asia
B A C K G R O U N D
C L A I R V O Y A N T S O F T . C O M
CLAIRVOYANT
A W A R D S & R E C O G N I T I O N

20. Questions
Strata Hadoop Word | New York City | September 29th, 2016Page 21
Principal @ Clairvoyant
Email: avinash@clairvoyantsoft.com
LinkedIn: https://www.linkedin.com/in/avinashramineni