1. Messaging Platform with
MongoDB Atlas
@eHarmony
Vijay Vangapandu
(Senior Principal Software Engineer)
Email: vijayvangapandu@eharmony.com
LinkedIn: http://bit.ly/vijay-ln
Elankumaran Srinivasan
(Senior Software Engineer)
Email: esrinivasan@eharmony.com
LinkedIn: http://bit.ly/elan-ln
June 20, 2017
2. • Our comprehensive & revealing
Relationship Questionnaire
• 29 Dimensions® of Compatibility
• 15 million matches per day globally
(10 billion since inception)
• 101 thousand photos uploaded per day
• 1 million communications per day
• 70 million registrants globally
Who is eHarmony?
7. Match Comm Drop Off (match level)
• Decline in Guided communication stages completion since Jan 2016
Users completing CEQ’s : -36.5%
Users completing MHCS : -36.0%
Users completing OEQ’s : -17.7%
Users reaching open comm: -36.8%
8. Goals of the Redesign
• Simplified communication flow
• No lockdown or dependent stages
• Real time messaging system - Chat
• Extend the system to support richer content types
• Giphy
• Video/Photos
• Stickers
• Support various custom business requirements
• Improved user experience
12. Limitations With Current Relational Store
• Antiquated
• Large number of tables
• Complex Joins
• Performance Concerns
• Rigid database model
13. Communication – Microservice architecture
Redis Oracle
XMPP
HTTPS
Aggregation
Service
Data API Service
SSL
Match ServiceUser Service
Auth
Service
API Gateway
NodeJS APP Stack
FIREWALL
???
14. Characteristics of eH Use Cases
• Data Consistency
• Read Heavy
• Scalable
• Flexible Schema
• High Performance
• Rich Query Support
15. Communication – Microservice architecture
Redis Oracle
XMPP
HTTPS
Aggregation
Service
Data API Service
SSL
Match ServiceUser Service
Auth
Service
API Gateway
NodeJS APP Stack
FIREWALL
MongoDB
16. Why MongoDB Atlas?
• NoSQL DB as a Service
• High Availability
• Low Management Overhead
• Monitoring & Alerting
• Automated Backups
• Disaster Recovery
• Workload Isolation
• Strong Security
• No need for on-site DBA
26. Query - Conversations
Conversations
User Match - 335567
Match - 112334
Match - 233456
Message 1
Message 2
Message N
Message N
Message N
Message 1
Message 1
....
........
Shard
Shard
Shard
31. Performance – Load Testing
eHarmony’s Data Center
Cloud Tests DAO as API Service
Mongo
Driver
MongoDB Atlas
• Network Latency with multiple hops
• Query Performance Impact
• Data Growth
• Sharding
32. Performance – Best Practices
• Indexing
– Too many or none.
– Compound Indexes
– Covered Queries
– Background Index creation.
• Leverage in place updates.
39. Alerting
MongoDB Atlas
• Replica set has no Primary
• Cluster missing active Mongos
• Number of connections
• Write Tickets
• Read Tickets
• Replication Lag
• Disk space
• Number of read
• Number of writes
43. Future Road Map
• Support other message types like videos/images/map location etc.
• Presentation ready feed.
• Monitor impact of additional shards on aggregate functions.
• Text Search.