Presentation on the usage of NoSQL technologies at Gumtree.com Covers experiences of using Riak, Redis and MongoDB

1. NoSQL
@
(or Not Only SQL)
Andy Summers, Software Engineer

4. We are the UK’s
#1 classifieds website
and in the UK
with over
top 30 websites
7 million
visitors a month*
* Average unique visitors according to ComScore data in 2013

5. A place which connects people
to the things they need…

6. • 1.7 million live adverts
• 150 million adverts posted in all time
• 100,000 new adverts per day
• Approx 300,000 replies per day
Source: Gumtree data, November 2013

7. Gumtree Tech

8. Gumtree Tech
•
•
•
•
•
•
•
15 Back End devs
6 Front End devs
3 Mobile App devs
5 Site Ops
5 Product Managers
6 QA
3 Designers

9. Riak
Cassandra
Redis
Neo4j
mongoDB
CouchDB

10. •Replies
•Sessions
•A few other things

11. Replies

12. v.1
Mobile
Apps
Mail
Server
REST API
Anonymizer
My
SQL
Cass
andra

13. • OSGI 
• Two sources of truth?!

14. y?
Scalabilit
MySQL
MySQL

15. Cassandra
?

16. „We need something better‟

17. v.2
RIAK
Elastic
Search

18. „Just a Key-Value Store‟
• Real-time Querying?
• Reporting?
• My manager wants to know how many of x
happened in y?

19. “But all of this
works fine in
MySQL…”

20. Use the Right Tool
for the Right Job

21. The German Story

22. R
Using
iak for
eplyTS
18
15
2
Nodes
TB
Data
centers

23. And then we Launched.
And everything was fine!

24. Well, at least 3 Months…

27. New Technology
always
fails!

28. Have a
Backup
... and if you can‟t:
have a backup plan

29. Dear Basho Devs,

30. So where are Gumtree in all this?
• Keeping a close eye on eBay
Kleinanzeigen‟s experiences
• Acknowledging that staying with the
MySQL and Cassandra solution, is not
suitable for Gumtree‟s long term needs

31. Sessions

32. Improving support for regular
software releases
Interweb
Richard
Gumtree servers

33. Oh… That‟s not good.
• Off-peak release times only
• Sticky sessions have a lot to answer for
• Have to restart instance => Kill everyone on it

34. redis

35. Redis
• „Just another key-value store‟
• Data typing
• TTL support
• Support for master-slave, monitoring,
publish-subscribe

36. Interweb
Richard
Success!
Gumtree servers
Redis

37. Redis – all good?
• Single master = single point of failure
• Sentinel – potential 66 seconds of
downtime when master fails?
• Sharding is never easy
• Not tested dual data centre setup

38. The Other Stuff

39. mongoDB

40. • Popular Search Tracking
– High realtime write-traffic
– Regretting this a lot
• Some low traffic use cases
– Low traffic, low read
• Some batch event analysis
– Batched reads and writes

41. In Conclusion
• Data consistency vs. High availability
• NoSQL comes in many different guises,
technologies suited for different use cases
• No technology is perfect

42. If you enjoyed this talk
and like Gumtree
• eBay Tech Meetups across Europe
– http://www.meetup.com/eBay-Europe-Technology
• Gumtree Careers via LinkedIn
– http://linkd.in/1eu4rK5

43. Thank You!
…Questions or comments?