Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
NoSQL

@
(or Not Only SQL)
Andy Summers, Software Engineer
We are the UK’s

#1 classifieds website

and in the UK

with over

top 30 websites

7 million

visitors a month*

* Averag...
A place which connects people
to the things they need…
• 1.7 million live adverts
• 150 million adverts posted in all time
• 100,000 new adverts per day
• Approx 300,000 replies...
Gumtree Tech
Gumtree Tech
•
•
•
•
•
•
•

15 Back End devs
6 Front End devs
3 Mobile App devs
5 Site Ops
5 Product Managers
6 QA
3 Desig...
Riak

Cassandra

Redis
Neo4j

mongoDB

CouchDB
•Replies
•Sessions
•A few other things
Replies
v.1
Mobile
Apps

Mail
Server

REST API
Anonymizer

My
SQL

Cass
andra
• OSGI 
• Two sources of truth?!
y?

Scalabilit

MySQL

MySQL
Cassandra

?
„We need something better‟
v.2

RIAK

Elastic
Search
„Just a Key-Value Store‟
• Real-time Querying?

• Reporting?
• My manager wants to know how many of x
happened in y?
“But all of this
works fine in

MySQL…”
Use the Right Tool
for the Right Job
The German Story
R

Using

iak for
eplyTS

18

15

2

Nodes

TB

Data
centers
And then we Launched.

And everything was fine!
Well, at least 3 Months…
New Technology

always
fails!
Have a

Backup
... and if you can‟t:
have a backup plan
Dear Basho Devs,
So where are Gumtree in all this?
• Keeping a close eye on eBay
Kleinanzeigen‟s experiences
• Acknowledging that staying w...
Sessions
Improving support for regular
software releases

Interweb

Richard

Gumtree servers
Oh… That‟s not good.
• Off-peak release times only

• Sticky sessions have a lot to answer for

• Have to restart instance...
redis
Redis
• „Just another key-value store‟

• Data typing
• TTL support
• Support for master-slave, monitoring,
publish-subscr...
Interweb

Richard

Success!
Gumtree servers

Redis
Redis – all good?
• Single master = single point of failure
• Sentinel – potential 66 seconds of
downtime when master fail...
The Other Stuff
mongoDB
• Popular Search Tracking
– High realtime write-traffic
– Regretting this a lot

• Some low traffic use cases
– Low traffi...
In Conclusion
• Data consistency vs. High availability
• NoSQL comes in many different guises,
technologies suited for dif...
If you enjoyed this talk
and like Gumtree
• eBay Tech Meetups across Europe
– http://www.meetup.com/eBay-Europe-Technology...
Thank You!

…Questions or comments?
NoSQL at Gumtree
NoSQL at Gumtree
NoSQL at Gumtree
NoSQL at Gumtree
Upcoming SlideShare
Loading in …5
×

NoSQL at Gumtree

790 views

Published on

Presentation on the usage of NoSQL technologies at Gumtree.com
Covers experiences of using Riak, Redis and MongoDB

Published in: Technology, Business
  • Be the first to comment

  • Be the first to like this

NoSQL at Gumtree

  1. 1. NoSQL @ (or Not Only SQL) Andy Summers, Software Engineer
  2. 2. We are the UK’s #1 classifieds website and in the UK with over top 30 websites 7 million visitors a month* * Average unique visitors according to ComScore data in 2013
  3. 3. A place which connects people to the things they need…
  4. 4. • 1.7 million live adverts • 150 million adverts posted in all time • 100,000 new adverts per day • Approx 300,000 replies per day Source: Gumtree data, November 2013
  5. 5. Gumtree Tech
  6. 6. Gumtree Tech • • • • • • • 15 Back End devs 6 Front End devs 3 Mobile App devs 5 Site Ops 5 Product Managers 6 QA 3 Designers
  7. 7. Riak Cassandra Redis Neo4j mongoDB CouchDB
  8. 8. •Replies •Sessions •A few other things
  9. 9. Replies
  10. 10. v.1 Mobile Apps Mail Server REST API Anonymizer My SQL Cass andra
  11. 11. • OSGI  • Two sources of truth?!
  12. 12. y? Scalabilit MySQL MySQL
  13. 13. Cassandra ?
  14. 14. „We need something better‟
  15. 15. v.2 RIAK Elastic Search
  16. 16. „Just a Key-Value Store‟ • Real-time Querying? • Reporting? • My manager wants to know how many of x happened in y?
  17. 17. “But all of this works fine in MySQL…”
  18. 18. Use the Right Tool for the Right Job
  19. 19. The German Story
  20. 20. R Using iak for eplyTS 18 15 2 Nodes TB Data centers
  21. 21. And then we Launched. And everything was fine!
  22. 22. Well, at least 3 Months…
  23. 23. New Technology always fails!
  24. 24. Have a Backup ... and if you can‟t: have a backup plan
  25. 25. Dear Basho Devs,
  26. 26. So where are Gumtree in all this? • Keeping a close eye on eBay Kleinanzeigen‟s experiences • Acknowledging that staying with the MySQL and Cassandra solution, is not suitable for Gumtree‟s long term needs
  27. 27. Sessions
  28. 28. Improving support for regular software releases Interweb Richard Gumtree servers
  29. 29. Oh… That‟s not good. • Off-peak release times only • Sticky sessions have a lot to answer for • Have to restart instance => Kill everyone on it
  30. 30. redis
  31. 31. Redis • „Just another key-value store‟ • Data typing • TTL support • Support for master-slave, monitoring, publish-subscribe
  32. 32. Interweb Richard Success! Gumtree servers Redis
  33. 33. Redis – all good? • Single master = single point of failure • Sentinel – potential 66 seconds of downtime when master fails? • Sharding is never easy • Not tested dual data centre setup
  34. 34. The Other Stuff
  35. 35. mongoDB
  36. 36. • Popular Search Tracking – High realtime write-traffic – Regretting this a lot • Some low traffic use cases – Low traffic, low read • Some batch event analysis – Batched reads and writes
  37. 37. In Conclusion • Data consistency vs. High availability • NoSQL comes in many different guises, technologies suited for different use cases • No technology is perfect
  38. 38. If you enjoyed this talk and like Gumtree • eBay Tech Meetups across Europe – http://www.meetup.com/eBay-Europe-Technology • Gumtree Careers via LinkedIn – http://linkd.in/1eu4rK5
  39. 39. Thank You! …Questions or comments?

×