NoSQL

@
(or Not Only SQL)
Andy Summers, Software Engineer
We are the UK’s

#1 classifieds website

and in the UK

with over

top 30 websites

7 million

visitors a month*

* Averag...
A place which connects people
to the things they need…
• 1.7 million live adverts
• 150 million adverts posted in all time
• 100,000 new adverts per day
• Approx 300,000 replies...
Gumtree Tech
Gumtree Tech
•
•
•
•
•
•
•

15 Back End devs
6 Front End devs
3 Mobile App devs
5 Site Ops
5 Product Managers
6 QA
3 Desig...
Riak

Cassandra

Redis
Neo4j

mongoDB

CouchDB
•Replies
•Sessions
•A few other things
Replies
v.1
Mobile
Apps

Mail
Server

REST API
Anonymizer

My
SQL

Cass
andra
• OSGI 
• Two sources of truth?!
y?

Scalabilit

MySQL

MySQL
Cassandra

?
„We need something better‟
v.2

RIAK

Elastic
Search
„Just a Key-Value Store‟
• Real-time Querying?

• Reporting?
• My manager wants to know how many of x
happened in y?
“But all of this
works fine in

MySQL…”
Use the Right Tool
for the Right Job
The German Story
R

Using

iak for
eplyTS

18

15

2

Nodes

TB

Data
centers
And then we Launched.

And everything was fine!
Well, at least 3 Months…
New Technology

always
fails!
Have a

Backup
... and if you can‟t:
have a backup plan
Dear Basho Devs,
So where are Gumtree in all this?
• Keeping a close eye on eBay
Kleinanzeigen‟s experiences
• Acknowledging that staying w...
Sessions
Improving support for regular
software releases

Interweb

Richard

Gumtree servers
Oh… That‟s not good.
• Off-peak release times only

• Sticky sessions have a lot to answer for

• Have to restart instance...
redis
Redis
• „Just another key-value store‟

• Data typing
• TTL support
• Support for master-slave, monitoring,
publish-subscr...
Interweb

Richard

Success!
Gumtree servers

Redis
Redis – all good?
• Single master = single point of failure
• Sentinel – potential 66 seconds of
downtime when master fail...
The Other Stuff
mongoDB
• Popular Search Tracking
– High realtime write-traffic
– Regretting this a lot

• Some low traffic use cases
– Low traffi...
In Conclusion
• Data consistency vs. High availability
• NoSQL comes in many different guises,
technologies suited for dif...
If you enjoyed this talk
and like Gumtree
• eBay Tech Meetups across Europe
– http://www.meetup.com/eBay-Europe-Technology...
Thank You!

…Questions or comments?
NoSQL at Gumtree
NoSQL at Gumtree
NoSQL at Gumtree
NoSQL at Gumtree
Upcoming SlideShare
Loading in …5
×

NoSQL at Gumtree

716 views

Published on

Presentation on the usage of NoSQL technologies at Gumtree.com
Covers experiences of using Riak, Redis and MongoDB

Published in: Technology, Business
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
716
On SlideShare
0
From Embeds
0
Number of Embeds
8
Actions
Shares
0
Downloads
13
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • Gumtree is under the umbrella of eBay Classifieds, part of eBay Inc who own many other big names
  • eBay Classifieds encapsulates many classifieds sites across many different countries
  • Data Source: Comscore
  • #1 Classifieds site in UKwww site, m. site, Android app, iPhone appJVM-based technology stackLean startup approach to development and releases
  • Founded in 2000eBay bought in 2005
  • What do we have to do with NoSQL?
  • Version 1 Problems:MultitenanncyOSGITwostoragebackendsNot really happy withcassandraMySQL not easily horizontal scalableTwodatastorages
  • A black box where nobody quite knows what goes on inside
  • NoSQL at Gumtree

    1. 1. NoSQL @ (or Not Only SQL) Andy Summers, Software Engineer
    2. 2. We are the UK’s #1 classifieds website and in the UK with over top 30 websites 7 million visitors a month* * Average unique visitors according to ComScore data in 2013
    3. 3. A place which connects people to the things they need…
    4. 4. • 1.7 million live adverts • 150 million adverts posted in all time • 100,000 new adverts per day • Approx 300,000 replies per day Source: Gumtree data, November 2013
    5. 5. Gumtree Tech
    6. 6. Gumtree Tech • • • • • • • 15 Back End devs 6 Front End devs 3 Mobile App devs 5 Site Ops 5 Product Managers 6 QA 3 Designers
    7. 7. Riak Cassandra Redis Neo4j mongoDB CouchDB
    8. 8. •Replies •Sessions •A few other things
    9. 9. Replies
    10. 10. v.1 Mobile Apps Mail Server REST API Anonymizer My SQL Cass andra
    11. 11. • OSGI  • Two sources of truth?!
    12. 12. y? Scalabilit MySQL MySQL
    13. 13. Cassandra ?
    14. 14. „We need something better‟
    15. 15. v.2 RIAK Elastic Search
    16. 16. „Just a Key-Value Store‟ • Real-time Querying? • Reporting? • My manager wants to know how many of x happened in y?
    17. 17. “But all of this works fine in MySQL…”
    18. 18. Use the Right Tool for the Right Job
    19. 19. The German Story
    20. 20. R Using iak for eplyTS 18 15 2 Nodes TB Data centers
    21. 21. And then we Launched. And everything was fine!
    22. 22. Well, at least 3 Months…
    23. 23. New Technology always fails!
    24. 24. Have a Backup ... and if you can‟t: have a backup plan
    25. 25. Dear Basho Devs,
    26. 26. So where are Gumtree in all this? • Keeping a close eye on eBay Kleinanzeigen‟s experiences • Acknowledging that staying with the MySQL and Cassandra solution, is not suitable for Gumtree‟s long term needs
    27. 27. Sessions
    28. 28. Improving support for regular software releases Interweb Richard Gumtree servers
    29. 29. Oh… That‟s not good. • Off-peak release times only • Sticky sessions have a lot to answer for • Have to restart instance => Kill everyone on it
    30. 30. redis
    31. 31. Redis • „Just another key-value store‟ • Data typing • TTL support • Support for master-slave, monitoring, publish-subscribe
    32. 32. Interweb Richard Success! Gumtree servers Redis
    33. 33. Redis – all good? • Single master = single point of failure • Sentinel – potential 66 seconds of downtime when master fails? • Sharding is never easy • Not tested dual data centre setup
    34. 34. The Other Stuff
    35. 35. mongoDB
    36. 36. • Popular Search Tracking – High realtime write-traffic – Regretting this a lot • Some low traffic use cases – Low traffic, low read • Some batch event analysis – Batched reads and writes
    37. 37. In Conclusion • Data consistency vs. High availability • NoSQL comes in many different guises, technologies suited for different use cases • No technology is perfect
    38. 38. If you enjoyed this talk and like Gumtree • eBay Tech Meetups across Europe – http://www.meetup.com/eBay-Europe-Technology • Gumtree Careers via LinkedIn – http://linkd.in/1eu4rK5
    39. 39. Thank You! …Questions or comments?

    ×