JonMoore-NoSQLEU-2010

Why Big Enterprises are Interested in NoSQL Jon Moore (@jon_moore) Chief Engineer

Who is Comcast?

Who am I? http://www.flickr.com/photos/mihaibojin/ / CC BY 2.0

Who am I? Board room http://www.flickr.com/photos/mihaibojin/ / CC BY 2.0

Who am I? Board room My office http://www.flickr.com/photos/mihaibojin/ / CC BY 2.0

Who am I? Board room My office …is on the back side http://www.flickr.com/photos/mihaibojin/ / CC BY 2.0

Who am I? } Board room ~40 floors My office …is on the back side http://www.flickr.com/photos/mihaibojin/ / CC BY 2.0

Who is Comcast Interactive Media (CIM)? • Online and cross-platform entertainment and media businesses

Who is Comcast Interactive Media (CIM)? • Online and cross-platform entertainment and media businesses • Top 200 traffic sites (worldwide); Top 50 in the USA • Almost 4 billion pageviews per month

Why are we interested in NoSQL?

aren’t Why are we interested in NoSQL?

aren’t Why are we interested in NoSQL? • Peak requests: ~10,000/sec

aren’t Why are we interested in NoSQL? • Peak requests: ~10,000/sec – 10.000/sec when measured in the EU

aren’t Why are we interested in NoSQL? • Peak requests: ~10,000/sec – 10.000/sec when measured in the EU • Peak ORIGIN requests: 700/sec

aren’t Why are we interested in NoSQL? • Peak requests: ~10,000/sec – 10.000/sec when measured in the EU • Peak ORIGIN requests: 700/sec – (Yay, HTTP caching!)

aren’t Why are we interested in NoSQL? • Peak requests: ~10,000/sec – 10.000/sec when measured in the EU • Peak ORIGIN requests: 700/sec – (Yay, HTTP caching!) • It is NOT difficult to achieve this scale, even with RDBMS technologies

aren’t Why are we interested in NoSQL? • Peak requests: ~10,000/sec – 10.000/sec when measured in the EU • Peak ORIGIN requests: 700/sec – (Yay, HTTP caching!) • It is NOT difficult to achieve this scale, even with RDBMS technologies • We don’t need NoSQL for massive scale

aren’t Why are we interested in NoSQL? • Performance targets

aren’t Why are we interested in NoSQL? • Performance targets – 500ms origin service time at the 99th percentile

aren’t Why are we interested in NoSQL? • Performance targets – 500ms origin service time at the 99th percentile – Plenty of time if the data is structured well

aren’t Why are we interested in NoSQL? • Performance targets – 500ms origin service time at the 99th percentile – Plenty of time if the data is structured well – We don’t need NoSQL for its high performance

aren’t Why are we interested in NoSQL? • Performance targets – 500ms origin service time at the 99th percentile – Plenty of time if the data is structured well – We don’t need NoSQL for its high performance • Large data sets?

aren’t Why are we interested in NoSQL? • Performance targets – 500ms origin service time at the 99th percentile – Plenty of time if the data is structured well – We don’t need NoSQL for its high performance • Large data sets? – Some of our largest barely break 350 GB

aren’t Why are we interested in NoSQL? • Performance targets – 500ms origin service time at the 99th percentile – Plenty of time if the data is structured well – We don’t need NoSQL for its high performance • Large data sets? – Some of our largest barely break 350 GB – We don’t need NoSQL to handle “big data”

aren’t Why are we interested in NoSQL? • Storage is a means to a business end

aren’t Why are we interested in NoSQL? • Storage is a means to a business end – Relational DBs are mature, stable technologies

aren’t Why are we interested in NoSQL? • Storage is a means to a business end – Relational DBs are mature, stable technologies – NoSQL, being newer, carries some intrinsic risks

aren’t Why are we interested in NoSQL? • Storage is a means to a business end – Relational DBs are mature, stable technologies – NoSQL, being newer, carries some intrinsic risks • Availability of skilled labor and toolsets

aren’t Why are we interested in NoSQL? • Storage is a means to a business end – Relational DBs are mature, stable technologies – NoSQL, being newer, carries some intrinsic risks • Availability of skilled labor and toolsets – We can hire experienced DBAs

aren’t Why are we interested in NoSQL? • Storage is a means to a business end – Relational DBs are mature, stable technologies – NoSQL, being newer, carries some intrinsic risks • Availability of skilled labor and toolsets – We can hire experienced DBAs – Presence of standards (SQL, ODBC, JDBC)

aren’t Why are we interested in NoSQL? • Storage is a means to a business end – Relational DBs are mature, stable technologies – NoSQL, being newer, carries some intrinsic risks • Availability of skilled labor and toolsets – We can hire experienced DBAs – Presence of standards (SQL, ODBC, JDBC) – NoSQL requires more ramp-up and investment

aren’t Why are we interested in NoSQL? • Storage is a means to a business end – Relational DBs are mature, stable technologies – NoSQL, being newer, carries some intrinsic risks • Availability of skilled labor and toolsets – We can hire experienced DBAs – Presence of standards (SQL, ODBC, JDBC) – NoSQL requires more ramp-up and investment • And yet, we ARE interested in NoSQL

State is the hard part

State is the hard part • We have to copy it around

State is the hard part • We have to copy it around – For high availability

State is the hard part • We have to copy it around – For high availability – For scale (includes caching)

State is the hard part • We have to copy it around – For high availability – For scale (includes caching) • Copying state is not a problem

State is the hard part • We have to copy it around – For high availability – For scale (includes caching) • Copying state is not a problem – A physical necessity

State is the hard part • We have to copy it around – For high availability – For scale (includes caching) • Copying state is not a problem – A physical necessity • How we think about distributed state is the problem

And now for something completely different…

And now for something completely different… A Brief History Of Computing Models

In the beginning… one processor, one thread of control, one place for the state *Excerpted from http://en.wikipedia.org/wiki/BASIC CC-BY-SA: http://creativecommons.org/licenses/by-sa/3.0/

Multiprogramming: the illusion of concurrency Still single-threaded execution Implicit interaction via sharing system resources

True physical concurrency

True physical concurrency Networking (c. 1940!) (map of internet)

True physical concurrency Networking (c. 1940!) (map of internet) SMP (c. 1960!)

True physical concurrency Commoditization Late ‘90s : Networking networking (c. 1940!) Early ‘00s: both! (map of internet) SMP (c. 1960!)

Modern physical reality of data

Modern physical reality of data RAID striped across multiple disks

Modern physical reality of data cached in multiple cores CP CP CP CP RAID striped across multiple disks

Modern physical reality of data …on multiple servers

Modern physical reality of data …in multiple application tiers …on multiple servers

Modern physical reality of data …in multiple data centers Internet …in multiple application tiers …on multiple servers

Modern physical reality of data …in multiple data centers ALL Internet AT …in multiple application tiers ONCE …on multiple servers

CAP Theorem: The Iron Triangle Partition Tolerance Consistency Availability 16

Volcano CAP Theorem: The Iron Triangle Partition Tolerance Consistency Availability 16

This is your data on ACID (or with the “CA” of “CAP”) Internet

Your application *is* CAP-Theorem compliant

Your application *is* CAP-Theorem compliant • With multiple datacenters, partitions are common

Your application *is* CAP-Theorem compliant • With multiple datacenters, partitions are common – BGP route ﬂaps

Your application *is* CAP-Theorem compliant • With multiple datacenters, partitions are common – BGP route ﬂaps – Backhoes!

Your application *is* CAP-Theorem compliant • With multiple datacenters, partitions are common – BGP route ﬂaps – Backhoes! – internal vs. external partitions

Your application *is* CAP-Theorem compliant • With multiple datacenters, partitions are common – BGP route ﬂaps – Backhoes! – internal vs. external partitions • Thus, most consumer-facing Internet apps must be partition-tolerant

Your application *is* CAP-Theorem compliant • With multiple datacenters, partitions are common – BGP route ﬂaps – Backhoes! – internal vs. external partitions • Thus, most consumer-facing Internet apps must be partition-tolerant – Choose CP or AP

Your application *is* CAP-Theorem compliant • With multiple datacenters, partitions are common – BGP route ﬂaps – Backhoes! – internal vs. external partitions • Thus, most consumer-facing Internet apps must be partition-tolerant – Choose CP or AP • CA across data centers is a leaky abstraction

Your application *is* CAP-Theorem compliant • With multiple datacenters, partitions are common – BGP route ﬂaps – Backhoes! – internal vs. external partitions • Thus, most consumer-facing Internet apps must be partition-tolerant – Choose CP or AP • CA across data centers is a leaky abstraction – CAP theorem says so

NoSQL is about a choice • Keep pretending your reality is simple, OR • See your data as it truly is.

Selection Criteria

Exposure of distributed reality

Exposure of distributed reality • Read repair : show me your inconsistencies

Exposure of distributed reality • Read repair : show me your inconsistencies – They will happen

Exposure of distributed reality • Read repair : show me your inconsistencies – They will happen – I might as well code for that case

Exposure of distributed reality • Read repair : show me your inconsistencies – They will happen – I might as well code for that case – Timestamp-based reconciliation == yucky

Exposure of distributed reality • Read repair : show me your inconsistencies – They will happen – I might as well code for that case – Timestamp-based reconciliation == yucky • Tunable quorum parameters (R + W > N)

Exposure of distributed reality • Read repair : show me your inconsistencies – They will happen – I might as well code for that case – Timestamp-based reconciliation == yucky • Tunable quorum parameters (R + W > N) – Ideally, per request and with partial results

Exposure of distributed reality • Read repair : show me your inconsistencies – They will happen – I might as well code for that case – Timestamp-based reconciliation == yucky • Tunable quorum parameters (R + W > N) – Ideally, per request and with partial results – Minimally, per-tenancy (e.g. bucket or table)

Exposure of distributed reality • Read repair : show me your inconsistencies – They will happen – I might as well code for that case – Timestamp-based reconciliation == yucky • Tunable quorum parameters (R + W > N) – Ideally, per request and with partial results – Minimally, per-tenancy (e.g. bucket or table) • Together: slide along the CP  AP continuum

Exposure of distributed reality • Read repair : show me your inconsistencies – They will happen – I might as well code for that case – Timestamp-based reconciliation == yucky • Tunable quorum parameters (R + W > N) – Ideally, per request and with partial results – Minimally, per-tenancy (e.g. bucket or table) • Together: slide along the CP  AP continuum – As application needs warrant

Idealized API ReadResult read(<args>, R, timeout); ReadResult { boolean success; List<ResultSet> results; } int write(<args>, W, timeout); 22

Operational Requirements James Hamilton, Windows Live Services Platform. “On Designing and Deploying Internet-Scale Systems” Large Installation System Administration (LISA ’07) http://www.mvdirona.com/jrh/ talksAndPapers/JamesRH_Lisa.pdf

Operational Requirements James Hamilton, Windows Live Services Platform. “On Designing and Deploying Internet-Scale Systems” Large Installation System Administration (LISA ’07) http://www.mvdirona.com/jrh/ talksAndPapers/JamesRH_Lisa.pdf We want to run a 24x7 service with an 8am-5pm staff.

Operational scalability

Operational scalability • “Can I add more capacity without adding too many more sysadmins?”

Operational scalability • “Can I add more capacity without adding too many more sysadmins?” • On-the-ﬂy node addition/subtraction

Operational scalability • “Can I add more capacity without adding too many more sysadmins?” • On-the-ﬂy node addition/subtraction • Automatic re-partitioning / balancing

Operational scalability • “Can I add more capacity without adding too many more sysadmins?” • On-the-ﬂy node addition/subtraction • Automatic re-partitioning / balancing • Disaster recovery / cold restart

More Ops-friendliness

More Ops-friendliness • Is there a company behind the product available to provide support? – vs. having to have in-house experts – Important until commoditization / stabilization of open source offerings

More Ops-friendliness • Is there a company behind the product available to provide support? – vs. having to have in-house experts – Important until commoditization / stabilization of open source offerings • Monitoring – JMX (preferred), SNMP (if we have to) – Detailed health and performance metrics

More Ops-friendliness • Is there a company behind the product available to provide support? – vs. having to have in-house experts – Important until commoditization / stabilization of open source offerings • Monitoring – JMX (preferred), SNMP (if we have to) – Detailed health and performance metrics • Performance – “Good enough” – We’ll run our own benchmarks anyway

Support for multiple data centers Text

Support for multiple data centers Text 1,760 mi

Support for multiple data centers Text 1,760 mi = 2,832 km

Support for multiple data centers Text 2.832 1,760 mi = 2,832 km

Support for multiple data centers Text 2.832 1,760 mi = 2,832 km = 100 ms

Support for multiple data centers Text 2.832 average 1,760 mi = 2,832 km = 100 ms

Support for multiple data centers Text 2.832 average 1,760 mi = 2,832 km = 100 ms (Philly -> London = 3,548 mi)

Support for multiple data centers “Active-active” Internet

Multi-DC support: Approach 0 Internet

Multi-DC support: Approach 0 VPN! IPSEC tunnels! Internet

Multi-DC support: Approach 0 VPN! IPSEC tunnels! Internet One Big, Topology-Agnostic Cluster

Multi-DC support: Approach 0 “This is NOT multi-DC support.” VPN! IPSEC tunnels! Internet One Big, Topology-Agnostic Cluster

Multi-DC support: Approach 1 Internet One Big, Topology-Aware Cluster

Multi-DC support: Approach 1 Internet One Big, Topology-Aware Cluster `

Multi-DC support: Approach 1 + 100ms Internet One Big, Topology-Aware Cluster `

Multi-DC support: Approach 1 Internet One Big, Topology-Aware Cluster

Multi-DC support: Approach 1 Internet Partition 1 Partition 2

Multi-DC support: Approach 1 “This is the CP approach to multi-DC” Internet Partition 1 Partition 2

Multi-DC support: Approach 2 Internet Cluster 1 Cluster 2

Multi-DC support: Approach 2 Internet Cluster 1 Cluster 2 `

Multi-DC support: Approach 2 Internet No 100 ms penalty! Cluster 1 Cluster 2 `

Multi-DC support: Approach 2 Internet No 100 ms penalty! Cluster 1 Cluster 2 ` asynchronous replication

Multi-DC support: Approach 2 Internet No 100 ms penalty! Cluster 1 Cluster 2 ` ` asynchronous replication

Multi-DC support: Approach 2 (Also good for setting up reporting instances) Internet No 100 ms penalty! Cluster 1 Cluster 2 ` ` asynchronous replication

Multi-DC support: Approach 2 Internet Cluster 1 Cluster 2 `

Multi-DC support: Approach 2 “This is the AP approach to multi-DC” Internet Cluster 1 Cluster 2

Are you a wallﬂower, or did you ask anyone to dance?

Are you a wallﬂower, or did you ask anyone to dance? • We are rolling some NoSQL into production now – And still investigating others…

Are you a wallﬂower, or did you ask anyone to dance? • We are rolling some NoSQL into production now – And still investigating others… • No one project ﬁts all of our needs exactly – But the pace is astounding – Now vs. 6 months ago (NoSQL East, Oct. 2009)

Are you a wallﬂower, or did you ask anyone to dance? • We are rolling some NoSQL into production now – And still investigating others… • No one project ﬁts all of our needs exactly – But the pace is astounding – Now vs. 6 months ago (NoSQL East, Oct. 2009) • More and more projects are gaining the needed feature sets for an enterprise world

Takeaways

Takeaways • Many enterprises do not have the scale or performance problems of the web titans

Takeaways • Many enterprises do not have the scale or performance problems of the web titans • However, many of them do have distributed systems for which “CA” is not appropriate

Takeaways • Many enterprises do not have the scale or performance problems of the web titans • However, many of them do have distributed systems for which “CA” is not appropriate • NoSQL projects, due to their focus on scale, have had to wrestle with distributed systems concepts

Conclusion: an empty stage CC By-SA 2.0: http://www.ﬂickr.com/photos/ 37622685@N02/3948536721/

Conclusion: an empty stage Enterprises are waiting for someone to take this stage with a mix of: • CAP visibility / tradeoffs • Multi-DC support • Easy operations CC By-SA 2.0: http://www.ﬂickr.com/photos/ 37622685@N02/3948536721/

http://nosql.pl	2162
http://nosql.mypopescu.com	252
http://nosqlpl.tumblr.com	16
http://www.slideshare.net	13
http://webcache.googleusercontent.com	2
https://team.theflowr.com	1
http://net.agrupalia.com	1
http://translate.googleusercontent.com	1
http://www.mefeedia.com	1
http://feeds.feedburner.com	1
http://web.archive.org	1
http://mbot-2.local	1

JonMoore-NoSQLEU-2010

by cimjmoore

Accessibility

Categories

Upload Details

Usage Rights

12 Embeds 2,452

Statistics

JonMoore-NoSQLEU-2010 Presentation Transcript