0
1
An Architectural
Overview of MapR
M. C. Srivas, CTO/Founder
2
 100% Apache Hadoop
 With significant enterprise-
grade enhancements
 Comprehensive
management
 Industry-standard
in...
3
MapR: Lights Out Data Center Ready
• Automated stateful failover
• Automated re-replication
• Self-healing from HW and S...
4
MapR does MapReduce (fast)
TeraSort Record
1 TB in 54 seconds
1003 nodes
MinuteSort Record
1.5 TB in 59 seconds
2103 nod...
5
MapR does MapReduce (faster)
TeraSort Record
1 TB in 54 seconds
1003 nodes
MinuteSort Record
1.5 TB in 59 seconds
2103 n...
6
Google chose MapR to
provide Hadoop on
Google Compute Engine
The Cloud Leaders Pick MapR
Amazon EMR is the largest
Hadoo...
7
1. Make the storage reliable
– Recover from disk and node failures
How to make a cluster reliable?
8
1. Make the storage reliable
– Recover from disk and node failures
2. Make services reliable
– Services need to checkpoi...
9
1. Make the storage reliable
– Recover from disk and node failures
2. Make services reliable
– Services need to checkpoi...
10
 No NVRAM
Reliability With Commodity Hardware
11
 No NVRAM
 Cannot assume special connectivity
– no separate data paths for "online" vs. replica traffic
Reliability W...
12
 No NVRAM
 Cannot assume special connectivity
– no separate data paths for "online" vs. replica traffic
 Cannot even...
13
 No NVRAM
 Cannot assume special connectivity
– no separate data paths for "online" vs. replica traffic
 Cannot even...
14
 No NVRAM
 Cannot assume special connectivity
– no separate data paths for "online" vs. replica traffic
 Cannot even...
15
 Replication is easy, right? All we have to do is send the same bits
to the master and replica.
Reliability via Replic...
16
 When the replica comes back, it is stale
– it must brought up-to-date
– until then, exposed to failure
But crashes oc...
17
 HDFS solves the problem a third way
Unless its Apache HDFS …
18
 HDFS solves the problem a third way
 Make everything read-only
– Nothing to re-sync
Unless its Apache HDFS …
19
 HDFS solves the problem a third way
 Make everything read-only
– Nothing to re-sync
 Single writer, no reads allowe...
20
 HDFS solves the problem a third way
 Make everything read-only
– Nothing to re-sync
 Single writer, no reads allowe...
21
 HDFS solves the problem a third way
 Make everything read-only
– Nothing to re-sync
 Single writer, no reads allowe...
22
 HDFS solves the problem a third way
 Make everything read-only
– Nothing to re-sync
 Single writer, no reads allowe...
23
 To support normal apps, need full read/write support
 Let's return to issue: resync the replica when it comes back
T...
24
 24 TB / server
– @ 1000MB/s = 7 hours
– practical terms, @ 200MB/s = 35 hours
How long to re-sync?
25
 24 TB / server
– @ 1000MB/s = 7 hours
– practical terms, @ 200MB/s = 35 hours
 Did you say you want to do this onlin...
26
 24 TB / server
– @ 1000MB/s = 7 hours
– practical terms, @ 200MB/s = 35 hours
 Did you say you want to do this onlin...
27
 24 TB / server
– @ 1000MB/s = 7 hours
– practical terms, @ 200MB/s = 35 hours
 Did you say you want to do this onlin...
28
 24 TB / server
– @ 1000MB/s = 7 hours
– practical terms, @ 200MB/s = 35 hours
 Did you say you want to do this onlin...
29
 Use dual-ported disk to side-step this problem
Traditional solutions
Clients
Primary
Server
Replica
Dual Ported
Disk ...
30
 Use dual-ported disk to side-step this problem
Traditional solutions
Clients
Primary
Server
Replica
Dual Ported
Disk ...
31
 Use dual-ported disk to side-step this problem
Traditional solutions
Clients
Primary
Server
Replica
Dual Ported
Disk ...
32
 Use dual-ported disk to side-step this problem
Traditional solutions
Clients
Primary
Server
Replica
Dual Ported
Disk ...
33
Forget Performance?
SAN/NAS
data data data
data data data
daa data data
data data data
function
RDBMS
Traditional Archi...
34
Forget Performance?
SAN/NAS
data data data
data data data
daa data data
data data data
function
RDBMS
Traditional Archi...
35
 Chop the data on each node to 1000's of pieces
– not millions of pieces, only 1000's
– pieces are called containers
W...
36
 Chop the data on each node to 1000's of pieces
– not millions of pieces, only 1000's
– pieces are called containers
W...
37
 Chop the data on each node to 1000's of pieces
– not millions of pieces, only 1000's
– pieces are called containers
...
38
 Chop the data on each node to 1000's of pieces
– not millions of pieces, only 1000's
– pieces are called containers
...
39
Why does it improve things?
40
 100-node cluster
 each node holds 1/100th of every node's data
MapR Replication Example
41
 100-node cluster
 each node holds 1/100th of every node's data
 when a server dies
MapR Replication Example
42
 100-node cluster
 each node holds 1/100th of every node's data
 when a server dies
MapR Replication Example
43
 100-node cluster
 each node holds 1/100th of every node's data
 when a server dies
MapR Replication Example
44
 100-node cluster
 each node holds 1/100th of every node's data
 when a server dies
 entire cluster resync's the de...
45
 100-node cluster
 each node holds 1/100th of every node's data
 when a server dies
 entire cluster resync's the de...
46
 99 nodes re-sync'ing in parallel
MapR Re-sync Speed
47
 99 nodes re-sync'ing in parallel
– 99x number of drives
– 99x number of ethernet ports
– 99x cpu's
MapR Re-sync Speed
48
 99 nodes re-sync'ing in parallel
– 99x number of drives
– 99x number of ethernet ports
– 99x cpu's
 Each is resync'i...
49
 99 nodes re-sync'ing in parallel
– 99x number of drives
– 99x number of ethernet ports
– 99x cpu's
 Each is resync'i...
50
 99 nodes re-sync'ing in parallel
– 99x number of drives
– 99x number of ethernet ports
– 99x cpu's
 Each is resync'i...
51
Why is this so difficult?
52
 Writes are synchronous
 Data is replicated in a "chain"
fashion
– utilizes full-duplex network
 Meta-data is replic...
53
 As data size increases, writes
spread more, like dropping a
pebble in a pond
 Larger pebbles spread the
ripples fart...
54
MapR Container Resync
 MapR is 100% random
write
– very tough problem
 On a complete crash, all
replicas diverge from...
55
MapR Container Resync
 MapR can detect exactly where
replicas diverged
– even at 2000 MB/s update rate
 Resync means
...
56
 Resync traffic is “secondary”
 Each node continuously measures RTT to all its peers
 More throttle to slower peers
...
57
Where/how does MapR exploit this
unique advantage?
58
NameNode
E F
NameNode
E F
NameNode
E F
MapR's No-NameNode Architecture
HDFS Federation MapR (distributed metadata)
• Mu...
59
Relative performance and scale
0
2000
4000
6000
8000
10000
12000
14000
16000
18000
0 1000 2000 3000 4000 5000 6000
File...
60
Where/how does MapR exploit this
unique advantage?
61
MapR’s NFS allows Direct Deposit
Connectors not needed
No extra scripts or clusters to deploy and maintain
Random Read/...
62
Where/how does MapR exploit this
unique advantage?
63
MapR Volumes
100K volumes are OK,
create as many as
desired!
/projects
/tahoe
/yosemite
/user
/msmith
/bjohnson
Volumes...
64
Where/how does MapR exploit this
unique advantage?
65
M7 Tables
 M7 tables integrated into storage
– always available on every node, zero admin
 Unlimited number of tables...
66
M7 vs. CDH: 50-50 Mix (Reads)
67
M7 vs. CDH: 50-50 load (read latency)
68
Where/how does MapR exploit this
unique advantage?
69
 ALL Hadoop components are Highly Available, eg, YARN
MapR makes Hadoop truly HA
70
 ALL Hadoop components are Highly Available, eg, YARN
 ApplicationMaster (old JT) and TaskTracker record their state ...
71
 ALL Hadoop components are Highly Available, eg, YARN
 ApplicationMaster (old JT) and TaskTracker record their state ...
72
 ALL Hadoop components are Highly Available, eg, YARN
 ApplicationMaster (old JT) and TaskTracker record their state ...
73
 ALL Hadoop components are Highly Available, eg, YARN
 ApplicationMaster (old JT) and TaskTracker record their state ...
74
Where/how can YOU exploit
MapR’s unique advantage?
 ALL your code can easily be scale-out HA
75
Where/how can YOU exploit
MapR’s unique advantage?
 ALL your code can easily be scale-out HA
 Save service-state in M...
76
Where/how can YOU exploit
MapR’s unique advantage?
 ALL your code can easily be scale-out HA
 Save service-state in M...
77
Where/how can YOU exploit
MapR’s unique advantage?
 ALL your code can easily be scale-out HA
 Save service-state in M...
78
Where/how can YOU exploit
MapR’s unique advantage?
 ALL your code can easily be scale-out HA
 Save service-state in M...
79
Where/how can YOU exploit
MapR’s unique advantage?
 ALL your code can easily be scale-out HA
 Save service-state in M...
80
Build cluster brick by brick, one node at a time
 Use commodity hardware at rock-bottom prices
 Get enterprise-class ...
Upcoming SlideShare
Loading in...5
×

Architectural Overview of MapR's Apache Hadoop Distribution

3,343

Published on

Describes the thinking behind MapR's architecture. MapR"s Hadoop achieves better reliability on commodity hardware compared to anything on the planet, including custom, proprietary hardware from other vendors. Apache HDFS and Cassandra replication is also discussed, as are SAN and NAS storage systems like Netapp and EMC.

Published in: Technology

Transcript of "Architectural Overview of MapR's Apache Hadoop Distribution"

  1. 1. 1 An Architectural Overview of MapR M. C. Srivas, CTO/Founder
  2. 2. 2  100% Apache Hadoop  With significant enterprise- grade enhancements  Comprehensive management  Industry-standard interfaces  Higher performance MapR Distribution for Apache Hadoop
  3. 3. 3 MapR: Lights Out Data Center Ready • Automated stateful failover • Automated re-replication • Self-healing from HW and SW failures • Load balancing • Rolling upgrades • No lost jobs or data • 99999’s of uptime Reliable Compute Dependable Storage • Business continuity with snapshots and mirrors • Recover to a point in time • End-to-end check summing • Strong consistency • Built-in compression • Mirror between two sites by RTO policy
  4. 4. 4 MapR does MapReduce (fast) TeraSort Record 1 TB in 54 seconds 1003 nodes MinuteSort Record 1.5 TB in 59 seconds 2103 nodes
  5. 5. 5 MapR does MapReduce (faster) TeraSort Record 1 TB in 54 seconds 1003 nodes MinuteSort Record 1.5 TB in 59 seconds 2103 nodes 1.65 300
  6. 6. 6 Google chose MapR to provide Hadoop on Google Compute Engine The Cloud Leaders Pick MapR Amazon EMR is the largest Hadoop provider in revenue and # of clusters Deploying OpenStack? MapR partnership with Canonical and Mirantis on OpenStack support.
  7. 7. 7 1. Make the storage reliable – Recover from disk and node failures How to make a cluster reliable?
  8. 8. 8 1. Make the storage reliable – Recover from disk and node failures 2. Make services reliable – Services need to checkpoint their state rapidly – Restart failed service, possibly on another node – Move check-pointed state to restarted service, using (1) above How to make a cluster reliable?
  9. 9. 9 1. Make the storage reliable – Recover from disk and node failures 2. Make services reliable – Services need to checkpoint their state rapidly – Restart failed service, possibly on another node – Move check-pointed state to restarted service, using (1) above 3. Do it fast – Instant-on … (1) and (2) must happen very, very fast – Without maintenance windows • No compactions (eg, Cassandra, Apache HBase) • No “anti-entropy” that periodically wipes out the cluster (eg, Cassandra) How to make a cluster reliable?
  10. 10. 10  No NVRAM Reliability With Commodity Hardware
  11. 11. 11  No NVRAM  Cannot assume special connectivity – no separate data paths for "online" vs. replica traffic Reliability With Commodity Hardware
  12. 12. 12  No NVRAM  Cannot assume special connectivity – no separate data paths for "online" vs. replica traffic  Cannot even assume more than 1 drive per node – no RAID possible Reliability With Commodity Hardware
  13. 13. 13  No NVRAM  Cannot assume special connectivity – no separate data paths for "online" vs. replica traffic  Cannot even assume more than 1 drive per node – no RAID possible  Use replication, but … – cannot assume peers have equal drive sizes – drive on first machine is 10x larger than drive on other? Reliability With Commodity Hardware
  14. 14. 14  No NVRAM  Cannot assume special connectivity – no separate data paths for "online" vs. replica traffic  Cannot even assume more than 1 drive per node – no RAID possible  Use replication, but … – cannot assume peers have equal drive sizes – drive on first machine is 10x larger than drive on other?  No choice but to replicate for reliability Reliability With Commodity Hardware
  15. 15. 15  Replication is easy, right? All we have to do is send the same bits to the master and replica. Reliability via Replication Clients Primary Server Replica Clients Primary Server Replica Normal replication, primary forwards Cassandra-style replication
  16. 16. 16  When the replica comes back, it is stale – it must brought up-to-date – until then, exposed to failure But crashes occur… Clients Primary Server Replica Clients Primary Server Replica Primary re-syncs replica Replica remains stale until "anti-entropy" process kicked off by administrator Who re-syncs?
  17. 17. 17  HDFS solves the problem a third way Unless its Apache HDFS …
  18. 18. 18  HDFS solves the problem a third way  Make everything read-only – Nothing to re-sync Unless its Apache HDFS …
  19. 19. 19  HDFS solves the problem a third way  Make everything read-only – Nothing to re-sync  Single writer, no reads allowed while writing Unless its Apache HDFS …
  20. 20. 20  HDFS solves the problem a third way  Make everything read-only – Nothing to re-sync  Single writer, no reads allowed while writing  File close is the transaction that allows readers to see data – unclosed files are lost – cannot write any further to closed file Unless its Apache HDFS …
  21. 21. 21  HDFS solves the problem a third way  Make everything read-only – Nothing to re-sync  Single writer, no reads allowed while writing  File close is the transaction that allows readers to see data – unclosed files are lost – cannot write any further to closed file  Real-time not possible with HDFS – to make data visible, must close file immediately after writing – Too many files is a serious problem with HDFS (a well documented limitation) Unless its Apache HDFS …
  22. 22. 22  HDFS solves the problem a third way  Make everything read-only – Nothing to re-sync  Single writer, no reads allowed while writing  File close is the transaction that allows readers to see data – unclosed files are lost – cannot write any further to closed file  Real-time not possible with HDFS – to make data visible, must close file immediately after writing – Too many files is a serious problem with HDFS (a well documented limitation)  HDFS therefore cannot do NFS, ever – No “close” in NFS … can lose data any time Unless its Apache HDFS …
  23. 23. 23  To support normal apps, need full read/write support  Let's return to issue: resync the replica when it comes back This is the 21st century… Clients Primary Server Replica Clients Primary Server Replica Primary re-syncs replica Replica remains stale until "anti-entropy" process kicked off by administrator Who re-syncs?
  24. 24. 24  24 TB / server – @ 1000MB/s = 7 hours – practical terms, @ 200MB/s = 35 hours How long to re-sync?
  25. 25. 25  24 TB / server – @ 1000MB/s = 7 hours – practical terms, @ 200MB/s = 35 hours  Did you say you want to do this online? How long to re-sync?
  26. 26. 26  24 TB / server – @ 1000MB/s = 7 hours – practical terms, @ 200MB/s = 35 hours  Did you say you want to do this online? – throttle re-sync rate to 1/10th – 350 hours to re-sync (= 15 days) How long to re-sync?
  27. 27. 27  24 TB / server – @ 1000MB/s = 7 hours – practical terms, @ 200MB/s = 35 hours  Did you say you want to do this online? – throttle re-sync rate to 1/10th – 350 hours to re-sync (= 15 days)  What is your Mean Time To Data Loss (MTTDL)? How long to re-sync?
  28. 28. 28  24 TB / server – @ 1000MB/s = 7 hours – practical terms, @ 200MB/s = 35 hours  Did you say you want to do this online? – throttle re-sync rate to 1/10th – 350 hours to re-sync (= 15 days)  What is your Mean Time To Data Loss (MTTDL)? – how long before a double disk failure? – a triple disk failure? How long to re-sync?
  29. 29. 29  Use dual-ported disk to side-step this problem Traditional solutions Clients Primary Server Replica Dual Ported Disk Array Raid-6 with idle spares Servers use NVRAM
  30. 30. 30  Use dual-ported disk to side-step this problem Traditional solutions Clients Primary Server Replica Dual Ported Disk Array Raid-6 with idle spares Servers use NVRAM COMMODITY HARDWARE LARGE SCALE CLUSTERING
  31. 31. 31  Use dual-ported disk to side-step this problem Traditional solutions Clients Primary Server Replica Dual Ported Disk Array Raid-6 with idle spares Servers use NVRAM COMMODITY HARDWARE LARGE SCALE CLUSTERING
  32. 32. 32  Use dual-ported disk to side-step this problem Traditional solutions Clients Primary Server Replica Dual Ported Disk Array Raid-6 with idle spares Servers use NVRAM COMMODITY HARDWARE LARGE SCALE CLUSTERING Large Purchase Contracts, 5-year spare-parts plan
  33. 33. 33 Forget Performance? SAN/NAS data data data data data data daa data data data data data function RDBMS Traditional Architecture function App function App function App
  34. 34. 34 Forget Performance? SAN/NAS data data data data data data daa data data data data data function RDBMS Traditional Architecture data function data function data function data function data function data function data function data function data function data function data function data function Hadoop function App function App function App Geographically dispersed also?
  35. 35. 35  Chop the data on each node to 1000's of pieces – not millions of pieces, only 1000's – pieces are called containers What MapR does
  36. 36. 36  Chop the data on each node to 1000's of pieces – not millions of pieces, only 1000's – pieces are called containers What MapR does
  37. 37. 37  Chop the data on each node to 1000's of pieces – not millions of pieces, only 1000's – pieces are called containers  Spread replicas of each container across the cluster What MapR does
  38. 38. 38  Chop the data on each node to 1000's of pieces – not millions of pieces, only 1000's – pieces are called containers  Spread replicas of each container across the cluster What MapR does
  39. 39. 39 Why does it improve things?
  40. 40. 40  100-node cluster  each node holds 1/100th of every node's data MapR Replication Example
  41. 41. 41  100-node cluster  each node holds 1/100th of every node's data  when a server dies MapR Replication Example
  42. 42. 42  100-node cluster  each node holds 1/100th of every node's data  when a server dies MapR Replication Example
  43. 43. 43  100-node cluster  each node holds 1/100th of every node's data  when a server dies MapR Replication Example
  44. 44. 44  100-node cluster  each node holds 1/100th of every node's data  when a server dies  entire cluster resync's the dead node's data MapR Replication Example
  45. 45. 45  100-node cluster  each node holds 1/100th of every node's data  when a server dies  entire cluster resync's the dead node's data MapR Replication Example
  46. 46. 46  99 nodes re-sync'ing in parallel MapR Re-sync Speed
  47. 47. 47  99 nodes re-sync'ing in parallel – 99x number of drives – 99x number of ethernet ports – 99x cpu's MapR Re-sync Speed
  48. 48. 48  99 nodes re-sync'ing in parallel – 99x number of drives – 99x number of ethernet ports – 99x cpu's  Each is resync'ing 1/100th of the data MapR Re-sync Speed
  49. 49. 49  99 nodes re-sync'ing in parallel – 99x number of drives – 99x number of ethernet ports – 99x cpu's  Each is resync'ing 1/100th of the data MapR Re-sync Speed  Net speed up is about 100x – 350 hours vs. 3.5
  50. 50. 50  99 nodes re-sync'ing in parallel – 99x number of drives – 99x number of ethernet ports – 99x cpu's  Each is resync'ing 1/100th of the data MapR Re-sync Speed  Net speed up is about 100x – 350 hours vs. 3.5  MTTDL is 100xbetter
  51. 51. 51 Why is this so difficult?
  52. 52. 52  Writes are synchronous  Data is replicated in a "chain" fashion – utilizes full-duplex network  Meta-data is replicated in a "star" manner – response time better MapR's Read-write Replication client1 client2 clientN client1 client2 clientN
  53. 53. 53  As data size increases, writes spread more, like dropping a pebble in a pond  Larger pebbles spread the ripples farther  Space balanced by moving idle containers Container Balancing • Servers keep a bunch of containers "ready to go". • Writes get distributed around the cluster.
  54. 54. 54 MapR Container Resync  MapR is 100% random write – very tough problem  On a complete crash, all replicas diverge from each other  On recovery, which one should be master? Complete crash
  55. 55. 55 MapR Container Resync  MapR can detect exactly where replicas diverged – even at 2000 MB/s update rate  Resync means – roll-back rest to divergence point – roll-forward to converge with chosen master  Done while online – with very little impact on normal operations New master after crash
  56. 56. 56  Resync traffic is “secondary”  Each node continuously measures RTT to all its peers  More throttle to slower peers – Idle system runs at full speed  All automatically MapR does Automatic Resync Throttling
  57. 57. 57 Where/how does MapR exploit this unique advantage?
  58. 58. 58 NameNode E F NameNode E F NameNode E F MapR's No-NameNode Architecture HDFS Federation MapR (distributed metadata) • Multiple single points of failure • Limited to 50-200 million files • Performance bottleneck • Commercial NAS required • HA w/ automatic failover • Instant cluster restart • Up to 1T files (> 5000x advantage) • 10-20x higher performance • 100% commodity hardware NAS appliance NameNode A B NameNode C D NameNode E F DataNode DataNode DataNode DataNode DataNode DataNode A F C D E D B C E B C F B F A B A D E
  59. 59. 59 Relative performance and scale 0 2000 4000 6000 8000 10000 12000 14000 16000 18000 0 1000 2000 3000 4000 5000 6000 Filecreates/s Files (M) 0 100 200 400 600 800 1000 MapR distribution Other distribution Benchmark: File creates (100B) Hardware: 10 nodes, 2 x 4 cores, 24 GB RAM, 12 x 1 TB 7200 RPM 0 50 100 150 200 250 300 350 400 0 0.5 1 1.5 Filecreates/s Files (M) Other distributionMapR Other Advantage Rate (creates/s) 14-16K 335-360 40x Scale (files) 6B 1.3M 4615x
  60. 60. 60 Where/how does MapR exploit this unique advantage?
  61. 61. 61 MapR’s NFS allows Direct Deposit Connectors not needed No extra scripts or clusters to deploy and maintain Random Read/Write Compression Distributed HA Web Server … Database Server Application Server
  62. 62. 62 Where/how does MapR exploit this unique advantage?
  63. 63. 63 MapR Volumes 100K volumes are OK, create as many as desired! /projects /tahoe /yosemite /user /msmith /bjohnson Volumes dramatically simplify the management of Big Data • Replication factor • Scheduled mirroring • Scheduled snapshots • Data placement control • User access and tracking • Administrative permissions
  64. 64. 64 Where/how does MapR exploit this unique advantage?
  65. 65. 65 M7 Tables  M7 tables integrated into storage – always available on every node, zero admin  Unlimited number of tables – Apache HBase is typically 10-20 tables (max 100)  No compactions  Instant-On – zero recovery time  5-10x better perf  Consistent low latency – At 95%-ile and 99%-ile MapR M7
  66. 66. 66 M7 vs. CDH: 50-50 Mix (Reads)
  67. 67. 67 M7 vs. CDH: 50-50 load (read latency)
  68. 68. 68 Where/how does MapR exploit this unique advantage?
  69. 69. 69  ALL Hadoop components are Highly Available, eg, YARN MapR makes Hadoop truly HA
  70. 70. 70  ALL Hadoop components are Highly Available, eg, YARN  ApplicationMaster (old JT) and TaskTracker record their state in MapR MapR makes Hadoop truly HA
  71. 71. 71  ALL Hadoop components are Highly Available, eg, YARN  ApplicationMaster (old JT) and TaskTracker record their state in MapR  On node-failure, AM recovers its state from MapR – Works even if entire cluster restarted MapR makes Hadoop truly HA
  72. 72. 72  ALL Hadoop components are Highly Available, eg, YARN  ApplicationMaster (old JT) and TaskTracker record their state in MapR  On node-failure, AM recovers its state from MapR – Works even if entire cluster restarted  All jobs resume from where they were – Only from MapR MapR makes Hadoop truly HA
  73. 73. 73  ALL Hadoop components are Highly Available, eg, YARN  ApplicationMaster (old JT) and TaskTracker record their state in MapR  On node-failure, AM recovers its state from MapR – Works even if entire cluster restarted  All jobs resume from where they were – Only from MapR  Allows pre-emption – MapR can pre-empt any job, without losing its progress – ExpressLane™ feature in MapR exploits it MapR makes Hadoop truly HA
  74. 74. 74 Where/how can YOU exploit MapR’s unique advantage?  ALL your code can easily be scale-out HA
  75. 75. 75 Where/how can YOU exploit MapR’s unique advantage?  ALL your code can easily be scale-out HA  Save service-state in MapR  Save data in MapR
  76. 76. 76 Where/how can YOU exploit MapR’s unique advantage?  ALL your code can easily be scale-out HA  Save service-state in MapR  Save data in MapR  Use Zookeeper to notice service failure
  77. 77. 77 Where/how can YOU exploit MapR’s unique advantage?  ALL your code can easily be scale-out HA  Save service-state in MapR  Save data in MapR  Use Zookeeper to notice service failure  Restart anywhere, data+state will move there automatically
  78. 78. 78 Where/how can YOU exploit MapR’s unique advantage?  ALL your code can easily be scale-out HA  Save service-state in MapR  Save data in MapR  Use Zookeeper to notice service failure  Restart anywhere, data+state will move there automatically  That’s what we did!
  79. 79. 79 Where/how can YOU exploit MapR’s unique advantage?  ALL your code can easily be scale-out HA  Save service-state in MapR  Save data in MapR  Use Zookeeper to notice service failure  Restart anywhere, data+state will move there automatically  That’s what we did!  Only from MapR: HA for Impala, Hive, Oozie, Storm, MySQL, SOLR/Lucene, Kafka, …
  80. 80. 80 Build cluster brick by brick, one node at a time  Use commodity hardware at rock-bottom prices  Get enterprise-class reliability: instant-restart, snapshots, mirrors, no-single-point-of-failure, …  Export via NFS, ODBC, Hadoop and other std protocols MapR: Unlimited Scale # files, # tables trillions # rows per table trillions # data 1-10 Exabytes # nodes 10,000+
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×