Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
1
2
Ashwin Kumar
Pivotal
Pivotal Hadoop on Cloud Foundry
3
 Open source software for reliable,
scalable, distributed computing
 HDFS - A distributed file system for
“large” I/O
...
4
 Enterprise grade Hadoop distribution
• Cluster Management and Monitoring
• Bulk Data Loader
• Extensions for Virtualiz...
5
 Provision Hadoop resources to power data-centric Cloud
Foundry Apps
• Park unstructured data on HDFS.
• Execute batch ...
6
Extensibility in PaaS
Pivotal CF
7
Extensibility in PaaS
Pivotal CF Pivotal HD
8
Cloud Foundry Service API
Pivotal CF Pivotal HD
PHD
Service
Broker
Cloud
Controller
9
Pivotal HD Service
HDFS
Hive
YARN
HBase
HAWQ
ZooKeeper
Slave
Node
Slave
Node
Slave
Node
Slave
Node
Slave
Node
Slave
Node...
10
Pivotal HD Service
PHD
Service
Broker
HDFS
Hive
YARN
HBase
HAWQ
ZooKeeper
Slave
Node
Slave
Node
Slave
Node
Slave
Node
S...
11
Pivotal HD Service
PHD
Service
Broker
HDFS
Hive
YARN
HBase
HAWQ
ZooKeeper
Slave
Node
Slave
Node
Slave
Node
Slave
Node
S...
12
 Shared Clusters
 Bare Metal Installs
 Negotiation across Multiple Clusters
 Exclusive Clusters
 Dynamic Provision...
13
 Shared Clusters
 Bare Metal Installs
 Negotiation across Multiple Clusters
 Exclusive Clusters
 Dynamic Provision...
14
Pivotal HD Service
PHD
Service
Broker
HDFS
Hive
YARN
HBase
HAWQ
ZooKeeper
Slave
Node
Slave
Node
Slave
Node
Slave
Node
S...
15
 Shared Clusters
 Bare Metal Installs
 Negotiation across Multiple Clusters
 Exclusive Clusters
 Dynamic Provision...
16
Pivotal HD Service
PHD
Service
Broker
HDFS YARN HAWQ
HDFS YARN HAWQ
HDFS YARN HAWQ
17
Pivotal HD Service
PHD
Service
Broker
HDFS YARN HAWQ
HDFS YARN HAWQ
HDFS YARN HAWQ
18
 Shared Clusters
 Bare Metal Installs
 Negotiation across Multiple Clusters
 Exclusive Clusters
 Dynamic Provision...
19
Pivotal HD Service
PHD
Service
Broker
HDFS YARN HAWQ
HDFS YARN HAWQ
HDFS YARN HAWQ
20
Pivotal HD Service
PHD
Service
Broker
HDFS YARN HAWQ
HDFS YARN HAWQ
HDFS YARN HAWQ
21
 Shared Clusters
 Bare Metal Installs
 Negotiation across Multiple Clusters
 Exclusive Clusters
 Dynamic Provision...
22
Pivotal HD Service
PHD
Service
Broker
HDFS YARN HAWQ
HDFS YARN HAWQ
23
Pivotal HD Service
PHD
Service
Broker
HDFS YARN HAWQ
HDFS YARN HAWQ
HDFS YARN HAWQ
24
 Pivotal HD
• Deployable by BOSH
• Exposed as a Cloud Foundry service
 Data intensive apps are coming
 Only possible...
25
Upcoming SlideShare
Loading in …5
×

Pivotal HD as a Cloud Foundry Service

5,178 views

Published on

Published in: Technology, News & Politics
  • Be the first to comment

Pivotal HD as a Cloud Foundry Service

  1. 1. 1
  2. 2. 2 Ashwin Kumar Pivotal Pivotal Hadoop on Cloud Foundry
  3. 3. 3  Open source software for reliable, scalable, distributed computing  HDFS - A distributed file system for “large” I/O  YARN – A framework for resource scheduling and management  MapReduce – Popular paradigm for parallel batch processing Apache Hadoop
  4. 4. 4  Enterprise grade Hadoop distribution • Cluster Management and Monitoring • Bulk Data Loader • Extensions for Virtualization  Advanced Database Services • World’s Fastest SQL on Hadoop • 100% SQL Compliance Pivotal Hadoop
  5. 5. 5  Provision Hadoop resources to power data-centric Cloud Foundry Apps • Park unstructured data on HDFS. • Execute batch processing via MapReduce. • Perform deep, complex analytics in SQL using HAWQ. Pivotal HD for Cloud Foundry
  6. 6. 6 Extensibility in PaaS Pivotal CF
  7. 7. 7 Extensibility in PaaS Pivotal CF Pivotal HD
  8. 8. 8 Cloud Foundry Service API Pivotal CF Pivotal HD PHD Service Broker Cloud Controller
  9. 9. 9 Pivotal HD Service HDFS Hive YARN HBase HAWQ ZooKeeper Slave Node Slave Node Slave Node Slave Node Slave Node Slave Node Slave Node Slave Node Slave Node PHD Service Broker
  10. 10. 10 Pivotal HD Service PHD Service Broker HDFS Hive YARN HBase HAWQ ZooKeeper Slave Node Slave Node Slave Node Slave Node Slave Node Slave Node Slave Node Slave Node Slave Node
  11. 11. 11 Pivotal HD Service PHD Service Broker HDFS Hive YARN HBase HAWQ ZooKeeper Slave Node Slave Node Slave Node Slave Node Slave Node Slave Node Slave Node Slave Node Slave Node
  12. 12. 12  Shared Clusters  Bare Metal Installs  Negotiation across Multiple Clusters  Exclusive Clusters  Dynamic Provisioning Pivotal HD Service
  13. 13. 13  Shared Clusters  Bare Metal Installs  Negotiation across Multiple Clusters  Exclusive Clusters  Dynamic Provisioning Pivotal HD Service
  14. 14. 14 Pivotal HD Service PHD Service Broker HDFS Hive YARN HBase HAWQ ZooKeeper Slave Node Slave Node Slave Node Slave Node Slave Node Slave Node Slave Node Slave Node Slave Node
  15. 15. 15  Shared Clusters  Bare Metal Installs  Negotiation across Multiple Clusters  Exclusive Clusters  Dynamic Provisioning Pivotal HD Service
  16. 16. 16 Pivotal HD Service PHD Service Broker HDFS YARN HAWQ HDFS YARN HAWQ HDFS YARN HAWQ
  17. 17. 17 Pivotal HD Service PHD Service Broker HDFS YARN HAWQ HDFS YARN HAWQ HDFS YARN HAWQ
  18. 18. 18  Shared Clusters  Bare Metal Installs  Negotiation across Multiple Clusters  Exclusive Clusters  Dynamic Provisioning Pivotal HD Service
  19. 19. 19 Pivotal HD Service PHD Service Broker HDFS YARN HAWQ HDFS YARN HAWQ HDFS YARN HAWQ
  20. 20. 20 Pivotal HD Service PHD Service Broker HDFS YARN HAWQ HDFS YARN HAWQ HDFS YARN HAWQ
  21. 21. 21  Shared Clusters  Bare Metal Installs  Negotiation across Multiple Clusters  Exclusive Clusters  Dynamic Provisioning Pivotal HD Service
  22. 22. 22 Pivotal HD Service PHD Service Broker HDFS YARN HAWQ HDFS YARN HAWQ
  23. 23. 23 Pivotal HD Service PHD Service Broker HDFS YARN HAWQ HDFS YARN HAWQ HDFS YARN HAWQ
  24. 24. 24  Pivotal HD • Deployable by BOSH • Exposed as a Cloud Foundry service  Data intensive apps are coming  Only possible through extensibility of Cloud Foundry Conclusion
  25. 25. 25

×