Your SlideShare is downloading. ×
Oracle Big Data Appliance - Zbyszek Swoczyna, Oracle
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Oracle Big Data Appliance - Zbyszek Swoczyna, Oracle

1,314
views

Published on

Oracle Extreme Analytic Summit, 22.03.2012 r.

Oracle Extreme Analytic Summit, 22.03.2012 r.

Published in: Technology, Business

0 Comments
3 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
1,314
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
0
Comments
0
Likes
3
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. Oracle Big Data ApplianceTechnical OverviewZbigniew SwoczynaSE Manager, Storage Architect
  • 2. The following is intended to outline our general product direction.It is intended for information purposes only, and may not beincorporated into any contract. It is not a commitment to deliverany material, code, or functionality, and should not be relied uponin making purchasing decisions. The development, release, andtiming of any features or functionality described for Oracle’sproducts remains at the sole discretion of Oracle.
  • 3. Oracle Big Data Platform Big Data Exadata Exalytics ApplianceACQUIRE ORGANIZE ANALYZE DECIDE
  • 4. Why Build A Hadoop Appliance? Time to Build Optimizations Maintenance
  • 5. Oracle Big Data ApplianceHardware Overview
  • 6. Oracle Big Data Appliance• 18 Sun X4270 M Servers – 216 CPU cores – 864 GB RAM – 648 TB disk• 40 Gb/s InfiniBand – Inter-rack, inter-node connectivity• 10 Gb/s Ethernet – Data center connectivity
  • 7. BDA Node – Sun Fire X4270 M2Processors 2 Six-Core Intel® Xeon® X5675 Processors 3.06 GHzMemory 48GB (6 * 8GB) expandable to 96 GB or 144Disks 12 x 3 TB 7.2K RPM High Capacity SAS hot-swapDisk Disk Controller HBA with 512MB Battery BackedController CacheNetwork 2 InfiniBand 4X QDR (40Gb/s) Ports (1 Dual-port PCIe 2.0 HCA) 4 Embedded Gigabit Ethernet PortsRemote ILOM with 1 Gigabit Ethernet portManagementPower Redundant Hot-Swappable power supplies andSupplies fans
  • 8. Memory Expansion Kit• Optional kit for BDA customers in case certain nodes require more memory (Name Node) – Changes memory configuration of the X4270 M2 server from 48GB to either 96Gb (12 x 8 GB) to 144 GB (18 x 8 GB) – Can be done on all nodes in the cluster or on some nodes (Name Node, HBase master etc.) – Upgrades to 144GB can lead to a minor reduction in performance due to lower memory bandwidth (the memory frequency drops from 1333 MHz to 800 MHz)
  • 9. InfiniBand Network• Uses Sun 4x QDR (40Gb/s) InfiniBand switches – Subnet manager automatically discovers network topology – 2 “leaf” switches (32 port QDR gateway switch) – 1 “internal spine” switch (36 port QDR switch) for scale out• Individual Servers – Each server has Dual-port QDR (40Gb/s) IB HCA – Active-Passive Bonding –Single IP address per dual-port HCA • Performance is limited by PCIe bus, so active-active not needed – Connect dual port HCA to bothleaf switches for redundancy – Connections pre-wired in the Factory
  • 10. Leaf Switches Port 1 Port 1 Nodes (1 trough 9) Nodes (10 through 18) 9 9L2 L1Leaf switch Leaf switch 9 7 9 Nodes (10 through 18) Nodes (1 through 9) Port 2 Port 2 • InfiniBand network with Redundancy • Servers connect to the two leaf switches • Active & Passive ports balanced across switches • Full Bandwidth even if switch fails • Connections pre-wired at factory
  • 11. Spine and Leaf InfiniBand Switch S BDA Nodes Spine switch 1 1 Leaf switch Leaf switch L1 7 L2• Use 3rd switch (S) as “spine” switch for expansion to multiple racks• Connect each leaf switch to spine switch (1 links wide)• Interconnect “leaf” switches with each other (7 links wide)• Enough bandwidth even if a switch fails• Pre-wired at factory• Allows connectivity to BDA racks as well as Exadata racks
  • 12. Horizontal Scale Out ModelScale with Infiniband
  • 13. Oracle Big Data ApplianceSoftware Overview
  • 14. Oracle Big Data Appliance Pre-Installed, Optimized • Oracle Linux 5.6 • Java Hotspot VM • Cloudera CDH • Cloudera Manager • Open Source R Distribution • Oracle NoSQL Database CE • Oracle Big Data Connectors ** Optional. Available and licensed separately.
  • 15. Why Cloudera CDH?• Fast evolution in critical features – Built by the Hadoop experts in the community – Practical instead of esoteric• Proven at very large scale – In production at all the large consumers of Hadoop – Extremely stable in those environments• Managed and Tested by Cloudera – Managed Open Source components – Contains a rich management GUI tool
  • 16. Cloudera CDH Apache Hadoop Apache Sqoop Apache Hive Apache Mahout Apache Pig Apache Whirr Apache HBase Apache Oozie Apache Zookeeper Fuse-DFS Apache Flume HueLatest details at: http://www.cloudera.com/hadoop-details/
  • 17. Hadoop Software Layout (Masters) • Node 1: – M: Name Node, Balancer & HBase Master – S: HDFS Data Node, NoSQL Database Storage Node* • Node 2: – M: Secondary Name Node, Cloudera Manager, Zookeeper, MySQL Slave – S: HDFS Data Node, NoSQL Database Storage Node* • Node 3: 3 – M: JobTracker, MySQL Master, ODI Agent, Hive Server 2 1 – S: HDFS Data Node, NoSQL Database Storage Node** Optional
  • 18. Oracle Big Data Connectors*Software Overview
  • 19. Oracle Data IntegratorSimplifying MapReduce Oracle Data Integrator Automatically generates MapReduce code Oracle Loader for Manages the process Hadoop Loads into Data Warehouse
  • 20. Oracle Loader for HadoopUse The Cluster ORACLE LOADER FOR HADOOP MAP REDUCE MAP Last stage in MapReduce MAP SHUFFLE /SORT REDUCE workflow Partitioned and non- MAP REDUCE partitioned tables MAP REDUCE SHUFFLE MAP /SORT REDUCE Online and offline loads
  • 21. Oracle Direct Connector for HDFSDirect Access from Oracle Database HDFS Oracle Database SQL Query SQL access to HDFS External Table External table view Data query or import DCH DCH HDFS Infini Band DCH Client
  • 22. Oracle R Hadoop Connector Native R Access to HadoopClient Host Oracle Big Data Oracle Exadata Appliance R Engine R Engine ORE ORHC ORHC Native R MapReduce Hadoop MapReduce R Engine Native R HDFS access Cluster Nodes ORE Software HDFS
  • 23. Oracle Big Data Appliance• Optimized and Complete – All you need to store and integrate big data• Integrated with Oracle Exadata – Analyze all your data• Easy to Deploy – Risk Free, Quick Installation and Setup• Single Vendor Support – Full Oracle support for all hardware and software

×