Oracle Big Data Appliance - Zbyszek Swoczyna, Oracle

Oracle Big Data ApplianceTechnical OverviewZbigniew SwoczynaSE Manager, Storage Architect

The following is intended to outline our general product direction.It is intended for information purposes only, and may not beincorporated into any contract. It is not a commitment to deliverany material, code, or functionality, and should not be relied uponin making purchasing decisions. The development, release, andtiming of any features or functionality described for Oracle’sproducts remains at the sole discretion of Oracle.

Oracle Big Data Platform Big Data Exadata Exalytics ApplianceACQUIRE ORGANIZE ANALYZE DECIDE

Why Build A Hadoop Appliance? Time to Build Optimizations Maintenance

Oracle Big Data ApplianceHardware Overview

Oracle Big Data Appliance• 18 Sun X4270 M Servers – 216 CPU cores – 864 GB RAM – 648 TB disk• 40 Gb/s InfiniBand – Inter-rack, inter-node connectivity• 10 Gb/s Ethernet – Data center connectivity

BDA Node – Sun Fire X4270 M2Processors 2 Six-Core Intel® Xeon® X5675 Processors 3.06 GHzMemory 48GB (6 * 8GB) expandable to 96 GB or 144Disks 12 x 3 TB 7.2K RPM High Capacity SAS hot-swapDisk Disk Controller HBA with 512MB Battery BackedController CacheNetwork 2 InfiniBand 4X QDR (40Gb/s) Ports (1 Dual-port PCIe 2.0 HCA) 4 Embedded Gigabit Ethernet PortsRemote ILOM with 1 Gigabit Ethernet portManagementPower Redundant Hot-Swappable power supplies andSupplies fans

Memory Expansion Kit• Optional kit for BDA customers in case certain nodes require more memory (Name Node) – Changes memory configuration of the X4270 M2 server from 48GB to either 96Gb (12 x 8 GB) to 144 GB (18 x 8 GB) – Can be done on all nodes in the cluster or on some nodes (Name Node, HBase master etc.) – Upgrades to 144GB can lead to a minor reduction in performance due to lower memory bandwidth (the memory frequency drops from 1333 MHz to 800 MHz)

InfiniBand Network• Uses Sun 4x QDR (40Gb/s) InfiniBand switches – Subnet manager automatically discovers network topology – 2 “leaf” switches (32 port QDR gateway switch) – 1 “internal spine” switch (36 port QDR switch) for scale out• Individual Servers – Each server has Dual-port QDR (40Gb/s) IB HCA – Active-Passive Bonding –Single IP address per dual-port HCA • Performance is limited by PCIe bus, so active-active not needed – Connect dual port HCA to bothleaf switches for redundancy – Connections pre-wired in the Factory

Leaf Switches Port 1 Port 1 Nodes (1 trough 9) Nodes (10 through 18) 9 9L2 L1Leaf switch Leaf switch 9 7 9 Nodes (10 through 18) Nodes (1 through 9) Port 2 Port 2 • InfiniBand network with Redundancy • Servers connect to the two leaf switches • Active & Passive ports balanced across switches • Full Bandwidth even if switch fails • Connections pre-wired at factory

Spine and Leaf InfiniBand Switch S BDA Nodes Spine switch 1 1 Leaf switch Leaf switch L1 7 L2• Use 3rd switch (S) as “spine” switch for expansion to multiple racks• Connect each leaf switch to spine switch (1 links wide)• Interconnect “leaf” switches with each other (7 links wide)• Enough bandwidth even if a switch fails• Pre-wired at factory• Allows connectivity to BDA racks as well as Exadata racks

Horizontal Scale Out ModelScale with Infiniband

Oracle Big Data ApplianceSoftware Overview

Oracle Big Data Appliance Pre-Installed, Optimized • Oracle Linux 5.6 • Java Hotspot VM • Cloudera CDH • Cloudera Manager • Open Source R Distribution • Oracle NoSQL Database CE • Oracle Big Data Connectors ** Optional. Available and licensed separately.

Why Cloudera CDH?• Fast evolution in critical features – Built by the Hadoop experts in the community – Practical instead of esoteric• Proven at very large scale – In production at all the large consumers of Hadoop – Extremely stable in those environments• Managed and Tested by Cloudera – Managed Open Source components – Contains a rich management GUI tool

Cloudera CDH Apache Hadoop Apache Sqoop Apache Hive Apache Mahout Apache Pig Apache Whirr Apache HBase Apache Oozie Apache Zookeeper Fuse-DFS Apache Flume HueLatest details at: http://www.cloudera.com/hadoop-details/

Hadoop Software Layout (Masters) • Node 1: – M: Name Node, Balancer & HBase Master – S: HDFS Data Node, NoSQL Database Storage Node* • Node 2: – M: Secondary Name Node, Cloudera Manager, Zookeeper, MySQL Slave – S: HDFS Data Node, NoSQL Database Storage Node* • Node 3: 3 – M: JobTracker, MySQL Master, ODI Agent, Hive Server 2 1 – S: HDFS Data Node, NoSQL Database Storage Node** Optional

Oracle Big Data Connectors*Software Overview

Oracle Data IntegratorSimplifying MapReduce Oracle Data Integrator Automatically generates MapReduce code Oracle Loader for Manages the process Hadoop Loads into Data Warehouse

Oracle Loader for HadoopUse The Cluster ORACLE LOADER FOR HADOOP MAP REDUCE MAP Last stage in MapReduce MAP SHUFFLE /SORT REDUCE workflow Partitioned and non- MAP REDUCE partitioned tables MAP REDUCE SHUFFLE MAP /SORT REDUCE Online and offline loads

Oracle Direct Connector for HDFSDirect Access from Oracle Database HDFS Oracle Database SQL Query SQL access to HDFS External Table External table view Data query or import DCH DCH HDFS Infini Band DCH Client

Oracle R Hadoop Connector Native R Access to HadoopClient Host Oracle Big Data Oracle Exadata Appliance R Engine R Engine ORE ORHC ORHC Native R MapReduce Hadoop MapReduce R Engine Native R HDFS access Cluster Nodes ORE Software HDFS

Oracle Big Data Appliance• Optimized and Complete – All you need to store and integrate big data• Integrated with Oracle Exadata – Analyze all your data• Easy to Deploy – Risk Free, Quick Installation and Setup• Single Vendor Support – Full Oracle support for all hardware and software

Oracle Big Data Appliance - Zbyszek Swoczyna, Oracle

by Marketing Oracle Polska on Mar 22, 2012

Accessibility

Categories

Upload Details

Usage Rights

Statistics

Oracle Big Data Appliance - Zbyszek Swoczyna, Oracle Presentation Transcript