Manage Your Data, Using Oracle Big Data Appliance - Tips & Tricksngest, process and manage the data, using Oracle Big Data Appliance (end-to-end BigData solution from Oracle): - Oracle BDA architecture and componets overview - Oracle platform, Cloudera CDH, Clodera Manager and specific Oracle components; - Advantages and additional value of an Oracle BDA; - Challenges, faced inside whole stack (BDA, Cloudera); - Challenges, which came from original Hadoop EcoSystem; - Customer case (anonymized): how to utilize a power of an Oracle BDA, including external Informatica Big Data Management tool.

1. Copyright 2018 Accenture. All rights reserved.
Manage Your Data using
Oracle Big Data Appliance
Tips & Tricks

2. Copyright 2018 Accenture. All rights reserved.Copyright 2018 Accenture. All rights reserved.
About me
Aleksejs Nemirovskis
Technology Architect
Over 23 years of experience in IT industry with main focus to
- DB architecture, design, analysis, development and performance tuning
- Data Integration / Data Warehousing / ETL / EL-T / BI / Data Analytics
- Oracle Technologies stack: DB, Data Integrations, Infrastructure, Cloud
- Big Data (Apache Hadoop, CDH, Cloudera Manager)
Experienced in all stages of software development life cycle (SDLC) from business
requirements and technical definitions to development, testing and production
support as well as in Agile software development techniques (Scrum, XP)

3. Copyright 2018 Accenture. All rights reserved.Copyright 2018 Accenture. All rights reserved.
Agenda
➢ Oracle Big Data Appliance architecture and components overview
➢ Oracle BDA System Overview
➢ Oracle BDA Hardware Overview
➢ Oracle BDA Software Overview
➢ Cloudera Distributed Hadoop (CDH)
➢ Advantages and additional value of an Oracle BDA
➢ End-to-end system
➢ Oracle XQuery 4 Hadoop
➢ Oracle DataSource for Apache Hadoop (OD4H)
➢ Challenges
➢ Oracle Big Data Appliance
➢ Cloudera
➢ Hadoop Eco-System
➢ Architecture & Implementation
➢ Customer case
➢ Infrastructure Setup
➢ Infrastructure Setup - PROD
➢ Data Processing
➢ Q&A

4. Copyright 2018 Accenture. All rights reserved.Copyright 2018 Accenture. All rights reserved.
Manage Your Data using Oracle BDA
➢Oracle BDA architecture and components overview
➢Advantages and additional value of an Oracle BDA
➢Challenges
➢Customer case
➢Q&A

5. Copyright 2018 Accenture. All rights reserved.
Oracle Big Data Appliance
➢ Cloudera Manager
▪ Cloudera Navigator
➢ Cloudera Distributed Hadoop
➢ Oracle Big Data Connectors
➢ Oracle Linux
➢ Oracle Server (hardware)
System Overview

6. Copyright 2018 Accenture. All rights reserved.
Oracle Big Data Appliance
Hardware OverviewStarter Rack - 6 x Compute / Storage Nodes
X7-2 Per Node
configuration
➢ 2 x 24-Core (2.1GHz) Intel
® Xeon ® 8160
➢ 8 x 32GB DDR4-2666 MHz
Memory, expandable to
1.5TB
➢ 12 x 10 TB 7,200 RPM High
Capacity SAS Drives
➢ 2 x 150GB M.2 SATA SSD
Drives
➢ 2 x QDR 40Gb/sec
InfiniBand Ports
➢ 1 x Dual-port InfiniBand
QDR CX3 (40 Gb/sec) PCIe
HCA
➢ 1 x Built-in RJ45 1 Gigabit
Ethernet port
Full Rack - 18 x Compute / Storage Nodes

7. Copyright 2018 Accenture. All rights reserved.
Oracle Big Data Appliance
Software Overview
Integrated Software
➢ Cloudera Enterprise 5 – Data Hub Edition with
support for:
▪ Cloudera’s Distribution including Apache
Hadoop (CDH)
▪ Cloudera Impala
▪ Cloudera Search
▪ Apache HBase
▪ Apache Spark
▪ Apache Kafka
▪ Cloudera Manager with support for:
o Cloudera Navigator
o Cloudera Back-up and Disaster
Recovery (BDR)
➢ Oracle Perfect Balance
➢ Oracle Table Access for Hadoop
Optional Software
➢ Oracle Big Data SQL
➢ Oracle Big Data Connectors:
▪ Oracle SQL Connector for Hadoop
▪ Oracle Loader for Hadoop
▪ Oracle XQuery for Hadoop
▪ Oracle R Advanced Analytics for Hadoop
▪ Oracle Data Integrator
➢ Oracle Audit Vault and Database Firewall for
Hadoop Auditing
➢ Oracle Data Integrator
➢ Oracle GoldenGate
➢ Oracle NoSQL Database Enterprise Edition
➢ Oracle Big Data Spatial and Graph
➢ Oracle Big Data Discovery

8. Copyright 2018 Accenture. All rights reserved.
Oracle Big Data Appliance
Cloudera Distributed Hadoop

9. Copyright 2018 Accenture. All rights reserved.Copyright 2018 Accenture. All rights reserved.
Manage Your Data using Oracle BDA
➢Oracle BDA architecture and components overview
➢Advantages and additional value of an Oracle BDA
➢Challenges
➢Customer case
➢Q&A

10. Copyright 2018 Accenture. All rights reserved.
Oracle Big Data Appliance
End-to-End System`
Maintained&SupportedbyOracle

11. Copyright 2018 Accenture. All rights reserved.
Oracle Big Data Appliance
Oracle XQuery 4 Hadoop

12. Copyright 2018 Accenture. All rights reserved.
Oracle Big Data Appliance
Oracle DataSource 4 Hadoop

13. Copyright 2018 Accenture. All rights reserved.Copyright 2018 Accenture. All rights reserved.
Manage Your Data using Oracle BDA
➢Oracle BDA architecture and components overview
➢Advantages and additional value of an Oracle BDA
➢Challenges
➢Customer case
➢Q&A

14. Copyright 2018 Accenture. All rights reserved.
Challenges
Oracle Big Data Appliance
Cloudera release: gap up to 2 years
Hadoop HFDS: dedicated Name Node
vs. Name/Data Node
Support: Oracle vs. Cloudera vs.
Hadoop

15. Copyright 2018 Accenture. All rights reserved.
Challenges
Cloudera
HIVE vs. Impala
Limitations
(e.g. DBs/Objects to replicate)
HDFS <> Sentry synchronization

16. Copyright 2018 Accenture. All rights reserved.
Challenges
Hadoop Eco-System
Parallel writing into the Parquet files
Spark and Sentry – how to deal with
security, using Spark

17. Copyright 2018 Accenture. All rights reserved.
Challenges
Architecture & Implementation
Partitioning – partitioning key challenge
Compression – always needed?
Fast processing > small files on HDFS
(e.g. Resilient Distributed Datasets – Spark RDD)

18. Copyright 2018 Accenture. All rights reserved.Copyright 2018 Accenture. All rights reserved.
Manage Your Data using Oracle BDA
➢Oracle BDA architecture and components overview
➢Advantages and additional value of an Oracle BDA
➢Challenges
➢Customer case
➢Q&A

19. Copyright 2018 Accenture. All rights reserved.
Infrastructure Setup
 Node01
 Node02
 Node03
Oracle BDA Informatica BDM
Customer Case
7 Clusters: BDA X6-2 Starter Rack (6 nodes)
➢ 264 CPU cores
➢ 1.5 TB RAM (+ 1.5 TB RAM per cluster is upcoming)
➢ 576 TB Storage
4 Domains: Informatica Big Data Management
➢ 10.2.1 in a full push-down mode
➢ 2 Domains: 3 nodes per domain
➢ 2 Domains: 3 + 3 (DR) nodes per domain

20. Copyright 2018 Accenture. All rights reserved.
Infrastructure Setup - PROD
 Node01
 Node02
 Node03
Main Cluster Main Nodes
Customer Case
BDA X6-2 Starter Rack cluster (6 nodes + 10 nodes are upcoming)
➢ 264 CPU cores (+ 440 CPU cores are upcoming)
➢ 1.5 TB RAM (+ 6.3 TB RAM is upcoming)
➢ 576 TB Storage (+ 960 TB is upcoming)
Informatica Big Data Management
➢ 10.2.1 in a full push-down mode

21. Copyright 2018 Accenture. All rights reserved.
Customer Case
Data Processing
Sources Ingestion/Processing Stores Usage
Cloudera Data
Science Workbench

22. Copyright 2018 Accenture. All rights reserved.Copyright 2018 Accenture. All rights reserved.
Manage Your Data using Oracle BDA
Q & A