Netezza pure data

IBM PureData System
for Analytics
Powered by Netezza
Hossein Sarshar

Agenda
• What is PureData and Netezza
o History
o Characteristics
o Product chain
• PureData Hardware Architecture
o Introduction
o Hardware architecture
o Paralleled structures
• Analytics with PureData
o Introduction
o In-database analytics tools
• Demo
IBM® PureData™ for Analytics 2

What is PureData and
Netezza
PureSystems
PureFlex PureApplication

In 2010, IBM bought a new analytics platform called
Netezza. It was founded in 2000 at Marlborough, CA.
IBM later rebranded it to PureData.
What is PureData and
Netezza
PureSystems
PureFlex PureApplication PureData

PureSystems Product
Family
PureFlex:
o Combines and optimizes compute, storage, networking and virtualization
capabilities under a single, unified management console into an
infrastructure system.
PureApplication:
o Is a platform system designed and tuned specifically for transactional web
and database applications.
PureData:
o Based on Netezza technology, PureData is all data experts need in a
single well tuned appliance.
PureData
Operational
Analytics
Transactions Analytics

PureSystems
Characteristics
• Built-in Experts
o No indexing/tuning/partitioning
o Fully parallel, optimized in-Database Analytics.
o No storage administration.
o No software installation.
• Integration by Design:
o Server, Storage, Database in one easy to use package.
o Automatic parallelization and resource optimization to scale economically
o Enterprise-class security and platform management
• Simplified Experience:
o Up and running in hours.
o Minimal ongoing administration.
o Standard interfaces to best of breed Analytics, BI, and data integration tools.
o Built-in analytics capabilities allow users to derive insight from data quickly.
o Easy connectivity to other Big Data Platform components
Each of these come as an appliance equal to
simplified yet strong private clouds with
minimal administration

PureData Introduction
• It is a datawarehousing and data analytics
appliance that is fast enough to process terabytes
of data in seconds. It is a fully parallel machine.
• Netezza’s main technology is using FPGA (Field
Programmable Gateway Array) to filter
unnecessary files in parallel manner.
• PureData uses Netezza technology to perform
deep analytics on huge amount of data in a
reasonable time.
• It is purpose-built for high performance analytics.
• It supports all DB structures (3NF, Star, De-Normalized
table)

PureData Architecture
Disk storage
RAID 1 disks
High speed data
streams
SMP Host
Redhat linux
servers
Optimizer
Compiler
A gateway to
the system
Snippet-Blades
Query accelerator
using FPGAs

S-Blades (SPU)

S-Blades
Intel Quad-Core
Dual-Core FPGADRAM
IBM BladeCenter Server Netezza DB Accelerator
SAS Expander
Module
SAS Expander
Module

S-Blades Overview
• There are 8 intel core on IBM Blade-Center Server
and 8 FPGA on Netezza DB accelerator.
o FPGA has similar dimensions a CPU has, consumes 5 times less power and
clock speed is about 5 times less
o More caching capability
o Low latency and high throughput
• Each of these S-Blades takes ownership of 6-8 disks.
• The queries are divided into subqueries that are
processed by S-Blades.

PureData AMPP (Shared-
Nothing) Architecture
12
Advanced
Analytics
Loader
ETL
BI
Applications
FPGA
Memory
CPU
FPGA
Memory
CPU
FPGA
Memory
CPU
Hosts
SMP
Host
Disk
Enclosures
S-Blades™
Network
Fabric
Netezza Appliance

FPGA Secret Sauce
FPGA Core CPU Core
Uncompress
Project Restrict,
Visibility
Complex ∑
Group by, …
select DISTRICT,
PRODUCTGRP,
sum(NRX)
from MTHLY_RX_TERR_DATA
where MONTH = '20091201'
and MARKET = 509123
and SPECIALTY = 'GASTRO'
Slice of table
MTHLY_RX_TERR_DATA
(compressed)
where MONTH = '20091201'
and MARKET = 509123
and SPECIALTY = 'GASTRO'
sum(NRX)
select DISTRICT,
PRODUCTGRP,
sum(NRX)
Using FPGA reduces a
tremendous among of
unnecessary data movement

PureData System
Configuration
14IBM® PureData™ for Analytics

PureData System
Configuration

PureData System
Configuration
Single Rack System Multi Rack System
Specs N3001-
002
N3001-
005
N3001-
010
N3001-
020
N3001-
040
N3001-
080
Racks 1 1 1 2 4 8
Active S-Blades 2 4 7 14 28 56
CPU Cores 40 80 140 280 560 1120
FPGA Cores 32 64 112 224 448 896
User Data in TB 32 98 192 384 768 1536
N3001 is the newest IBM PureData

What is Achievable
• Having agile analytics platform.
• No administration effort to install/manage
• Scalability in petabyte level
• Linear speedup scalability by adding additional
racks.
• Big Data Meets Deep Analytics => No need to
sample

High Performance
Analytics Architecture

PureData Analytics
Modules

Netezza In-Database
Analytics Options
Classification Time Series Clustering
Associate
Rules
Simulation
and Monte
Carlo Analysis
Geospatial

Demo
• Installation
• Client Tool Exploration
• Command Execution

Summary
• A system for analytics
• Out-of-the-box solution
• It uses FPGA technology to boost query execution
• It uses nothing-shared approach.
• PureData uses open standards to communicate to
outside world
• It has many NZ in-database and 3rd party in-
database options to enrich our analytics

References
• http://www-01.ibm.com/software/data/netezza/
• http://www.ibm.com/ibm/puresystems/ca/en/

Netezza pure data

More Related Content

What's hot

Viewers also liked

Similar to Netezza pure data

Recently uploaded

Netezza pure data

Editor's Notes