The document summarizes a presentation about data warehouse appliances and the principles of designing a data warehouse on an "EDWH appliance" platform. It discusses how appliances provide optimized, pre-tuned systems for BI workloads. It also presents the architecture of a massively parallel processing (MPP) data warehouse for operational data warehousing, including features like shared-nothing architecture and parallel query execution.

1. BI Forum 2009
Principy architektury MPP datového
skladu
Václav Hubka - Hewlett-Packard
26. listopadu 2009
© 2009 Hewlett-Packard Development Company, L.P.
The information contained herein is subject to change without notice.

2. Agenda
• Spotřebiče pro enterprise datové sklady (EDWH)
• Principy návrhu datového skladu na platfrormě
„EDWH spotřebičů“
• Představení Operational DWH
• Architektura MPP datového skladu pro Operational
DWH
27 November
2 2009

3. Data Warehouse
Appliances
© 2009 Hewlett-Packard Development Company, L.P.
The information contained herein is subject to change without notice.

4. DWH appliances
• Provide:
• Systems that are packaged (tightly integrated stack, bundled, balanced, pre-
tuned, pre-installed; single support contact) and optimized for BI workloads
• Low maintenance and support through a single source
• Fast installation
• Integrated management and automated system administration. Increased
functionality can not mean increased administrative complexity.
• Easy incremental expansion
• Guaranteed performance for specific purposes, use of new technologies to
drive high performance
• Lower TCO and faster ROI
• Have established market acceptance
• 10TB or more in first phase is common today
• Have begun to host EDWs
• Satisfy real-time, on-demand, and/or Operational BI requirements outside of
EDW
• 38% of TDWI research survey respondents have deployed, are currently
evaluating, or plan to evaluate soon

5. Data Warehouse Appliance Pros
What do you think is the leading benefit of
a data warehouse appliance?
6% = Other
6% = Low cost
8% = Easy incremental expansion
39% = Pre-tuned for
11% = Fast installation data warehousing
12% = Reduced system integration
18% = Fast query performance
Sources: TDWI Tech Survey, August 2005, 119 responses
TDWI Tech Survey, February 2007, 112 responses

6. Traditional data warehouse approach
Physical Index
Analyze Load Query Ongoing
database and
data data data tuning
design aggregate
• Performance is dependent on getting a good
physical design
• Time to market with new data is limited by skills
and resources
• Query performance is poor when queries don’t
take advantage of the design (no index scans)

7. Neoview platform performance
Load Query
data data
• High-power-to-data ratio can operate without database
tuning
− Load a TB in 1 hour
− Scan a TB in 30 seconds
• Neoview features take performance beyond scanning
− Next-generation optimizer can resolve complex queries efficiently
− pMesh dual switch fabrics provide massive bandwidth between
nodes
− “Skewbusting” technology to resolves the traditional skew issues
of MPP system

8. Simply put…
• Overpowering a workload with inexpensive power
has many benefits
− The ability to perform queries that no one anticipated
− “Load-and-go” simplicity of design
− Reduced indices, materialized views, etc., to manage
and tune
− Enough power to quickly drop, reload, and restructure
tables

9. What the appliance model is not good
at…
• Random I/O
− Scan a 1 TB table to find 1,000 rows using 256 inexpensive
processors in 30 seconds
− Same access with an index takes a fraction of a second using only
one of the 256 inexpensive processors
• Common aggregations
− Brute force does wonders for aggregating data on the fly so that
you don’t need to prebuild materialized views
− The same brute force can build a MV quickly, reducing CPU
consumption 1,000x at runtime
• Both limit concurrency
− How many times can one table scan in a day?

10. Beyond the database appliance with
the Neoview platform
Improve 10–1,000x faster
Load Query
database
data data
design Much higher concurrency
Performance repository
• Outperforms a pure appliance with little or no design
decisions
• Permits you to “graduate” tables to a more enhanced
design that supports extremely high concurrency
• Mixed workload support allows both optimized and
nonoptimized workloads to coexist

11. Operational DWH
© 2009 Hewlett-Packard Development Company, L.P.
The information contained herein is subject to change without notice.

12. Operational BI
Know your customer and Timely, operational decisions
Call Operating
partners center priorities • Real-time analytics
• 1000s of concurrent users • 24 x 7 availability
• Single version of the truth
Self- Market
service signals
Rich Responding
interactions to business
across all events as
touch Next they
points occur
Suppliers generation Real-Time
streams
EDW
Leveraging information as a
strategic asset
Richer data inputs Web
• Large data volumes
• Complex, mixed workloads
Operational Unstructured
System data
12

13. Performing multiple decisions by a large
number of users characterizes operational
BI
High
Strategic Operational
Business Business Intelligence
Intelligence
Collective
impact of
decisions
Tactical Business Tactical Business
Intelligence Intelligence
Low
Low Number of decisions High
13

14. The evolution of data warehouses in
operational BI environments
Traditional BI Operational BI
Back room analysis Built into business processes
Reporting Automating actions
Offline batch updates Continuous online updates
Departmental data marts Enterprise-wide resource
Few users doing strategic Thousands of users performing many
analysis types of tasks
Availability not critical Mission-critical
14

15. HP Neoview & Operational Business Intelligence –
Real-time insight for your business
• Enterprise data warehouse, for large enterprise operational needs
• Scales to thousands of users, terabytes of data
• Rapidly deployed, easily managed, compatible with existing BI tools
• Integrated solution, HP innovation and flexible- standards-based
components
Data Integration Enterprise Data Warehouse Query Tools
HP Neoview
Integrated
Real time Concurrent
Updates Hardware Users
OS
DBMS
Mixed
Workloads
15

16. Architected for availability, scalability,
and performance
• Shared-nothing MPP
− Each processor a unit of parallel work
• Database virtualization
BI client
− Data transparently hashed across all disks
• Parallel query execution
− Queries divided into subtasks and executed in
parallel with results streamed through memory
• Real-time data warehousing
− Mixed workload & transactional heritage
ETL clients
• Unrivaled availability
− Continuously available in spite of any single
point failure; online database operations
• Extreme processing power
− 1 Intel® Itanium® processor to 2 RAID 1
volumes
16

17. Neoview is designed
for changing customer requirements
SLA: 20 min/2 hours SLA: 5 Seconds
Neoview: 1.6sec
Neoview: 2min to 46min
Report
Analytical queries
queries 300 concurrent
Neoview: 0.5sec
1.85 million
120 concurrent
queries
220 queries SLA: 2
Tactical-1
320 concurrent Seconds
SLA: 10 min Online 3.6 million
queries
ingest
Neoview: 7min 7.5m rows of data 91 billion rows
every 10 min
of data (20TB)
Tactical-2
Adhoc 350 concurrent
surprise 6.4 million
4 concurrent queries
8 queries
Tactical-3 SLA: 2 Seconds
400 concurrent
13.4 million Neoview: 200ms
queries
All workloads run
concurrently SLA: 200 ms
Neoview: 167ms

18. Architecture of an MPP
DWH – HP Neoview
© 2009 Hewlett-Packard Development Company, L.P.
The information contained herein is subject to change without notice.

19. Neoview segment architecture
Blade Blade
Node 1 Node 2 Node 3 Node 4 Node 5 Node 6 Node 7 Node 8 Node 9 Node 10 Node 11 Node 12 Node 13 Node 14 Node 15 Node 16
P01,P02 P03,P04 P05 P06 P07,P08 P09,P10 P11,P12 P13,P14 P15,P16 P17,P18 P19,P20 P21,P22 P23,P24 P25,P26 P27,P28 P29,P30 P31,P32
B27,B30 B05,B28 B01,B31 B02,B32 B03,B06 B04,B13 B07,B09 B08,B10 B11,B14 B12,B21 B15,B17 B16,B18 B19,B22 B20,B29 B23,B25 B24,B26
X Fabric Y Fabric
CS PS PS CS
• 16 nodes per P01 P14 • RAID1 (mirrored)
segment disk protection
• Dual active (X,Y) M01 M14 • Active reading from
interconnect both RAID1 copies
fabrics P15 P28
• Separate controller
• Multiple fat I/O
pipes writes for integrity
M15 M28
• End-to-end disk
• Dual cluster
switches for P29 P42
checksum integrity
inter-segment
I/O M29 M42
19

20. Neoview multi-segment architecture
Neoview Segment Neoview Segment
FT Clustered Mesh Fabric 1 to 16 segments
Neoview Segment Neoview Segment
• Active dual fault tolerant fabrics
• Multi-layered clustering
(>128p)
• 500 MB/sec dedicated links
• Each segment adds bandwidth
• Cross sectional bandwidth up
20 to 128 GB/sec

21. Highly Parallelized Database
• The key is transparently hashed to identify data
Table Table Table placement
A B C
• Balanced data distributions across all disks
• Balanced SQL execution across all processors
• Table, index, and materialized view support
Partitioning key
Hash of partitioning key
21 -
6 - 21

22. Neoview Shared Nothing Architecture
22

23. Indexed Clustering for Performance
− Data indexed for fast access by the clustering key
− Clustered data for fast sequential access
Partitioning determined
Hash by Data indexed and clustered
by hash of partitioning
order # by clustering key
key
Cluster by
Order order date, order number
Cluster by
Line item
order date,
order number,
item number
Hash by
order #
23 -
6 - 23

24. Thank you
Questions?
© 2009 Hewlett-Packard Development Company, L.P.
The information contained herein is subject to change without notice.

25. Co-locating Index and Base Table Data
− Eliminates cross-processor messaging overhead
− Fast and efficient indexing for query speed-up
Hash by
order #
Index on Cluster by
line item item number
Line item Cluster by
order date,
order number,
item number
Hash by
order #
25 -
6 - 25

26. Parallel UOW drives MPP performance
• Measured performance
Itanium 2 processor Itanium 2 processor − Scan: 286 MB/sec/CPU
2.34 GB/sec/segment
− Ingest: 1MB/sec/CPU
SCAN to 256 CPUs using 3 loaders
− Extract: 2.5MB/sec/CPU
to 64 CPUs
Data
Data Data
Data
Data
− Insert: 1MB/sec/connection
Data
Management Management
Management
management Management
management
at 128 connections
− Fetch: 1.5MB/sec/connection
at 128 connections
LDV1-P LDV1-B LDV3-P LDV3-B
RAID 1 RAID 1
26