Your SlideShare is downloading. ×
Big Data i CSC's optik, CSC Representative
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×

Saving this for later?

Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime - even offline.

Text the download link to your phone

Standard text messaging rates apply

Big Data i CSC's optik, CSC Representative

1,217
views

Published on

Præsentation fra Smarter Business 2012

Præsentation fra Smarter Business 2012

Published in: Technology, Business

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
1,217
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
44
Comments
0
Likes
0
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. CSC REFERENCE CASE IBM NETEZZA John V FabienkeChief Architect, Nordic Architecture Practice
  • 2. Agenda CSC Enterprise Business IntelligenceI. capability overview 3II. Zürich Netezza case study 7III. CSC Nordics Netezza PoC capability 22
  • 3. Capability Overview
  • 4. CSC Enterprise Business Intelligence Framework CSC Proprietary and Confidential 9/18/2012 9:02 AM 4
  • 5. CSC Enterprise Performance and Information ManagementPractice (EPIM) EPIM Practice (40+ SMEs) BI Capability Data Management Capability GLOBAL GOVERNANCE PRACTICE Strategy BI SOLUTION and SERVICE DATA SOLUTION and SERVICES Tower of Competency Tower of Competency Planning & Knowledge Management Data Solutions GLOBAL BI Solution Integration OFFERINGS Analytics Solutions Architecture & Disruptive Technologies Solutions BI Services Data Services Analytics Services Integration Services GLOBAL EMEA Enterprise Information Capability Group (120+ SMEs) DELIVERY BI Capability Data Management BI Services Data Services Analytics Services Integration Services Offshore Capability (800+ SMEs) BI Capability Data Management BI Services Data Services Analytics Services Integration Services CSC Proprietary and Confidential 9/18/2012 9:02 AM 5
  • 6. CSC’s Global Business Intelligence Alliances CSC Proprietary and Confidential 9/18/2012 9:02 AM 6
  • 7. Zürich Case StudyIBM Netezza solution
  • 8. Netezza OverviewBusiness problem to solve• Client‟s current mainframe and DB2 based infrastructure that their financial close processing, data warehouse processing, regulatory reporting and Business Intelligence reporting runs on was over 4 years old• Client was at full capacity on multiple fronts and the DB2 environment required a significant investment to upgrade• With the introduction of the U2Z program data into client‟s month-end close the system was no longer able to meet SLAs for the month-end close process• Even after 9 months of tuning the DB2 environment SLAs were still unable to be met CSC Proprietary and Confidential 9/18/2012 9:02 AM 8
  • 9. Netezza OverviewWhat scenarios were considered?• Netezza was suggested as a simple, high-performance solution• Teradata was included due to their being the long-term industry leader• IBM‟s traditional offering was considered as well• Oracle was dismissed due to TCO, lack of existing footprint and skill sets CSC Proprietary and Confidential 9/18/2012 9:02 AM 9
  • 10. Netezza OverviewPoC performance results• Netezza and Teradata reporting consistently in the 10x – 200x performance improvement range across the board• Analytic run queries ran 150 to 18,000 times faster than the current DB2 platform on both Netezza and Teradata• Compared to DB2 performance differences between Teradata and Netezza were negligible CSC Proprietary and Confidential 9/18/2012 9:02 AM 10
  • 11. Netezza OverviewReference calls• Teradata is usually used as an Enterprise Data Warehouse “workhorse”• Netezza is usually used as a reporting engine• Cost was the major factor in choosing Netezza over Teradata ‒ A few references had purchased both platforms for contract negotiations• Both platforms returned over 95% of user queries within 60 seconds CSC Proprietary and Confidential 9/18/2012 9:02 AM 11
  • 12. Netezza OverviewNetezza was chosen• Teradata‟s version 13 was not compatible with the client‟s version of Business Objects and Informatica• Teradata ran out of time to provide data points used to measure the month-end processing performance• The Netezza platform was a less complex solution than Teradata ‒ Better fit for client‟s out-sourced model CSC Proprietary and Confidential 9/18/2012 9:02 AM 12
  • 13. Netezza Overview Slice of User Data Disk Enclosures Swap and Mirror partitions High speed data streaming SQL Compiler Query Plan SMP Hosts Optimize Admin Snippet Blades™ (S-Blades™) High-performance database engine streaming joins, aggregations, sorts, etc. CSC Proprietary and Confidential 9/18/2012 9:02 AM 13
  • 14. EDW Architecture Overview Application Architecture: EDW INPUT OUTPUT DMS Mainframe CESAR CIID EDW on DB2 COS Informatica RADIUS/PALM EDW BO ZNAW ZDW DICE ZEA C Class Work ZDW Bench/EDB Netezza CORE WORKBENCH BIW Farmers nzload CSC Proprietary and Confidential 9/18/2012 9:02 AM 14
  • 15. Netezza OverviewPoC Query TimingsBusiness Objects Reports & SAS queries DB2 Timings (sec) NZ Timing ROWSComm Analysis Template – Dec 2 „08 - 23 113.516Comm Analysis Template – Dec ‟08 - AON 227 60 56.134HPY AY Loss Ratio Analysis 1.380 51 190.688Policy Effective year Template 2.460 14 888.324Comm Analysis Template – Dec 2 „08 7.166 41 341.234Comm Analysis Template – Dec 2 „08 - Marsh 2.150 15 44.308Direct Written Premium - 2006 300 118 1.322.590Construction Expiration List – Date Promt 2.024 22 29.510Domestic Linked to Customer with Foreign 2009 1.800 17 20.597International Report – Final – Dec Results 2.100 36 29.755NSBs Large Account Renewals 660 40 3.567Comm Analysis Template – Dec 2 „08 - AJG 7.826 15 33.263Comm Analysis Template – Dec 2 „08 - Wells Fargo 6.854 13 25.290HPA Comm Analysis 5.971 16 4.182Comm Analysis Template – Dec 2 „08 - HUB 2.840 14 15.508Producer Code to Dist ID - 17 29.780Query 1 38.317 43 1.480.812Query 2 54.707 5 4.169 CSC Proprietary and Confidential 9/18/2012 9:02 AM 15
  • 16. Netezza OverviewDB2 and Netezza landscape • DB2 has not gone away, a few applications still live on DB2 • DB2 environment was moved to InfoSphere Change Data Capture (CDC) • Informatica environment is completely on DB2 • Working on shrinking the DB2 environment • Netezza replication to Reston can be accomplished via Netezza utilities ‒ NZ_MIGRATE ‒ Data is sent compressed ‒ 2 TB per hour throughput CSC Proprietary and Confidential 9/18/2012 9:02 AM 16
  • 17. Netezza OverviewHow have things improved with Netezza? • Month-end processing is down from 9 days to 3 days ‒ Within 6 months client anticipate to have this down to 2 days • ADS processing does not stop for month-end (no catch-up time) • Queries can run on the Netezza platform during month-end processing • SAP feed down from day 3 to day 1 of Month-end processing • Netezza load and unload speed of 2 TB per hour • Developer mindset change from sequential to set processing • Business Object query time has dropped significantly ‒ Average BO query on DB2 took over 3,100 seconds ‒ Average BO query on Netezza now runs in 6.5 seconds CSC Proprietary and Confidential 9/18/2012 9:02 AM 17
  • 18. Netezza OverviewHow have things improved with Netezza? • On November 18, 2011 16.172 queries ran on the TF12 ‒ Largest number of queries run on a single day ‒ Average query time was 0.83 seconds ‒ Longest query time was 47 minutes (Informatica) ‒ Peek of 8,243 queries completed per hour ‒ Host CPU utilization never went above 10% ‒ SPU (blade) utilization never went above 17% • EDW conversion PoC ‒ 4.1 million updates ran in 18 seconds ‒ Executed on TF6 ‒ Informatica & post session task CSC Proprietary and Confidential 9/18/2012 9:02 AM 18
  • 19. Netezza OverviewBumps along the way • Netezza and Linux upgrades have caused issues • Netezza off-hours support is poor • Technical Account Manager (TAM) knowledge is lacking • Integration error debugging knowledge is lacking • 48 concurrent query limit on each Netezza appliance have caused issues • Service provider requirements not accommodated for ‒ Features lacking for guaranteeing SLA‟s ‒ Limited multi-tenancy features • Netezza documentation is lacking or sometimes incorrectLight at the end of the tunnel • IBM will bring stability to these issues • Client is member of the Netezza Advisory Board to escalate issues CSC Proprietary and Confidential 9/18/2012 9:02 AM 19
  • 20. Netezza OverviewWhat’s Next? • More projects being developed on Netezza • Plan Migration of ZEA to Netezza • Look at other systems to migrate (EDW + CDW) • Netezza upgraded in 2012 ‒ 2x performance ‒ 5x or better compression ‒ 4TB/hour load and unload speeds ‒ Concurrent Query limit raised to 100 ‒ Smart Caching ‒ Query Mix: large percentage of short, tactical queries ‒ Granular Zone Maps (potential 24x reduction in reads) CSC Proprietary and Confidential 9/18/2012 9:02 AM 20
  • 21. Netezza OverviewWhat’s Next? • Informatica Pushdown Optimization (PDO) ‒ Informatica mapping converted to SQL ‒ SQL is submitted to Netezza ‒ All data stays on Netezza ‒ All processing happens within Netezza • PDO example – Creation of Combined Mart ‒ Before: DB2 only method – 18 to 20 hours ‒ After: Netezza conversion – 6 to 8 hours ‒ Future: Netezza with PDO – expected 1 to 2 hours CSC Proprietary and Confidential 9/18/2012 9:02 AM 21
  • 22. Netezza PoC capability CSC Nordics
  • 23. Scandihealth POC Setup Selected Near Real Tables Push Time Xfer Reports Changes Mainframe + DB2 CDC server Netezza Cognos + Targit CSC Proprietary and Confidential 9/18/2012 9:02 AM 23
  • 24. Business caseOptimize hospital beds• Calculate availability of hospital beds more frequently in order to optimize utilization• Utilization of beds is an important KPI for a hospital• Before – At best the customers would get calculations once a day• After – Calculations performed every 15 minutes CSC Proprietary and Confidential 9/18/2012 9:02 AM 24
  • 25. FindingsAll business requirements met• Data is updated in near real-time• Queries can now be very frequent• No incremental MIPS cost for queries and reports• The load CDC imposed on mainframe MIPS was insignificant CSC Proprietary and Confidential 9/18/2012 9:02 AM 25
  • 26. Thank You Q&A