• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
Business Intelligence Architecture
 

Business Intelligence Architecture

on

  • 9,177 views

Business Intelligence and data warehousing architecture and sizing methodology.

Business Intelligence and data warehousing architecture and sizing methodology.

Statistics

Views

Total Views
9,177
Views on SlideShare
9,084
Embed Views
93

Actions

Likes
14
Downloads
555
Comments
0

5 Embeds 93

http://www.slideshare.net 53
http://www.linkedin.com 21
http://www.techgig.com 11
http://blogs.sun.com 6
http://192.168.6.184 2

Accessibility

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    Business Intelligence Architecture Business Intelligence Architecture Presentation Transcript

    • Open for Business…
    • WHO AM I • Big Data / Analytics / BI & Cloud Solutions Specialist • http://www.linkedin.com/in/JulioPhilippe • Skills Architecture Business Intelligence IT Transformation Cloud Computing IT Solutions Management Mentoring Big Data Analytics Business Development Hadoop Datacenter Optimization Data Warehousing 2 Business Intelligence Architecture
    • « The data are not created relevant, they become though! » 3 Business Intelligence Architecture
    • KEY BUSINESS DRIVERS Manufacturing Telecommunications Information & Technology Internationalization Costs Reduction Reduce time to market Costs Reduction Banking & Finance Transportation & Travel Energy Regulations observance Increase risks management Costs Reduction Collecting and Sharing experiences Costs Reduction Increase refining capacity in traditional petroleum Investments in the renewable energy sector Cost Reduction Education & Research Pharmaceutical Media & Entertainment Create Intelligent Buildings Costs Reduction Improved lifecycle management of products Enhanced information dissemination Focus on products that have the best chance of getting to market Costs Reduction Government Retail Healthcare Provide timely access to decision support Accomplish more work with fewer resources Costs Reduction Need to manage profitability and control expenses Costs Reduction Accelerating Employers-Led Initiatives New Consumers-centric Technologies Costs Reduction Manufacturer / Supplier collaboration Reengineering Sales Reengineering Distribution Intensified focus on customer Costs Reduction 4 Business Intelligence Architecture Evolving consumer behaviors Costs Reduction
    • KEY IT DRIVERS • Backlog of report requests and need to provide self- service access to intelligence • Resource constraints inhibit ability to broaden BI deployment to meet demand • Complex IT environments – Disparate data, applications, and integration challenges • Push to reduce costs and standardize tools 5 Business Intelligence Architecture
    • VALUE PROPOSITION • Scale without impacting the response time Scalability / Performance Costs / Complexity Flexibility • To handle the data volume and throughput demands of users • Scalable infrastructure that supports growth • Costs effective to purchase, support and manage and proven ROI • Effortless to design and implement • Leverage your existing investment • Server, Storage and network integration • Dynamic infrastructure with virtualization technologies • BI solutions able to provide information to users when they need it Availability 6 • Extensive non-disruptive services and upgrade Business Intelligence Architecture
    • DATA MODEL • OLTP Model - Transactional - Third Normal Form Product Id-product Name Region Sale Tables Customer Id-product Id-company Price Margin Creation date Id-region Name Id-customer Name Extraction Transformation Load Product • OLAP Model - Business Intelligence Star Schema Time Management Aggregation Id-product Name Id-period Name Sale Dimension Tables Region Id-region Name 7 Period Business Intelligence Architecture Id-product Id-customer Id-period Id-region CA Margin Fact Table Customer Id-customer Name
    • MANAGEMENT, ADMINISTRATION & SUPPORT • Fast Predictable Performance - Monitor and ensure database performance with automated diagnostics and tuning • Lower Ongoing Costs - Drive down the cost of operations with automated change and configuration management • The Fastest Time to Value & Lowest Risk - Automate testing of patches, changes and upgrades while keeping data secure 8 Business Intelligence Architecture
    • NEW BUSINESS INTELLIGENCE Compute Trends Storage Trends New Analytics New Data Structure (Massively Parallel Processing, Algorithms…) Distributed File Systems, NoSQL Database, NewSQL…) Logical data warehouse Master/Slave Enterprise data warehouse Objects storage Multi-structured Data Master/Master General purpose data warehouse Proprietary and dedicated data warehouse Distributed File Systems OLTP is the data warehouse Master Data Management, Data Quality, Data Integration 9 Business Intelligence Architecture Federated/ Sharded
    • MOVING COMPUTATION TO STORAGE General Purpose Storage Servers • Combine server with disks & networking for reducing latency • Specialized software enables general purpose systems designs to provide high performance data services Moving Data processing to Storage Legacy Emerging Next Gen. Application Application Application Data Processing Data Processing Metadata Mgmt Network Data Processing Metadata Mgmt Storage Metadata Mgmt Storage Storage Storage Array (SAN, NAS) 10 Servers Business Intelligence Architecture
    • DATA WAREHOUSE • Data Warehouse Appliances – EMC Greenplum • SQL Database • Massively Parallel Processing – Microsoft Parallel Data Warehouse • Hadoop Connectivity – IBM Netezza • Column-Oriented database – Oracle Exadata – SAP HANA – ParAccel Analytic Database • In-Memory database – Teradata – HP Vertica 11 Business Intelligence Architecture
    • BI SIZING METHODOLOGY Data Volume Concurrent Users Extract volume Complexity Concurrent Requests Data Processing Complexity S/W Components Batch Time Range DB Structure Costs Users Time Range Benchmarks Results A balanced configuration from Processors to Disks CPUs 12 Memory I/O Disks Business Intelligence Architecture
    • KEY QUESTIONS • What is the Web servers sizing ? – http server, proxy server, application server, portal server, LDAP server • What is the ETL server sizing ? • What is the Database servers sizing ? – Staging area, data warehouse, data marts • What is the data volume ? – Index, metadata, aggregates, Temp space… • What is the backup server sizing – Drives • What are the key elements affecting the sizing ? – User, data, timing, SLA… 13 Business Intelligence Architecture
    • DATA FLOWS DIAGRAM Operational Data Source ETL Flow Data Processing complexity ETL time range h1 - h2 DWH SA %Index WORK TEMP, LOG... %index, %metadata Data Processing Complexity ETL Flow DATA MARTS % Aggregats re-building External Flow %index, %aggregates, %axis Users time range h1 – h2 * Usable data, indexes, aggregates and metadata included... n1 Low users 14 n2 Medium users Business Intelligence Architecture n3 High users
    • DATA VOLUME QUALIFICATION • Software : Oracle DB , Sybase DB, DB2, MS SQL Server… • Business data is the data source resulting from the operational systems (CRM, RH, BILLING, PURCHASES, SCM...) • Usable data is the result of business data and technical data according database organization, like indexes, aggregates, metadata, axis, indicators and data work • The database is structured on 3 levels - Staging Area is the storage area for data validation - Data Warehouse is the storage area for business data, data details and metadata - Data Marts is the storage area for business data, axis, indicators and aggregates 15 Business Intelligence Architecture
    • USERS QUALIFICATION • Software : SAP BO Explorer, Oracle BI, SAS analytics… • Named user have the rights to use Business Intelligence system • Concurrent users have access to BI system resources simultaneously • Users are classified by typology - Low users view predefined and static reports - Medium users navigate within reports, do slicing and dicing, but usually hit aggregates - High users run ad-hoc queries with a high probability of full table scans • Operational period (ex. from Monday to Friday) • Associated time frame (ex. 08:00 AM - 07:00 PM) 16 Business Intelligence Architecture
    • DATA INTEGRATION QUALIFICATION • Software : Oracle Data Integrator, PowerCenter Informatica… • ETL processing are classified by typology : - Simple processing represents simple calculations and concatenations - Medium processing represents average calculations, medium concatenations - High processing represents heavy calculations, statistical, complex algorithms and heavy concatenations • Data retention duration (number of years, months or days) • Operational period (ex. from Monday to Friday) • Operation frequency (daily, weekly, monthly) • Time range (ex. 08:00 PM to 06:00 AM) 17 Business Intelligence Architecture
    • BI ARCHITECTURE Primary Site Secondary Site Apps Servers LAN Switches DB Servers SAN Switches Storage Array • • • • • 18 Data Replication SQL based High availability Enterprise database Right design for structured data Hardware storage (SAN, NAS, DAS) Business Intelligence Architecture Storage Array
    • NEW BI ARCHITECTURE Network Switches Apps Servers • • • • 19 Data Nodes – Compute and storage integrated Not only SQL based High scalability, availability and flexibility Compute and storage in the same box for reducing the network latency Right design for semi-structured and unstructured data Business Intelligence Architecture
    • THANK YOU