ANALYSIS SERVICES  ANALYSIS SERVICES  ANALYSIS SERVICES  ANALYSIS SERVICES  ANALYSIS SERVICES ANALYSIS SERVICES ANALYSIS SERVICES ANALYSIS SERVICES ANALYSIS SERVICES  ANALYSIS SERVICESSQL SERVER SQL SERVER SQL SERVER SQL SERVER DATA MINING  DATA MINING DATA MINING DATA MINING DATA MINING DATA MINING DATA MINING DATA MININGINTEGRATION SERVICES INTEGRATION SERVICES INTEGRATION SERVICES   INTEGRATION SERVICES  INTEGRATION  SERVICES  INTEGRATION SERVICES Business Intelligence Architecture and Conceptual FrameworkINTEGRATION SERVICES INTEGRATION SERVICES INTEGRATION SERVICES   INTEGRATION SERVICES  INTEGRATION  SERVICES  INTEGRATION SERVICES ANALYSIS SERVICES  ANALYSIS SERVICES  ANALYSIS SERVICES  ANALYSIS SERVICES  ANALYSIS SERVICES ANALYSIS SERVICES ANALYSIS SERVICES ANALYSIS SERVICES ANALYSIS SERVICES  ANALYSIS SERVICESSQL SERVER  SQL SERVER  SQL SERVER SQL SERVERDATA MINING  DATA MINING DATA MINING DATA MINING DATA MINING DATA MINING DATA MINING DATA MINING
About MeSlava KokaevGroup Leader at Boston Business Intelligence User GroupPrincipal BI Developer/ Architect at Industrial Defendervkokaev@bostonbi.orgwww.bostonbi.org/blog.aspx
Agenda
Drive Corporate PerformanceGiving a purpose to business intelligence“You can’t manage what you can’t measure. You can’t measure what you can’t describe”Robert Kaplan and David NortonAuthors of “The Balanced Scorecard”
Enterprise BI Strategy and VisionTo improve organizational KeyBusiness Processes and Operations by providing critical to business Information at RightTime and RightFormat  to all levels of employees.Goals
Understanding The Business System Microsoft BI PlatformBusiness Intelligence SystemManagement SystemEnterpriseData WarehouseSystemBusinessAnalysisSystemOperational System
Microsoft’s BI platformCOLLABORATIONCONTENT MANAGEMENTSharePoint ServerSEARCHReportsDashboardsExcel WorkbooksAnalyticViewsScorecardsPlansEND USER TOOLS & PERFORMANCE MANAGEMENT APPSExcelPower PivotBI PLATFORMSQL Server Reporting ServicesSQL Server Analysis ServicesSQL Server DBMSSQL Server Integration Services
IntegrateReportAnalyzeCovered in Module 03Covered in Modules 06, 07, and 08Covered in Modules 04, 05, and 07SQL Server 2008 BI Platform ComponentsData acquisition from source systems and integration
Data transformation and synthesis
Data enrichment, with business logic, hierarchical views
Data discovery via data mining
Data presentation and distribution
Data access for the massesBusiness Intelligence Conceptual FrameworkSource SystemETL SystemDW  SystemDA System
Sales Business ProcessBalance ScorecardsSales corrections and ImprovementPlan SalesSales QuotaStock DataSale Orders (Facts /Measures)Resellers SalesReseller (Dimension)Sales ResultMonitor SalesSales  SummarySales  TransactionAnalyze SalesSQL Server DBSales RepresentativeSales Manager
Bike FactoryTiresFactoryStill FactoryAdventureWorksHeadquarterPlastic FactoryColor FactoryAccessory FactoryWarehouseResellers
ETL SystemOLTPSSISDWOLAP
Enterprise Data Source StructureCall CenterWeb AppsCRMInventoryFinanceData WarehouseERPHR
ETL SystemExtract, Transform, Load (ETL)ETL is a process in Business Intelligence that:Extract data from the source systemsTransform the data to convert it to a desired stateLoad the data into the data warehouse
ETL Framework and Logical ArchitectureCheck System StateETL PackagesExtract DataFile SystemLoad StagingExtract from StagingOLTPSTAGING SchemaSend NotificationLog ETL Process Transform DataLoad DimensionsETL SchemaLoad FactsDWH SchemaDatabaseProcess CubeCube
ETL Benefits ProductivityCoding ETL scripts using a metadata-driven graphical tool with built-in data cleansing and transformation functions is generally faster than hand coding. Mappings, extract rules, cleansing rules, transformation rules, aggregation logic and loading rules are generally handled as separate objects in an ETL tool. This means that you can change one object in an ETL "string" without affecting the other objects. For example, you can change the loading logic for a particular target table (say, from direct insert to generating a flat loader file) without affecting the cleansing and/or transformation logic for that table. This compartmentalization eases maintenance, and reduces the need for retesting. Objects in an ETL tool (e.g., transformation rules) can be reused. ETL tools facilitate impact analysis when modifying or enhancing a data warehouse. Methodology•ETL tools impose a certain level of structure, rigor, and consistency in your development approach. Documentation•The meta data trapped by an ETL tool graphically documents source and target database structures, mappings (a.k.a. "data genealogy"), cleansing rules (a.k.a. "business rules") and transformation rules.
GoalsImplement many routines quickly, with limited developer resourcesReliability and AccuracyAbility to introduce \ modify \ remove transformation rulesAbility to maintain and apply logical business rules on dataSupport for scheduled and user-initiated package execution
Relational Data Warehouse Architecture and dimensional model ?
Star Schema vs. OLTPETL
Fact and Dimensions together or “Star Schema” Database
DimensionsDimensions are the foundation of the dimensional model, describing the objects of the business, such as employee, product, customer, service.They describe the surrounding measurement events. The business processes (facts) or actions of the business in which the dimensions participate. Each dimension table links to all the business processes in which it participates. A single dimension that is shared across all these processes is called a conformed dimension.
Fact TablesEach fact table contains the measurements associated with a specific business process. A record in a fact table is a measurement, and a measurement event can always produce a fact table record. These events usually have numeric measurements that quantify the magnitude of the event, such as quantity ordered, sale amount, or call duration. These numbers are called facts(or measuresin Analysis Services).The key to the fact table is a multi-part key made up of a subset of the foreign keys from each dimension table involved in the business event.
Reviewing Star Schema Benefits Transforms normalized data into a simpler modelDelivers high-performance queriesDelivers higher performing queries using Star Join Query OptimizationUses mature modeling techniques that are widely supported by many BI toolsRequires low maintenance as the data warehouse design evolves
Vendors, Suppliers,Channel partnersCustomersBusiness partnersMonitoring Systems Analysis SystemsBusiness Processes  and Operations  Controlling SystemsStrategy and Planning SystemsIT providersFinancial service providersEnterprise Business Analysis System
Business Process System CycleKaizen BPM
Abstract Functional Business ModelIDEF0 Modeling NotationFeedback (Improvement)PlanPlans, Business Rule and KPIInput DataProcess Output (Facts /Measures)DoResourcesCheckResult DataActData MiningReporting ServicesSQL ServerAnalysis Services
Business Frameworks and  Logical Business Levels
ON-LINE Analytical ProcessingMultidimensional databases are also called online analytical processing (OLAP) databases and…Contain structures optimized for rapid ad hoc information retrievalPre-calculate and store aggregated valuesInclude calculation engines for fast, flexible transformation of base dataAre designed to reveal business trends and statistics not directly visible in the data retrieved from a data warehouseData mining models discover patterns in data, typically for prediction analysisProductAssociationSalesFinanceProduction
Understanding Cube Structure116499518931455194513769451553187412451576445147918741245295415751479157630071575232229541383300724553007Accessories1654Australia645136521456459882012Product  Line2012CountryUnited States845Bikes7457002751082234905Canada905345ClothingQuarter 1761875Quarter 2745745FranceComponentsSemester 1Quarter 3Quarter 4TXSemester 2MACalendar Year - 2009COVTState
Data Visualization SystemClient access and distribution mechanisms can include:Static report viewers and browsersAd hoc query toolsReport writersModeling applicationsScorecard applicationsPortals and dashboardsDelivering data is a process of continuous business improvement:MonitorAnalyzePlan
Data VisualizationBasic Business IntelligenceCustom Business Intelligence Web PortalPowerPivot for SharePoint and Self Services BI
Basic Intelligence Static ReportsReport ServerCUBEExport Report to ExcelTechnology:ASP.NET
SQL Server 2008 R2Business User Tools:Excel 2007

Bi Architecture And Conceptual Framework

  • 1.
    ANALYSIS SERVICES ANALYSIS SERVICES ANALYSIS SERVICES ANALYSIS SERVICES ANALYSIS SERVICES ANALYSIS SERVICES ANALYSIS SERVICES ANALYSIS SERVICES ANALYSIS SERVICES ANALYSIS SERVICESSQL SERVER SQL SERVER SQL SERVER SQL SERVER DATA MINING DATA MINING DATA MINING DATA MINING DATA MINING DATA MINING DATA MINING DATA MININGINTEGRATION SERVICES INTEGRATION SERVICES INTEGRATION SERVICES INTEGRATION SERVICES INTEGRATION SERVICES INTEGRATION SERVICES Business Intelligence Architecture and Conceptual FrameworkINTEGRATION SERVICES INTEGRATION SERVICES INTEGRATION SERVICES INTEGRATION SERVICES INTEGRATION SERVICES INTEGRATION SERVICES ANALYSIS SERVICES ANALYSIS SERVICES ANALYSIS SERVICES ANALYSIS SERVICES ANALYSIS SERVICES ANALYSIS SERVICES ANALYSIS SERVICES ANALYSIS SERVICES ANALYSIS SERVICES ANALYSIS SERVICESSQL SERVER SQL SERVER SQL SERVER SQL SERVERDATA MINING DATA MINING DATA MINING DATA MINING DATA MINING DATA MINING DATA MINING DATA MINING
  • 2.
    About MeSlava KokaevGroupLeader at Boston Business Intelligence User GroupPrincipal BI Developer/ Architect at Industrial Defendervkokaev@bostonbi.orgwww.bostonbi.org/blog.aspx
  • 3.
  • 4.
    Drive Corporate PerformanceGivinga purpose to business intelligence“You can’t manage what you can’t measure. You can’t measure what you can’t describe”Robert Kaplan and David NortonAuthors of “The Balanced Scorecard”
  • 5.
    Enterprise BI Strategyand VisionTo improve organizational KeyBusiness Processes and Operations by providing critical to business Information at RightTime and RightFormat to all levels of employees.Goals
  • 6.
    Understanding The BusinessSystem Microsoft BI PlatformBusiness Intelligence SystemManagement SystemEnterpriseData WarehouseSystemBusinessAnalysisSystemOperational System
  • 7.
    Microsoft’s BI platformCOLLABORATIONCONTENTMANAGEMENTSharePoint ServerSEARCHReportsDashboardsExcel WorkbooksAnalyticViewsScorecardsPlansEND USER TOOLS & PERFORMANCE MANAGEMENT APPSExcelPower PivotBI PLATFORMSQL Server Reporting ServicesSQL Server Analysis ServicesSQL Server DBMSSQL Server Integration Services
  • 8.
    IntegrateReportAnalyzeCovered in Module03Covered in Modules 06, 07, and 08Covered in Modules 04, 05, and 07SQL Server 2008 BI Platform ComponentsData acquisition from source systems and integration
  • 9.
  • 10.
    Data enrichment, withbusiness logic, hierarchical views
  • 11.
  • 12.
  • 13.
    Data access forthe massesBusiness Intelligence Conceptual FrameworkSource SystemETL SystemDW SystemDA System
  • 14.
    Sales Business ProcessBalanceScorecardsSales corrections and ImprovementPlan SalesSales QuotaStock DataSale Orders (Facts /Measures)Resellers SalesReseller (Dimension)Sales ResultMonitor SalesSales SummarySales TransactionAnalyze SalesSQL Server DBSales RepresentativeSales Manager
  • 15.
    Bike FactoryTiresFactoryStill FactoryAdventureWorksHeadquarterPlasticFactoryColor FactoryAccessory FactoryWarehouseResellers
  • 16.
  • 17.
    Enterprise Data SourceStructureCall CenterWeb AppsCRMInventoryFinanceData WarehouseERPHR
  • 18.
    ETL SystemExtract, Transform,Load (ETL)ETL is a process in Business Intelligence that:Extract data from the source systemsTransform the data to convert it to a desired stateLoad the data into the data warehouse
  • 19.
    ETL Framework andLogical ArchitectureCheck System StateETL PackagesExtract DataFile SystemLoad StagingExtract from StagingOLTPSTAGING SchemaSend NotificationLog ETL Process Transform DataLoad DimensionsETL SchemaLoad FactsDWH SchemaDatabaseProcess CubeCube
  • 20.
    ETL Benefits ProductivityCodingETL scripts using a metadata-driven graphical tool with built-in data cleansing and transformation functions is generally faster than hand coding. Mappings, extract rules, cleansing rules, transformation rules, aggregation logic and loading rules are generally handled as separate objects in an ETL tool. This means that you can change one object in an ETL "string" without affecting the other objects. For example, you can change the loading logic for a particular target table (say, from direct insert to generating a flat loader file) without affecting the cleansing and/or transformation logic for that table. This compartmentalization eases maintenance, and reduces the need for retesting. Objects in an ETL tool (e.g., transformation rules) can be reused. ETL tools facilitate impact analysis when modifying or enhancing a data warehouse. Methodology•ETL tools impose a certain level of structure, rigor, and consistency in your development approach. Documentation•The meta data trapped by an ETL tool graphically documents source and target database structures, mappings (a.k.a. "data genealogy"), cleansing rules (a.k.a. "business rules") and transformation rules.
  • 21.
    GoalsImplement many routinesquickly, with limited developer resourcesReliability and AccuracyAbility to introduce \ modify \ remove transformation rulesAbility to maintain and apply logical business rules on dataSupport for scheduled and user-initiated package execution
  • 22.
    Relational Data WarehouseArchitecture and dimensional model ?
  • 23.
  • 24.
    Fact and Dimensionstogether or “Star Schema” Database
  • 25.
    DimensionsDimensions are thefoundation of the dimensional model, describing the objects of the business, such as employee, product, customer, service.They describe the surrounding measurement events. The business processes (facts) or actions of the business in which the dimensions participate. Each dimension table links to all the business processes in which it participates. A single dimension that is shared across all these processes is called a conformed dimension.
  • 26.
    Fact TablesEach facttable contains the measurements associated with a specific business process. A record in a fact table is a measurement, and a measurement event can always produce a fact table record. These events usually have numeric measurements that quantify the magnitude of the event, such as quantity ordered, sale amount, or call duration. These numbers are called facts(or measuresin Analysis Services).The key to the fact table is a multi-part key made up of a subset of the foreign keys from each dimension table involved in the business event.
  • 27.
    Reviewing Star SchemaBenefits Transforms normalized data into a simpler modelDelivers high-performance queriesDelivers higher performing queries using Star Join Query OptimizationUses mature modeling techniques that are widely supported by many BI toolsRequires low maintenance as the data warehouse design evolves
  • 28.
    Vendors, Suppliers,Channel partnersCustomersBusinesspartnersMonitoring Systems Analysis SystemsBusiness Processes and Operations Controlling SystemsStrategy and Planning SystemsIT providersFinancial service providersEnterprise Business Analysis System
  • 29.
  • 30.
    Abstract Functional BusinessModelIDEF0 Modeling NotationFeedback (Improvement)PlanPlans, Business Rule and KPIInput DataProcess Output (Facts /Measures)DoResourcesCheckResult DataActData MiningReporting ServicesSQL ServerAnalysis Services
  • 31.
    Business Frameworks and Logical Business Levels
  • 32.
    ON-LINE Analytical ProcessingMultidimensionaldatabases are also called online analytical processing (OLAP) databases and…Contain structures optimized for rapid ad hoc information retrievalPre-calculate and store aggregated valuesInclude calculation engines for fast, flexible transformation of base dataAre designed to reveal business trends and statistics not directly visible in the data retrieved from a data warehouseData mining models discover patterns in data, typically for prediction analysisProductAssociationSalesFinanceProduction
  • 33.
    Understanding Cube Structure116499518931455194513769451553187412451576445147918741245295415751479157630071575232229541383300724553007Accessories1654Australia645136521456459882012Product Line2012CountryUnited States845Bikes7457002751082234905Canada905345ClothingQuarter 1761875Quarter 2745745FranceComponentsSemester 1Quarter 3Quarter 4TXSemester 2MACalendar Year - 2009COVTState
  • 34.
    Data Visualization SystemClientaccess and distribution mechanisms can include:Static report viewers and browsersAd hoc query toolsReport writersModeling applicationsScorecard applicationsPortals and dashboardsDelivering data is a process of continuous business improvement:MonitorAnalyzePlan
  • 35.
    Data VisualizationBasic BusinessIntelligenceCustom Business Intelligence Web PortalPowerPivot for SharePoint and Self Services BI
  • 36.
    Basic Intelligence StaticReportsReport ServerCUBEExport Report to ExcelTechnology:ASP.NET
  • 37.
    SQL Server 2008R2Business User Tools:Excel 2007