Bw360 en col62

  • 7,696 views
Uploaded on

Bw360 en col62

Bw360 en col62

More in: Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads

Views

Total Views
7,696
On Slideshare
0
From Embeds
0
Number of Embeds
1

Actions

Shares
Downloads
1,176
Comments
0
Likes
6

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. BW360 SAP BI Performance & Administration InternalOnly SAP NetWeaver Information IntegrationPartner Use Date Training Center SAPSAP Instructors Education Website PartnerUseInternal Only Participant Handbook Course Version: 2006 Q2 Course Duration: 5 Day(s) Material Number: 50081099 An SAP course - use it to learn, reference it for work
  • 2. Copyright Copyright © 2006 SAP AG. All rights reserved. No part of this publication may be reproduced or transmitted in any form or for any purpose without the express permission of SAP AG. The information contained herein may be changed without prior notice. Some software products marketed by SAP AG and its distributors contain proprietary software components of other software vendors. Trademarks • Microsoft®, WINDOWS®, NT®, EXCEL®, Word®, PowerPoint® and SQL Server® are registered trademarks of Microsoft Corporation. InternalOnly • IBM®, DB2®, OS/2®, DB2/6000®, Parallel Sysplex®, MVS/ESA®, RS/6000®, AIX®, S/390®, AS/400®, OS/390®, and OS/400® are registered trademarks of IBM Corporation. • ORACLE® is a registered trademark of ORACLE Corporation. • INFORMIX®-OnLine for SAP and INFORMIX® Dynamic ServerTM are registered trademarks of Informix Software Incorporated.Partner • UNIX®, X/Open®, OSF/1®, and Motif® are registered trademarks of the Open Group. • Citrix®, the Citrix logo, ICA®, Program Neighborhood®, MetaFrame®, WinFrame®, VideoFrame®, MultiWin® and other Citrix product names referenced herein are trademarks of Use Citrix Systems, Inc. • HTML, DHTML, XML, XHTML are trademarks or registered trademarks of W3C®, World Wide Web Consortium, Massachusetts Institute of Technology. • JAVA® is a registered trademark of Sun Microsystems, Inc. SAPSAP • JAVASCRIPT® is a registered trademark of Sun Microsystems, Inc., used under license for technology invented and implemented by Netscape. • SAP, SAP Logo, R/2, RIVA, R/3, SAP ArchiveLink, SAP Business Workflow, WebFlow, SAP EarlyWatch, BAPI, SAPPHIRE, Management Cockpit, mySAP.com Logo and mySAP.com are Partner trademarks or registered trademarks of SAP AG in Germany and in several other countries allUse over the world. All other products mentioned are trademarks or registered trademarks of their respective companies. DisclaimerInternal THESE MATERIALS ARE PROVIDED BY SAP ON AN "AS IS" BASIS, AND SAP EXPRESSLY DISCLAIMS ANY AND ALL WARRANTIES, EXPRESS OR APPLIED, INCLUDING WITHOUT Only LIMITATION WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE, WITH RESPECT TO THESE MATERIALS AND THE SERVICE, INFORMATION, TEXT, GRAPHICS, LINKS, OR ANY OTHER MATERIALS AND PRODUCTS CONTAINED HEREIN. IN NO EVENT SHALL SAP BE LIABLE FOR ANY DIRECT, INDIRECT, SPECIAL, INCIDENTAL, CONSEQUENTIAL, OR PUNITIVE DAMAGES OF ANY KIND WHATSOEVER, INCLUDING WITHOUT LIMITATION LOST REVENUES OR LOST PROFITS, WHICH MAY RESULT FROM THE USE OF THESE MATERIALS OR INCLUDED SOFTWARE COMPONENTS.
  • 3. About This Handbook This handbook is intended to complement the instructor-led presentation of this course, and serve as a source of reference. It is not suitable for self-study. Typographic Conventions InternalOnly American English is the standard used in this handbook. The following typographic conventions are also used. Type Style DescriptionPartner Example text Words or characters that appear on the screen. These include field names, screen titles, pushbuttons as well as menu names, paths, and options. Use Also used for cross-references to other documentation both internal (in this documentation) and external (in other locations, such as SAPNet). SAPSAP Example text Emphasized words or phrases in body text, titles of graphics, and tables EXAMPLE TEXT Names of elements in the system. These include report PartnerUse names, program names, transaction codes, table names, and individual key words of a programming language, when surrounded by body text, for example SELECT and INCLUDE.Internal Example text Screen output. This includes file and directory names and their paths, messages, names of variables and parameters, and passages of the source text of a program. Only Example text Exact user entry. These are words and characters that you enter in the system exactly as they appear in the documentation. <Example text> Variable user entry. Pointed brackets indicate that you replace these words and characters with appropriate entries. 2006/Q2 © 2006 SAP AG. All rights reserved. iii
  • 4. About This Handbook BW360 Icons in Body Text The following icons are used in this handbook. Icon Meaning For more information, tips, or background Note or further explanation of previous point InternalOnly Exception or cautionPartner Procedures Use Indicates that the item is displayed in the instructors presentation. SAPSAP PartnerUseInternal Only iv © 2006 SAP AG. All rights reserved. 2006/Q2
  • 5. Contents Course Overview ............................................................................. vii Course Goals.................................................................................vii Course Objectives ...........................................................................vii InternalOnly Unit 1: SAP NetWeaver .......................................................................1 SAP NetWeaver: IT Practices, IT Scenarios, and Usage Types ....................... 2 Unit 2: InfoProviders......................................................................... 13Partner InfoCube Data Model ...................................................................... 15 Data Store Object Architecture ........................................................... 33 MultiProvider ................................................................................ 36 Use Virtual Providers ............................................................................ 42 Real-time InfoCubes ....................................................................... 48 Data Store Object for Direct Update ..................................................... 60 SAPSAP Unit 3: Process Chains...................................................................... 65 Process Chains Design .................................................................... 66 Managing Process Chains ................................................................ 76 PartnerUse Unit 4: InfoProvider Administration and Tuning .....................................103 Managing InfoCubes ...................................................................... 105 Managing Data Store Objects............................................................ 114Internal Relational DBMS Indexes in BI .......................................................... 125 Indexing for InfoProviders ................................................................ 141 Table Partitioning in BI .................................................................... 158 Only Unit 5: Reporting Performance...........................................................171 Design Guidelines for Optimal Query Performance................................... 172 OLAP Caching ............................................................................ 179 Unit 6: BI Statistics and Monitoring .....................................................201 BI Statistics ................................................................................. 203 BI Statistics Technical Content........................................................... 210 BI Administrator Cockpit .................................................................. 223 BI Trace Tool ............................................................................... 232 2006/Q2 © 2006 SAP AG. All rights reserved. v
  • 6. Contents BW360 Unit 7: Aggregates ..........................................................................239 Relational DBMS Aggregate Basics..................................................... 241 Missing Aggregates ....................................................................... 252 Analyzing Aggregate Usage.............................................................. 262 Aggregate Maintenance .................................................................. 270 Unit 8: Data Acquisition Performance ..................................................313 Data Load Cycle ........................................................................... 314 InternalOnly Technical Details ........................................................................... 318 Parallelization .............................................................................. 336 Optimizing Data Acquisition Performance.............................................. 344 Unit 9: Information Life Cycle Management...........................................365Partner Information Life Cycle Management in BI .............................................. 366 Unit 10: Transport Management .........................................................393 Use System Landscape ........................................................................ 395 Standard Transport System .............................................................. 401 Object Versions and Metadata ........................................................... 405 SAP Transporting in BI .......................................................................... 410SAP BEx Objects ................................................................................ 422 Transporting Roles and Workbooks ..................................................... 426 System Copy ............................................................................... 429 Transport Strategies and Troubleshooting ............................................. 434 PartnerUse Unit 11: BI Accelerator .....................................................................441 BI Accelerator Introduction ............................................................... 443 BI Accelerator Architecture ............................................................... 453Internal Implementing BI Accelerator Indexes ................................................... 464 BI Accelerator Monitoring................................................................. 481 Only Unit 12: Appendix ...........................................................................495 BI and OLTP System Comparison....................................................... 496 BI Database Settings...................................................................... 499 Web Application Server Settings ........................................................ 503 Reorganizing Tables....................................................................... 521 Sizing BI Systems ......................................................................... 526 Index ............................................................................................535 vi © 2006 SAP AG. All rights reserved. 2006/Q2
  • 7. Course Overview The focus of this course in on advanced BI administration and performance. This course provides a detailed look at the BI architecture and system landscape. Initially the various types of InfoProviders are discussed, then more details are covered that include the managing of InfoProviders for optimal performance. In order to manage the Enterprise Data Warehousing environment, sophisticated monitoring tools are InternalOnly essential. This course provides a detailed look at various monitoring tools in BI. Important statistics are collected for system wide analysis. This course discusses how the statistics are collected and analyzed. Participants will learn techniques to improve both query performance and data acquisition performance. Participants will learn how to create and maintain effective aggregates. This course also discusses the TransportPartner Management system, and the different methods that are used to handle Transports in BI. Participants will also learn about Information Life Cycle management and archiving in BI. A new technology to enhance query performance is introduced with Use the BI Accelerator. Target Audience SAP This course is intended for the following audiences:SAP • Project team members with extensive BI knowledge. Course Prerequisites PartnerUse Required Knowledge • Knowledge of the subject matter covered by BW310 (BI Data Warehousing).Internal Course Goals Only This course will prepare you to: • Administer and monitor BI systems. • Improve query performance. • Improve data acquisition performance. • Transport data in BI systems. Course Objectives After completing this course, you will be able to: 2006/Q2 © 2006 SAP AG. All rights reserved. vii
  • 8. Course Overview BW360 • Describe the BI architecture and data models. • List techniques to enhance query performance. • Describe how implement BI Statistics Technical Content. • Use BI monitoring and tracing tools to analyze BI performance. • List techniques to improve data acquisition performance. • Create effective aggregates to improve query performance. • Implement partitioning techniques for optimal performance. • Describe how the BI Accelerator can be used to improve query performance. InternalOnly SAP Software Component Information The information in this course pertains to the following SAP Software Components and releases:Partner Use SAPSAP PartnerUseInternal Only viii © 2006 SAP AG. All rights reserved. 2006/Q2
  • 9. Unit 1 SAP NetWeaver InternalOnly Unit Overview NetWeaver is SAPs application and integration platform that forms the technical foundation for mySAP Business Suite and SAP xApps. It delivers a complete, open, and flexible infrastructure that easily enables integration to SAP and non-SAPPartner applications. As of SAP NetWeaver 2004s, SAP retired the component terms as a method of Use describing the bundling features and functions. Now the concept for SAP NetWeaver and its enablement of IT is through the new cross-organizational and cross-technology IT practices and IT scenarios. SAPSAP Unit Objectives After completing this unit, you will be able to: • Describe SAP software is organized. PartnerUse • Define IT practices. • Define IT scenarios and scenario variants. • Define usage types.Internal Unit Contents Lesson: SAP NetWeaver: IT Practices, IT Scenarios, and Usage Types ...... 2 Only 2006/Q2 © 2006 SAP AG. All rights reserved. 1
  • 10. Unit 1: SAP NetWeaver BW360 Lesson: SAP NetWeaver: IT Practices, IT Scenarios, and Usage Types Lesson Overview With SAP Netweaver 2004s, a new evolution of SAP software is introduced. SAP software is now organized around the new concepts of IT Scenarios, IT strategies and Usage Types. Separate components such as, Enterprise Portal, Business Intelligence, Process Integration, Mobile Intelligence, Web Application Server, and Master Data InternalOnly Management, are now organized together as a single platform. This extremely comprehensive technical platform provides a powerful base for deploying the Enterprise Service-Oriented Architecture. This new platform enhances BI in many ways. Business reporting and analyticalPartner capabilities are expanded by the integration of the BEx frontend with Enterprise Portal. This new platform also enables enhanced Enterprise Data Warehousing features including new objects for modeling and implementing the EDW layers. Use This lesson provides information about the new concept and the corresponding new terminology. SAP Lesson ObjectivesSAP After completing this lesson, you will be able to: • Describe SAP software is organized. Partner • Define IT practices.Use • Define IT scenarios and scenario variants. • Define usage types.Internal Business Example Your company is now running SAP Netweaver 2004s. You need to become familiar Only with the new concepts and terminology that are introduced in this release. Terms Used by SAP Software Organization and Delivery IT Practices, Scenarios, and Scenario Variants IT practices identify how you can use SAP NetWeaver to solve specific business problems by deploying integrated IT scenarios in a way that does not disrupt your existing operations. IT practices look at the overall SAP NetWeaver platform in vertical slices that focus on key business issues, rather than on isolated technology 2 © 2006 SAP AG. All rights reserved. 2006/Q2
  • 11. BW360 Lesson: SAP NetWeaver: IT Practices, IT Scenarios, and Usage Types components. These include goals familiar to almost any IT organization, for example, the need to combine different integration technologies, to develop composite applications leveraging existing system investments, or to build new business processes in a flexible way. Other examples of IT practices include data unification and Business Information Management. InternalOnlyPartner Use SAPSAP Figure 1: SAP NetWeaver 2004s and IT Practices PartnerUse To address the broad challenges in each IT practice, SAP NetWeaver provides predefined IT scenarios, introduced fully with SAP NetWeaver 2004s. By implementing IT scenarios, customers can adopt the core functionality of SAP NetWeaver in incremental phases. Each practice can be broken into one or multiple ITInternal scenarios, providing organizations with a process-oriented approach to making best use of SAP NetWeaver. Only IT practices serve as high-level starting points. They give structure to IT scenarios as the IT implemental things connected to documentation and configuration. An IT scenario is a set of IT processes, grouped to attain a defined business goal. Targeting the business of IT, IT scenarios help the technical community to more quickly and to easily develop, compose, and run applications. They also help IT build, enhance, 2006/Q2 © 2006 SAP AG. All rights reserved. 3
  • 12. Unit 1: SAP NetWeaver BW360 and operate an IT infrastructure that integrates people, information, and processes. In many cases, further grouping is necessary, and the next level of organization is a scenario variant, which is a smaller grouping of IT processes. Note: IT scenarios replace the previous component view of SAP NetWeaver. All product information will be rolled out based on IT scenarios. InternalOnlyPartner Use SAPSAP Figure 2: IT Practices, IT Scenarios, and Variants PartnerUse The aim of IT scenarios is to help customers and partners install and operate SAP NetWeaver. You utilize IT scenarios to run business applications – both custom-built and packaged applications – or to implement a defined IT concept, such as evolving a companys system landscape into a services-based architecture.Internal Focusing on the flow of activities, IT scenarios are meaningful groupings of processes typically performed by one department. In this context; application development, adaptation and configuration, or business process management are typical examples of Only IT scenarios. In summary, we chose a scenario-based approach to ease the transition from your business requirements to a system landscape that will enable them. 4 © 2006 SAP AG. All rights reserved. 2006/Q2
  • 13. BW360 Lesson: SAP NetWeaver: IT Practices, IT Scenarios, and Usage Types InternalOnlyPartner Use Figure 3: IT Scenarios and IT Practices IT scenarios can cross IT practices. The figure below demonstrates this using Enterprise Data Warehousing as an example. A scenario-based approach to ease the SAPSAP transition from your business requirements to a system landscape that enable them. The IT Scenario “Enterprise Data Warehousing (EDW)” detailed in the preceding graphic and the scenario variants “Modeling ”and “Running” the EDW detailed below are the focus of this class. PartnerUseInternal Only Figure 4: Scenario Variants for Enterprise Data Warehousing 2006/Q2 © 2006 SAP AG. All rights reserved. 5
  • 14. Unit 1: SAP NetWeaver BW360 Usage Types In previous releases, SAP delivered and organized around IT-specific software components. The key capabilities of SAP NetWeaver were delivered as software components. Note: Although components exist, and are shown when you access the System → Status path on the SAP GUI, they are not relevant in business discussions. In addition, implementation is focused on usage types not components. InternalOnlyPartner Use SAPSAP PartnerUse Figure 5: How Should We Group and Deliver SAP NetWeaver? Now, instead of an IT focus, SAP is moving to a business-based focus and deliveringInternal the actual software you need to run an IT scenario. This business focused delivery of software crosses IT focused “software components”and is easier to understand for all. Only 6 © 2006 SAP AG. All rights reserved. 2006/Q2
  • 15. BW360 Lesson: SAP NetWeaver: IT Practices, IT Scenarios, and Usage Types InternalOnlyPartner Use Figure 6: Business-Focused Usage Types Usage types as an organizational tool have many advantages over software components by themselves. The figure below lists some of the advantages of usage types. SAPSAP Note: This figure also shows that some usage types are dependant on others. For example, usage type EP requires usage type AS Java. PartnerUseInternal Only Figure 7: Usage Types 2006/Q2 © 2006 SAP AG. All rights reserved. 7
  • 16. Unit 1: SAP NetWeaver BW360 In addition to SAP NetWeaver software components being packaged and delivered under usage types, other software is needed in an SAP implementation. Standalone engines are additional, installable software units that do not work as full-blown systems of SAP NetWeaver, but provide a specific (server) function in combination with one or multiple SAP NetWeaver systems. They are not part of a usage type, and do not run on Web AS ABAP or Web AS Java. One example of an engine is the Search and Classification (TREX) engine, used for search and retrieval in large document collections, text mining, automatic document classification, and search and aggregation over structured data in SAP applications InternalOnly Client programs are, again, additional installable programs not considered as usage types in any way. This category of software either resides on local front-end PC accessed by users or on back-end systems acting as a client program within a SAP NetWeaver system landscape. Front-end and back-end clients are part of the systemPartner or standalone engine installation for various reasons. Usage types and client programs and dedicated engines categorize all the software needed in an SAP environment. This is shown in the figure below. Use SAPSAP PartnerUseInternal Figure 8: Usage Types and Client Programs and Dedicated Engines Only The organization of multiple usage types is often needed to build up a usable system. The figure below shows the interaction between clients, systems, and standalone engines. 8 © 2006 SAP AG. All rights reserved. 2006/Q2
  • 17. BW360 Lesson: SAP NetWeaver: IT Practices, IT Scenarios, and Usage Types InternalOnlyPartner Use Figure 9: Clients, Systems, and Standalone Engines The concepts in this lesson are critical for your success. One example, installing the software, is shown in the following two figures. SAPSAP PartnerUseInternal Only Figure 10: Usage Types Drive Installation of SAP NetWeaver 2006/Q2 © 2006 SAP AG. All rights reserved. 9
  • 18. Unit 1: SAP NetWeaver BW360 InternalOnlyPartner Use Figure 11: Installation: Selection of Usage Type Although not every link below is focused solely on implementation, usage types, or IT scenarios, they will be very helpful in your future. SAPSAP PartnerUseInternal Only Figure 12: Where to Go for More Information 10 © 2006 SAP AG. All rights reserved. 2006/Q2
  • 19. BW360 Lesson: SAP NetWeaver: IT Practices, IT Scenarios, and Usage Types Lesson Summary You should now be able to: • Describe SAP software is organized. • Define IT practices. • Define IT scenarios and scenario variants. • Define usage types. Related Information InternalOnly • Use the last figure in this lesson to find links to a wealth of information about BI. • Note 917950 has a lot of detail on BEx related usage types.Partner Use SAPSAP PartnerUseInternal Only 2006/Q2 © 2006 SAP AG. All rights reserved. 11
  • 20. Unit Summary BW360 Unit Summary You should now be able to: • Describe SAP software is organized. • Define IT practices. • Define IT scenarios and scenario variants. • Define usage types. InternalOnlyPartner Use SAPSAP PartnerUseInternal Only 12 © 2006 SAP AG. All rights reserved. 2006/Q2
  • 21. Unit 2 InfoProviders InternalOnly Unit Overview In this unit, you will learn about BI InfoProviders. The BI Extended Star Schema is discussed in detail. You will learn how to design the star schema for performance. You will also learn about Data Store Objects and the important role they play in BI.Partner Other InfoProviders discussed in this unit include MultiProviders, Virtual Providers and Real Time InfoProviders. Use Unit Objectives After completing this unit, you will be able to: SAPSAP • Describe the SAP BI Extended Star Schema. • Describe the performance aspects of the Star Schema. • Explain the difference between a display attribute and a navigational attribute. • Describe the purpose of line item dimensions. PartnerUse • Describe the Data Store Object Architecture. • Describe the process of activating Data Store Objects. • Describe how to create a MultiProvider.Internal • Explain the performance implications of query processing on MultiProviders. • List the administration and performance benefits of utilizing Multiproviders. Only • Create Virtual Providers. • Describe the benefits of Virtual Providers. • Explain the impact on query performance when Virtual Providers are read. • Create a Real-time InfoCube and write into it • Create a Data Store Object for Direct Update. Unit Contents Lesson: InfoCube Data Model ..................................................... 15 Exercise 1: Examine Data Model ............................................. 23 2006/Q2 © 2006 SAP AG. All rights reserved. 13
  • 22. Unit 2: InfoProviders BW360 Exercise 2: Create an InfoCube ............................................... 29 Lesson: Data Store Object Architecture .......................................... 33 Lesson: MultiProvider ............................................................... 36 Lesson: Virtual Providers ........................................................... 42 Lesson: Real-time InfoCubes ...................................................... 48 Exercise 3: Real-Time InfoCubes ............................................. 55 Lesson: Data Store Object for Direct Update .................................... 60 InternalOnlyPartner Use SAPSAP PartnerUseInternal Only 14 © 2006 SAP AG. All rights reserved. 2006/Q2
  • 23. BW360 Lesson: InfoCube Data Model Lesson: InfoCube Data Model Lesson Overview In this lesson, you will learn about the need for data models. You will also learn about the BI Extended Star Schema and how to design the star schema. Lesson Objectives After completing this lesson, you will be able to: InternalOnly • Describe the SAP BI Extended Star Schema. • Describe the performance aspects of the Star Schema. • Explain the difference between a display attribute and a navigational attribute.Partner • Describe the purpose of line item dimensions. Business Example Use You are working in the Data Warehousing department of a chemical company. You need to load data separately into the master data tables, text tables, and hierarchy tables. SAPSAP Introducing Data Models • Do not underestimate the importance of the data model. PartnerUse – It is the basis of all BI application tuning. – Create the blueprint very carefully. – The effects of a bad data model are visible first with mass data load.Internal Only 2006/Q2 © 2006 SAP AG. All rights reserved. 15
  • 24. Unit 2: InfoProviders BW360 InternalOnlyPartner Figure 13: Data Model Design Use BI performance begins with the data model. The design phase is the ideal time to discuss the performance implications of BI SAP models. A poorly designed model will cause overall performance problems in terms ofSAP reading and loading BI objects. The design of BI models should include performance considerations from the beginning. PartnerUseInternal Only 16 © 2006 SAP AG. All rights reserved. 2006/Q2
  • 25. BW360 Lesson: InfoCube Data Model BI Extended Star Schema InternalOnlyPartner Use Figure 14: SAP BI Star Schema SAPSAP The SAP BI Star Schema is made up of a Fact table and Dimension tables. There can be a maximum of 16 dimension tables in one Star Schema. The Star Schema is also called and InfoCube. The InfoObjects in the dimension tables are called characteristics. The InfoObjects in PartnerUse the fact table are key figures. All characteristics have a SID (surrogate ID) table. The characteristic SID table gets created when the characteristic is activated. The SID table contains a SIDID for each characteristic value. The SIDID represents the characteristic value in numericInternal INT4 format and is generated by the system when values are loaded. Because each characteristic in a dimension table is represented by a numeric SIDID, the read performance on the InfoCube is greatly enhanced. (Reading numeric values is much Only faster than reading alphanumeric values.) If a characteristic, such as Customer or Product, also has master data tables, then the SID table is also the link between the master data and the characteristic in the dimension tables. Use transaction LISTSCHEMA to display the tables in the Star Schema. The BI extended star schema is subdivided into a solution-dependent part (InfoCube) and a solution-independent part: attribute tables, text tables, and hierarchy table. The attribute, text and hierarchy tables are also shared among the other InfoCubes. 2006/Q2 © 2006 SAP AG. All rights reserved. 17
  • 26. Unit 2: InfoProviders BW360 InternalOnlyPartner Use Figure 15: Extended Star Schema The attributes of the characteristics are located in the master data tables. SAP A great challenge when designing a solution is to decide whether to store data as aSAP characteristic in a dimension table (and therefore in the InfoCube) or as an attribute in a master data table. Data is loaded separately into the master data tables (attribute tables) text tables and Partner hierarchy tables.Use AttributesInternal Only 18 © 2006 SAP AG. All rights reserved. 2006/Q2
  • 27. BW360 Lesson: InfoCube Data Model InternalOnly Figure 16: Display and Navigational Attributes • A display attribute is an attribute of an InfoObject that can be displayed together with the InfoObject. It cannot be used on its own. • A navigation attribute is an attribute of an InfoObject that can be displayedPartner independently of the InfoObject and also used to navigate. For example it can be used to filter data. Use • It has to be enabled as a navigation attribute in the InfoObject that it is based on and switched on for each InfoCube. • The naming convention for a navigational attribute is: InfoObject__Attribute, such as 0customer__0country. SAPSAP PartnerUseInternal Only Figure 17: Navigational Attribute: Read Execution 2006/Q2 © 2006 SAP AG. All rights reserved. 19
  • 28. Unit 2: InfoProviders BW360 Designing the Star Schema • Small dimensions. • Few dimensions (less important than small dimensions). • Only as many details as necessary. • Hierarchies only if necessary. • Time-dependent structures only if necessary. • Avoid MIN, MAX - Aggregation for key figures in huge InfoCubes InternalOnly If MIN or MAX Aggregation is used in key figures, no delta changerun in Aggregates is possible. For each change run, a total rebuild of the aggregate is done. When modeling real world processes, different interests generally conflict with one other. A good design is therefore a compromise that allows the most important partsPartner of each aspect to be represented without disregarding the other aspects. Some questions to ask when determining requirements: Use • Will delivered business content meet many requirements? • What granularity of data do the users really need to access most frequently? • Which user requirements are, in reality, exception reporting? SAPSAP • Are users basing their requirements on simply replicating reports from a legacy system? Granularity is the level of detail of data. A decision about the granularity of modeling is one of the main results of the data modeling phase. Granularity deeply influences: PartnerUse • Reporting capabilities • Query Performance • Required Disk SpaceInternal • Data load performance Only Figure 18: Designing the Data Model 20 © 2006 SAP AG. All rights reserved. 2006/Q2
  • 29. BW360 Lesson: InfoCube Data Model Line Item Dimensions For good performance, the number of rows in the dimension tables should be small compared to the number of rows in the fact table. A general rule of thumb is the number of dimension table rows should be less than 15% of the number of fact table rows. Line item dimensions solve the problem of too many rows in the dimension table. The prerequisite is that there can be only one characteristic in the dimension. Line item dimensions improve both reporting and data load performance. InternalOnly F4 Help usually uses the dimension table for selection. In a line item dimension, there is no dimension table. Therefore, the S-table has to be used. An InfoCube can for example only uses 4 material numbers, but the BI system knows 10.000 material numbers. The material numbers, which are stored in the InfoCube, can be found in the dimension table. All material numbers of the BI system can be found in thePartner S-table. This means the F4-help can access a table with 10.000 entries instead of a table with 4 entries. Use SAPSAP PartnerUseInternal Figure 19: Line Item Dimension Only 2006/Q2 © 2006 SAP AG. All rights reserved. 21
  • 30. Unit 2: InfoProviders BW360 InternalOnly Figure 20: Create a Line Item DimensionPartner Use SAPSAP PartnerUseInternal Only Figure 21: Aspects of Design During the design phase of the project, there is an important balance between analysis aspects and performance aspects. 22 © 2006 SAP AG. All rights reserved. 2006/Q2
  • 31. BW360 Lesson: InfoCube Data Model Exercise 1: Examine Data Model Exercise Objectives After completing this exercise, you will be able to: • Examine the data model of InfoCubes Business Example InternalOnly Check for the data model of an existing InfoCube. Task 1: Examine the data model of InfoCube T_PU_SLOW (technical name).Partner 1. Analyze the InfoCube T_PU_SLOW. Find and display the data model of the InfoCube Use 2. Transaction LISTSCHEMA Open another window to call up the schema viewer for InfoCubes. What is the name of the database table for the InfoCube F fact table? SAPSAP Expand to view the names of the dimension tables. 3. Analyze the InfoCube’s F fact table and dimension tables Go to transaction RSRV and determine the number of records for the F fact PartnerUse table and dimension tables. Which one of these dimensions tables is the best candidate to be a Line Item dimension?Internal Task 2: Learn more about data models., Choose the InfoCube with the technical name Only “T_ATTR”. For this InfoCube two queries exist, run both queries (technical names “T_ATTR1” and “T_ATTR2”). Describe what you see, why do you see this difference? 1. Open the BEx Analyzer (call transaction RRMX). 2006/Q2 © 2006 SAP AG. All rights reserved. 23
  • 32. Unit 2: InfoProviders BW360 Solution 1: Examine Data Model Task 1: Examine the data model of InfoCube T_PU_SLOW (technical name). 1. Analyze the InfoCube T_PU_SLOW. Find and display the data model of the InfoCube a) Go to transaction RSA1. Choose Modeling, then InfoProvider. Goto InternalOnly InfoArea BW Training → BW Customer Training → BW360 Performance and Administration → Unit02. Highlight the InfoCube, context menu, and choose Display Data Model. Expand the Key Figures folder, and note the key figures that make up thisPartner InfoCube’s fact table. Expand each dimension (three triangles icon), to see what characteristics make up each dimension table of the InfoCube. List the dimensions that were built by the designer of the InfoCube. Use 2. Transaction LISTSCHEMA Open another window to call up the schema viewer for InfoCubes. What is the name of the database table for the InfoCube F fact table? SAPSAP PartnerUseInternal Only Continued on next page 24 © 2006 SAP AG. All rights reserved. 2006/Q2
  • 33. BW360 Lesson: InfoCube Data Model Expand to view the names of the dimension tables. a) In the box at the top left of the screen, type in “/oLISTSCHEMA”. In the field InfoCube Type, type in “B”, and type in the technical name T_PU_SLOW of the InfoCube, then select the Execute icon. The name of the database F fact table for the InfoCube is /BIC/FT_PU_SLOW. These are the dimension and their tables: InternalOnly T_PU_SLOWP: /BIC/DT_PU_SLOWP T_PU_SLOWT: /BIC/DT_PU_SLOWT T_PU_SLOWU: /BIC/DT_PU_SLOWU T_PU_SLOW1: /BIC/DT_PU_SLOW1 T_PU_SLOW2: /BIC/DT_PU_SLOW2Partner T_PU_SLOW3: /BIC/DT_PU_SLOW3 T_PU_SLOW4: /BIC/DT_PU_SLOW4 T_PU_SLOW5: /BIC/DT_PU_SLOW5 Use T_PU_SLOW6: /BIC/DT_PU_SLOW6 T_PU_SLOW8: /BIC/DT_PU_SLOW8 T_PU_SLOWA: /BIC/DT_PU_SLOWA SAP 3. Analyze the InfoCube’s F fact table and dimension tablesSAP Go to transaction RSRV and determine the number of records for the F fact table and dimension tables. PartnerUseInternal Only Continued on next page 2006/Q2 © 2006 SAP AG. All rights reserved. 25
  • 34. Unit 2: InfoProviders BW360 Which one of these dimensions tables is the best candidate to be a Line Item dimension? a) Go to transaction RSRV. Goto to Tests → All Elementary Tests → Database. Highlight the Test Database Information about InfoProvider Tables, right-click (context menu), and choose Select Test. On the right side, expand the test and make the parameter entry InfoProvider T_PU_SLOW. Choose the button “Transfer”. Execute the test by selecting the “Execute” button in the top of the left side. Display the protocol with the button “Display”. Expand your whole protocol. Now you InternalOnly can see the tables with their records. The dimension T_PU_SLOW8 with 188557 entries (23% of F fact table) in the table /BIC/DT_PU_SLOW8 is the best candidate to be a “High Cardinality” dimension and a “Line-Item” dimension. It can be a linePartner item dimension because, as you have seen in the data model, there is only one characteristic inside that dimension. All dimensions with only one characteristic can be a good candidate for a line item dimension. Use SAPSAP PartnerUseInternal Only Continued on next page 26 © 2006 SAP AG. All rights reserved. 2006/Q2
  • 35. BW360 Lesson: InfoCube Data Model Task 2: Learn more about data models., Choose the InfoCube with the technical name “T_ATTR”. For this InfoCube two queries exist, run both queries (technical names “T_ATTR1” and “T_ATTR2”). Describe what you see, why do you see this difference? 1. Open the BEx Analyzer (call transaction RRMX). a) Open → Query → InfoAreas → BW Training → BW Customer Training → BW360 Performance and Administration → Unit 02 → Query with InternalOnly attribute country 1 → OK. Open the second query “Query with attribute country 2” the same way you did for the first query. The first query displays EN for the country. The second query displays DE for country. Both queries look the same, toPartner see the difference (it doesn’t matter if you can open the change dialog only in display mode): Change → Change Query (global definition) → Technical Names on. Use The “country” characteristic used in query one is the country InfoObject itself. The “country” characteristic used in query 2 is the navigational attribute of InfoObject Airport. Hence the second query displays the country from the master data table (/BIC/PT_AIRPORT) and the first query SAPSAP takes the value for the country out of the InfoCube. The queries themselves display in both cases just “country” as the name of the object, in order to avoid mistakes always choose a different name for the navigational attribute! PartnerUseInternal Only 2006/Q2 © 2006 SAP AG. All rights reserved. 27
  • 36. Unit 2: InfoProviders BW360 InternalOnlyPartner Use SAPSAP PartnerUseInternal Only 28 © 2006 SAP AG. All rights reserved. 2006/Q2
  • 37. BW360 Lesson: InfoCube Data Model Exercise 2: Create an InfoCube Exercise Objectives After completing this exercise, you will be able to: • Create InfoObjects. • Create an InfoCube. • Identify the database tables that are created. InternalOnly Business Example Business requirements determine that a new InfoCube must be created to hold flight data. (The transaction data will be loaded later from the OLTP system.}Partner New InfoObjects are also needed to be included in the InfoCube. For performance reasons the InfoCube should have two dimensions, and each dimension should be a line item dimension with high cardinality. Use Task: Before you create your InfoObjects you need to create an InfoObject catalog. After SAP you have created your catalog create your own InfoObjects.SAP 1. Create an InfoObject catalog, Group##_char. In the catalog create characteristic InfoObjects: T_CARID##, and T_CONID##. 2. Create InfoCube TRANSP##, with two line item dimensions: CARRID PartnerUse and CONNID. Make each a line item dimension with high cardinality. In the CARRID dimension insert InfoObject T_CARID##, and in the CONNID dimension insert InfoObject T_CONID##. Include the Key Figure T_FLPRICE and 0CALDAY in your InfoCube.Internal Only 2006/Q2 © 2006 SAP AG. All rights reserved. 29
  • 38. Unit 2: InfoProviders BW360 Solution 2: Create an InfoCube Task: Before you create your InfoObjects you need to create an InfoObject catalog. After you have created your catalog create your own InfoObjects. 1. Create an InfoObject catalog, Group##_char. In the catalog create characteristic InfoObjects: T_CARID##, and T_CONID##. a) In transaction RSA1 choose InfoObjects and go to InfoArea: BW Training InternalOnly → BW Customer training → BW360 → Unit02→ Group##. b) From the context menu of your InfoArea Group##, choose Create InfoObject Catalog. Enter the name GR##_char and check InfoObject Type Chars.Partner c) Activate your InfoObject catalog. d) From the context menu of your InfoObject catalog choose Create Use InfoObject. Field Name Values SAP InfoObject T_CARID##SAP Description Group ## Carrid Template T_CARID98 PartnerUse e) Activate. f) From the context menu of your InfoObject catalog choose Create InfoObject..Internal Field Name Values InfoObject T_CONID## Only Description Group ## Connid Template T_CONID98 g) Activate. Continued on next page 30 © 2006 SAP AG. All rights reserved. 2006/Q2
  • 39. BW360 Lesson: InfoCube Data Model 2. Create InfoCube TRANSP##, with two line item dimensions: CARRID and CONNID. Make each a line item dimension with high cardinality. In the CARRID dimension insert InfoObject T_CARID##, and in the CONNID dimension insert InfoObject T_CONID##. Include the Key Figure T_FLPRICE and 0CALDAY in your InfoCube. a) From RSA1, choose Modeling → InfoProvider. Go to InfoArea BW Training → BW Customer training → BW360 → Unit 01 →Group##. b) From the context menu of your InfoArea choose Create InfoCube. InternalOnly Field Name Values InfoCube TRANSP## Description Group ## InfoCubePartner c) Press Create. d) Rename Dimension1 to CARRID and change properties. From the context Use menu of your dimension choose Properties and select Line Item and High Cardinality. e) Press Continue. SAPSAP f) From the context menu of your CARRID dimension choose InfoObject.Direct Input. Enter T_CARID##. g) From the context menu of the Dimensions folder, select Create new Partner dimensions. Enter CONNID for the description and select Line Item andUse High Cardinality. Press continue. h) From the context menu of the TIME dimension choose InfoObject.Direct Input. Enter 0CALDAY.Internal i) From the context menu of the Key Figures folder choose InfoObject.Direct Input. Enter T_FLPRICE. Only j) Activate your InfoCube. From the context menu of your InfoCube choose Display Data Model. k) Go to transaction LISTCHEMA. Enter TRANSP##. Expand the hierarchy and view the tables that were created. 2006/Q2 © 2006 SAP AG. All rights reserved. 31
  • 40. Unit 2: InfoProviders BW360 Lesson Summary You should now be able to: • Describe the SAP BI Extended Star Schema. • Describe the performance aspects of the Star Schema. • Explain the difference between a display attribute and a navigational attribute. • Describe the purpose of line item dimensions. InternalOnlyPartner Use SAPSAP PartnerUseInternal Only 32 © 2006 SAP AG. All rights reserved. 2006/Q2
  • 41. BW360 Lesson: Data Store Object Architecture Lesson: Data Store Object Architecture Lesson Overview This lesson will give you an overview of the Data Store Object Architecture. It will also describe the processes of loading and activation of Data Store Objects Objects. Lesson Objectives After completing this lesson, you will be able to: InternalOnly • Describe the Data Store Object Architecture. • Describe the process of activating Data Store Objects.Partner Business Example Your organization uses Data Store Objects to store huge amounts of data. As an employee of this chemical company, you need to understand the Data Store Use architecture. Data Store Architecture SAPSAP PartnerUseInternal Only Figure 22: Data Store Object Architecture 2006/Q2 © 2006 SAP AG. All rights reserved. 33
  • 42. Unit 2: InfoProviders BW360 Data Store Objects consist of three tables: • Activation Queue: In this table the new data is stored before activation. The structure is like a PSA table. The key is made up of the request, data package, and record number. After activation, a request is deleted from the Activation Queue. • Table with active data: In this table the currently active data is stored. The table has a semant key defined by the data modeler for example.order number, order item). Reporting is done on that table. • Change Log: During activation, changes in the active data are stored in the Change Log. You can find the entire history of activations for the Data Store InternalOnly Object in that table because data is not automatically deleted from that table. If data targets are supplied with data from the Data Store Object, in a delta process the data is read from the Change Log. The Change Log is a PSA table and can be maintained in the PSA tree of the administrator workbench. That isPartner why the Change Log also has a technical key that consists of the request, data package, and record number. Upload and activation process: New data is loaded to the Data Store Object and the Use technical key is added to the records. Requests can be loaded independently from each other (sequentially or in parallel). All requests are stored in the activaton queue first. Activation can be triggered manually or automatically. At the beginning of the activation, the data is sorted by the semantic key and the technical key. Therefore, SAPSAP records with identical semantic keys are activated in the same data package and in the right order. Data packages for activation are then created. The size of those packages can be customized. Those packages can be processed in parallel. The maximum number of packages and the server group the activation runs on can also PartnerUse be customized.Internal Only 34 © 2006 SAP AG. All rights reserved. 2006/Q2
  • 43. BW360 Lesson: Data Store Object Architecture Lesson Summary You should now be able to: • Describe the Data Store Object Architecture. • Describe the process of activating Data Store Objects. InternalOnlyPartner Use SAPSAP PartnerUseInternal Only 2006/Q2 © 2006 SAP AG. All rights reserved. 35
  • 44. Unit 2: InfoProviders BW360 Lesson: MultiProvider Lesson Overview This lesson will help you understand the concept of MultiProviders and explain the types of queries on MultiProvider. It will also help you describe high-level partitioning. Lesson Objectives InternalOnly After completing this lesson, you will be able to: • Describe how to create a MultiProvider. • Explain the performance implications of query processing on MultiProviders.Partner • List the administration and performance benefits of utilizing Multiproviders. Business Example Use As an employee of the Data Warehousing department of a chemical company, you need to improve the performance of a query. A smaller-sized query results in faster read performance. To do this, you can divide a query into sub-queries that run SAP against the smaller basic InfoCubes. This can be achieved by using MultiProvidersSAP that break reporting scenarios down into smaller units. As a result, you need to create MultiProviders. Partner Overview on MultiProviderUse BI Partitioning: Concepts MultiProviders are a type of partitioning on a conceptual level.Internal Only 36 © 2006 SAP AG. All rights reserved. 2006/Q2
  • 45. BW360 Lesson: MultiProvider InternalOnlyPartner Use Figure 23: MultiProvider Example Plan InfoCube has a few columns. SAPSAP Actual InfoCube has several columns. If you use both in one Cube, you would end up with a sparse InfoCube. Reporting Benefits PartnerUse • MultiProviders break reporting scenarios down into smaller units. Specific queries run fast against the smaller basic InfoCubes, and a small amount of data is read. Smaller sized datasets result in faster read performance.Internal • A query against a MultiProvider is divided into sub-queries that run against the smaller basic InfoCubes.These sub-queries are a form of parallel query processing. Parallel processing is generally faster than queries executed Only sequentially. These sub-queries also can run against the aggregates built for the BasicCubes. • Disadvantages:Overhead for splitting queries (may be minimal). 2006/Q2 © 2006 SAP AG. All rights reserved. 37
  • 46. Unit 2: InfoProviders BW360 Data Load Benefits • Each load request is a relatively small set of data being loaded into a relatively small InfoCube. Fewer records to load, and a smaller data target imply better load performance. • Disadvantages- None. Sub-queries can run against the aggregates of the Basic Cube, but the OLAP processor does not combine several aggregates as it sometimes does for normal query processing. In rare cases, this can lead to degraded performance. InternalOnly Multi Providers • MultiProviders are a combination of InfoProviders (database union). NoPartner additional data storage. • Queries on MultiProviders are split automatically and distributed to Use InfoProviders.Usage is transparent for reporting. SAPSAP PartnerUseInternal Figure 24: Query on MultiProvider - PARALLEL Only If enough system resources are available, queries on MultiProviders are split up to access in parallel the different BasicCubes. Every sub-query runs with its own workprocess, so you need more system resources while they run. 38 © 2006 SAP AG. All rights reserved. 2006/Q2
  • 47. BW360 Lesson: MultiProvider The run time of the query is like the longest access to one BasicCube, in this example 40 seconds. InternalOnlyPartner Use Figure 25: Query on MultiProvider - NOPARALLEL Runtime of the query is now the sum of all accesses to the different Aggregates or InfoCubes, in this example (15+1+3+2+2) → 23 seconds. SAPSAP PartnerUseInternal Only Figure 26: Non-Parallel Reads 2006/Q2 © 2006 SAP AG. All rights reserved. 39
  • 48. Unit 2: InfoProviders BW360 In the following cases, the system chooses sequential (non-parallel) processing: You want to • In table RSADMIN, entry QUERY_MAX_WP_DIAG has value (column value) 1. • The query is configured for non-parallel processing in RSRT. • The entire query consists of one sub-access only. • The query is running in a batch process. • The query was started from the query monitor (transaction RSRT) using various InternalOnly debug options (for example, SQL query display, execution plan display). • The query requests non-cumulative key figures. • Insufficient dialog processes are available when the query is executed. These are required for parallel processing.Partner • The result of the query is saved in a file or a table. Use SAPSAP PartnerUseInternal Only 40 © 2006 SAP AG. All rights reserved. 2006/Q2
  • 49. BW360 Lesson: MultiProvider Lesson Summary You should now be able to: • Describe how to create a MultiProvider. • Explain the performance implications of query processing on MultiProviders. • List the administration and performance benefits of utilizing Multiproviders. InternalOnlyPartner Use SAPSAP PartnerUseInternal Only 2006/Q2 © 2006 SAP AG. All rights reserved. 41
  • 50. Unit 2: InfoProviders BW360 Lesson: Virtual Providers Lesson Overview This lesson describes the purpose and benefits of creating VirtualProvides. Lesson Objectives After completing this lesson, you will be able to: InternalOnly • Create Virtual Providers. • Describe the benefits of Virtual Providers. • Explain the impact on query performance when Virtual Providers are read.Partner Business Example The data that you need for analysis resides on the source system. You need to be able Use to read the data, real time, from the source system. Virtual Providers SAPSAP PartnerUseInternal Only Figure 27: SAP Virtual Provider Architecture A Virtual Provider is a type of InfoCube where the data is not managed in BI. The data is managed externally. Only the structure of the VirtualProvider is defined in BI. The data is read for reporting using an interface with another system. Using a VirtualProvider , reporting can be done on data from external systems without having to physically store transaction data in BI. A VirtualProvider can be created to read data from an external SAP system or from an external non-SAP system. 42 © 2006 SAP AG. All rights reserved. 2006/Q2
  • 51. BW360 Lesson: Virtual Providers InternalOnlyPartner Use Figure 28: Example: Using a Virtual Provider for Data Reconciliation Data reconciliation for DataSources allows you to ensure the consistency of data that has been loaded into BI and is available and used productively there. Data SAPSAP reconciliation is based on a comparison of the data loaded into BI and the application data in the source system. The data in the source system is accessed directly to perform this comparison. The term Productive DataSource is a DataSource that is used for data transfer in the productive operation of BI. Data reconciliation Partner DataSource is a DataSource that is used as a format for accessing the application dataUse in the source system directly. With this method, BI transaction data can be compared with the transaction data on the source system. The data from the DataStore object can be combined with the dataInternal that is read directly by creating a Multiprovider. Queries based on the MultiProvider, can compare the loaded data with the application data in the source system. Only 2006/Q2 © 2006 SAP AG. All rights reserved. 43
  • 52. Unit 2: InfoProviders BW360 There are 3 types of VirtualProviders: • VirtualProvider based on Data Transfer Process This type of VirtualProvider is defined based on a DataSource or an InfoProvider and uses the same characteristics and key figures. There is no need to program interfaces in the source system. The same extractors are used to select data in the source system as you use to load data into the BI system. This type of Virtual InfoProvider is used when an SAP system is the source system, • VirtualProvider with BAPIs InternalOnly VirtualProviders whose transaction data for reporting is read from an external system using a BAPI. Any read tool that supports the interface for a non-SAP system can be used. Since the transaction data is not managed in the BI system, there is very little administrative effort on the BI side.Partner • VirtualProvider using Function Modules This Virtual InfoProvider displays data from non-BI data sources in BI without Use having to copy the dataset into the BI structures. The data can be local or remote. This method can transform the data before it is passed to the OLAP processor. In comparison to other VirtualProviders, this Virtual InfoProvider is more generic. It offers more flexibility, but also requires a higher implementation effort SAPSAP PartnerUseInternal Only 44 © 2006 SAP AG. All rights reserved. 2006/Q2
  • 53. BW360 Lesson: Virtual Providers InternalOnlyPartner Use Figure 29: Virtual Provider using BAPI • During query execution, the data is read on the source system. SAPSAP – The structures which store the data are designed for transactional performance not optimized for reporting performance. • Virtual Providers offer one option for providing operational reporting without Partner storing transaction data within BI.Use – Summarized data can be stored within BI InfoCubes for analysis reporting. – Reporting on transactional data might be less utilized, and therefore a performance trade-off may be desirable. Most queries would run fastInternal since they go against summarized data, but the rare operational report will require a longer time to run. • The data is loaded when the report is executed. Only – The total amount of data loaded to the Data Targets is reduced, thereby improving the performance of the normal data loads. – Loading data when a report is run adds another component to the response time observed by the end user. 2006/Q2 © 2006 SAP AG. All rights reserved. 45
  • 54. Unit 2: InfoProviders BW360 Direct Access Any DataSource (transaction datasource or master data datasource), that supports the relevant mode in the metadata can be accessed directly using the data transfer process. DataSources for source systems of type File and DB Connect are also available for direct access. The Data Transfer Process can be used to access data in an InfoProvider directly. The prerequisite is that the transformation and dataflow must be established between the source and target object. InternalOnlyPartner Use SAPSAP PartnerUseInternal Only 46 © 2006 SAP AG. All rights reserved. 2006/Q2
  • 55. BW360 Lesson: Virtual Providers Lesson Summary You should now be able to: • Create Virtual Providers. • Describe the benefits of Virtual Providers. • Explain the impact on query performance when Virtual Providers are read. Related Information InternalOnly • See https://service.sap.com/bi How to....Create Virtual InfoProvider with services.Partner Use SAPSAP PartnerUseInternal Only 2006/Q2 © 2006 SAP AG. All rights reserved. 47
  • 56. Unit 2: InfoProviders BW360 Lesson: Real-time InfoCubes Lesson Overview In this lesson, you will learn how to create a Real-time InfoCube and load data into it. Lesson Objectives After completing this lesson, you will be able to: InternalOnly • Create a Real-time InfoCube and write into it Business Example Your organization currently uses InfoCube. While writing data to these, each userPartner creates his own request ID. As a result, the F-fact table consists of many small requests, which in turn leads to bad performance. As an employee of this chemical company, you need to create a transaction InfoCube then write data to it. Use Real-time InfoCubes SAPSAP • Requested functionality: – Users would like to write their own data into data targets, usually small amounts of data. – Real-time updates are done in BI. PartnerUse • Problem – Writing into InfoCube: each user would create own request ID, F-fact table would consist of many small requests → bad performance.Internal • Solution – Real-time InfoCube → all users will write into the same request Only The Real-time InfoCube is a BI function, although it is heavily used in applications like Business Planning and Simulation, BPS, during planning functions. The idea behind a Real-time InfoCube is that in some applications only small amounts of data need to be written to an InfoCube. The point about this requirement is that an InfoCube is designed for mass data loads. Hence, SAP needed a technology to deal with lots of small updates in an efficient way. The Real-time InfoCube does not create a new Request ID for each data load. All users share the same request in an InfoCube. 48 © 2006 SAP AG. All rights reserved. 2006/Q2
  • 57. BW360 Lesson: Real-time InfoCubes InternalOnlyPartner Figure 30: Business Scenario - BPS Use On this graphic, you can a see a typical scenario where a Real-time InfoCube is used. The scenario above is Business Planning and Simulation, BPS, with a Business Consolidation System (BCS) scenario. There are two types of data entries for this application: (1) user enters the data manually, (2) run automatic planning functions SAPSAP which might copy or revaluate data from existing records. In both upload cases, the BPS application calls an Real Time interface in order to write the data directly into the InfoCube without any data staging technologies. The Real Time interface is a function module which was originally developed for Real Time PartnerUse applications, but can be used for other applications as well.Internal Only Figure 31: Creating Real-Time InfoCubes 2006/Q2 © 2006 SAP AG. All rights reserved. 49
  • 58. Unit 2: InfoProviders BW360 The screen shot shows how to create a Real-time InfoCube. The creation of a normal InfoCube and a Real-time InfoCube is similar. For a Real-time InfoCube the check box Real-time must be selected. InternalOnlyPartner Use SAP Figure 32: Writing into a Real-time InfoCubeSAP An F-Fact table consists of different requests. In the example above, the InfoCube consists of three requests, the first two requests are closed requests, but the third request is an open request. All users write into the same (open) request, it can also PartnerUse happen that one User writes multiple times into that request. The open request in the Data Warehousing Workbench is displayed with a yellow status information. The short text for all the InfoPackages which were or are a open request just indicates that the request was created by the Real Time interface.Internal The closed requests do have status information, but they do not have any monitor entries. Only 50 © 2006 SAP AG. All rights reserved. 2006/Q2
  • 59. BW360 Lesson: Real-time InfoCubes Accessing a Real-time InfoCube 1. Possibility: BI Interface • Normal BI technology • InfoPackage 2. Possibility: Real Time interface • Read function modules InternalOnly – RSDRI_INFOPROV_READ – RSDRI_INFOPROV_READ_RFC • Write function modules – RSDRI_CUBE_WRITE_PACKAGEPartner – RSDRI_CUBE_WRITE_PACKAGE_RFC • No InfoPackages, Use Basically there are two possibilities to load data to a Real-time InfoCube. The first possibility is to load the data via the BI interface with standard data staging. The second possibility is to load the data using the Real Time interface. To load data to a Real-time InfoCube using he Real Time interface, different function modules must SAP be called.SAP If these function modules are used, no InfoPackages are used. The function modules provide direct access to the Real-time InfoCube. Partner See note 302937 for debugging possibilities.Use Switching Data Load Technologies • Switch Real Time Interface → BI InterfaceInternal – Unsaved data will be lost • Switch BI Interface → Real Time Interface Only – All requests have to be green or red – Not possible if yellow request exist • Switch via 1. DataWarehousing Workbench manual (see screenshot on the next graphic) 2. Function module RSAPO_SWITCH_TRANS_TO_BATCH RSAPO_SWITCH_BATCH_TO_TRANS 2006/Q2 © 2006 SAP AG. All rights reserved. 51
  • 60. Unit 2: InfoProviders BW360 Different things have to be considered if the data load technology will be switched in order to guarantee a problem free operation. The switch from Real Time Interface to BI interface will close the open request and all data which is not saved will be lost (for example, manual data entry in BPS-BCS). The user does not recognize the switch until the user will try to save his or her data. In order to avoid losing data, this maintenance function should be done only when there are no users actively planning. InternalOnlyPartner Use SAPSAP Figure 33: Switching a Real-Time InfoCube PartnerUse The screen shot above just shows how the data load technology of a Real-time InfoCube can be switched. Regardless of the data load technology, the InfoCube is in both cases a Real-time InfoCube.Internal Only 52 © 2006 SAP AG. All rights reserved. 2006/Q2
  • 61. BW360 Lesson: Real-time InfoCubes InternalOnlyPartner Figure 34: Closing of an Open Request Use The open request will be closed automatically after 50,000 data records. This threshold value cannot be changed. If the open request should be closed earlier, a function module (rsapo_close_trans_re- quest) can be used. SAPSAP Furthermore, the request will be closed if the data load technology is switched. PartnerUseInternal Only Figure 35: Aggregates and Real-time InfoCube 2006/Q2 © 2006 SAP AG. All rights reserved. 53
  • 62. Unit 2: InfoProviders BW360 In order to see data of an open request, the variable 0s_rqmrc has to be activated from business content. Afterwards, this variable has to be used in in the free characteristics or filter section. The variable can be found in the packet dimension in BEx. See note 382744 for a description of the explanations above. If that variable is used for reporting, the system can access data from an aggregate and the data which is missing in the aggregate is taken out of the open request. This functionality guarantees that the technique of performance tuning with aggregates can still be used for Real-time InfoCubes. InternalOnlyPartner Use SAPSAP PartnerUseInternal Only 54 © 2006 SAP AG. All rights reserved. 2006/Q2
  • 63. BW360 Lesson: Real-time InfoCubes Exercise 3: Real-Time InfoCubes Exercise Objectives After completing this exercise, you will be able to: • Create and load data into a Real-Time InfoCube. Business Example InternalOnly A possible business scenario would be realtime updates into BI where some application (in this exercise a simple ABAP) writes data directly into a Real-Time InfoCube. Task 1:Partner Creation of a Real-Time InfoCube 1. Create your Real-Time InfoCube. Use Copy the Real-Time InfoCube T_TIC00 to T_RTC##, choose “Real-Time InfoCube ##” for the description. Choose the same InfoArea for your InfoCube as the InfoArea used for T_TIC00. Although you copy the Real-Time InfoCube you have to set the flag SAPSAP “Real-Time”, otherwise you do not have a Real-Time InfoCube. 2. Create your Real-time BI application. Copy the report ZT_BW360_FILL_TCUBE to ZT_BW360_FILL_TCUBE_## PartnerUse Open the ABAP editor and change the necessary lines of coding. Change every T_TIC00 to T_RTC## in the coding. (It has to be changed three times.)Internal Check and activate your ABAP, execute it twice. Task 2: Only Technology of open requests 1. From the Data Warehousing Workbench, check how the open request is displayed. Check the fact table of your InfoCube. Note that both data loads have the same request ID. 2. Close your open request. 2006/Q2 © 2006 SAP AG. All rights reserved. 55
  • 64. Unit 2: InfoProviders BW360 Solution 3: Real-Time InfoCubes Task 1: Creation of a Real-Time InfoCube 1. Create your Real-Time InfoCube. Copy the Real-Time InfoCube T_TIC00 to T_RTC##, choose “Real-Time InfoCube ##” for the description. InternalOnly Choose the same InfoArea for your InfoCube as the InfoArea used for T_TIC00. Although you copy the Real-Time InfoCube you have to set the flag “Real-Time”, otherwise you do not have a Real-Time InfoCube. a) Creation of Realtime BI InfoCube.Partner Goto transaction RSA1 choose Modeling then InfoProvider. Goto BW Training → BW360 Performance and Administration → Unit1. Use Right mouse on Unit1 then choose Create InfoCube. Field Name Values SAP InfoCube T_RTC##SAP InfoCube Description Real-Time InfoCube ## Copy from T_TIC00 Partner InfoCube Type BasicCube, Real-TimeUse 2. Create your Real-time BI application. Copy the report ZT_BW360_FILL_TCUBE to ZT_BW360_FILL_TCUBE_##Internal Open the ABAP editor and change the necessary lines of coding. Change every T_TIC00 to T_RTC## in the coding. (It has to be changed three Only times.) Continued on next page 56 © 2006 SAP AG. All rights reserved. 2006/Q2
  • 65. BW360 Lesson: Real-time InfoCubes Check and activate your ABAP, execute it twice. a) Goto transaction SE38 and enter ZT_BW360_FILL_TCUBE, then choose copy. Field Name Values Target program ZT_BW360_FILL_TCUBE_## Press Enter. Exchange in lines 18, 20 and 23 “T_TIC00” with “T_RTC##”. InternalOnly Press save, checkand activate. Go back to transaction SE38 and press the execute (F8). Press execute(F8) once more. You have now loaded two sets of data into your new InfoCube. ThisPartner effectively simulates planning steps that users would carry out in an BPS planning scenario. Use Task 2: Technology of open requests SAP 1. From the Data Warehousing Workbench, check how the open request isSAP displayed. Check the fact table of your InfoCube. Note that both data loads have the same request ID. a) Goto transaction RSA1 choose Modeling then InfoProvider. PartnerUse Goto InfoArea BW Training → BW360 Performance and Administration → Unit1. Right mouse click on your InfoCube “Real-Time InfoCube ##”. Choose manage and the tabstrip Requests.Internal You see one request without a QM status, this is your open request. Goto transaction SE16. Only Field Name Values Table Name /BIC/FT_RTC## Press Table Content (F7) and then Execute (F8). In the column for the InfoPackage you see that all data records have the same request number. Continued on next page 2006/Q2 © 2006 SAP AG. All rights reserved. 57
  • 66. Unit 2: InfoProviders BW360 2. Close your open request. a) To close the request go to transaction SE37. Field Name Values Function module RSAPO_CLOSE_TRANS_REQUEST Press single test (F8). InternalOnly Field Name Values I_INFOCUBE T_RTC## Press execute (F8). You see exactly the data you expect compared to thePartner fact table. Use SAPSAP PartnerUseInternal Only 58 © 2006 SAP AG. All rights reserved. 2006/Q2
  • 67. BW360 Lesson: Real-time InfoCubes Lesson Summary You should now be able to: • Create a Real-time InfoCube and write into it InternalOnlyPartner Use SAPSAP PartnerUseInternal Only 2006/Q2 © 2006 SAP AG. All rights reserved. 59
  • 68. Unit 2: InfoProviders BW360 Lesson: Data Store Object for Direct Update Lesson Overview In this lesson, you will learn to create a Data Store Object for Direct Update and write to it. Lesson Objectives After completing this lesson, you will be able to: InternalOnly • Create a Data Store Object for Direct Update. Business ExamplePartner Your organization currently uses an Data Store Object. While writing data to Data Store Object, each user writes into the same activation queue and starts the activation. This in turn leads to bad performance. You need to create a Data Store Object for Use Direct Update. Creation of a Data Store Object for Direct Update SAPSAP PartnerUseInternal Only Figure 36: Create Data Store Object for Direct Update A Data Store Object for Direct Update is created like any other Data Store Object. In the definition of the Data Store Object, you simply have to change the type from standard to Direct Update. 60 © 2006 SAP AG. All rights reserved. 2006/Q2
  • 69. BW360 Lesson: Data Store Object for Direct Update InternalOnlyPartner Use Figure 37: Writing into a Data Store Object for Direct Update A Data Store Object for Direct Update just consists of the active data table. Although the system creates the delta queue table and activation queue table, it won‘t use these tables. SAPSAP This restriction means that no delta update from a Data Store Object for Direct Update to another data target is possible. The restriction itself is due to the fact that no standard staging techniques are used. PartnerUse Writing into a Data Store Object for Direct Update • RSDRI_ODSO_INSERT • RSDRI_ODSO_INSERT_RFCInternal • RSDRI_ODSO_MODIFY • RSDRI_ODSO_MODIFY_RFC Only • RSDRI_ODSO_UPDATE • RSDRI_ODSO_UPDATE_RFC • RSDRI_ODSO_DELETE_RFC Due to performance reasons, it should be considered carefully which function module will be used. The insert function module should only be used if it can be ensured that the operation will be an insert and not an update. The same is true for the update function module 2006/Q2 © 2006 SAP AG. All rights reserved. 61
  • 70. Unit 2: InfoProviders BW360 However, when either an Update or an Insert could occur, the function module modify should be used. The modify function module will first try to update an existing set of data, and if such a data record does not exist, an insert will be performed. There is no RSDRI_ODSO_DELETE function module. Additional Information • Reading from Data Store Object for Direct Update BAPI_ODSO_READ_DATA InternalOnly InfoSet Query No BEx Query In order to read from a Data Store Object for Direct Update, either a BAPI or anPartner InfoSet Query can be used, but no standard BEx Query. Switching a Data Store Object for Direct Update is only possible if it is empty. Use SAPSAP PartnerUseInternal Only 62 © 2006 SAP AG. All rights reserved. 2006/Q2
  • 71. BW360 Lesson: Data Store Object for Direct Update Lesson Summary You should now be able to: • Create a Data Store Object for Direct Update. InternalOnlyPartner Use SAPSAP PartnerUseInternal Only 2006/Q2 © 2006 SAP AG. All rights reserved. 63
  • 72. Unit Summary BW360 Unit Summary You should now be able to: • Describe the SAP BI Extended Star Schema. • Describe the performance aspects of the Star Schema. • Explain the difference between a display attribute and a navigational attribute. • Describe the purpose of line item dimensions. • Describe the Data Store Object Architecture. • Describe the process of activating Data Store Objects. InternalOnly • Describe how to create a MultiProvider. • Explain the performance implications of query processing on MultiProviders. • List the administration and performance benefits of utilizing Multiproviders. • Create Virtual Providers.Partner • Describe the benefits of Virtual Providers. • Explain the impact on query performance when Virtual Providers are read. Use • Create a Real-time InfoCube and write into it • Create a Data Store Object for Direct Update. SAPSAP PartnerUseInternal Only 64 © 2006 SAP AG. All rights reserved. 2006/Q2
  • 73. Unit 3 Process Chains InternalOnly Unit Overview In this lesson, you will learn about the design, properties and structure of process chains. Next, you will learn about process chain maintenance.Partner Unit Objectives After completing this unit, you will be able to: Use • Describe the design properties of process chains. • Describe the structure of a process chain. • Create a Process Chain. SAPSAP • Describe the three different views to process chains • Explain process chains and data target administration • Describe system-wide monitoring of process chains PartnerUse Unit Contents Lesson: Process Chains Design ................................................... 66 Lesson: Managing Process Chains ............................................... 76Internal Exercise 4: Create a Process Chain .......................................... 93 Only 2006/Q2 © 2006 SAP AG. All rights reserved. 65
  • 74. Unit 3: Process Chains BW360 Lesson: Process Chains Design Lesson Overview In this lesson, you will learn about the design and properties of process chains. Next, you will learn about Process Chain Maintenance. Finally, you will learn about the structure of a process. Lesson Objectives InternalOnly After completing this lesson, you will be able to: • Describe the design properties of process chains. • Describe the structure of a process chain.Partner • Create a Process Chain. Business Example Use Your organization performs different activities that have to be scheduled to prepare the data for reporting. The sequence of activities is critical and complex scheduling is necessary because of the dependencies of the different activities. As an employee of SAP the company, you need to build self-defined event chains to solve the dependenciesSAP between the different processes. PartnerUseInternal Only 66 © 2006 SAP AG. All rights reserved. 2006/Q2
  • 75. BW360 Lesson: Process Chains Design Design Properties InternalOnlyPartner Use SAPSAP Figure 38: Introduction: Typical Data Load Cycle PartnerUseInternal Only 2006/Q2 © 2006 SAP AG. All rights reserved. 67
  • 76. Unit 3: Process Chains BW360 Transaction RSPC: Process Chain Maintenance InternalOnlyPartner Use SAP Figure 39: Transaction RSPC: Process Chains MaintenanceSAP Transaction RSPC: Process Chain Maintenance enables: • Easy creation of process chains via Drag & Drop PartnerUse • Creation of items When you connect processes, you are prompted to indicate whether the subsequent process should execute based on the success or the failure of the predecessor process. In other words, it is possible to schedule a process to run only if the predecessorInternal process fails. Only 68 © 2006 SAP AG. All rights reserved. 2006/Q2
  • 77. BW360 Lesson: Process Chains Design Collector Processes • Collectors are used to manage multiple processes that feed into the same subsequent process. The collectors available for BW are: – AND: All of the processes that are direct predecessors must send an event in order for subsequent processes to be executed. – OR: At least one predecessor process must send an event. The first predecessor process that sends an event triggers the subsequent InternalOnly process. – EXOR: Exclusive “OR” Similar to regular “OR”, but there is only ONE execution of the successor processes, even if several predecessor processes raise an eventPartner Collector processes allow the designer of a process chain to trigger a subsequent process based on whether certain conditions are met by multiple predecessor processes. Use Application processes are the other type of processes – these represent BW activities such as aggregate rollup. Application Processes SAPSAP • Application processes represent BW activities that are typically performed as part of BW operations. Examples include: – Data load PartnerUse – Attribute/Hierarchy Change run – Aggregate rollup – Reporting Agent Settings • Other special types of application processes exist:Internal – Starter – a process that exists to trigger process chain execution – ABAP program Only – Another process chain – Remote process chains – Operating System command – Customer built processes A starter process is part of every process chain. 2006/Q2 © 2006 SAP AG. All rights reserved. 69
  • 78. Unit 3: Process Chains BW360 InternalOnlyPartner Use Figure 40: Start Process The process variant could be used just by one process chain. SAPSAP Each process chain could be started manually or via RFC connection with the function module, RSPC_API_CHAIN_START. Basic Principles PartnerUse • Openness ⇒ abstract meaning of “process”: “Any activity with defined start and defined ending.” • Security ⇒ founding on the batch administration:Internal – Processes get planned before they run and can be viewed with standard batch-monitor. – Dumps and Aborts are caught and thus can be treated as failures. Only • Responsibility: – Each process have to care for all necessary information and dependencies for its own, when it is added to the process chain. – A predecessor process is not responsible to start the correct successors and provide them with necessary information. 70 © 2006 SAP AG. All rights reserved. 2006/Q2
  • 79. BW360 Lesson: Process Chains Design Functions of the predecessor and successor processes: • The predecessor process runs then signals when it has completed, writing information about the completed task to a database table (RSPCPROCESSLOG). • The successor process reacts to the event which is triggered by the predecessor, reads the database table (RSPCVARIANT, RSPCVARIANTATTR) to obtain any needed information, then executes. • Additional administrative process is to check the sequence of the processes. InternalOnly Structure of a ProcessPartner Use SAPSAP PartnerUseInternal Figure 41: Structure of a Process A process type is an ABAP OO object. Only The object is initiated when it is time for a process to run. The variant holds the specific configuration information for a process. Variant – Configuration • Each process can have one or several variants. • The maintenance of variant is specific for every process type. 2006/Q2 © 2006 SAP AG. All rights reserved. 71
  • 80. Unit 3: Process Chains BW360 Instance: (instance for process is like request for infopackage) • A snapshot of the variant configuration (table RSPCVARIANT*) at runtime and written to table RSPCINSTANCE. • Log information about the different processes collected at runtime and written to table RSPCPROCESSLOG. InternalOnlyPartner Use SAPSAP Figure 42: Example: Structure of a Process - Process Type PartnerUse Standard process types can be changed depending on their name space (transaction SE06). You can create your own process types (see HOW TO - Guides on service.sap.com/biInternal → service → How To ... Guide). RSPC → Settings → Maintain Process type. Table/maintenance view RSPROCESSTYPES) (SM30)] contains all information about the defined process Only types. 72 © 2006 SAP AG. All rights reserved. 2006/Q2
  • 81. BW360 Lesson: Process Chains Design InternalOnlyPartner Use Figure 43: Example: Structure of a Process - Variant Variants have to be created for the process types. With variant settings, the process type gets necessary information for execution. SAPSAP For example, if you assign the process type Attribute Change run to a process chain, you must define a variant. With this variant, you have to define the InfoObjects for which you want to activate the master data. There are four different possibilities: Partner • HIERARCHY: Direct selection of the hierarchy, which need to be activatedUse • INFOOBJECT: Direct selection of the infoObjects, which need to be activated • LOADING: Indirect selection: Reference to an InfoPackage, which must be loaded before in the process chain. Combined with the metadata of these objectsInternal and the instance information the system derives the affected InfoObjects and hierarchies. If the chosen LOADING object is not in the process chain, the system automatically inserts the chosen infopackage in the process chain. Only • REPORTVARIANT: Indirect selection: Reference to a change run variant which you can define with RSDDS_AGGREGATES_MAINTAIN (SE38) or TCode RSATTR → Executing the Attribute/hierarchy change run with Variant: Instead of applying the InfoObjects or hierarachies directly to the process variant in RSPC you can create a central variant for the report RSDDS_AGGREGATES_MAINTAIN. You can assign InfoObjects and hierarchies to this report variant. This central variant could be used by several process variants in RSPC. The benefits are central maintenance. 2006/Q2 © 2006 SAP AG. All rights reserved. 73
  • 82. Unit 3: Process Chains BW360 Using a reference object like LOADING or REPORTVARIANT is sometimes more flexible. InternalOnlyPartner Use SAPSAP PartnerUseInternal Only 74 © 2006 SAP AG. All rights reserved. 2006/Q2
  • 83. BW360 Lesson: Process Chains Design Lesson Summary You should now be able to: • Describe the design properties of process chains. • Describe the structure of a process chain. • Create a Process Chain. InternalOnlyPartner Use SAPSAP PartnerUseInternal Only 2006/Q2 © 2006 SAP AG. All rights reserved. 75
  • 84. Unit 3: Process Chains BW360 Lesson: Managing Process Chains Lesson Overview This lesson will focus on the three different views to process chains. It will then discuss process chains and data target administration. Finally, it will give you an overview of system-wide monitoring of process chains. Lesson Objectives InternalOnly After completing this lesson, you will be able to: • Describe the three different views to process chains • Explain process chains and data target administrationPartner • Describe system-wide monitoring of process chains Business Example Use Your organization uses process chains to perform activities in which sequence and scheduling is critical. As an employee of the organization, you need to perform maintenance checks, such as ensuring consistency and checking the plan status of SAP the process chains. You can use different views and display object trees for suchSAP maintenance activities. Designing Process Chains PartnerUse A process chain is a sequence of processes that are organized to perform certain functions, such as loading master data or transaction data. Process chains can be scheduled or triggered by events. Some processes may trigger an event that can, in turn, start other processes.Internal Process Chains: • Automate the complex job schedules in B.I Only • Visualize the processes by using network graphics . • Centrally control and monitor the processes. A process chain consists of a start process, individual application processes and the collection processes. Process Chains all have a Start process. All other chain processes are scheduled to wait for an event. 76 © 2006 SAP AG. All rights reserved. 2006/Q2
  • 85. BW360 Lesson: Managing Process Chains The application processes are the actual processes. BI supports process types of the following categories: • General services • Load process and post processing processes • Data target administration processes, · • Reporting Agent processes • Other BI processes, as well as custom created processes. InternalOnlyPartner Use SAPSAP Figure 44: Process Chain General Services PartnerUse The OS Command allows you to check, for example, whether a particular file exists in a directory before you load data to it. If the file is not in the directory, the load process can be repeated at pre-determined intervals. The workflow and decision process types support the event Process ends with complexInternal status. This process type controls the process chain process on the basis of multi-value decisions. The process does not have to end simply successfully or with errors; for example, the week day can be used to decide that the process was successful and Only determine how the process chain is processed further. With the workflow option, the user can make this decision. With the decision process type, the final status of the process, and therefore the decision, is determined on the basis of conditions. These conditions are stored as formulas. 2006/Q2 © 2006 SAP AG. All rights reserved. 77
  • 86. Unit 3: Process Chains BW360 InternalOnlyPartner Figure 45: 3 Different Views to Process Chains Use There are three main views in the icon bar with different views: • Planning view (shows if the process chain is active) SAPSAP – Grey: Unplanned processes (such as not activated process chain) – Green: Planned processes (process chain is active and the start process has been released) – Yellow: Planned but unknown processes PartnerUse – Red: Multiple planned processes • Checking view (consistency check such as double used start variants, missing index deletion and recreation, wrong references in variants.Internal – Green: Error-free processes – Yellow: Process with warnings – Red: Process with errors Only • Log view – Grey: Not yet run – Green: Finished without error – Yellow: Running – Red: Aborted or failed 78 © 2006 SAP AG. All rights reserved. 2006/Q2
  • 87. BW360 Lesson: Managing Process Chains Note that the log information is usually a mixture of : • Log information of maintenance activities for example new process, change of design, activation) • Log information of activities (such as, previous executions of process chains) InternalOnlyPartner Use SAPSAP Figure 46: Different Object Trees for Process Chain Administration (1) Partner Folders in the process chain are called display components.Use For maintenance of display components you have to use process chains → attributes → display components.Internal • For reassignment of process chain to different display component • For creation of new display component Only 2006/Q2 © 2006 SAP AG. All rights reserved. 79
  • 88. Unit 3: Process Chains BW360 InternalOnlyPartner Use Figure 47: Different Object Trees for Process Chain Administration (2) SAPSAP PartnerUseInternal Figure 48: Maintain Process Chains - Detail View Only When collector processes are built into a process chain, there are actually several background jobs scheduled with events in order to construct the conditional nature of collector processes. 80 © 2006 SAP AG. All rights reserved. 2006/Q2
  • 89. BW360 Lesson: Managing Process Chains The simple view displays the processing chain as it exists logically. The detail view displays the processing chain with the extra collector processes. InternalOnlyPartner Use SAP Figure 49: Maintain MessageSAP Email can be sent to indicate the successful completion of a process too. PartnerUseInternal Only Figure 50: Monitoring Features of the Log View (RSPC) 2006/Q2 © 2006 SAP AG. All rights reserved. 81
  • 90. Unit 3: Process Chains BW360 Process Chains can be copied by selecting Copy from the Process Chain menu. When a Process Chain is copied, the original Process Chain and all process variants are saved with a new name. Process Chain attributes are displayed by selecting Attributes from the Process Chain menu. InternalOnlyPartner Use Figure 51: Process Chain Attributes SAPSAP If errors occur while processing alerts can be sent. Process chain maintenance uses alert management for this purpose. The alert is sent to the user who scheduled the process chain. Additional recipients Partner may be assigned by roles or specifying individual users in alert category maintenanceUse (transaction ALRTCATDEF). The recipients display program displays the alerts. The display program can be the Universal Worklist (UWL) in the portal, the alert inbox (transaction ALRTINBOX) orInternal an application that calls alerts using API. Alerts can also be sent by email, fax or SMS. This is dependent on the configuration. Only If you are using Process Chains in a client-specific application, you use the client-specific display to determine the clients in which the chain can be used. When the client number is entered here, the Process Chain can only be displayed, executed, edited and scheduled from this client. When a Process Chain is executed the user can be displayed by selecting Execution user from the Process Chain menu. The default setting Background user can also be changed to execution user here. 82 © 2006 SAP AG. All rights reserved. 2006/Q2
  • 91. BW360 Lesson: Managing Process Chains During the check, the system calculates the number of parallel processes according to the structure of the tree. It compares the result with the number of background processes on the selected server (or the total number of all available servers if no server is specified in the attributes of the process chain). If the number of parallel processes is greater than the number of available background processes, the system highlights every level of the process chain where the number of processes is too high, and produces a warning. Maintenance of Process Chains is protected by the authorization object S_RS_PC. This authorization checks process chain maintenance and locks the process chain, and InternalOnly the processes of the chain, against actions by unauthorized users. Process Chains and Data Target AdministrationPartner Use SAPSAP PartnerUseInternal Figure 52: Process Chain and Data Target Administration Only 2006/Q2 © 2006 SAP AG. All rights reserved. 83
  • 92. Unit 3: Process Chains BW360 Process Chains: Facts • Process chains are transportable. – There are icons for writing to a change request when maintaining a process chain in RSPC. – Process chains can be selected from the transport connection wizard within the Data Warehousing Workbench. • Special background jobs are run to facilitate the execution of the process chain: InternalOnly – BI_PROCESS_TRIGGER – Generated jobs BI_EVEN* Activation, Execution, and RuntimePartner • Activation of process chain (Pressing the “activation” button) – The process chain is copied to an active version. Use • Execution of process chain (Pressing the “execution” button) – Process BI_PROCESS_TRIGGER is scheduled and released with the configured options (only if direct scheduling is applied for the start SAP process).SAP – The subsequent application processes are scheduled and released. Event triggered Partner As program RSPROCESS with type and variant given as parametersUse With job name BI_PROCESS_<TYPE> (except AND-process, which is BI_EVEN<GUID>) • Runtime of process chainInternal – The Instance information is written to table RSPCINSTANCE. – The Log information is written to table RSPCPROCESSLOG. Only Activation: • Process chain is set to version ‘active’. • RSPCVARIANT, RSPCVARIANTATTR, ...contain information about the configuration 84 © 2006 SAP AG. All rights reserved. 2006/Q2
  • 93. BW360 Lesson: Managing Process Chains Execution: • The start process is set to ‘direct scheduling’: – BI_PROCESS_TRIGGER is released with configured start options. – Subsequent application processes are scheduled and released as event triggered jobs. The start process is set to ‘Start Using Meta Chain or API.’ • No BI_PROCESS_TRIGGER is scheduled or released. You have to start InternalOnly the process chain via another process chain or API (Function module: RSPC_API_CHAIN_START). • Subsequent application processes are scheduled and released as event triggered jobs.Partner All scheduled jobs use ABAP Report ‘RSPROCESS’ with certain parameters. Separation between Activation and Execution with two buttons available. Use The application processes themselves care for sending log information. There are no further specific process chain jobs in background for controlling the application processes (except: periodic CCMS job: SAP_CCMS_MONI_BATCH_DP). SAPSAP PartnerUseInternal Only Figure 53: Polling Flag 2006/Q2 © 2006 SAP AG. All rights reserved. 85
  • 94. Unit 3: Process Chains BW360 When you set the polling flag, you can control the behavior of the main process when you have distributed processes. Distributed processes, such as the loading process, are characterized as having different work processes involved in specific tasks. With the polling flag you determine whether the main process needs to be kept as long as the actual process has ended. Setting the indicator has the following advantages: • A high level of process security is guaranteed. • External scheduling tools that react only to the SAP internal event “Batch InternalOnly Process Complete”, are also informed about the status of distributed processes. Setting the indicator also has the following disadvantages: • There is an increased demand on resources. Although the CPU is not underPartner pressure during the waiting time, it is during the status checks that run at two minute intervals. • You require one more background process. Use SAPSAP PartnerUseInternal Figure 54: Process Chains and Data Flow Only Process Chains are mandatory in the Data Flow. Process Chains are necessary to bring together InfoPackages and Data Transfer Processes. Process chains may define a Data Transfer Process. It is also possible to define a Data Transfer Process for an InfoProvider in the Data Warehousing Workbench. 86 © 2006 SAP AG. All rights reserved. 2006/Q2
  • 95. BW360 Lesson: Managing Process Chains The recommendation is to use process chains: the data transfer process is executed when it is triggered by an event in the predecessor process in the process chain. Monitoring Process Chains InternalOnlyPartner Use SAPSAP Figure 55: Monitoring of Individual Process Chain Partner Click the right mouse button in order to see the available functions.Use With the log view of a process chain, monitoring of special BW processes (such as infopackages, change run, and precalculation) and common activities (such as drop/create index) is possible. When you click the right mouse button on a processInternal icon, you get a list of the available log functions. Different processes are clearly arranged in one view and directly linked to the corresponding logs. Only Some processes (depending on the process type) are restartable. 2006/Q2 © 2006 SAP AG. All rights reserved. 87
  • 96. Unit 3: Process Chains BW360 InternalOnlyPartner Use Figure 56: Restarting of Process Types Overview: System Wide Monitoring of Process Chains SAPSAP PartnerUseInternal Only Figure 57: Overview: Systemwide Monitoring of Process Chains 88 © 2006 SAP AG. All rights reserved. 2006/Q2
  • 97. BW360 Lesson: Managing Process Chains You can monitor all process chains in four different ways: • Via CCMS (either T-Code: RZ20 or BWCCMS) • Via Metachain: create a matachain including all the different process chains used in your company. With status colors such as green, yellow, or red you get a quick overview of the overall status of processes on your system. • Via monitoring different processes in process chains for themselves • Via external monitoring tools (alias: softwarepartner on www.service.sap.com) InternalOnlyPartner Use SAPSAP PartnerUse Figure 58: System wide Monitoring - BWCCMS Process chains are included automatically in BWCCMS.Internal Only 2006/Q2 © 2006 SAP AG. All rights reserved. 89
  • 98. Unit 3: Process Chains BW360 The job SAP_CCMS_MONI_BATCH_DP checks hourly the status of process chains. • The activation of SAP_CCMS_MONI_BATCH_DP is necessary for CCMS monitoring generally once after installation – RZ21: technical infrastructure → method execution → activate background dispatching • Advantages: – Self-definition of alerts possible InternalOnly – Clearly arranged – Alerts linked to detailed monitors – Application log and BW log combined • Disadvantages:Partner – Maintenance not easy Process chains are assigned to BWCCMS per default. Use SAPSAP PartnerUseInternal Figure 59: Systemwide Monitoring - Metachain Only Advantages: • Graphical view • Easy to find the logs 90 © 2006 SAP AG. All rights reserved. 2006/Q2
  • 99. BW360 Lesson: Managing Process Chains Disadvantages: • It is not possible to filter the terminated parts of the metachain. Therefore, the view gets confusing if you build a complex scenario with too many different process chains. • Performance is influenced by graphical design (too much navigation to see the detailed logs). • Sometimes, there is low performance in execution log views. Advantages: InternalOnly • Allows fast access to detailed logs. Disadvantage: • Each log is only part of the entire process. You have to look for the differentPartner logs yourself. • Monitoring of the entire process is not possible. • Becomes very complex in large environments. Use • Detailed expert knowledge about the different log areas is necessary. SAPSAP PartnerUseInternal Only 2006/Q2 © 2006 SAP AG. All rights reserved. 91
  • 100. Unit 3: Process Chains BW360 InternalOnlyPartner Use SAPSAP PartnerUseInternal Only 92 © 2006 SAP AG. All rights reserved. 2006/Q2
  • 101. BW360 Lesson: Managing Process Chains Exercise 4: Create a Process Chain Exercise Objectives After completing this exercise, you will be able to: • Create an Generic DataSource. • Create a Data Transfer Process. • Create and InfoPackage. InternalOnly • Create a Process Chain to load data to an InfoCube. Business Example You created your InfoCube, TRANSP##, in the last unit. Now you must loadPartner transaction data from the OLTP system to your InfoCube. You need to create all of the objects needed for the data acquisition. You also need to create a Process Chain to run the data acquisition. Use Task 1: Create a generic DataSource for the source system T90CLNT090 using the table SAP SFLIGHT as a basis.SAP 1. Create a generic DataSource for the source system T90CLNT090. Task 2: PartnerUse Create a Transformation between your Data Source T_BW360_SFLIGHT_## and your InfoCube TRANSP##. 1. From the context menu of your DataSource create a Transformation.Internal Task 3: Create a Data Transfer Process between your InfoCube TRANSP## and your Only DataSource T_BW360_SFLIGHT_##. 1. Create a Data Transfer Process between your InfoCube TRANSP## and your DataSource T_BW360_SFLIGHT_##. Task 4: Create an InfoPackage for use in loading data. 1. Create an InfoPackage for use in loading data from your InfoCube. Continued on next page 2006/Q2 © 2006 SAP AG. All rights reserved. 93
  • 102. Unit 3: Process Chains BW360 Task 5: Create a Process Chain to extract data for your InfoCube, 1. Create a Process Chain to extract data to your InfoCube TRANSP##. 2. Create a Start process in your Process Chain PC_GR## for scheduling purposes. 3. Insert the Execute InfoPackage process into your Process Chain PC_GR##. 4. Load data into your InfoCube using the Process Chain you have just activated. InternalOnlyPartner Use SAPSAP PartnerUseInternal Only 94 © 2006 SAP AG. All rights reserved. 2006/Q2
  • 103. BW360 Lesson: Managing Process Chains Solution 4: Create a Process Chain Task 1: Create a generic DataSource for the source system T90CLNT090 using the table SFLIGHT as a basis. 1. Create a generic DataSource for the source system T90CLNT090. a) Use the menu Data Warehousing Workbench: Modeling → Source Systems and open the folder BI to find the source system T90CLNT090. InternalOnly From the context menu of T90CLNT090 choose the option Customizing Extractors . b) Use the menu Data Transfer to the SAP Business Information WarehousePartner → Generic DataSources → Maintain Generic DataSources and choose the Execute icon. In the field Transaction Data enter the value T_BW360_SFLIGHT_## Use and choose the Create icon. c) In the screen “Create DataSource for Transaction data: T_BW360_SFLIGHT_##” enter the following values: SAPSAP Applic. Component BW360-## Short description GR## Flight DataSource Medium description GR## Flight DataSource PartnerUse Long description GR## Flight DataSource View/Table SFLIGHTInternal Choose the Saveicon . In the dialog Create Object Directory Entry choose the Local Object button. Only In the screen “DataSource: Customer version Edit” select the checkboxes for the fields CARRID and CONNID. Choose the Saveicon . Choose the Back icon three times to return to BI and you should be in the Source Systems tree. d) From the context menu of source system T90CLNT090 choose Display DataSource Tree and find your application component BW360-## Continued on next page 2006/Q2 © 2006 SAP AG. All rights reserved. 95
  • 104. Unit 3: Process Chains BW360 From the context menu of your application component BW360-##choose Replicate Metadata. In response to the dialog Unknown DataSource choose the radio button DataSource and then click the Continue (Enter) icon. e) Activate your DataSource by pressing the activate icon. Task 2: Create a Transformation between your Data Source T_BW360_SFLIGHT_## and InternalOnly your InfoCube TRANSP##. 1. From the context menu of your DataSource create a Transformation. a) Use menu Data Warehousing Workbench: Modeling → DataSources → BW Training → BW Customer Training → BW360 → to your DataSourcePartner T_BW360_SFLIGHT_##. b) From the context menu of your DataSource choose Create Transformation. Use c) In the Create Transformation dialog box enter the following values: Object Type InfoCube SAPSAP Name TRANSP## Accept all other defaults and then click the Transfer (Enter) icon. d) Map all fields as follows by dragging a line between the fields on the left PartnerUse and the fields on the right. CARRID CARR_IDInternal CONNID CONN_ID FLDATE 0CALDAY PRICE Only PRICE CURRENCY PRICE Activate the Tranformation by choosing the Activate icon. Continued on next page 96 © 2006 SAP AG. All rights reserved. 2006/Q2
  • 105. BW360 Lesson: Managing Process Chains Task 3: Create a Data Transfer Process between your InfoCube TRANSP## and your DataSource T_BW360_SFLIGHT_##. 1. Create a Data Transfer Process between your InfoCube TRANSP## and your DataSource T_BW360_SFLIGHT_##. a) From the menu, choose Data Warehousing Workbench: Modeling → InfoProvider. Expand the InfoArea tree by choosing InfoProvider → BW_TRAINING InternalOnly → BW_CUSTOMER TRAINING → BW360 and locate your InfoCube TRANSP##. b) From the context menu of your InfoCube TRANSP## choose Create Data Transfer Process.Partner In the dialog enter the following values: Use Object Type DataSource DataSource T_BW360_SFLIGHT_## Source System T90CLNT090 SAPSAP Choose the Continue (Enter) icon. c) On the Extraction tab, enter the following value from the table and leave the defaults: PartnerUse Extraction Mode Full d) On the Update tab accept the defaults.Internal e) Activate the Data Transfer Process by choosing the Activate icon. Only Continued on next page 2006/Q2 © 2006 SAP AG. All rights reserved. 97
  • 106. Unit 3: Process Chains BW360 Task 4: Create an InfoPackage for use in loading data. 1. Create an InfoPackage for use in loading data from your InfoCube. a) Expand the Data Transfer Process under your InfoCube until you see the DataSource for which you created the Data Transfer Process. From the context menu of the DataSource choose Create InfoPackage and in the presented dialog enter the name Flight data for GR##. InternalOnly b) On the Data Selection tab you should not see any fields available for selection criteria. c) Go to the Extraction tab to make sure that the entry in the Adapter field is Access to SAP Data through Service API.Partner d) On the Processing tab you will see that the option Only PSA is not available for change. In BI, InfoPackages only extract data to the PSA table. Use e) On the Data Trargets tab, your InfoCube should be the only InfoProvider in the list. f) On the Update tab the only option should be a Full update. SAP g) On the Schedule tab, do not schedule. Only save the InfoPackage bySAP choosing the Save icon. Task 5: PartnerUse Create a Process Chain to extract data for your InfoCube, 1. Create a Process Chain to extract data to your InfoCube TRANSP##. a) Either use transaction RSPC or choose the Process Chains icon.Internal b) In the Process Chain Display Planning View choose the Create icon. In the New Process Chaindialog, enter the values: Only Process Chain PC_GR## Long description Load flight data for GR## then choose the Continue (Enter) icon. Continued on next page 98 © 2006 SAP AG. All rights reserved. 2006/Q2
  • 107. BW360 Lesson: Managing Process Chains 2. Create a Start process in your Process Chain PC_GR## for scheduling purposes. a) When you are presented with the dialog to create a Start process. Click the Create icon. Enter the following values: Process Variants Start_GR## Long description Start Process GR## InternalOnly b) Choose the Continue (Enter) icon. In the Maintain Start Processscreen, make sure that the Scheduling Option is set to DirectScheduling which should be the default and choose the Change Selections button.Partner In the Start Time dialog choose the Immediate button and then the Save icon. Use In the Maintain Start Process screen, choose the Save icon. Choose the Back icon and then the Continue (Enter) icon. SAPSAP PartnerUseInternal Only Continued on next page 2006/Q2 © 2006 SAP AG. All rights reserved. 99
  • 108. Unit 3: Process Chains BW360 3. Insert the Execute InfoPackage process into your Process Chain PC_GR##. a) To access other processes choose the Process Type icon located just above the Navigation block Descript./Descript.. This will open a list of folders which contains the allowed processes for a Process Chain. b) Open the folder Load Process and Post Processing and find the process Execute InfoPackage. Using the mouse, select and then drag and drop this process over to the work area for your Process Chain. c) In the dialog Insert Execute InfoPackage find and choose your InfoPackage InternalOnly Flight data for GR## so that it is inserted into the field Load Data. The BI system will insert the system-generated technical name into the field. Choose the Continue (Enter) icon. The system will insert not only the Execute InfoPackage process but alsoPartner the Data Transfer Process. d) Connect the Start process with the Execute InfoPackage process by Use dragging a link between them by clicking on the Start process and holding the primary mouse button down and then drag to the Execute InfoPackage process and release the mouse button. e) Check your Process Chain by switching to the Checking View using the SAPSAP menuGoto → Checking View. All the processes in your Process chain should be green in color indicating that your Process Chain is executable. Finally activate your Process Chain using the Activate icon. PartnerUse 4. Load data into your InfoCube using the Process Chain you have just activated. a) Execute your Process Chain by choosing the Execute icon. b) Switch to the Log view using the menu Goto → Log ViewInternal In the Date Selection dialog, choose the radio button Today and then choose the Transfer (Enter) icon. Only You may need to choose the Refresh icon several times until the Process Chain is finished. c) Check the data in your InfoCube by using the transaction LISTCUBE and enter the technical name of your InfoCube and choose the Execute icon. On the following screen choose the button Fld Selectn for Output and choose several Characteristics. The Key Figures are selected by default. then choose the Execute icon and you will be returned to the previous screen. Choose the Execute icon again to see the data in your InfoCube. 100 © 2006 SAP AG. All rights reserved. 2006/Q2
  • 109. BW360 Lesson: Managing Process Chains Lesson Summary You should now be able to: • Describe the three different views to process chains • Explain process chains and data target administration • Describe system-wide monitoring of process chains InternalOnlyPartner Use SAPSAP PartnerUseInternal Only 2006/Q2 © 2006 SAP AG. All rights reserved. 101
  • 110. Unit Summary BW360 Unit Summary You should now be able to: • Describe the design properties of process chains. • Describe the structure of a process chain. • Create a Process Chain. • Describe the three different views to process chains • Explain process chains and data target administration • Describe system-wide monitoring of process chains InternalOnlyPartner Use SAPSAP PartnerUseInternal Only 102 © 2006 SAP AG. All rights reserved. 2006/Q2
  • 111. Unit 4 InfoProvider Administration and Tuning InternalOnly Unit Overview This unit focuses on the administration of InfoProviders. There are many tasks that must be performed to keep InfoProviders running as optimally as possible. Incorrect data must be deleted and InfoCubes must be compressed.Partner This unit, also discusses the use of DB Statistics for database optimization and for indexing databases. Next, you will learn about indexes used in the InfoCube Star Schema. You will also learn about creating indexes in the BI system for other Use InfoProviders, like DataStore Objects and master data tables. This unit also discusses how to partition tables in BI for improved performance. SAPSAP Unit Objectives After completing this unit, you will be able to: • Display InfoCube Content. PartnerUse • Compress InfoCubes. • Delete data from an InfoCube. • Reconstruct an InfoCube.Internal • Remodel an InfoCube. • Explain the process of reading from an Data Store Object. Only • Describe the process of reporting on an Data Store Object. • Delete data from a Data Store Object. • Describe a B-tree index. • Explain how indexes are read at query runtime. • List several BI tools that support the maintenance of indexes. • Describe the indexes that are created for InfoCube tables. • Compare indexes used in an Basic InfoCube with Real Time InfoCube indexes. • Describe how InfoCube indexes are read during query runtime. • Describe how table partitioning can be used to optimize performance. 2006/Q2 © 2006 SAP AG. All rights reserved. 103
  • 112. Unit 4: InfoProvider Administration and Tuning BW360 • Partition InfoCubes. • Repartition InfoCubes. Unit Contents Lesson: Managing InfoCubes ..................................................... 105 Lesson: Managing Data Store Objects .......................................... 114 Lesson: Relational DBMS Indexes in BI ......................................... 125 Lesson: Indexing for InfoProviders ............................................... 141 InternalOnly Lesson: Table Partitioning in BI ................................................... 158Partner Use SAPSAP PartnerUseInternal Only 104 © 2006 SAP AG. All rights reserved. 2006/Q2
  • 113. BW360 Lesson: Managing InfoCubes Lesson: Managing InfoCubes Lesson Overview This lesson discusses the techniques used to manage InfoCubes for optimal performance. Lesson Objectives After completing this lesson, you will be able to: InternalOnly • Display InfoCube Content. • Compress InfoCubes. • Delete data from an InfoCube.Partner • Reconstruct an InfoCube. • Remodel an InfoCube. Use Business Example You are managing a number of InfoCubes. You have been asked to remove and then reload the data in the InfoCubes. You also must compress the data after it is loaded. SAPSAP InfoCube Content. In this section, participants will learn how to display and delete InfoCube content. PartnerUse After loading InfoCube data, it is possible to view the contents immediately for the Data Warehousing Workbench, RSA1. From the context menu of the InfoCube, choose Manage, Contents.Internal Only 2006/Q2 © 2006 SAP AG. All rights reserved. 105
  • 114. Unit 4: InfoProvider Administration and Tuning BW360 InternalOnlyPartner Use Figure 60: Display InfoCube Content All characteristics, navigation attributes and key figures are available for the selection. The characteristics and key figures that are selected are displayed in the output list. SAPSAP Aggregation is carried out using the characteristics that are not selected. The maximum number of hits determines the amount of data that is returned. By selecting the option DB aggregation, several records are taken from the database and brought together in one row of the list. The option Output number of hits generates PartnerUse an additional key figure in the output list that shows from how many records were rolled up.Internal Only 106 © 2006 SAP AG. All rights reserved. 2006/Q2
  • 115. BW360 Lesson: Managing InfoCubes InternalOnlyPartner Use Figure 61: Display SQL and Execution Plan The SQL statement is generated according to the field selection and restrictions, and is sent to the database. For each statement the database optimizer decides on how to SAPSAP retrieve the data. The execution plan shows the path that was taken by the database optimizer. The tables that were read and the join statements can be analyzed here. PartnerUseInternal Only Figure 62: Display InfoCube Indexes 2006/Q2 © 2006 SAP AG. All rights reserved. 107
  • 116. Unit 4: InfoProvider Administration and Tuning BW360 By choosing Table Detail the Indexes of the InfoCube can be displayed. From here the database statistics can be viewed and updated if needed. InternalOnlyPartner Figure 63: Deleting InfoCube Contents Use In the Data Warehousing Workbench, from the InfoCube context menu, choose Delete to delete the data from the InfoCube. There is a prompt that warns that all logs and related entries will be deleted as well. After the deletion has completed a report is SAPSAP displayed that shows all of the activity of the deletion process. All deleted objects are listed on this report. PartnerUseInternal Only Figure 64: Deleting InfoCube Data by Requests Data can be removed from an InfoCube by Request ID. Select the request and choose Delete.A background process is used to execute this deletion. Deletion by requests can also be managed in process chains. 108 © 2006 SAP AG. All rights reserved. 2006/Q2
  • 117. BW360 Lesson: Managing InfoCubes To load data back into the InfoCube, choose the Reconstruct tab, and then select the requests that you want to use to reconstruct the InfoCube. After selecting the corresponding requests from the list, choose Reconstruct. InternalOnlyPartner Use Figure 65: Selective Deletes from an InfoCube InfoCube data can be deleted according to business-orientated needs. For example, you can delete the data for a specific cost center or customer even if though the data is SAPSAP spread over several requests in the InfoCube. PartnerUseInternal Only Figure 66: InfoCube Compression 2006/Q2 © 2006 SAP AG. All rights reserved. 109
  • 118. Unit 4: InfoProvider Administration and Tuning BW360 When data is loaded into an InfoCube, the data is organized by requests (InfoPackage IDs). Each request has its own request ID, which is included in the primary key of fact table and in the packet dimension. With request IDs, it is possible delete or reconstruct InfoCube data by requests. However, because the request ID is included in the primary key of the fact table, it is possible that the same data record (all characteristics agree, with the exception of the request ID) can be written more than once in the fact table. This unnecessarily increases the volume of data, and reduces performance in reporting. The system must perform aggregation using the request ID every time a query is executed. InternalOnly Compression solves this problem. Compressing the InfoCube eliminates the duplicate records by converting all request IDs to 0 and rolling up the key figures for records that have common values for all characteristics. After InfoCube data is compressed, it can no longer be deleted from the InfoCube using its request ID.Partner Use SAPSAP PartnerUse Figure 67: Scheduling InfoCube CompressionInternal It is possible to manually choose request IDs and release them to be compressed immediately, or the compression can be scheduled in the background. Compression can be scheduled with a process chain. Only Choose zero-elimination to delete fact table entries where all key figures are equal to 0. Compressing one request takes approx. 2.5 ms per data record. With non-cumulative InfoCubes, compression has an additional effect on query performance. Also, the marker for non-cumulatives in non-cumulative InfoCubes is updated. This means that, on the whole, less data is read for a non-cumulative query, and the reply time is therefore reduced. The compression for a non-cumulative InfoCube, the summarization time (including the time to update the markers) will be about 5 ms per data record. 110 © 2006 SAP AG. All rights reserved. 2006/Q2
  • 119. BW360 Lesson: Managing InfoCubes For performance reasons, it is recommended that InfoCube requests be compressed as soon it is determined that they are correct, and that there is no need to delete the request, by requiest ID, from the InfoCube. Remodeling InfoCubes InternalOnlyPartner Use Figure 68: Remodeling InfoCubes SAPSAP If it is necessary to change an InfoCube, and no data has been loaded to this InfoCube, the change can be done in InfoCube maintenance. However, if changes are necessary for an InfoCube that has already been loaded the task may not be possible in InfoCube maintenance. In this case, InfoCube remodeling may be used. PartnerUse With InfoCube Remodeling, it is possible to change the structure of the InfoCube without losing data. For example, it may be necessary to replace an InfoObject in an InfoCube with another, similar InfoObject. Remodeling manages, Add, Change or Delete functions for any characteristic or key figure in the InfoCube.Internal Caution: As a precaution, it recommended that a back-up be made of the Only InfoCube before the Remodeling process begins. If there are process chains that run periodically and include the corresponding InfoCube, do not restart these process chains until the remodeling is completed. After remodeling, check all the BI objects that are connected to the InfoCube (transformation rules, MultiProviders, queries and so on) to verify that they are still active. These objects may have to reactivated manually after remodeling. 2006/Q2 © 2006 SAP AG. All rights reserved. 111
  • 120. Unit 4: InfoProvider Administration and Tuning BW360 InternalOnlyPartner Figure 69: Schedule Remodeling Use Based on the remodeling definition, a Remodeling Rule is created. The Check option checks and validates all the InfoObjects. The Impact Analysis option displays a list of the objects affected by the remodeling. These objects are deactivated during remodeling and will need to be reactivated later. SAPSAP Choose Schedule to either start remodeling immediately, or schedule it for later. By choosing Execute Steps in Parallel, the process can be executed in parallel. This should only be selected if the system resources are available to handle this operation. PartnerUse After the process has successfully completed, the InfoCube will be remodeled and active.Internal Only 112 © 2006 SAP AG. All rights reserved. 2006/Q2
  • 121. BW360 Lesson: Managing InfoCubes Lesson Summary You should now be able to: • Display InfoCube Content. • Compress InfoCubes. • Delete data from an InfoCube. • Reconstruct an InfoCube. • Remodel an InfoCube. InternalOnly Related Information • Related SAP Notes: 792435 Selective deletion: Using temporary tablesPartner 647512 Metadata changes to InfoCube and table conversion Use SAPSAP PartnerUseInternal Only 2006/Q2 © 2006 SAP AG. All rights reserved. 113
  • 122. Unit 4: InfoProvider Administration and Tuning BW360 Lesson: Managing Data Store Objects Lesson Overview This lesson will focus on the management of Data Store Objects. Lesson Objectives After completing this lesson, you will be able to: InternalOnly • Explain the process of reading from an Data Store Object. • Describe the process of reporting on an Data Store Object. • Delete data from a Data Store Object.Partner Business Example Your organization uses Data Store Objects for storing large amounts of data. As an Use employee of the chemical company, you need to perform activities, such as reporting, loading from Data Store Objects to data targets, or deleting data. To do this, you read Data Store Objects. SAPSAP Reading from an Data Store Object Data is read from an Data Store Object for different reasons. Reporting can be done on a single (or few) records level with key access or randomly Partner with any selection.Use Data in an Data Store Object is often loaded to other data targets. All of the active data in a full upload or entire requests in a delta upload can be read. Selection criteria are also possible for loading from an Data Store Object.Internal Deletion of data also means reading of data. Deletion is possible on a request level in the change log or in the active data. Selective deletion is possible. Selective deletion Only after reading for archiving purposes may be used quite often. 114 © 2006 SAP AG. All rights reserved. 2006/Q2
  • 123. BW360 Lesson: Managing Data Store Objects Reading from an Data Store Object • Reading from an Data Store Object is done during: – Key access (one document) – Random access • Loading from Data Store Object to Data Target – Entire request/active data – Selective InternalOnly • Deletion – Request Change logPartner Active data – Selective Use – Archiving • The highlighted accesses are optimized by the way the Data Store Object is designed (tables and indexes). Usually, no additional tuning is required for SAP those actions.SAP • Single line reporting with key access is fast because of the primary index of the active data table. • Full uploads require a full table scan of the active data. Delta uploads read from PartnerUse the change where the request number is the first field of the primary key. • For the same reason request deletions are fast.Internal Only 2006/Q2 © 2006 SAP AG. All rights reserved. 115
  • 124. Unit 4: InfoProvider Administration and Tuning BW360 InternalOnlyPartner Figure 70: Reading Performance Improvements with Indexes Use Indexes on tables can help to improve the reading performance for all types of reading described above. The order of the key fields determines the order of the fields for the primary index. The fields most frequently used for selection should be first in the order. SAPSAP Additional indexes can be defined for other reading accesses, for example: • Calendar month for selective deletion/archiving • Sales Document number for reporting PartnerUse In transaction DB05, you can determine the selectivity (distinct values) of table fields (single fields as well as fields in combination) and the distribution of field values. This information helps to define suitable indexes.Internal More information about indexes can be found in the indexes unit. Reporting on an Data Store Object Only • Data Store Objects are not designed for all reporting requirements. – By default, only one primary index is defined. – No pre-aggregation is possible (no aggregates), aggregations are always done from the lowest level. – No star-schema. – Often designed for storing detailed (mass) data 116 © 2006 SAP AG. All rights reserved. 2006/Q2
  • 125. BW360 Lesson: Managing Data Store Objects There are no technical restrictions in regard to reporting options on Data Store Objects. You can run the same kind of queries on an Data Store Object as on an InfoCube. However, Data Store Objects are not optimized for all kinds of reporting requirements. Aggregations will always be done from the lowest level stored in the Data Store Object. The data in the Data Store Object is stored in one table, not in a star schema. Typically, these tables can become very large because they are often designed to store data on a detailed level. Reporting on a high level means reading of a huge amount of data from the database and aggregation in memory or temporary tablespace. InternalOnlyPartner Use SAPSAP PartnerUse Figure 71: Reporting Performance (Joins with SID Tables)Internal The Data Store Object doesn’t have dimension tables, so the SID tables of all characteristics are joined directly with the Data Store Object table during reporting. This can be a performance issue if there are many characteristics and/or if there are Only large SID tables. 2006/Q2 © 2006 SAP AG. All rights reserved. 117
  • 126. Unit 4: InfoProvider Administration and Tuning BW360 InternalOnlyPartner Use Figure 72: Reporting Performance SAPSAP There are no technical restrictions in regards to reporting option on an Data Store Object. You can run the same kind of queries on an Data Store Object as on an InfoCube. However, Data Store Objects are not optimized for all kinds of reporting requirements. PartnerUse For aggregated reporting connect InfoCubes to the Data Store Object as subsequent data targets. You can drill down from an InfoCube to an Data Store Object to see details that are not available in the InfoCube. Reporting directly on the Data Store Object should be done on a detailed level. As inInternal operational reporting, viewing the details of one or a few items makes sense for an Data Store Object. Only 118 © 2006 SAP AG. All rights reserved. 2006/Q2
  • 127. BW360 Lesson: Managing Data Store Objects Overview of Data Store Object Size • The size of the tables of a Data Store Object is important for performance. • Loading – Activation Queue – Active Data • Loading from a Data Store Object to Data Target – Change Log InternalOnly – Active Data • Reporting – Active DataPartner • There is also an impact on table maintenance and management. The size of the tables of a Data Store Object affects the performance of all operations with the Data Store Object. Use For loading, the size of Activation Queue and Active Data are important, because the Activation Queue is filled during the upload and read during activation. The Active Data is both read and filled or updated during activation. SAPSAP Loading from a Data Store Object to connected data targets reads the active data for full uploads and the change log for delta uploads. For reporting, the active data is read. PartnerUse In general, reading from a table and writing to a table (especially updates, inserts are less critical) is slower depending on the size of the table. Large tables are harder to handle for the database. For example, table conversions in case of changes and deletions of large sets of data can get critical.Internal Only 2006/Q2 © 2006 SAP AG. All rights reserved. 119
  • 128. Unit 4: InfoProvider Administration and Tuning BW360 Reducing a Data Store Object Tables Size • The size of the tables of an Object can be reduced. • Data Modelling – Only necessary level of detail – Only necessary InfoObjects • Frequent activation – Keep amount of data in Activation Queue small. InternalOnly – Small Requests to load to connected data targets (easier to handle) • Deletion of data The size of the tables of a Data Store Object affects performance of all operations withPartner the Data Store Object. For loading the size of Activation Queue and Active Data are important, because the Activation Queue is filled during the upload and read during activation. The Active Use Data is both read and filled or updated during activation. Loading from a Data Store Object to connected data targets reads the active data for full uploads and the change log for delta uploads. SAPSAP For reporting the active data is read. In general, reading from a table and writing to a table (especially updates, inserts are less critical) is slower the larger the table is. PartnerUseInternal Only Figure 73: Data Store Object: Deletion by Request 120 © 2006 SAP AG. All rights reserved. 2006/Q2
  • 129. BW360 Lesson: Managing Data Store Objects Deletion of requests can only be done in the right order of loading because there are updates, not only inserts. The change log is needed to roll back to the data before the request was loaded (before image). Deletion per Request • To be deleted: – Unactivated requests → Request will be deleted from the activation queue. InternalOnly – Already activated requests (Rollback) → the complete Change Log request is deleted as well from the active table as from the Change Log. Note: Only Rollback: Return to the state before the erroneous request has been posted, i.e. all requests that have been posted afterwards arePartner also deleted. • Usage: Use – Deletion of erroneous requests even after the activation, since an error in the data will mostly be noticed after the activation/update of the respective data target. SAPSAP PartnerUseInternal Only Figure 74: Selective Deletion 2006/Q2 © 2006 SAP AG. All rights reserved. 121
  • 130. Unit 4: InfoProvider Administration and Tuning BW360 Selective Deletion • To be deleted: – Only data from the active table is deleted. The Change Log remains unaffected. • Usage: – Selective deletion is recommended if only single fields are not needed anymore and therefore should be deleted from the Data Store Object. As InternalOnly an example, you might want to delete only the data within a specific time period, which is not needed anymore. Another scenario could be that it is impossible to determine which request the erroneous data is related to (otherwise you could also delete the request). –Partner Note: This function may affect subsequent loads of the Data Store Object (since the Change Log is not updated, e.g. balance values). Use SAPSAP PartnerUseInternal Only Figure 75: Data Store Object: Delete Change Log Data 122 © 2006 SAP AG. All rights reserved. 2006/Q2
  • 131. BW360 Lesson: Managing Data Store Objects Delete Change Log • To be deleted: – Only the data from the Change Log is deleted. • Usage: – There have already been many requests loaded into the system → the Change Log is growing and impacts system performance. The data can be deleted selectively in case it is sufficient to keep only a restricted history, InternalOnly and the data is not needed for deltas or reconstruction.Partner Use SAPSAP PartnerUseInternal Only 2006/Q2 © 2006 SAP AG. All rights reserved. 123
  • 132. Unit 4: InfoProvider Administration and Tuning BW360 Lesson Summary You should now be able to: • Explain the process of reading from an Data Store Object. • Describe the process of reporting on an Data Store Object. • Delete data from a Data Store Object. InternalOnlyPartner Use SAPSAP PartnerUseInternal Only 124 © 2006 SAP AG. All rights reserved. 2006/Q2
  • 133. BW360 Lesson: Relational DBMS Indexes in BI Lesson: Relational DBMS Indexes in BI Lesson Overview This lesson discusses how Relational DBMS indexes are implemented in BI to help provide optimal query performance. Lesson Objectives After completing this lesson, you will be able to: InternalOnly • Describe a B-tree index. • Explain how indexes are read at query runtime. • List several BI tools that support the maintenance of indexes.Partner Business Example Use You are working in the Data Warehousing department of a chemical company. You need to improve query performance. You need to determine if you need additional indexes for optimal performance. SAPSAP Using Indexes • An Index is a data structure sorted by values containing pointers to data records of a table. The index function is available in all RDBMS in different flavors. PartnerUse Index Performance Impact • An Index can improve reading performance when data is searched for values of fields contained in the index.Internal • Indexes are maintained during writing to a table. This decreases writing performance unless the index is used for the write access itself. Only Indexes contain information about the data stored in the table. If the table contents change, then the index also needs to be changed in order to ensure that the new information is available in the index as well. This information can be used to access data faster whereas its maintenance slows down changes to data. Usage Of An Index 2006/Q2 © 2006 SAP AG. All rights reserved. 125
  • 134. Unit 4: InfoProvider Administration and Tuning BW360 InternalOnlyPartner Figure 76: The Database Optimizer Determines the Optimal Access Path Use • An Index can improve the following operations: – Select...where <table fields> = <value> – Update...where <table fields> = <value SAPSAP – Delete...where <table fields> = <value – Table Joins (<table1.field1> = <table2.field2> Usage of an Index – Influences PartnerUse • The database optimizer decides if an index is used and selects the appropriate index based on these criteria: – Fields in the index compared to fields in the statementInternal – Order of the fields in the index – Selectivity of the fields Only – Quality of the index – Size of the table • Information such as Size, Quality, and Selectivity is stored in DB Statistics. Combination of the right indexes and up to date DB Statistics usually gives the best performance. 126 © 2006 SAP AG. All rights reserved. 2006/Q2
  • 135. BW360 Lesson: Relational DBMS Indexes in BI An index can be useful if: • Fields in the index are used for selections or joins. • The fields in the index are selective, that is, they have a reasonably high cardinality (exception: bitmap index). • The quality, such as clustering factor, is good. • The table is reasonably large because, for small tables, reading the entire table is usually cheaper than additionally reading an index. The database collects statistical information about the tables and uses this information InternalOnly to decide on the best indexes to be used. • The database might not find the best execution plan if the DB statistics are not up to date. • Even if they are, the database may make a “wrong” decision. In this case,Partner a hint can help. Dropping Indexes Use • If a large amount of data is written to a table, dropping of indexes before and recreation afterwards may be faster. SAP – Exception:SAP Index is needed in write transactions, e.g. Update/Select record where key = “XYZ” Partner If sy-subrc <> 0Use Insert record key“XYZ” – Example:Internal Index on SIDs in Dimensions Dropping of indexes improves writing performance because the indexes does not need Only to be maintained then. Recreation of an index will usually be faster than single record maintenance if large amounts of data are written to the table. An Index needed in a loading process must not be dropped. Typically, a table that is updated or read during a loading procedure will be read with a unique key. If the record with a specific key does not exist, it will be created. For example, in BI, this is done for dimension tables (key not necessarily unique), Data Store Object tables for active data, and in the E-Fact table. 2006/Q2 © 2006 SAP AG. All rights reserved. 127
  • 136. Unit 4: InfoProvider Administration and Tuning BW360 InternalOnly Figure 77: Binary Search Finding an entry in a sorted array can be done with binary search. In order to perform binary search, the array to be searched should be first sorted. WePartner start by looking at the middle component. If the value it holds is too high, go to the middle of the bottom half of the array and look again. If the middle component value was too low, go to the middle of the top half instead. Then, repeat this “split the Use remainder”step until you find the value you want, or decide that it does not exist. SAPSAP PartnerUseInternal Only Figure 78: Binary Tree In order to support such a binary search, a sorted structure can be stored as a binary tree In a tree, records are stored in locations called leaves. This name derives from the fact that records always exist at end points; there is nothing beyond them. Branch points are called nodes. The order of a tree is the number of branches (called children) 128 © 2006 SAP AG. All rights reserved. 2006/Q2
  • 137. BW360 Lesson: Relational DBMS Indexes in BI per node. In a binary tree, there are always two children per node, so the order is 2. The number of access operations required to reach the desired record is called the depth of the tree. Binary trees are used when all the data is in RAM. InternalOnlyPartner Use SAPSAP Figure 79: Unbalanced Binary Tree versus Maintenance Effort Maintenance of a binary tree can become expensive if it should be balanced. If it is not maintained to be balanced, searches in the deeper parts of the tree become expensive. PartnerUseInternal Only 2006/Q2 © 2006 SAP AG. All rights reserved. 129
  • 138. Unit 4: InfoProvider Administration and Tuning BW360 InternalOnlyPartner Figure 80: B-Tree Index Overview Use B-trees are preferred when decision points, called nodes, are on disk rather than in random-access memory. It takes thousands of times longer to access a data element from hard disk as compared with accessing it from RAM. B-trees save time by using SAPSAP nodes with many branches (children), compared with binary trees, in which each node has only two children. When there are many children per node, a record can be found by passing through fewer nodes than if there are two children per node. PartnerUseInternal Only 130 © 2006 SAP AG. All rights reserved. 2006/Q2
  • 139. BW360 Lesson: Relational DBMS Indexes in BI InternalOnlyPartner Use Figure 81: B-Tree Index (Add Entry 24) Adding new values into a B-Tree is done by adding them to the right block in the sort order if there are still vacancies. Otherwise, the block is split up into two blocks. SAPSAP PartnerUseInternal Only Figure 82: Bitmap Index 2006/Q2 © 2006 SAP AG. All rights reserved. 131
  • 140. Unit 4: InfoProvider Administration and Tuning BW360 A bitmap index has one bit for every record or value combination. The bit is 1 if the record has this value in the column(s) otherwise it is 0. The size of the bitmap index depends on the number of values. InternalOnlyPartner Use SAPSAP Figure 83: Bitmap Index • B-Tree Index PartnerUse – Useful on columns with high cardinality – Typically, only one index used (exception: dynamic bitmap index creation, higher costs) – More robust during changes (degeneration possible)Internal – Size of the index independent of the number of distinct values – Clustering factor for evaluation of quality Only • Bitmap Index – Useful on columns with low cardinality – Several indexes can be used on the same table for one SQL statement, “bitmap merge join” with lower costs – Susceptible for degeneration or deadlocks – Size of the index depends on the number of distinct values (usually, smaller than B-tree) – No clustering factor 132 © 2006 SAP AG. All rights reserved. 2006/Q2
  • 141. BW360 Lesson: Relational DBMS Indexes in BI Clustered Index: IBM DB2 Universal Database for UNIX and Windows. • A clustered index is used to physically sort a table. • The performance impacts of using such an index are: – Higher maintenance effort – Faster access to data if selected by the cluster index – Less I/O (fewer physical blocks read) – Faster I/O (physical blocks located close to each other) InternalOnly DB2 UDB for UNIX and Windows can sort rows in physical data pages in the order of an index that is also known as the clustering index. The clustering index is created automatically for fact tables and is a compound index containing all dimension key columns. The order of the clustering index is determined by the InfoCube layout,Partner respectively by the order of the InfoCube dimensions. Clustering allows you to save the sorted data records in the fact table of an InfoCube. Use Data records with the same dimension keys are saved in the same extents (related database storage unit). This prevents the same data records from being spread over a large memory area and thereby reduces the number of extents to be read upon accessing tables. This greatly increases read, write and delete access to the fact table. SAPSAP Index clustering organizes the data records of a fact table according to the sort sequence of an index. There is a linear organization according to the values of the index field. If a data record cannot be inserted according to sorting because the relevant extent is already full, the data record is inserted into an empty extent at the Partner end of the table. Sorting is not guaranteed and can degenerate during many insert andUse delete operations. Reorganizing the table restores the sorting and releases memory space that is no longer required. The clustering index of an F fact table is, by default, the secondary index in the timeInternal dimension. The clustering index of an E fact table is, by default, the acting primary index (P index). Only Managing BI Indexes The primary index is distinguished from the secondary indexes of a table. The primary index contains the key fields of the table and a pointer to the non-key fields of the table. The primary index is created automatically when the table is created in the database. Secondary indexes are needed if the table is frequently accessed in a way that does not take advantage of the sorting of the primary index for the access. If the index fields have key function, that is if they already uniquely identify each record of the table, an index can be defined as a unique index. 2006/Q2 © 2006 SAP AG. All rights reserved. 133
  • 142. Unit 4: InfoProvider Administration and Tuning BW360 InternalOnlyPartner Use Figure 84: RSA1 - DataWarehousing Workbench One interface for index handling can be found in the BI DataWarehousing Workbench SAPSAP (transaction RSA1). To locate the interface: 1. Go to the respective InfoCubes. 2. Choose Manage. Partner 3. Go to the tabstrip “Performance”.Use This will provide you with the screen that is shown on the graphic. The operations that are offered on this screen apply to: • all secondary indexes on an InfoCubes F- and E-fact tablesInternal • but not indexes that have been defined by a user, i.e. that contain at least a letter in their name Only • and not primary indexes. 134 © 2006 SAP AG. All rights reserved. 2006/Q2
  • 143. BW360 Lesson: Relational DBMS Indexes in BI In the context of aggregates, they apply to the corresponding indexes on fact tables of all the aggregates of that InfoCube. The following operations are available: • check: This checks the status of the indexes, in particular whether they exist and if yes then if they are of the expected type (bitmap or B-tree). Typically non-existing indexes will lead to a red light whereas wrong index types lead to a yellow light. More details of the checks can be seen via transaction RSRV. • delete: This will drop all the indexes. • repair: This operation will recreate non-existing indexes; for Oracle on existing indexes it will perform a coalesce operation which is supposed to tackle the InternalOnly problem of degenerated indexes.Partner Use SAPSAP PartnerUse Figure 85: RSRV - BI Check RoutinesInternal You get to this screen by: • Calling transaction RSRV, Only • Clicking on the node “Database” • Dragging the check “Indices of ...” to the right hand side, and • Providing the technical name of the respective InfoCube 2006/Q2 © 2006 SAP AG. All rights reserved. 135
  • 144. Unit 4: InfoProvider Administration and Tuning BW360 There buttons help you: • Check the indexes: Press “Analysis” (or F8). • List the messages that has been collected by the check routine: press “Results” (or F6). • Repair the indexes: Press “Repair” (or Strg + F6). The check and repair operations have been described in the context of RSA1 (Data Warehousing Workbench). InternalOnlyPartner Use SAPSAP Figure 86: SE14 - Customizing Indexes PartnerUse Transaction SE14 is not BI-specific but a general transaction provided by the ABAP data dictionary to customize technical settings of database tables and indexes. This transaction is not easy to handle. It is therefore recommended only for experiencedInternal users. Here is how to get to the situation that index parameters are to be changed: • Call transaction SE14. • Provide the name of the database table and press“Edit.” Only • Press “Indexes” (or F5). • Choose the respective index from the pop-up. • Press“Storage parameters” (or Strg + F6). This will lead you to the screen as shown above. Typically, the parameter values as currently set on the database are shown. There are two more options: • The parameters as they were originally set (“technical settings”) • The parameters as they have been set by a user (“For new creation”) 136 © 2006 SAP AG. All rights reserved. 2006/Q2
  • 145. BW360 Lesson: Relational DBMS Indexes in BI If you want to change a parameter value, say the tablespace, you need to do the following (assuming no user has previously changed anything for that index): • Press “For new creation”. • Mark an index. • Press “Create parameter values” (or Strg + F4). • PSAPibly choose the template for the user-defined values, e.g. use the current DB settings. • Now, you can change the parameter. InternalOnly • Press “Save”. Now, the parameters are set for the next time the index is created. That means that the current setting remains. In order to activate the new settings, you have to drop the index and re-create it.Partner BI employs a cascading algorithm to determine the parameters for an index: Use the settings provided in SE14 under “For new creation”. Use If such settings are not available then use the settings provided by the underlying TABART (only applies to the tablespace). If such settings are not available, use the default settings. SAPSAP If one parameter has changed, option 1 applies to all other parameters too. PartnerUseInternal Only Figure 87: DB02 - Missing Indexes on the Database 2006/Q2 © 2006 SAP AG. All rights reserved. 137
  • 146. Unit 4: InfoProvider Administration and Tuning BW360 Transaction DB02 is frequently used by administrators to check for missing indexes: simply call transaction DB02 and press the button “Missing Indexes”. This will lead you to the screen shown in the lower left-hand corner of the graphic. If many fact table primary or secondary indexes are missing then you might apply SAP note 157918. Then run the ABAP reports: • SAP_UPDATE_DBDIFF, and • SAP_INFOCUBE_INDEXES_REPAIR InternalOnlyPartner Use SAPSAP PartnerUseInternal Only 138 © 2006 SAP AG. All rights reserved. 2006/Q2
  • 147. BW360 Lesson: Relational DBMS Indexes in BI and check DB02 again. Details of the reports are provided on the following graphic. • SAP_UPDATE_DBDIFF: updates table DBDIFF – Fact tables without primary index – Temporary views – Other temporary objects (see SAP note 308533 for temp. obj.) – Needs to be run only once • SAP_INFOCUBE_INDEXES_REPAIRs InternalOnly – Applies to all active InfoCubes and aggregates – Drops existing primary index – Re-creates missing P-indexes – Re-creates missing secondary indexesPartner – Should be run as a batch job • Check SAP note 157918 Use • BI provides two ABAP reports that are relevant for indexing problems. Further details of the context are described in SAP note 157918. • SAP_UPDATE_DBDIFF SAPSAP – This report updates the table DBDIFF which lists tables or table spaces that do not comply with ABAP dictionary standards. In the context of BI these are fact tables without primary index and temporary objects, such as tables, views, and triggers. This report will make sure that fact tables PartnerUse and temporary objects are properly listed in DBDIFF which – in turn – is used by DB02. DB02 will take differences into account that are listed in DBDIFF and therefore will not list those differences as errors. • SAP_INFOCUBE_INDEXES_REPAIRInternal – This report applies to all active InfoCubes and their aggregates. It removes existing primary indexes, recreates missing P-indexes and recreates missing Only secondary indexes. This report can be used to repair InfoCube indexes system-wide, that is, without looking at each InfoCube individually. This is particularly interesting for administrators. The report should be run as a batch job as it might run for a while. 2006/Q2 © 2006 SAP AG. All rights reserved. 139
  • 148. Unit 4: InfoProvider Administration and Tuning BW360 Lesson Summary You should now be able to: • Describe a B-tree index. • Explain how indexes are read at query runtime. • List several BI tools that support the maintenance of indexes. Related Information Related SAP Notes: InternalOnly 402469 Additional indexes on master data tables 356333 ORA Fragmentation 178275 ORA bitmap indexesPartner 157918 Missing indexes 323090 Generated indexes Use 304861 Generated indexes 335725 Change indexing parameters SAP 383325 Create bitmap index manuallySAP 401469 Indexes 401242 Analyze index errors PartnerUseInternal Only 140 © 2006 SAP AG. All rights reserved. 2006/Q2
  • 149. BW360 Lesson: Indexing for InfoProviders Lesson: Indexing for InfoProviders Lesson Overview This lesson discusses the indexes on the tables in an InfoCube, including the dimension tables, F fact table and E fact table. This lesson also describes how the indexes are used during query runtime to improve query performance. During query runtime, master tables are also read. This unit also discusses the indexes on Master tables. InternalOnly Secondary indexes are not created automatically for Data Store Objects. This lesson also discusses indexes for Data Store Objects. Lesson ObjectivesPartner After completing this lesson, you will be able to: • Describe the indexes that are created for InfoCube tables. Use • Compare indexes used in an Basic InfoCube with Real Time InfoCube indexes. • Describe how InfoCube indexes are read during query runtime. SAPSAP Business Example As an employee of the Data Warehousing department in a chemical company, you need to use an InfoCube. An InfoCube is a set of DB tables that are related to one another in a star schema. Using an InfoCube helps you to recognize star schema PartnerUse scenarios by providing techniques, such as star joins and bitmap indexing schemes.Internal Only 2006/Q2 © 2006 SAP AG. All rights reserved. 141
  • 150. Unit 4: InfoProvider Administration and Tuning BW360 Indexes in an InfoCube InternalOnlyPartner Use Figure 88: An InfoCubes Star Schema An InfoCube is a multi-dimensional reporting scenario built from: SAP • characteristics, such as “month”, “product”, “city”, and “sales organization”SAP • key figures, such as “costs”, “profit”, and “sales”. Technically, an InfoCube is a set of DB tables that are related to one another in a star schema. This is a very common technique in data warehousing and many database PartnerUse vendors have tuned their various DBMS products to recognize star schema scenarios by providing techniques such as star joins and bitmap indexing schemes. Tables in the InfoCube, fact tables and dimension tables, are automatically indexed. There are two fact tables:F fact table and E fact table (compressed). There is noInternal unique index (using the table primary key) defined for the fact tables. There is a secondary index for each dimension table and a composite index across Only all the columns of DIM IDs. The composite index is the P index. In Oracle and Informix database management systems, the P index is only on the E fact table and facilitates compression. In MS SQL Server database management systems the E fact table is indexed using a unique ) index. In DB2-UDB database management systems, the P index is created as a clustering index on the E and F fact tables. A clustering index sorts all data records of a table physically in the database, which accelerates read and delete accesses. 142 © 2006 SAP AG. All rights reserved. 2006/Q2
  • 151. BW360 Lesson: Indexing for InfoProviders The boxes in this graphic represent various tables. The lines show the foreign key – key relationships between those tables. In the center of a star schema, lies the fact table which contains a huge amount of data; in particular it holds all the information on the key figures. All the other tables are relatively small in comparison to the fact table. The dimension tables group related characteristics such as “city”, “region” and “country”. Finally, the master data tables hold information on the characteristics, for example, attributes like the “color ”or the “price” of a product. In this example, there are dimensions on time, region, product and sales organization. Therefore, the fact table holds information on sales figures, profit, costs etc. per InternalOnly day, product, city and sales organization. Each query on such a star schema uses/materializes a certain subset of those relationships.Partner Use SAPSAP PartnerUseInternal Figure 89: Example: InfoCube IUSALES Only 2006/Q2 © 2006 SAP AG. All rights reserved. 143
  • 152. Unit 4: InfoProvider Administration and Tuning BW360 Here, we see a subset of tables of the InfoCube IUSALESs star schema. The red arrows connect the respective foreign key column (end of the arrow) with the corresponding key column (head of arrow): • The fact table contains one foreign key column per InfoCube dimension and a column per key figure (of the InfoCube). • The dimension table consists of a dimension id (DIMID) column which constitutes the primary key of the dimension plus a column per characteristic in that dimension. Those columns hold SID (surrogate id) values of the corresponding characteristic. InternalOnly • In the third layer, there are SID-tables of the characteristics. This can be a standard S-table, containing only relationship between SID and characteristic key, an X-table (SID-key relationship plus SID columns per time-independent navigational attribute), or a Y-table (SID-key relationship, timestamp, SIDPartner columns per time-dependent navigational attribute). • In the fourth layer, there are standard S-tables for navigational attributes. Use SAPSAP PartnerUseInternal Only Figure 90: Indexing Common For All Databases 144 © 2006 SAP AG. All rights reserved. 2006/Q2
  • 153. BW360 Lesson: Indexing for InfoProviders This graphic shows the indexing scheme for an InfoCube common for all databases: • In the standard case, the fact table in Oracle has single column indexes on the dimension columns. • For Oracle these are usually bitmap indexes An exception are columns of high cardinality dimensions which have a (non-unique) B-tree rather than a bitmap index. • BI on DB2/400 uses the so called Encode Vector Indexes (EVI). EVIs are a further development of bitmap indices for DB2/400 which are compact in size, InternalOnly fast to build and very efficient for the access to the table data. EVIs are important in an ad hoc query environment where combining multiple indexes is a bigger requirement than having perfect index just for one pre-canned query. http://www-919.ibm.com/servers/eserver/iseries/developer/bi/evi.htmlPartner How to setup the EVIs for the fact tables? Please refer to the following SAP notes: 501572“EVI stage 2 support”; 541508 Use “Checking the system parameters” • A dimension table has a primary index (i.e. a unique B-tree) on its DIMID column. SAP • X- and Y-tables only have a primary key.SAP • SID tables have a primary key index – the primary key comprises the characteristic key column(s) – and a unique index on the SID column. The uniqueness of the latter is a kind of additional check to assert the uniqueness Partner of the SIDs.UseInternal Only 2006/Q2 © 2006 SAP AG. All rights reserved. 145
  • 154. Unit 4: InfoProvider Administration and Tuning BW360 InternalOnlyPartner Figure 91: Fact Table Indexing Use This graphic shows the indexing scheme of the standard InfoCube fact tables. Both fact tables do not have primary indexes but a “P-index” (such as /BIC/EIUSALES~P). It comprises all dimension columns of the fact table. However, it does not enforce SAPSAP a unique constraint. This is the only difference with the originally existing primary index. On the E fact table the P-index is specifically designed to support the InfoCube compression process. In such a situation, BI has to check the E fact table for existing PartnerUse dimension id combinations in order to decide whether the currently processedInternal Only 146 © 2006 SAP AG. All rights reserved. 2006/Q2
  • 155. BW360 Lesson: Indexing for InfoProviders record (which originates in the F fact table) is to be inserted (when ist dimension id combination does not exist in the E fact table so far) or updated (in the other case). A missing P-index can cause a very considerable decrease in compression performance. • Index over all dimensions keys – Usually non-unique ““P-Index” (/BIC/F<InfoCube>~P, /BIC/E<InfoCube>~P – Oracle and Informix InternalOnly Not on F-Fact table – MSS Non-clusteredPartner Unique 0-Index on E-Fact table – DB2/UDB Use Clustered index Order: Time Dimension, Dimension 1, Dimension 2, … – DB2/390 SAPSAP Unique 0-Index on E-Fact table PartnerUseInternal Only Figure 92: InfoCube (Real Time, Oracle) 2006/Q2 © 2006 SAP AG. All rights reserved. 147
  • 156. Unit 4: InfoProvider Administration and Tuning BW360 This graphic shows the indexing scheme of a Real Time InfoCube for Oracle. Such InfoCubes are currently only used in the context of SEM – check SAP note 333260 for details. The only difference with a standard InfoCube lies in the indexing of the F fact table. Here, bitmap indexes are replaced by standard B-tree indexes. The reason behind this is that such InfoCubes are used in a transactional context with many concurrent users reading from and writing into the InfoCube (i.e. into the F fact table). Bitmap indexes cannot support such a scenario because Oracle provides no row-level locking for bitmap indexes. This leads to deadlock situations (ORA-60) if users concurrently update a bitmap indexed fact table. InternalOnlyPartner Use SAPSAP PartnerUse Figure 93: InfoCube (Partitioned, Oracle only) This graphic shows the indexing scheme of fact tables of a partitioned InfoCube.Internal The significant issue here is an additional bitmap index on the column of the F fact table that corresponds to the partitioning column (time characteristic the InfoCube is partitioned by) of the E fact table (see red box). This indexs name is “900”, i.e. Only “/BIC/F...~900” on the database. The reason behind that index is to support restrictions that are likely to exist on this column. In case of a partitioned E fact table, the BI SQL-generator translates time restrictions into restrictions on the partitioning column whenever possible. This allows the Oracle query optimizer to prune the query to the relevant partitions on the E fact table. The “900” index allows the F fact table to benefit from those additional (and redundant) restrictions too. 148 © 2006 SAP AG. All rights reserved. 2006/Q2
  • 157. BW360 Lesson: Indexing for InfoProviders There is no architectural difference between Basic Cubes and Real Time InfoCubes. However, in Real Time InfoCubes there is only the bitmap index on the F fact table. Otherwise, transactional write accesses would result in deadlock situations. Please refer to the discussion of deadlocking on the Real Time InfoCube graphic. Index Usage The status of InfoCube tables and indexes can be checked in RSRV. The use of indexes during query runtime can be viewed from the Execution Plan. One way to view the execution plan is from RSRT → Execute +Debug → InternalOnly DataBase→Display Run Schedule.Partner Use SAPSAP PartnerUse Figure 94: InfoCube Index AnalysisInternal For every table called in the execution plan, the indexes and additional information about table statistics can be displayed. The execution plan also shows the access types Only and join operations chosen by the database optimizer. 2006/Q2 © 2006 SAP AG. All rights reserved. 149
  • 158. Unit 4: InfoProvider Administration and Tuning BW360 InternalOnlyPartner Figure 95: Star Join Execution Plan (Oracle Example) Use Here, a typical star query execution plan for a BI query is shown. The single column bitmap indexes are used to propagate query restrictions to the fact table, thereby minimizing the amount of relevant facts at a very early stage of query processing. This SAPSAP is the reason why those operations are found in the lower part of the execution tree. The execution plan on this graphic is a typical example. Almost every BI query should show this kind of execution pattern. Obviously, it highly depends on using bitmap indexes. If those indexes are not employed for whatever reason (they do not exist, PartnerUse there are no restrictions, Oracles query optimizer chooses an execution plan without bitmap indexes etc.) then you will typically experience very high run times. In the case of Real Time InfoCubes and single column B-tree indexes, Oracle is supposed to transform the relevant B-tree indexes into bitmap indexes at run timeInternal (“B-tree to bitmap conversion”). This is an additional computational step which consumes resources, thus star queries on B-tree indexed fact tables are less efficient than similar queries supported by bitmap indexes. Only 150 © 2006 SAP AG. All rights reserved. 2006/Q2
  • 159. BW360 Lesson: Indexing for InfoProviders InternalOnlyPartner Figure 96: Indexing of a Dimension Table Use A dimension table has the primary index and the following secondary indexes: • Index over all SID columns supporting loading SAP • Index on each individual column but the first one supporting querying. The firstSAP SID column does not need a single index because it is the first column in the index over all SID columns. • The order of the fields in the dimensions is determined by the order of the Partner InfoObjects in the InfoCube which depends on the transactional data load.UseInternal Only 2006/Q2 © 2006 SAP AG. All rights reserved. 151
  • 160. Unit 4: InfoProvider Administration and Tuning BW360 InternalOnlyPartner Use Figure 97: Indexing of Master Data (X-, Y-) Tables This graphic shows an example of a master data table, in this case an X-table (characteristic SIDs and SIDs of time-independent navigational attributes). What is SAP said about the X-table similarly applies to the Y-table (characteristic SIDs and SIDs ofSAP time-dependent navigational attributes). An X-table only has a primary index. However, depending on actual queries it might be useful to set up additional indexes: PartnerUse • On the column(s) of the characteristic values, in this example on the column /BIC/IUCITY • Single column indexes on the columns of the navigational attributes.Internal The decision whether to create such additional indexes or not depends on the actual queries and whether those queries impose restrictions on the respective characteristics or navigational attributes. BI usually translates restrictions to SID-based restriction. Only Thus, indexing the characteristic value column(s) is rarely necessary, e.g. if the restriction is interval-based as such restrictions usually cannot be mapped to an equivalent SID-based restriction. Single column indexes on restricted navigational attributes are more likely to be helpful. Depending on the cardinality of the attribute you might decide to use a bitmap index (low cardinality, e.g. an attribute “color”with a handful of values) or a standard B-tree index (medium or high cardinality, e.g. population numbers as in the example in the graphic). 152 © 2006 SAP AG. All rights reserved. 2006/Q2
  • 161. BW360 Lesson: Indexing for InfoProviders Obviously, one has to find a good trade-off when defining a number of supportive indexes: too many additional indexes can harm data upload processes as those indexes have to be maintained. Therefore, it is not recommended that such indexes are defined arbitrarily. On the other hand, such new indexes can enhance the query performance. InternalOnlyPartner Use Figure 98: Indexing of Master Data SID-Tables SAPSAP An SID-table for a characteristic has two unique B-tree indexes: the primary index on the characteristic value column(s) and an index on the SID column. The unique constraint on the SID column is a double-check to avoid a SID value being inserted twice (multiple SID entries for the same value can have terrible effects in BIs OLAP Partner engine). Both indexes support the frequently used function that converts a given SIDUse into the corresponding characteristic value and vice versa. There is no need for additional indexes on SID-tables. Line Item Dimension and High CardinalityInternal Only 2006/Q2 © 2006 SAP AG. All rights reserved. 153
  • 162. Unit 4: InfoProvider Administration and Tuning BW360 InternalOnly Figure 99: Line Item Dimension and High CardinalityPartner When a line item dimension is created, there is an option for high cardinality. Following is information relating to the cardinality flag for line item dimensions. Use • High Cardinality Cardinality refers to the number of distinct values in the column of a table. SAP ExampleSAP – Column Gender can have only 2 values – M or F this is low cardinality – A column holding a unique document number has a high cardinality • High cardinality flag PartnerUse This should only be set when the dimension table is at least 20% of the size of the fact table. On non-Oracle systems this setting has no effect on index type or queryInternal performance. It may improve loading. Oracle systems (only) - there are two types of indexes, bitmap and b-tree Only – Bitmap indexes are best for read performance on an Infocube. – Bitmap indexes are created by default on each dimension column of a fact table. – Setting the high cardinality flag for a dimension creates a b-tree index instead of bitmap. This is recommended on dimensions which are not used in queries. 154 © 2006 SAP AG. All rights reserved. 2006/Q2
  • 163. BW360 Lesson: Indexing for InfoProviders For Non-Oracle based Systems, setting the high cardinality flag has no impact on the index type, because Bitmap Index are only used with Oracle, but there are some internal checks during the data load which decide on different internal loading strategies. This means for non-Oracle system that the high cardinality flag is used for internal checks and therefore it should be set in case of “high cardinality” on non-Oracle systems. But for non-Oracle systems there is no negative impact of the high cardinality flag on the query performance. InternalOnly The line item dimension flag affects the InfoCube’s structure. It should be used when a dimension has only one characteristic. The high cardinality flag can affect the type of index used on a dimension. It shouldPartner not be selected unless the data model meets specific criteria. Data Store Object Indexing Use Secondary indexes are not created automatically on Data Store Objects. Depending on the query requirements, the indexes must be created manually. SAPSAP PartnerUseInternal Only Figure 100: Data Store Objects Secondary Indexes Under Indexes, you can create secondary indexes from the Index context menu in RSA1. Primary indexes are created automatically by the system. If the values in the index fields uniquely identify each record in the table, select Unique Index from the dialog box. Note that errors can occur during activation if the values are not unique. The description of the indexes is specified by the system. To create a folder for the 2006/Q2 © 2006 SAP AG. All rights reserved. 155
  • 164. Unit 4: InfoProvider Administration and Tuning BW360 indexes, choose Continue from the dialog box. Now you can transfer the required key fields into the index folder using Drag. You can create a maximum of 16 secondary indexes, but the recommendation is less than 5. • Possible additional secondary index on F fact table – SAPnote 363092 – Supports APO-mass-update jobs – Index name contains a letter, e.g. /BIC/FIUSALES~A12 InternalOnly • Primary index for APO – APO InfoCubes APO InfoCubes = InfoCubes with “BI application” – APO cannot cope with duplicates in InfoCubes – APO InfoCubes have a primary index “P” on F fact tablePartner • Unique (rather than non-unique) index on dimension tables – Same reason as for primary index Use • APO includes a BI-system: APO 2.0 is based on BI 1.2, APO 3.0 is based on BI 2.0. There are three APO-specific issues related to indexing an InfoCube: – In order to support a mass-update job, it might be useful to define an SAPSAP additional secondary index on the F fact table of the respective InfoCube. SAP note 363092 describes how to define such an index. Normally, it is not necessary to define additional indexes on InfoCube fact tables and we strongly discourage customers to do that. However, this APO-specific Partner situation is a notable exception. Please note that additionally definedUse indexes should contain at least one letter in ist name. In that way, such indexes are excluded from the regular index dropping and recreation during data uploads (see note 157169).Internal – APO InfoCubes cannot – like BIs OLAP engine – cope with duplicates in the fact tables. This imposes unique constraints on the F- and E-fact tables. While BI compression algorithm guarantees uniqueness in the E-fact table, Only a primary index on the F-fact table is required. Therefore APO InfoCubes have F-fact tables with a primary index (whose name is “P” rather than the standard “0”). Note: APO InfoCubes are InfoCube whose“BI application” is set to “APO”. You can check column BIAPPL in table RSDCUBE to be sure if an InfoCubes BI application is APO. – For the same reason, one has to avoid multiple dimension Ids for the same combination of characteristic values. This can be achieved by defining the concatenated index on a dimension table to be unique. See graphic on indexing of dimension tables. 156 © 2006 SAP AG. All rights reserved. 2006/Q2
  • 165. BW360 Lesson: Indexing for InfoProviders Lesson Summary You should now be able to: • Describe the indexes that are created for InfoCube tables. • Compare indexes used in an Basic InfoCube with Real Time InfoCube indexes. • Describe how InfoCube indexes are read during query runtime. InternalOnlyPartner Use SAPSAP PartnerUseInternal Only 2006/Q2 © 2006 SAP AG. All rights reserved. 157
  • 166. Unit 4: InfoProvider Administration and Tuning BW360 Lesson: Table Partitioning in BI Lesson Overview This lesson describes how table partitioning can be used to optimize performance. Participants will learn how to partition and repartition InfoCubes. Lesson Objectives After completing this lesson, you will be able to: InternalOnly • Describe how table partitioning can be used to optimize performance. • Partition InfoCubes. • Repartition InfoCubes.Partner Business Example Use As an employee of the Data Warehousing department of a chemical company, you need to partition InfoCubes by using Table Partitioning. This is required as partitioning leads to well balanced partitions/fragments, and the same amount of data is loaded into the cube at regular points in time. In addition, partitioning improves SAP read performance and enables you to remove obsolete by simply dropping partitions.SAP PartnerUseInternal Only 158 © 2006 SAP AG. All rights reserved. 2006/Q2
  • 167. BW360 Lesson: Table Partitioning in BI InfoCube Partitioning InternalOnlyPartner Use Figure 101: Table Partitioning SAPSAP PartnerUseInternal Figure 102: Supporting Databases Only This graphic provides an overview of DB platforms that provide table partitioning. 2006/Q2 © 2006 SAP AG. All rights reserved. 159
  • 168. Unit 4: InfoProvider Administration and Tuning BW360 In BI there are two types of Table partitioning: • Range partitioning • Multi-Dimensional Clustering Range Partitioning is supported on these databases: • DB2/OS390 • DB2 for z/OS • DB2/AS400 InternalOnly • Oracle • MSSQL Multi-Dimensional Clustering is only supported on DB2 UDB for Linux Unix Windows.Partner Use SAPSAP PartnerUseInternal Only Figure 103: Low Level: InfoCube Partitioning Example (1) This graphic illustrates the star schema of an InfoCube with a range-partitioned fact table. The boxes show various tables; the lines show foreign key - key relationships between those tables. Each query on this star schema materializes a certain subset of those relationships. In BI, fact tables can be partitioned over a time characteristic either 0CALMONTH (as in the example here) or 0FISCPER. The reasons for this approach are the following: 160 © 2006 SAP AG. All rights reserved. 2006/Q2
  • 169. BW360 Lesson: Table Partitioning in BI The DBMS requires that ranges are defined when the fact table is created, so you need to know the future values of the partition column in advance. This is straightforward for 0CALMONTH or 0FISCPER, as users normally have a clear idea of the time frame their data. In other words, when you specify an InfoCube to be partitioned, you only have to know the time interval for which data will be loaded into the cube. Almost every reasonable OLAP query has a restriction based on time. Such restrictions can be exploited by the query optimizer to focus on those fact table partitions/fragments that contain the relevant data. The technical term for this is partition pruning, as the query processing is pruned to the relevant partitions/fragments InternalOnly of the fact table. Irrelevant partitions/fragments can be discarded at an early stage. Partitioning over time usually leads to well-balanced partitions/fragments, as the same amount of data is loaded into the cube at regular points in time. Archiving works over time: data before a certain date is archived rather than dataPartner for a certain sales region. Use SAPSAP PartnerUseInternal Only Figure 104: Low Level: InfoCube Partitioning Example (2) When selecting data for 05.1999, only the partition “Apr – Jun 99” is accessed. 2006/Q2 © 2006 SAP AG. All rights reserved. 161
  • 170. Unit 4: InfoProvider Administration and Tuning BW360 InternalOnlyPartner Figure 105: Table Partitioning: E Fact Table Use SAPSAP PartnerUseInternal Figure 106: Table Partitioning: E Fact Table Ranges Only 162 © 2006 SAP AG. All rights reserved. 2006/Q2
  • 171. BW360 Lesson: Table Partitioning in BI InternalOnlyPartner Figure 107: Table Partitioning: Column Added to Fact Tables This graphic shows the indexing scheme of fact tables of a partitioned InfoCube. The Use significant issue here is that an additional column (time characteristic the InfoCube is partitioned by) is added to both the F fact table and the E fact table.. SAPSAP PartnerUseInternal Only Figure 108: Table Partitioning: Storage (SE14) 2006/Q2 © 2006 SAP AG. All rights reserved. 163
  • 172. Unit 4: InfoProvider Administration and Tuning BW360 Low Level: InfoCube Partitioning Tips • Partition InfoCubes in the development system at the time of initial design. • Use the time characteristic that corresponds with most of the reporting scenarios for an InfoCube. • Aggregates are also partitioned like the InfoCube if they contain the partitioning characteristic. Even if the InfoCube is never compressed and the aggregates may be. • Partitioning does not allocate empty space. Instead it determines a way of InternalOnly grouping data as it is loaded into the fact table. It is recommended that partitioning be done before loading any data. However, in many cases it is necessary to partition after data is loaded or if there is a requirement to modify the original partitioning definition.Partner Repartitioning Use With Repartitioning: • non-partitioned InfoCubes and DataStore objects may be partitioned after data has been loaded. SAPSAP • existing partitioning schema for already partitioned InfoCubes and DataStore objects can be modified. PartnerUseInternal Only Figure 109: Databases that support repartitioning. Caution: Backups should be made before Repartitioning. 164 © 2006 SAP AG. All rights reserved. 2006/Q2
  • 173. BW360 Lesson: Table Partitioning in BI InternalOnlyPartner Use Figure 110: Repartitioning There are three options for Repartitioning: Add Partitions, Merge Partitions and Complete Repartitionings.. SAPSAP PartnerUseInternal Only Figure 111: Add Partitions Add Partitions add partitions at the top (split). 2006/Q2 © 2006 SAP AG. All rights reserved. 165
  • 174. Unit 4: InfoProvider Administration and Tuning BW360 Merge Partitions merge partitions at the bottom end of the partitioning schema (merge). InternalOnlyPartner Use Figure 112: Merge Partitions SAPSAP Complete Partitioning completely converts the fact tables of the InfoCube. The system creates shadow tables with the new partitioning schema and copies all of the data from the original tables into the shadow tables. As soon as the copy operation is completed, the system creates indexes and the original table replaces the shadow table. Partner After the system has successfully completed the partitioning request, both fact tablesUse exist in the original state (shadow table), as well as in the modified state with the new partitioning schema (original table). You can manually delete the shadow tables after repartitioning has been successfully completed in order to free up the memory. Shadow tables have the name space /BIC/4F (F fact table) or /BIC/4E (E Fact table).Internal Complete Partitioning is only available for InfoCubes. Note: The repartitioning of DataStore Objects is currently supported only on Only DB2 UDB for Linux, UNIX, WINDOWS. 166 © 2006 SAP AG. All rights reserved. 2006/Q2
  • 175. BW360 Lesson: Table Partitioning in BI InternalOnlyPartner Figure 113: Complete Repartitioning Use Persistent Staging Area Partitioning SAPSAP PartnerUseInternal Only Figure 114: Table Partitioning: Persistent Staging Area The persistent staging area (PSA) is intended for temporary data storage within the BI system. The PSA is a transparent table for storing the detailed requests in the format of the DataSource 2006/Q2 © 2006 SAP AG. All rights reserved. 167
  • 176. Unit 4: InfoProvider Administration and Tuning BW360 The PSA is defined according to DataSource and stores the key request number, packet number, and record number. Updates of PSA and InfoCube can run sequentially or in parallel. If there are errors in the data that flows to the Data Store Object, this data can be deleted and reloaded from the PSA. To avoid large tables, data should be deleted periodically from the PSA. Data from an extraction in the PSA should exist for only a few weeks after a successful load. the PSA is not intended to hold data permanently. Routine jobs InternalOnly should be scheduled to delete data from the PSA, after the data is no longer needed by the applications.Partner Use SAPSAP PartnerUseInternal Figure 115: PSA Partitioning Example In this example, using Oracle, partitioning of the PSA is a range partition based on a Only partition number that is assigned to each request. Partitioning provides for faster data load performance. Array inserts are possible. Inserts occur into smaller database objects. Partitioning provides the ability to easily maintain PSA as a temporary data staging area. 168 © 2006 SAP AG. All rights reserved. 2006/Q2
  • 177. BW360 Lesson: Table Partitioning in BI Lesson Summary You should now be able to: • Describe how table partitioning can be used to optimize performance. • Partition InfoCubes. • Repartition InfoCubes. Related Information Related SAP Notes: InternalOnly 943988 Repartitioning feature is not accessible from RSA1. 895539 Appending/extending partitions to an E fact table. 869407 Partitioning on MS SQL Server 2005.Partner 590370 Too many uncompressed request (f table partitions). 524456 Informix Use 485878 DB2 UDB OS390 SAPSAP PartnerUseInternal Only 2006/Q2 © 2006 SAP AG. All rights reserved. 169
  • 178. Unit Summary BW360 Unit Summary You should now be able to: • Display InfoCube Content. • Compress InfoCubes. • Delete data from an InfoCube. • Reconstruct an InfoCube. • Remodel an InfoCube. • Explain the process of reading from an Data Store Object. InternalOnly • Describe the process of reporting on an Data Store Object. • Delete data from a Data Store Object. • Describe a B-tree index. • Explain how indexes are read at query runtime.Partner • List several BI tools that support the maintenance of indexes. • Describe the indexes that are created for InfoCube tables. Use • Compare indexes used in an Basic InfoCube with Real Time InfoCube indexes. • Describe how InfoCube indexes are read during query runtime. • Describe how table partitioning can be used to optimize performance. SAPSAP • Partition InfoCubes. • Repartition InfoCubes. PartnerUseInternal Only 170 © 2006 SAP AG. All rights reserved. 2006/Q2
  • 179. Unit 5 Reporting Performance InternalOnly Unit Overview The focus of this unit is BI query performance. Query performance begins with a good design. This unit presents some general guidelines for defining queries for optimal performance. This unit also introduces the concept of caching for queryPartner performance. OLAP caching, Read Mode and methods to support pre-calculation are also discussed in this unit. Use Unit Objectives After completing this unit, you will be able to: SAPSAP • List several design techniques for optimal query performance. • Describe how the OLAP cache is used for query performance. • Discuss how the OLAP processor reads data from the InfoProvider during navigation. PartnerUse • Describe how to configure OLAP cache settings. • List the different OLAP cache options. • List the different types of Read Modes.Internal • Describe how to set Read Mode at the InfoProvider level and at the query level. Unit Contents Only Lesson: Design Guidelines for Optimal Query Performance.................. 172 Lesson: OLAP Caching ........................................................... 179 2006/Q2 © 2006 SAP AG. All rights reserved. 171
  • 180. Unit 5: Reporting Performance BW360 Lesson: Design Guidelines for Optimal Query Performance Lesson Overview This lesson focuses on query design for perfromance. The BEx environment is very flexible and offers a wide range of options for query development. This lesson provides some general guidelines for query performance. Lesson Objectives InternalOnly After completing this lesson, you will be able to: • List several design techniques for optimal query performance. Business ExamplePartner As a BI query developer, you need to follow design guidelines for optimal query performance. Use Business Explorer and Users Recent analyst studies have concluded that 70 percent of business users belong to the SAPSAP information consumers group. Information consumers require less analytical power than information analysts and information authors. The majority of business users, therefore, need an interface that is simple to use, and gives them access to all the information they need. PartnerUse Information analysts are users who require far more analysis functions than information consumers. Information authors have the most sophisticated requirements of all.Internal The main goal for site builders must be good support: • Consumers need predefined data collections. Only • Consumers need predefined navigation paths. • Consumers need self-explanatory ways to navigate. Authors create queries depending on the roles of consumers and analysts. Analysts work with reports, analyze them, and use slicing and dicing techniques to uncover important information. Consumers need very specific information presented to them in a way that enables them to access the information easily. 172 © 2006 SAP AG. All rights reserved. 2006/Q2
  • 181. BW360 Lesson: Design Guidelines for Optimal Query Performance InternalOnlyPartner Use Figure 116: Business Explorer for Reporting and Analysis Business Explorer enhances and extends BI by providing an integrated and state-of-the-art modeling environment. Business Explorer also provides a seamless integration with SAP Netweaver Portal. SAPSAP PartnerUseInternal Only Figure 117: Variants of IT Scenario: Enterprise Reporting, Query and Analysis 2006/Q2 © 2006 SAP AG. All rights reserved. 173
  • 182. Unit 5: Reporting Performance BW360 In BI there are several variants of IT Scenario, Enterprise Reporting Query and Analysis, to support the business requirements of customers and developers. This IT scenario shows how business experts can use the reporting and analysis tools available in BI to provide information customized for the different categories of user within your company. The tools in the BI Suite, Business Explorer (BEx), allow business experts to create and distribute the necessary reports and analyses. These tools are integrated with each other and have user-friendly, intuitive interfaces. Detailed analysis of BI information can be performed both on the Web and in Microsoft InternalOnly Excel. Formatted reports can be defined to optimize presentation and printing. Web applications and formatted reports can be easily converted into PDF files and printed. Users access BI information using the portal. Users can execute information broadcasting functions, for example, to send this information by e-mail, or broadcastPartner it to Knowledge Management (KM folders) and collaboration rooms. In addition, the integration of BI in SAP NetWeaver Visual Composer provides the Use option of including BI data in composite applications, which users can access in the portal. Business experts create iViews or portal pages based on data from SAP or third-party systems. The data can be stored in relational or multidimensional format. A set of query templates that incorporate specific business aspects, and user-friendly SAP operation in SAP NetWeaver Visual Composer both support business experts inSAP creating BI applications. Query Design for Optimal Performance PartnerUse The design of queries can have a significant impact on the performance. Sometimes long running queries are the result of poor design, not just the amount of data. There are a number of design techniques that developers can use to provide optimal query performance.Internal For example, in most cases characteristics should be placed in the rows and key figures in the columns. A characteristic should only be used in the columns in certain Only circumstances (like time). Characteristics having potentially many values (such as 0MATERIAL) must not be added to the columns without a filter or variables. Alternatively, it can be integrated into the query as a free characteristic – enabling it to be used in navigation. If a relatively detailed time characteristic, such as calendar day (0CALDAY) is added to the rows, the more aggregated time characteristics (such as calendar month (0CALMONTH)) and calendar year (0CALYEAR) should also be included in the free characteristics of the query. For most reports, a current period of time (current month, previous or current calendar year) is useful. For this reason, the use of variables is particularly relevant for time characteristics. 174 © 2006 SAP AG. All rights reserved. 2006/Q2
  • 183. BW360 Lesson: Design Guidelines for Optimal Query Performance InternalOnlyPartner Use Figure 118: Query Guidelines: Variables and Drop Down Lists Variables and drop down lists can improve query performance by making the data SAP request more specific. This is very important for queries against Data Store ObjectsSAP and InfoSets, which are not aggregated like InfoCubes. When queries are executed on an InfoSet with a large number of single value selections, some selections are often missing in the where condition of the generated PartnerUse SQL statement. This either results in poor system performance or in termination due to limitations given by the relevant database platform. In the IMG, RSCUSTV19, it is possible to set a numerical value to represent the maximum number of single value selections allowed (300, for example). If aInternal characteristic has 300 single value selections they are still displayed in the form of a where condition. If a characteristic has 305 single value selections, a virtual InfoProvider is generated dynamically and a join is added. The single value selections Only are no longer processed by the where statement but by the dynamically generated InfoProvider. Also see SAP Note 673575. 2006/Q2 © 2006 SAP AG. All rights reserved. 175
  • 184. Unit 5: Reporting Performance BW360 InternalOnly Figure 119: Selections: Include and Exclude When using restricted key figures, filters or selections, try to avoid the Exclusion option if possible. Only characteristics in the inclusion can use database indexes. Characteristics in the exclusion cannot use indexes.Partner Use SAPSAP PartnerUseInternal Figure 120: Queries on MultiProviders When a query is run against a MultiProvider, all of InfoProviders in that MultiProvider Only are read. The selection of the InfoProviders in a MultiProvider query can be controlled by restricting the virtual characteristic 0INFOPROVIDER to only read the InfoProviders that are needed. In this way, there will be no unnecessary database reads. 176 © 2006 SAP AG. All rights reserved. 2006/Q2
  • 185. BW360 Lesson: Design Guidelines for Optimal Query Performance InternalOnlyPartner Figure 121: Summary: General Guidelines for Optimal Query Performance Use Note: Defining calculated key figures at the InfoProvider level instead of the query level will improve query runtime performance, but may add time for data loads. SAPSAP Additional Query Design Tips: Cell calculation by means of the cell editor generates separate queries at query Partner runtime. Be cautious with cell calculations.Use Customer-specific code is necessary for virtual key figures and characteristics. Check Code in Customer Exits. Using graphics in queries, such as charts, can have a performance impact.Internal Only 2006/Q2 © 2006 SAP AG. All rights reserved. 177
  • 186. Unit 5: Reporting Performance BW360 Lesson Summary You should now be able to: • List several design techniques for optimal query performance. InternalOnlyPartner Use SAPSAP PartnerUseInternal Only 178 © 2006 SAP AG. All rights reserved. 2006/Q2
  • 187. BW360 Lesson: OLAP Caching Lesson: OLAP Caching Lesson Overview This lesson discusses the use of the OLAP cache to improve BI query performance. Participants will learn how to configure the OLAP cache for optimal query preformance, as well as, analyze the cache contents. This lesson also discusses the different methods the OLAP processor uses to read data from the InfoProvider during navigation. InternalOnly Lesson Objectives After completing this lesson, you will be able to: • Describe how the OLAP cache is used for query performance.Partner • Discuss how the OLAP processor reads data from the InfoProvider during navigation. Use • Describe how to configure OLAP cache settings. • List the different OLAP cache options. • List the different types of Read Modes. • Describe how to set Read Mode at the InfoProvider level and at the query level. SAPSAP Business Example You need to analyze and improve query performance. One important method is to use PartnerUse OLAP caching. You need to configure and analyze OLAP caching for your queries.Internal Only 2006/Q2 © 2006 SAP AG. All rights reserved. 179
  • 188. Unit 5: Reporting Performance BW360 OLAP Caching InternalOnlyPartner Use Figure 122: Query Cache: InfoCubes and Queries SAP BI data is analyzed by defining queries on InfoProviders. Queries are defined bySAP selecting characteristics and key figures. Because of the multidimensional nature of the query data, you can create various views of a dataset. The OLAP processor structures queries from the InfoProvider data PartnerUse and provides methods for navigating through the data in several dimensions. The Business Explorer requests the data from the InfoProvider and presents the current view of the stored data. Only the query data that is actually required is transferred. If you want a different view of the data when navigating, you can get it from theInternal InfoProvider with the OLAP processor. The data from the current and previous drill downs always remains on the application server. Only The global OLAP Cache buffers query result data, in order to provide them for further accesses. Thereby query performance can be improved significantly. To optimize the usage of the OLAP Cache there are two possibilities to store keep query result set data: either in the main memory (distributed over one or more application servers) or persistently. 180 © 2006 SAP AG. All rights reserved. 2006/Q2
  • 189. BW360 Lesson: OLAP Caching The option you choose depends on the following questions: • How often is the query requested? Result data of Queries that are requested frequently should be held in cache. • How complex is the query? Complex result data of queries, processed by the OLAP Processor should be held in cache. • How often is data loaded? If query-relevant data is often changed and therefore has to be loaded frequently, caching is barely advantageous, because the cache has to be regenerated every time. InternalOnly If caching is not desired, you have the possibility to globally switch off caching in customizing, RSCUSTV14.Partner Use SAPSAP PartnerUse Figure 123: Customizing: Switching Caching off or onInternal Cache Modes Only The cache mode determines whether and in what way the query results and navigational states (calculated by the OLAP processor as highly compressed data) are to be saved in the OLAP Cache. In order to achieve an ideal usage of the OLAP Cache, there are the following modes: Cache is inactive (0) The cross-transactional caching is deactivated. That means that no result data will be held in cache. Main Memory Cache without Swapping (1) 2006/Q2 © 2006 SAP AG. All rights reserved. 181
  • 190. Unit 5: Reporting Performance BW360 The cached data is stored in the main memory. In case that memory is exhausted, excess data will be removed according to the LRU algorithm, i.e deleted from the memory. When a query is subsequently requested, the result has to be read from the InfoProvider once again. Main Memory Cache with Swapping (2) The cached data is stored in the main memory as well. If cache memory is used up, the excess data will be written to a background store (file, clustered table or BLOB InternalOnly table; see next chapter) and could be loaded back into the OLAP Cache memory when executing another request. After new data has been loaded into a query-relevant InfoProvider again, the background store will no longer be used.Partner Cluster/BLOB/FlatFile Cache per Application Server The cached data is stored persistently as cluster tables or BLOB tables (BLOB = Use binary large object) in a database or as a file in a directory attainable from the application server. It’s generally recommended to choose a directory that is proximate to the application server. This table or file can only be accessed or written by the associated application server. SAPSAP That also means that there is an own table or file for each application server. Cross-Application Server Cache Cluster/BLOB/Flatfile Cache The cached data is stored persistently as a cross-application server cluster, BLOB table PartnerUse or file in a file system in the network, accessible from all application servers. In that cache mode, only one cluster/blob table or Flat File is existing, but can be accessed and written by all application servers.Internal Only 182 © 2006 SAP AG. All rights reserved. 2006/Q2
  • 191. BW360 Lesson: OLAP Caching InternalOnly Figure 124: Cache ModesPartner LRU Algorithm If main memory cache is exhausted, but more data has to be written into the cache, the Use Least-Recently-Used-Algorithm removes or swaps query result data. To get a closer look to the LRU you should first imagine that the entries were ordered like in the figure below. SAPSAP PartnerUseInternal Only Figure 125: Ring Buffer and Ring Pointer of the LRU 2006/Q2 © 2006 SAP AG. All rights reserved. 183
  • 192. Unit 5: Reporting Performance BW360 The first cache entry is signed with a so-called Ring Pointer. He tells the LRU where to start looking for entries that can be removed or swapped. In case that cache is exhausted the LRU will pass the entries clockwise. If a suitable one is found, the Ring Pointer is placed on the subsequent Cache-Entry. The next figure shows a part of the Cache Monitor screen for Main Memory Caching. InternalOnlyPartner Use Figure 126: Cache Monitor; Flags SAP The meaning of the relevant flags that can be seen in the figure is as followsSAP Flag Description Read shows whether the entry is read or not PartnerUse no flag means the entry has been written to the cache but not read again Write shows that an entry is written but has not been read already Dirty signal that tells the LRU, that this entry has to be swapped into aInternal background memory (flatfile, cluster or BLOB) Swapped shows that the entry is swapped into a flatfile, cluster or BLOB table Only Because of some differences of the LRU between Main Memory Cache without Swapping and Main Memory Cache with Swapping, they will be discussed separately in the following chapters. Main Memory Cache without Swapping In this Cache Mode only the READ-Flag is used, WRITE, DIRTY and SWAPPED are meaningless. The LRU checks each entry of the ring, whether the flag is set or not. 184 © 2006 SAP AG. All rights reserved. 2006/Q2
  • 193. BW360 Lesson: OLAP Caching There are two possible cases: • READ-Flag is set → Flag will be removed • READ-Flag is not set → Cache-Entry will be removed InternalOnlyPartner Figure 127: LRU, Main Memory Cache without Swapping Main Memory with Swapping Use For this Cache Mode all flags are important. The following situations can occur: Only WRITE is flagged: SAPSAP WRITE will be removed and DIRTY will be set Only DIRTY is set: The cache entry will be swapped into a background store, DIRTY will be removed, Partner SWAPPED is setUse READ and WRITE are flagged: WRITE will be removed, DIRTY is setInternal READ and DIRTY are flagged: READ and DIRTY will be removed, SWAPPED will be set and the cache entry is written to the background store Only 2006/Q2 © 2006 SAP AG. All rights reserved. 185
  • 194. Unit 5: Reporting Performance BW360 InternalOnly Figure 128: LRU, Main Memory with Swapping Query Monitor (RSRT)Partner The query monitor tests, checks and manages BI queries. By using the query monitor, you can test queries or query views, as well as regenerate, check, or change query properties. Use With the query monitor display of technical information about queries is possible and additionally it allows the entry into the cache monitor. Cache Monitor (RSRCACHE) SAPSAP From the Cache Monitor screen you are able to get an overview about the global cache parameters, the amount of memory used by the query runtime objects and the current underlying cache structure. PartnerUseInternal Only Figure 129: Cache Monitor To reach the OLAP cache monitor ( RSRCACHE) from the SAP Easy Access screen in the Business Explorer → BEx Monitor → OLAP: Cache Monitor. 186 © 2006 SAP AG. All rights reserved. 2006/Q2
  • 195. BW360 Lesson: OLAP Caching The starting screen of the cache monitor shows the different parameters, set for the OLAP Cache. The structure in the various screen always looks equal and is shown and described in the following. In a logical point of view the OLAP Cache is created hierarchically and therefore is equal to the hierarchical display of buffered objects. InternalOnlyPartner Use SAPSAP Figure 130: Historical cache structure The directory tree of the OLAP Cache is divided into four levels. PartnerUse For every executed query an own directory will be created. The name of this directory is determined by the technical name of the Info Provider and the query. The entries of these directories contain the actual result data of the different queries.Internal By a double click on an entry a dialog appears which shows information about hierarchies or variables included in the query. The physical view of the OLAP Cache can be seen in the list display of buffered Only objects. All entries are shown in a flat structure and their order depends on the chronological order in which the data has been read on the database. Like in the historical view you get detailed information about hierarchies and variables. 2006/Q2 © 2006 SAP AG. All rights reserved. 187
  • 196. Unit 5: Reporting Performance BW360 InternalOnlyPartner Figure 131: List View of Cache Structure Use The Technical Information gives an overview of the maximum cache size, current cache size, reserved cache and current cache entries for the runtime object. Additionally the buffer poll time, reserved buffer and buffer setting cache of the SAPSAP shared memory are displayed. PartnerUseInternal Only Figure 132: Technical Information 188 © 2006 SAP AG. All rights reserved. 2006/Q2
  • 197. BW360 Lesson: OLAP Caching Query Properties In the Query Properties dialog box of the query monitor settings can be selected for a BI query with regard to the read mode, the cache mode, the selection of structure elements, the optimization mode and the calculation accuracy. By default the cache setting is turned on, except for queries that use virtual characteristics or key figures. For queries with virtual characteristics or key figures cache is turned off by default. From RSRT, the use of cache for queries with virtual characteristics or key figures can be activated. InternalOnly The delta process enables individual data from a query to be inserted into the cache. In this way, when they are changes to the data basis, the complete data for the query no longer has to be read from the database again. The results of a query are thus independent of whether the data is read directly from the database or from the cache.Partner A complete restructuring of the cache is always necessary, however, if metadata, master data , hierarchies, exchange rates, etc. change. But a complete rebuild of the cache object also has to be performed for InfoProviders that are not suitable for the Use delta process when there are changes. InfoProviders are delta process enabled when a time stamp, sequential number, or, as with InfoCubes, the request shows which entries have been added. SAPSAP PartnerUseInternal Only Figure 133: Delta Caching Parallel Processing during Query Execution 2006/Q2 © 2006 SAP AG. All rights reserved. 189
  • 198. Unit 5: Reporting Performance BW360 A query can be divided into sub-queries by the system. If dividing the query results in more than one sub-query, the read operation is performed in parallel by default. Note: Queries that are based on non-cumulative InfoCubes have a high memory consumption and cannot be processed in parallel. For this reason, all queries that are based on non-cumulative InfoCubes are divided into sub-queries that are separated from the other sub-queries. The ‘normal’ sub-queries are processed in parallel first and the sub-queries that are based on the non-cumulatives are processed sequentially afterwards. InternalOnlyPartner Use SAPSAP PartnerUse Figure 134: Query Splits Parallelized Several dialog work processes are required in order to execute queries in parallel. The maximum degree of parallelism determines the maximum number of work processesInternal that are used for each query. This value is restricted to 6 by default. The maximum value can be changed to a value between 1 and 100 in the QUERY_MAX_WP_DIAG entry in table RSADMIN. Only The actual degree to which queries are executed in parallel depends on the load on the system at any given time and lies between 1 (sequential processing) and the maximum value. If the number of sub-queries is greater than the maximum level of parallelism, all existing sub-queries are divided between the work processes determined by the degree of parallelism. The results of all sub-queries are collected at a synchronization point and collated to form an overall result. In sequential processing, the sub-queries are processed one after another. The interim result is immediately passed on to the analytic engine. 190 © 2006 SAP AG. All rights reserved. 2006/Q2
  • 199. BW360 Lesson: OLAP Caching A query on a MultiProvider is deconstructed internally into sub-queries. These are processed in parallel by default. In most cases parallel processing has benefits in terms of performance. However, in some cases sequential processing can be more advantageous for query performance. Parallel processing for a query on a MultiProvider can be switched off in the Query Monitor Query Properties. MultiProvider OLAP cache settings can be made using the field partitioning type: • No Partitioning • Partitioning in Groups InternalOnly • Partitioning in Groups: Basis Cubes Separate • Each InfoProvider SeparatePartner Use SAPSAP PartnerUseInternal Figure 135: OLAP Cache for MultiProviders Only 2006/Q2 © 2006 SAP AG. All rights reserved. 191
  • 200. Unit 5: Reporting Performance BW360 The OLAP processor determines the optimal initial size of the table for caching query results. This is done after the first execution and again after a period of 31 days since the last optimization. It automatically generates a report afterwards. In the Query Properties dialog box in the Query Monitor, you can implement, modify, or deactivate this function. Choose one of the following options: • (0) Query Is Optimized after Generation • (1) Query Optimization with Individual Periods in Days. Flexible option to determine the optimization period. • (9) Query Optimization Inactive InternalOnly Queries may also be regenerated. To view the reasons why a query was generated again, press the generation log push button afterwards. Read ModePartner Use SAPSAP PartnerUseInternal Only Figure 136: Read Modes The read mode determines how often the OLAP processor reads data from the database during navigation. For a query, the OLAP processor can read the data from the fact table in one of three ways: • Read all the data. • Read the data during navigation. • Read during navigation and when expanding the hierarchy. 192 © 2006 SAP AG. All rights reserved. 2006/Q2
  • 201. BW360 Lesson: OLAP Caching Only presentation hierarchies have an influence on the read mode, not hierarchies which are used as a filter condition in the query. Run your query with different read modes to verify the right read mode. Switching the read mode can slow down the runtime of a query if you dont have fitting aggregates for this read mode. InternalOnlyPartner Use SAPSAP Figure 137: Read All Data PartnerUse If the query uses the read mode Read all data, all the data that may be required is read on the database. In the first screen of this example, only Information grouped by customer is needed.Internal Drill downs and navigations are possible based on country. In this query, product information is not necessary. With this read mode, the query collects the information grouped by customer and country from the database. Further drill downs can be Only performed in memory. Since a large amount of information is kept in memory, a large amount of memory is consumed. In this case, if you do not want to drill down, you do not need the information in memory but you must still wait a long time for all this information to be collected from the database. For small InfoCubes, it is sometimes better to use the read mode Read all data. 2006/Q2 © 2006 SAP AG. All rights reserved. 193
  • 202. Unit 5: Reporting Performance BW360 InternalOnlyPartner Figure 138: Read During Navigation and Hierarchy Expansion Use If the query uses the read mode Read during navigation or Read during navigation and during hierarchy expansion, only the data that may be required for the initial view is read from the database. SAPSAP In the first screen of this example, only Information grouped by country to the third level of the country hierarchy is needed. Drill downs and navigations are possible only for country. With this read mode, the query only collects the information grouped by country from the database. Further drill downs on country or customer require an PartnerUse access of the database. Smaller amounts of information are kept in extended memory, so less extended memory is consumed. In this case, if you do not want to drill down, you do not need the information in extended memory, so you will not have to wait a long time for all this information toInternal be collected from the database. Only 194 © 2006 SAP AG. All rights reserved. 2006/Q2
  • 203. BW360 Lesson: OLAP Caching InternalOnlyPartner Use Figure 139: Trade-Offs The main differences between the three read modes are as follows: • Read all data SAPSAP A query accesses the database only once, but a large number of rows are read (even if you do not want the use this information). This results in a high database runtime and consumption of a large amount of extended memory. Partner • Read data during navigationUse A query accesses the database during navigation, but if presentation hierarchies, which can be expanded, are used, the information is read grouped by the characteristic this presentation hierarchy belongs to. As a result, shorter databaseInternal runtimes and less extended memory consumption. • Read data during navigation & expanding the hierarchy Only A query accesses the database to collect only the data actually needed, so long as appropriate aggregates are available. As a result, the shortest database runtime and the least extended memory consumption. How to Set Read Modes Query Read Mode can be set at the InfoProvider level in InfoProvider Maintenance (RSA1), or at the query level in the Query Monitor( RSRT). 2006/Q2 © 2006 SAP AG. All rights reserved. 195
  • 204. Unit 5: Reporting Performance BW360 In InfoProvider maintenance, you can specify the standard settings for an InfoProvider read mode. These settings will be valid for future queries that are created for the corresponding InfoProvider. Choose Environment→InfoProvider Properties→ Change and specify the desired read mode on the Query/Cache tab page. For the InfoProviders for which data changes do not occur under the control of the BI system (for example, VirtualProviders and transactional DataStore objects), specify the cache validity, meaning the data retention period in the cache. Table RSDCUBE enables you to check the default read mode of all your InfoCubes, table RSRREPDIR for each specific query. To get only the InfoCubes that are InternalOnly currently active and usable in the system, restrict the values of OBJVERS to “A” and OBJSTAT to “ACT”. To see the correct names of the InfoCubes, make sure that flag Check conversion exit is set: choose Settings → User parameters and in tab Data browser make sure that the flag is set.Partner In column READMODE the read mode is specified: H: Query to Read Data on Demand when Navigation/Expanding Hier. Use X: Query to Read Data During Navigation Blank/A: Query to Read All Data At Once SAPSAP PartnerUseInternal Figure 140: Table RSDCUBE Only Table RSDCUBE enables you to check the default read mode of all your InfoCubes. To get only the InfoCubes that are currently active and usable in the system, restrict the values of OBJVERS to “A” and OBJSTAT to “ACT”. 196 © 2006 SAP AG. All rights reserved. 2006/Q2
  • 205. BW360 Lesson: OLAP Caching To see the correct names of the InfoCubes, make sure that flag Check conversion exit is set: Choose Settings → User parameters and in tab Data browser make sure that the flag is set. In column READMODE, the read mode is specified: At the Query level, the predefined default is Read during navigation and when expanding the hierarchy. The default read mode for queries on InfoSets is Read the data during navigation. Read mode can be set in the Query Monitor transaction RSRT. Read mode for each Query is also set in table RSRREPDIR. InternalOnlyPartner Use SAPSAP PartnerUseInternal Only 2006/Q2 © 2006 SAP AG. All rights reserved. 197
  • 206. Unit 5: Reporting Performance BW360 Lesson Summary You should now be able to: • Describe how the OLAP cache is used for query performance. • Discuss how the OLAP processor reads data from the InfoProvider during navigation. • Describe how to configure OLAP cache settings. • List the different OLAP cache options. InternalOnly • List the different types of Read Modes. • Describe how to set Read Mode at the InfoProvider level and at the query level.Partner Use SAPSAP PartnerUseInternal Only 198 © 2006 SAP AG. All rights reserved. 2006/Q2
  • 207. BW360 Unit Summary Unit Summary You should now be able to: • List several design techniques for optimal query performance. • Describe how the OLAP cache is used for query performance. • Discuss how the OLAP processor reads data from the InfoProvider during navigation. • Describe how to configure OLAP cache settings. • List the different OLAP cache options. InternalOnly • List the different types of Read Modes. • Describe how to set Read Mode at the InfoProvider level and at the query level.Partner Use SAPSAP PartnerUseInternal Only 2006/Q2 © 2006 SAP AG. All rights reserved. 199
  • 208. Unit Summary BW360 InternalOnlyPartner Use SAPSAP PartnerUseInternal Only 200 © 2006 SAP AG. All rights reserved. 2006/Q2
  • 209. Unit 6 BI Statistics and Monitoring InternalOnly Unit Overview The focus of the unit is the analysis of BI statistics. To begin with, you will learn about the differences between statistics in the OLTP system and statistics in BI. Next, you will learn about the activation of BI Statistics Technical Content. Finally, you willPartner learn how to use the delivered tools to analyze BI specific workload. Unit Objectives Use After completing this unit, you will be able to: • Describe how to configure the collection of BI statistics. SAPSAP • Define the concept of Event ID. • List several components of query runtime statistics. • Describe the statistics that are gathered during data load. • Analyze Query Runtime statistics using the Query Monitor. PartnerUse • Explain how to activate BI Statistics Technical Content. • Describe the components of BI StatisticsTechnical Content. • Describe the architecture for collecting BI statistics.Internal • List several methods for analyzing BI Statistics. • Describe the motivating factors for BI Administrator Cockpit. Only • List the prerequisites for BI Administrator Cockpit. • Describe how to install the BI Administrator Cockpit. • List the capabilities of the BI trace tool. • Explain how to log and playback a trace. • Describe how trace attributes are used. Unit Contents Lesson: BI Statistics ................................................................ 203 Lesson: BI Statistics Technical Content.......................................... 210 2006/Q2 © 2006 SAP AG. All rights reserved. 201
  • 210. Unit 6: BI Statistics and Monitoring BW360 Lesson: BI Administrator Cockpit ................................................. 223 Lesson: BI Trace Tool .............................................................. 232 InternalOnlyPartner Use SAPSAP PartnerUseInternal Only 202 © 2006 SAP AG. All rights reserved. 2006/Q2
  • 211. BW360 Lesson: BI Statistics Lesson: BI Statistics Lesson Overview This lesson discusses the implementation and use of BI Statistics. Participants will learn how to configure and maintain BI Statistics. Lesson Objectives After completing this lesson, you will be able to: InternalOnly • Describe how to configure the collection of BI statistics. • Define the concept of Event ID. • List several components of query runtime statistics.Partner • Describe the statistics that are gathered during data load. • Analyze Query Runtime statistics using the Query Monitor. Use Business Example Both query performance and data load performance must be improved on your systems. You must begin by doing some overall analysis of query runtime and data SAPSAP load cycles. It is necessary for you to have access to important statistics that provide the required information for your analysis. Comparison of OLTP systems with BI PartnerUse • Features of OLTP (mySAP ERP) : Mixture of read and write activitiesInternal Many different transactions, such as VA01, FD01, and batchinput. • Features of BI: Only Execution of queries is the dominant type of processing with a clearly defined sequence, but customer-defined content. Customer defined queries, which have to be tuned by the customer. More detailed data about the different steps of a query execution is required for tuning activities. The standard workload monitor, ST03 (Administrator), does not provide the statistics needed for analysis in BI. 2006/Q2 © 2006 SAP AG. All rights reserved. 203
  • 212. Unit 6: BI Statistics and Monitoring BW360 Special BI statistics are necessary for tuning BI objects and analyzing data load performance. The average response time statistics in ST03 (Administrator) are not a valuable performance measurement in BI, because other factors, such as the structure of a query and size and design of the corresponding InfoProvider, influence the runtime and therefore the response time. While defining an maximum average response time in mySAP ERP is useful, this type of measurement is not applicable in BI. InternalOnly Statistics for BIPartner Use SAPSAP PartnerUseInternal Figure 141: Query Runtime Statistics Only 204 © 2006 SAP AG. All rights reserved. 2006/Q2
  • 213. BW360 Lesson: BI Statistics For each query, the OLAP processor collects statistical data from many levels. • Front end / OLAP Statistics and Data Manager Statistics: Splitting the BW 3.x OLAP statistics into Frontend / OLAP statistics and Data Manager statistics. Serial execution (Front-End/OLAP) vs. parallel execution (Data Manager, mostly). Many different events (FE/OLAP) vs. small number of events (Data Manager) InternalOnly • Statistic detail levels Statistic detail levels on query level. The level of detail of statistic data collection for a certain object is customizable.Partner • Event Concept Event ID (characteristic) to uniquely identify each step in the process. Use Flexible and extensible framework. • Enhanced coverage of BI functionality Enhanced: BEx Web, Planning SAPSAP New: BI Accelerator PartnerUseInternal Only 2006/Q2 © 2006 SAP AG. All rights reserved. 205
  • 214. Unit 6: BI Statistics and Monitoring BW360 InternalOnlyPartner Use Figure 142: Data Load Statistics SAPSAP PartnerUseInternal Only Figure 143: BI Statistics 206 © 2006 SAP AG. All rights reserved. 2006/Q2
  • 215. BW360 Lesson: BI Statistics Default settings for statistic data collection • (Delivered) System Default: D = ‘X’ = On, Statistic Detail level: D = ‘2’ = All details • System Default for all queries can be customized: Extras, ChangeDefault. • InfoProvider Default for queries can be customized. All new queries of this InfoProvider will inherit the new setting. Statistic Data Management InternalOnly • Review the (delivered) system default and disable statistic data collection where performance monitoring is not necessary. Example: InfoProviders with low data volume. • Setting Statistic Detail Level ‘2’ for all queries will result in large amounts of data in statistic data collection tables (rule of thumb: ~ 30 - 40 records perPartner query navigation step). • Recommend: Use Frequent transfer to Technical Content InfoProviders. Frequent deletion of Statistics Data (in transaction RSDDSTAT). SAPSAP PartnerUseInternal Only Figure 144: Event ID 2006/Q2 © 2006 SAP AG. All rights reserved. 207
  • 216. Unit 6: BI Statistics and Monitoring BW360 InternalOnlyPartner Use Figure 145: Query Monitor (RSRT): Analyze Query Runtime Statistics SAPSAP PartnerUseInternal Only 208 © 2006 SAP AG. All rights reserved. 2006/Q2
  • 217. BW360 Lesson: BI Statistics Lesson Summary You should now be able to: • Describe how to configure the collection of BI statistics. • Define the concept of Event ID. • List several components of query runtime statistics. • Describe the statistics that are gathered during data load. • Analyze Query Runtime statistics using the Query Monitor. InternalOnlyPartner Use SAPSAP PartnerUseInternal Only 2006/Q2 © 2006 SAP AG. All rights reserved. 209
  • 218. Unit 6: BI Statistics and Monitoring BW360 Lesson: BI Statistics Technical Content Lesson Overview This lesson discusses the activation and implementation of BI statistics Technical Content. Lesson Objectives After completing this lesson, you will be able to: InternalOnly • Explain how to activate BI Statistics Technical Content. • Describe the components of BI StatisticsTechnical Content. • Describe the architecture for collecting BI statistics.Partner • List several methods for analyzing BI Statistics. Business Example Use Technical Business Content for BI statistics is delivered with BI, and must be activated. This is a mandatory procedure, and it is your responsibility to activate all BI statistics Technical Content. You need to understand the purpose of Technical Content SAPSAP and how to implement Technical Content in your environment. Activation of Technical Content PartnerUseInternal Only Figure 146: Activate Technical Content 210 © 2006 SAP AG. All rights reserved. 2006/Q2
  • 219. BW360 Lesson: BI Statistics Technical Content It is important to keep the Enterprise Data Warehouse running as optimally as possible. In order to do this, system administrators rely on accurate statistics that provide information on the overall status of this dynamic environment. Statistics are needed to provide information on the BI reporting environment and OLAP processes. Statistics are also needed to monitor the status of data load processes in the Warehouse Management areas. The tools and objects needed to provide BI statistics are delivered as part of technical InternalOnly content. The Technical Content role, SAP_BW_TCONT, contains all the objects that are required to implement BI statistics. Installation of BI Statistics Technical Content is a mandatory procedure and can be done at the individual InfoCube or MultiProvider level or at the role level.Partner BI statistics Technical Content is installed the same way as BI Content: Data Warehousing Workbench -> Business Content. BI statistics Technical Content includes all the MetaData required to collect, load, Use analyze and monitor BI statistics, including: InfoProviders, InfoObjects, Process Chains, DataSources, DataTransfer Processes, queries and web applications. Installation may take a long time. Therefore, it is recommended to execute in the background. SAPSAP BI Statistics are provided to analyze and monitor: • Query Runtime Statistics • Process Chain and DTP Statistics PartnerUse • BI Object Request and Process Status There is also Technical Content for direct access and analysis on persistent data. By default, queries from the Technical Content filter on reading from persistentInternal InfoProviders only. Customers can enable reading from Virtual Providers on the query level. Technical Content is available on both detailed and aggregated levels for Query Only Runtime Statistics. In order to collect BI statistics for an object, the object must be enabled for statistics. This is done in the maintenance for statistics data collection, transaction RSDDSTAT. In this transaction objects, such as queries and infoproviders, can be enabled for statistics collection. The detail level of the statistics can be determined here as well. Query runtime statistics, are used to determine the execution time of user actions, at the front end, as well as, at the OLAP processor level. The system records the performance-critical parts of the processing (statistics events). It calculates the net times by calculating the runtime of an event using the difference between the start 2006/Q2 © 2006 SAP AG. All rights reserved. 211
  • 220. Unit 6: BI Statistics and Monitoring BW360 and end times (minus the times for other events called from within the event). The statistics data is stored in various tables, and then loaded into the technical content InfoProviders for analysis. InternalOnlyPartner Use SAPSAP Figure 147: Example Query Runtime In this example, the runtime of a BI Web Application is displayed. The data is retrieved from InfoProviders and passed through to the frontend. As the query runs, PartnerUse many statistics are collected: query runtime statistics.Internal Only 212 © 2006 SAP AG. All rights reserved. 2006/Q2
  • 221. BW360 Lesson: BI Statistics Technical Content InternalOnlyPartner Use Figure 148: Technical Content: Query runtime statistics The Technical Content to support collection of query runtime statistics are displayed above. MultiProvider 0TCT_MC01 is created based on the following two SAPSAP InfoProviders: 0TCT_C01 0TCT_VC01 The purpose of having a basic cube and a virtual cube is to provide historical as well as real time or recent data. PartnerUseInternal Only 2006/Q2 © 2006 SAP AG. All rights reserved. 213
  • 222. Unit 6: BI Statistics and Monitoring BW360 InternalOnlyPartner Use Figure 149: Technical Content: Data Load Statistics The Technical Content to support the collection of statistics for data loads is displayed above. MultiProvider 0TCT_MC21 is created based on the following two SAPSAP InfoProviders: 0TCT_C21 0TCT_VC21 The purpose of having a basic cube and a virtual cube is to provide historical as well as real time or recent data for the process chains in the monitors. All the process chain runtime monitors and process runtime monitors access this MultiProvider. PartnerUseInternal Only 214 © 2006 SAP AG. All rights reserved. 2006/Q2
  • 223. BW360 Lesson: BI Statistics Technical Content InternalOnlyPartner Use Figure 150: Technical Content: Data load status This examples shows the Technical Content that supports Data Load status. The MultiProvider 0TCT_MC11 is created based on the InfoProvider 0TCT_VC11. The MultiProvider offers better flexibility. SAPSAP BI Statistics Migration Due to the changes in the OLAP and front-end architecture, previously collected statistics data, from BW 3.X systems, can only be partially used with the new BI PartnerUse release. The structure of the new tables differs greatly from that of the RSDDSTAT table used in BW 3.x systems, therefore data cannot be loaded directly to the new InfoProviders. The BI statistics data must be migrated from the BW 3.X InfoProviders to the new BI InfoProviders.Internal Only 2006/Q2 © 2006 SAP AG. All rights reserved. 215
  • 224. Unit 6: BI Statistics and Monitoring BW360 InternalOnlyPartner Figure 151: Technical Content Migration: Query runtime statistics Use The Technical Content InfoCube, 0BWTC_C02, is no longer used in BI. It is replaced completely with new InfoProviders. SAPSAP PartnerUseInternal Figure 152: Technical Content Migration: Data Load statistics Only 216 © 2006 SAP AG. All rights reserved. 2006/Q2
  • 225. BW360 Lesson: BI Statistics Technical Content Figure 153: Technical Content Migration: Data Load status InternalOnly BI Statistics Technical Content Analysis To analyze BI OLAP/WHM performance, the following tools can be used: • Queries of BI statisticsTechnical Content InfoCubes ( filled by InfoSources thatPartner extract data from RSDDSTAT* tables.) • ST03 for BW. Use • Function module RSDDCVER_RFC_BW_STATISTICS not user friendly, but it is available for RFC • Administrator Cockpit SAPSAP PartnerUseInternal Only Figure 154: BI Statistics Technical Content Views BI statistics data is distributed across various database tables and views. Two predefined database views can be used to analyze query statistics directly. 2006/Q2 © 2006 SAP AG. All rights reserved. 217
  • 226. Unit 6: BI Statistics and Monitoring BW360 The database view RSDDSTAT_DM contains data for the aggregation layer and analytical engine. For example, statistics for database records selected and records transferred to query cache are statistics provided by this view. The database view RSDDSTAT_OLAP contains statistical data related to the frontend and calculation layer of the analytical engine. For example, event ids, navigational step count, and step type are statistics provided by this view. These database views can be read using transaction SE16. InternalOnlyPartner Use SAPSAP PartnerUse Figure 155: Example: Queries in the Technical Content BI Statistics Technical Content also contains a predefined workbook with predefined queries. These queries provide information about BI performance.Internal In order to have access to this workbook, the composite role SAP_BW_TECHNI- CAL_CONTENT must be assigned to your User ID. Only 218 © 2006 SAP AG. All rights reserved. 2006/Q2
  • 227. BW360 Lesson: BI Statistics Technical Content InternalOnlyPartner Use Figure 156: Workload Analysis, ST03, Expert Mode To view the BI workload, choose Expert Mode. BI statistics displayed in the Workload Monitor, ST03, are read from the BI Statistics SAPSAP InfoCubes. It is important to schedule the data loads for these InfoCubes on a regular basis. All necessary objects to load the InfoCubes are delivered with Business Content. PartnerUseInternal Only Figure 157: BI Workload Analysis, ST03 2006/Q2 © 2006 SAP AG. All rights reserved. 219
  • 228. Unit 6: BI Statistics and Monitoring BW360 InternalOnlyPartner Use Figure 158: BI Statistics in BEx Web using Profiling mode Using the display for BI query runtime statistics, the technical information relating to the performance of a Web application can be displayed directly on the Web. Profiling SAPSAP statistics are also displayed. If performance problems occur at the runtime for a Web application, it is possible to send this information to SAP in a customer message for error analysis purposes. Partner Statistics data can be displayed on the Web, by adding parameter PROFILING=X toUse the start URL of the Web application. The dialog for displaying the statistics results appears. In the dialog box, the following data is displayed in the corresponding columns:Internal • individual steps at the runtime for the Web application • summary of each step Only • Java events • OLAP events • data manager events To navigate to the print view for the statistics results, choose Open Print View. 220 © 2006 SAP AG. All rights reserved. 2006/Q2
  • 229. BW360 Lesson: BI Statistics Technical Content InternalOnlyPartner Use Figure 159: Summary: Analysis of BI Statistics using Technical Content • Built in: SAPSAP BI Statistics: Detailed Runtime Statistics Data collection for various BI Objects in Data Warehousing, Enterprise Reporting and Planning PartnerUse • Mandatory: Technical Content (InfoProviders and DataSources): Central Data for BI Administration Cockpit and BI system load, transaction ST03.Internal Persistent Data Storage and Remote Access to BI Statistics Information. Only • Recommended: BI Administration Cockpit (Business Package): Single point of entry and integration with other (non BI related) portal content (example: Universal Work List) Technical Content (Web Application and Queries): Flexible analysis of statistics data and sophisticated presentation of information (graphs, charts, tables) 2006/Q2 © 2006 SAP AG. All rights reserved. 221
  • 230. Unit 6: BI Statistics and Monitoring BW360 Lesson Summary You should now be able to: • Explain how to activate BI Statistics Technical Content. • Describe the components of BI StatisticsTechnical Content. • Describe the architecture for collecting BI statistics. • List several methods for analyzing BI Statistics. InternalOnlyPartner Use SAPSAP PartnerUseInternal Only 222 © 2006 SAP AG. All rights reserved. 2006/Q2
  • 231. BW360 Lesson: BI Administrator Cockpit Lesson: BI Administrator Cockpit Lesson Overview This lesson introduces participants to the Administrator Cockpit, a system monitoring environment, accessible through the SAP Netweaver Portal. Lesson Objectives After completing this lesson, you will be able to: InternalOnly • Describe the motivating factors for BI Administrator Cockpit. • List the prerequisites for BI Administrator Cockpit. • Describe how to install the BI Administrator Cockpit.Partner Business Example Use Administration and monitoring of an Enterprise Data Warehousing environment requires sophisticated tools. The Administrator Cockpit provides system administrators with centralized tools that facilitate system monitoring and error handling across the BI landscape. SAPSAP BI Administrator Cockpit The BI Admististrator Cockpit is a collection of web reports and applications that read Technical Content InfoCubes to display statistics related to system status and PartnerUse performance.Internal Only 2006/Q2 © 2006 SAP AG. All rights reserved. 223
  • 232. Unit 6: BI Statistics and Monitoring BW360 InternalOnlyPartner Use Figure 160: BI Administrator Cockpit Motivation The BI Administrator Cockpit facilitates BI administration tasks by providing concise system related information in the format of easy to read web reports and applications. SAPSAP The BI Administration Cockpit is accessed using the BI Administration 1.0 Business Package in SAP Netweaver 2004s Portal. This Business Package includes the portal content for the Administrator Cockpit, including iViews. worksets, portal pages and roles. The proper role must be assigned to the BI administrator. With this role, the BI Partner administrator will have access to the Administrator Cockpit: System AdministrationUse → Monitoring→ BI.Internal Only 224 © 2006 SAP AG. All rights reserved. 2006/Q2
  • 233. BW360 Lesson: BI Administrator Cockpit InternalOnlyPartner Use Figure 161: BI Administrator Cockpit Overview BI administrators use the BI Administrator Cockpit to monitor system status and performance. The Administrator Cockpit provides an overview of the objects and SAPSAP processes in BI systems. The BI Administrator Cockpit provides system administrators with a central point of access that is focused on critical situations and allows for navigation to more detailed Partner information. The alert framework provides notification for serious problems andUse error handling.Internal Only 2006/Q2 © 2006 SAP AG. All rights reserved. 225
  • 234. Unit 6: BI Statistics and Monitoring BW360 InternalOnlyPartner Use Figure 162: BI Administrator Cockpit Architecture The BI Administration Cockpit is dependent on the activation of delivered BI SAPSAP Technical Content. While the installation of the Administrator Cockpit is optional, the installation of the BI Technical Content is mandatory. The BI Technical Content includes InfoProviders, Process Chains, DataSources, InfoObjects and key figures for all the statistics that are collected. The Technical Content also includes many Partner queries and web applications. These web applications are read from the AdministratorUse Cockpit. Note: Customers may choose to incorporate the Technical Content web queries and applications into their own portal environment.Internal The Technical Content InfoProviders must be loaded by executing the Technical Content Process chains. The data displayed in the Administrator Cockpit applications Only is read directly from the Techincal Content InfoProviders, so it is recommended that these process chains be executed on a regular basis. In order to connect the Administrator Cockpit in the portal environment to the BI system there are settings that must be configured. To set up the BI Administration Cockpit in the BI system, open the IMG (SPRO) and go to Settings for BI Content -> Business Intelligence -> BI Administration Cockpit. 226 © 2006 SAP AG. All rights reserved. 2006/Q2
  • 235. BW360 Lesson: BI Administrator Cockpit InternalOnlyPartner Figure 163: BI Administrator Cockpit Configuration Use Steps to install Administrator Cockpit: 1. Activate the new technical content in SAP NetWeaver BI. Collect and activate all relevant objects from the technical content by selecting the role SAPSAP SAP_BW_BI_ADMINISTRATOR and using the Grouping option In Data Flow Before from the Data Warehousing Workbench -> Modeling->Business Content. 2. Assign the BI Administrator role, SAP_BW_BI_ADMINISTRATOR, to all BI administrators (users) in transaction SU01 . PartnerUse 3. Check updating of statistics in transaction RSDDSTAT, for InfoProviders, Queries and Data Transfer Process. The system default is X (= statistics turned on) and detail level 2 (= all details). If for certain objects no logging of performance statistics isInternal needed please switch them off in this transaction in order to avoid unnecessary load on statistics data storage and analysis. 4. Schedule the technical content process chains (0TCT*) in transaction RSPC. INIT Only process chains need to be scheduled once and DELTA process chains should be scheduled periodically. 5. Connect BI System and Portal. 6. Activate Business Package BI Administration 1.0. 2006/Q2 © 2006 SAP AG. All rights reserved. 227
  • 236. Unit 6: BI Statistics and Monitoring BW360 Download the Business Package BI Administration 1.0 from the Portal Content Portfolio on http://sdn.sap.com or from http://service.sap.com/swdc and import it into your SAP NetWeaver 2004s Portal (minimum: Support Package Stack 5). Then assign the roles System Administrator and BI Administrator to your BI administrator users in the portal. 7. Set Up call to BI Administration Cockpit. If you want to call the BI Administration Cockpit from BI transactions such as the Data Warehousing Workbench, you have to configure the call to the BI Administration Cockpit in the IMG. InternalOnly 8. (optional) Within the IMG, it is possible to assign importances to BI objects. These settings can take into account importances in technical content queries during analysis (for example for filtering or sorting).Partner Use SAPSAP PartnerUseInternal Figure 164: Assign Importances to BI Business Content Objects Only This optional configuration assigns importances to certain objects from Business Content. These settings determine importances (priorities) which are taken into account in the queries during analysis (for example for filtering or sorting). Importance settings can be made on: queries, MultiProviders, InfoSets, InfoCubes, DataStore objects, Process chains. 228 © 2006 SAP AG. All rights reserved. 2006/Q2
  • 237. BW360 Lesson: BI Administrator Cockpit InternalOnlyPartner Use Figure 165: Integration with SAP NetWeaver SAPSAP PartnerUseInternal Only Figure 166: Administrator Cockpit: Query Runtime Statistics 2006/Q2 © 2006 SAP AG. All rights reserved. 229
  • 238. Unit 6: BI Statistics and Monitoring BW360 Examples of Administrator Cockpit reports for query runtime: • BI Alerts can be configured from the SAP Alert Framework. Any alert subset can be configured, even non-BI alerts. • Administrators can configure thresholds for acceptable response time per query. Queries that go beyond the thresholds are highlighted. It is also possible to raise alerts during pre-calculation. • During regular checks the system can detect aggregates that have values that differ from the aggregated original values. InternalOnlyPartner Use SAPSAP Figure 167: Administrator Cockpit: Data Load Status Examples of Administrator Cockpit reports for data loads: PartnerUse • Load time per process chain can be displayed. • The durations of the last run of each process chain is compared to average times to load data. Any deviation that is higher than a set threshold is highlighted.Internal • A list of potential obsolete PSA data is available. Only 230 © 2006 SAP AG. All rights reserved. 2006/Q2
  • 239. BW360 Lesson: BI Administrator Cockpit Lesson Summary You should now be able to: • Describe the motivating factors for BI Administrator Cockpit. • List the prerequisites for BI Administrator Cockpit. • Describe how to install the BI Administrator Cockpit. Related Information InternalOnly • Recommended SAP Notes: Administrator Cockpit FAQ, 934848.Partner Use SAPSAP PartnerUseInternal Only 2006/Q2 © 2006 SAP AG. All rights reserved. 231
  • 240. Unit 6: BI Statistics and Monitoring BW360 Lesson: BI Trace Tool Lesson Overview This lesson introduces the new BI trace tool, RSTT. This new trace tool is a complete revision of the functions of the past trace tools (Transaction RSRTRACE, RSRCATTTRACE) and includes an enhanced application area. Lesson Objectives InternalOnly After completing this lesson, you will be able to: • List the capabilities of the BI trace tool. • Explain how to log and playback a trace.Partner • Describe how trace attributes are used. Business Example Use A number of obscure errors have been reported during the execution of a specific query. High level analysis cannot determine the cause of the errors. It is necessary to gather more detailed information relating to the query runtime process. You need SAP a robust tracing tool to fully understand the problem.SAP BI Trace Tool The BI trace tool application provides all the functionality needed to log and play back PartnerUse processes executed in the Business Explorer and BI Integrated Planning areas. With the BI trace tool it is also possible to define, execute and evaluate automatic regression tests with the CAT (computer aided testing) tool.Internal The BI trace tool can facilitate error analysis and the monitoring of inefficient process flows. Using the BI trace tool, selected navigation sequences, such as query navigation steps, can be saved and executed with different settings. Only Users must be activated in the BI trace tool before recording starts and deactivated after recording ends. The lifetime of a trace depends on the lifetime of the session (the processes to be recorded). As soon as a session is ended, the trace is also closed. 232 © 2006 SAP AG. All rights reserved. 2006/Q2
  • 241. BW360 Lesson: BI Trace Tool The standard authorization object S_RS_RSTT protects the trace tool environment, RSTT. This authorization includes three fields: • RSTTBOBJ: Specifies a particular object type or multiple object types which the selected user is authorized for a particular activity. • USER: Identifies a user or multiple users who are the authors and persons responsible for object types. This entry authorizes the current user to execute certain activities for their object type. • ACTVT: Specifies the activity or a series of activities that a user or user group can perform to interact with the selected object type. InternalOnlyPartner Use SAPSAP PartnerUse Figure 168: BI Trace Tool CapabilitiesInternal The Trace ID is a key with a name that the system generates using the following pattern: systemname/xxxxxx (xxxxxx represents a 6-digit sequence number). It is possible to change the generated trace ID to any 20-digit name. Note that the trace (ID) Only must be unique within the system. The trace (ID)s of deleted traces may be reused. With the BI trace tool, it is possible to monitor the system quality over a longer period of time with an automatic regression test. When testing for accuracy of the query results, the trace tool provides a tool that can check the quality throughout the entire support package cycle with system support, called the CAT Tool. Automatic tests (called CATT traces) are created with the support of a wizard. Users are guided through the individual definition steps: decisions are made regarding the storage of the test reference data and the assignments for the data structures to be tested, and store descriptions of the navigation steps. 2006/Q2 © 2006 SAP AG. All rights reserved. 233
  • 242. Unit 6: BI Statistics and Monitoring BW360 When the tests are executed, the CATT trace is executed internally and the current query results are compared with the test reference data stored in the definition. If the traces are displayed successfully and the current results values agree with those of the test reference data, the test was successful. In all other cases the test was not successful. The CAT tool also provides a user interface for visualizing the tested data contents. Test packages can be created to combine large amounts of CATT traces. These packages can be limited by using certain selection criteria (such as query name, InfoProvider name, author, trace ID). Test packages can be scheduled as background InternalOnly jobs (test jobs) that can be repeated. Logs (job logs and application logs) are stored for the status of the test run. The results of the tests are written directly to InfoObjects or InfoProviders as master data or transaction data and are immediately available for reporting.Partner Use SAPSAP PartnerUseInternal Figure 169: BI Trace Tool Attributes Only The Trace Attribute screen displays information on a selected trace. In the change mode, certain properties can be changed. For example, it is possible to provide a trace description, or change the name or trace ID. It is also possible to execute the trace up until a certain program object. 234 © 2006 SAP AG. All rights reserved. 2006/Q2
  • 243. BW360 Lesson: BI Trace Tool InternalOnlyPartner Figure 170: BI Trace Tool: Recorded Program Objects Use The display of a trace depends on the controlled execution of the recorded sequence of program calls. The trace can be executed completely or stopped at a given location in order to branch to the ABAP Debugger directly at this location. The latter option SAPSAP is recommended for a detailed analysis of the recorded processing (for example, error analysis). Double-click on a table entry to start the Trace Execution. If the program object of this table entry is in the application layer chosen on the Basis Attributes tab page, PartnerUse the ABAP Debugger appears. The system stops directly before calling the program object. If the program object is in a different application layer, the system executes the entire trace. The run is documented in corresponding messages.Internal Only 2006/Q2 © 2006 SAP AG. All rights reserved. 235
  • 244. Unit 6: BI Statistics and Monitoring BW360 InternalOnlyPartner Use Figure 171: Link to the Query Monitor from RSTT From the BI trace tool, there is a link directly to the Query Monitor, RSRV. This direct SAPSAP link facilitates further detailed analysis using query runtime statistics. PartnerUseInternal Only 236 © 2006 SAP AG. All rights reserved. 2006/Q2
  • 245. BW360 Lesson: BI Trace Tool Lesson Summary You should now be able to: • List the capabilities of the BI trace tool. • Explain how to log and playback a trace. • Describe how trace attributes are used. InternalOnlyPartner Use SAPSAP PartnerUseInternal Only 2006/Q2 © 2006 SAP AG. All rights reserved. 237
  • 246. Unit Summary BW360 Unit Summary You should now be able to: • Describe how to configure the collection of BI statistics. • Define the concept of Event ID. • List several components of query runtime statistics. • Describe the statistics that are gathered during data load. • Analyze Query Runtime statistics using the Query Monitor. • Explain how to activate BI Statistics Technical Content. InternalOnly • Describe the components of BI StatisticsTechnical Content. • Describe the architecture for collecting BI statistics. • List several methods for analyzing BI Statistics. • Describe the motivating factors for BI Administrator Cockpit.Partner • List the prerequisites for BI Administrator Cockpit. • Describe how to install the BI Administrator Cockpit. Use • List the capabilities of the BI trace tool. • Explain how to log and playback a trace. • Describe how trace attributes are used. SAPSAP PartnerUseInternal Only 238 © 2006 SAP AG. All rights reserved. 2006/Q2
  • 247. Unit 7 Aggregates InternalOnly Unit Overview The focus of this unit is the use of aggregates to improve BI query performance. The basic concepts and design of aggregates is discussed. Next, you will understand the impact of missing aggregates on query performance. Then you will learn aboutPartner various types of proposals that can be generated by the system. Finally, you will understand the significance of aggregate maintenance. Use Unit Objectives After completing this unit, you will be able to: SAPSAP • Define an aggregate. • Explain the use of filters. • Discuss the use of hierarchies in aggregates. Partner • Describe flat aggregates.Use • Discuss the impact of missing aggregates on query performance. • Define the indicators of missing indexes. • Describe tools that can be used to detect missing aggregates.Internal • Discuss the use of the query monitor to monitor query performance. • Explain the steps in aggregate rollup. Only • Describe the relationship between rollups and consistency. • Explain the purpose of change runs. • Identify the steps in serial change runs. • Identify the steps in parallel change runs. Unit Contents Lesson: Relational DBMS Aggregate Basics ................................... 241 Lesson: Missing Aggregates ...................................................... 252 Lesson: Analyzing Aggregate Usage ............................................ 262 2006/Q2 © 2006 SAP AG. All rights reserved. 239
  • 248. Unit 7: Aggregates BW360 Lesson: Aggregate Maintenance ................................................. 270 Exercise 5: Working with Aggregates........................................ 295 InternalOnlyPartner Use SAPSAP PartnerUseInternal Only 240 © 2006 SAP AG. All rights reserved. 2006/Q2
  • 249. BW360 Lesson: Relational DBMS Aggregate Basics Lesson: Relational DBMS Aggregate Basics Lesson Overview This lesson will help you understand the concept of relational DBMS aggregates. You will learn to define an aggregate. In addition, you will learn about the use of filters and hierarchies in aggregates. You will also learn about flat aggregates. Lesson Objectives InternalOnly After completing this lesson, you will be able to: • Define an aggregate. • Explain the use of filters.Partner • Discuss the use of hierarchies in aggregates. • Describe flat aggregates. Use Business Example You are working in the Data Warehousing department of a chemical company. For an analysis, you need to identify aggregates and create filters and hierarchies in SAPSAP aggregates. PartnerUseInternal Only 2006/Q2 © 2006 SAP AG. All rights reserved. 241
  • 250. Unit 7: Aggregates BW360 Aggregates: Definition InternalOnlyPartner Use Figure 172: Aggregates: Definition SAPSAP An aggregate is a roll up of fact data where a total value is sufficient and no detailed information is needed. PartnerUseInternal Figure 173: Aggregates are Created on Basic InfoCubes Only Aggregates are like InfoCubes except that they summarize or aggregate data from an InfoCube. When you use an aggregate, the summarization it represents does not need to be done during runtime. Aggregate functions happen in the background. They are not visible to the end user. 242 © 2006 SAP AG. All rights reserved. 2006/Q2
  • 251. BW360 Lesson: Relational DBMS Aggregate Basics InternalOnlyPartner Figure 174: Aggregates: Concept Use InfoCube access: The four rows have to be summarized to two rows, since the query is focused on revenue per month and not on material. Aggregate access: The database no longer has to summarize the number of rows. Only the required information is read from the database. SAPSAP There is no difference in the result. The records transferred to the BI instance are always the same, no matter if the data is read from the InfoCube or from the aggregate. Aggregates with Time-Dependent Components PartnerUse • Aggregates with a time-dependent component (navigational attribute or hierarachy) are calculated for a keydate (corresponding to the keydate of a query). • The keydate can be determined by:Internal – A BEx variable which is filled via a SAP or User Exit A fixed date – This date is retrieved when the aggregates is filled Only • Important for the use of aggregates is not the variable but the processed keydate because only data for this keydate is available in the aggregate and thus available for a query. The aggregate with a time-dependent component only contains data for a snapshot of the InfoCube/Master Data. This snapshot is determined by the keydate. 2006/Q2 © 2006 SAP AG. All rights reserved. 243
  • 252. Unit 7: Aggregates BW360 Aggregates: Different Aggregation Levels • Different type of aggregation can be used. An example of each type follows: – *: Group according to characteristic or attribute value – H: Group according to nodes of a hierarchy level – F: Filter according to fixed value InternalOnlyPartner Use Figure 175: Grouped by Characteristic • From the aggregate (country *), you can read out data for queries about: SAP – Sales in different countriesSAP – Sales worldwide • Because there is no information about customers available in the aggregate, the aggregate cannot be used for queries about sales made to a particular customer. PartnerUse – Sales for a special customer • If you need information about a special customer, you have to access the InfoCube.Internal Only 244 © 2006 SAP AG. All rights reserved. 2006/Q2
  • 253. BW360 Lesson: Relational DBMS Aggregate Basics InternalOnlyPartner Use Figure 176: Grouped by Characteristic: Technical View Technically speaking, an aggregate is a separate InfoCube with its own fact table SAPSAP and dimension tables. When an aggregate is created, it is given a six-digit number <1NNNNN> that starts with a “1”. The table name for an aggregate is derived in the same way from this number as InfoCube table names are derived from InfoCube names. PartnerUse For example, if an aggregate has the technical name 100001, its fact tables are called /BIC/E100001 and /BIC/F100001. Its dimensions have the table names /BIC/D100001P, /BIC/D100001T, and so on.Internal Dimension tables can be shared between an InfoCube and an aggregate. In this example, dimension 2 (the country dimension) is shared between the InfoCube and the aggregate. It is not necessary to create a new dimension table. A link to this Only dimension table is created in the aggregate fact table. Dimensions are only shared if all characteristics of the InfoCube-dimension are also used in the aggregate. Otherwise, a new dimension table is created for the aggregate. There is no longer a link from the new aggregate to the dimension customer (dimension 1), since the aggregate does not contain any information about the customer. There are two fact tables: the F-fact table and the E-fact table. If you upload data into an InfoCube, it is always written into the F-fact table. If you compress the data, the data is shifted from the F-fact table to the E-fact table. The F-fact tables for aggregates are usually empty, since aggregates are compressed automatically. 2006/Q2 © 2006 SAP AG. All rights reserved. 245
  • 254. Unit 7: Aggregates BW360 Filters in Aggregates InternalOnly Figure 177: Using Filters From the aggregate (country F, Germany; customer *), you can read out data forPartner queries about: • Sales in Germany for different customers Use • Overall sales in Germany Because there is no information about countries other than Germany available in the aggregate, the aggregate cannot be used for queries about: SAPSAP • Sales for all customers in the USA • Sales for all customers in Austria • Sales made to a particular customer • Sales worldwide PartnerUse Aggregates with filters are only useful for queries with the same filter. If you need summarized information about a special customer, you have to access the InfoCube, because the customer can have bought products in different countries.Internal Only 246 © 2006 SAP AG. All rights reserved. 2006/Q2
  • 255. BW360 Lesson: Relational DBMS Aggregate Basics InternalOnlyPartner Use Figure 178: Using Filters: Technical View SAP Dimension tables that are identical are shared between an InfoCube and an aggregate.SAP In this example, both dimension tables (country and customer) are shared between the InfoCube and the aggregate. It is not necessary to create a new dimension table for country, although only one value (Germany) is used. PartnerUseInternal Only 2006/Q2 © 2006 SAP AG. All rights reserved. 247
  • 256. Unit 7: Aggregates BW360 Hierarchies in Aggregates InternalOnlyPartner Use Figure 179: Using Hierarchies SAPSAP Aggregate (country H, level 2) may be used for queries about: • Sales for Europe • Sales for America PartnerUse • Sales for All (Overall sales or Sales worldwide) • Sales for all countries ordered by the country hierarchy up to level 1 or 2 If the hierarchy has double leafs {or leaves} such that some characteristic values appear more than once in the hierarchy or if the hierarchy has the attribute “noInternal restnode” (so that the remaining node is explicitly hidden), the query Sales for All (Overall sales) cannot use the hierarchy aggregate. Only Aggregates with a hierarchy are useful for queries that use nodes of the hierarchy as a filter or that use the hierarchy as a presentation hierarchy. For exceptions, see SAP Note 198568. The level of the desired nodes must be less than or equal to the level in the aggregate. 248 © 2006 SAP AG. All rights reserved. 2006/Q2
  • 257. BW360 Lesson: Relational DBMS Aggregate Basics Flat Aggregates • In case of flat aggregates: • If an aggregate has equal or less than 16 characteristics (including time characteristics, unit and package), each characteristic is put into a separate dimension. • The dimensions (except the package and unit) are marked as “Line Item”. • Performance issues: InternalOnly – “Flat Aggregates” are filled/rolled up without loading the data on the application server (just movement of data on database) – No generation of DIM IDs necessary during rebuild • Line item/high cardinality dimensions already specified in the InfoCube are also used for aggregates for corresponding dimensions (also for non-flat aggregates).Partner Use SAPSAP PartnerUseInternal Only Figure 180: Flat Aggregates This screenshot of an aggregate fact table shows that all key fields (other than the P and U dimension keys) are in fact master data IDs (SIDs). This meansthat there are no separate dimensions but the master data ID is directly stored in the fact table (line item). 2006/Q2 © 2006 SAP AG. All rights reserved. 249
  • 258. Unit 7: Aggregates BW360 A database view is created instead of a dimension table. This view is based on the SID table. InternalOnlyPartner Use SAPSAP PartnerUseInternal Only 250 © 2006 SAP AG. All rights reserved. 2006/Q2
  • 259. BW360 Lesson: Relational DBMS Aggregate Basics Lesson Summary You should now be able to: • Define an aggregate. • Explain the use of filters. • Discuss the use of hierarchies in aggregates. • Describe flat aggregates. InternalOnlyPartner Use SAPSAP PartnerUseInternal Only 2006/Q2 © 2006 SAP AG. All rights reserved. 251
  • 260. Unit 7: Aggregates BW360 Lesson: Missing Aggregates Lesson Overview This lesson will help you understand the impact of missing relational DBMS aggregates on query performance in BI. Lesson Objectives After completing this lesson, you will be able to: InternalOnly • Discuss the impact of missing aggregates on query performance. • Define the indicators of missing indexes. • Describe tools that can be used to detect missing aggregates.Partner Business Example Use While working with a chemical company, you need to design and create queries. These queries are used to extract data from huge databases maintained by its Data Warehousing department. You can use relation DBMS aggregates to improve the query performance. However, in case of missing or inappropriate aggregates, the SAP performance of queries can be affected. Therefore, you need to detect missingSAP aggregates. PartnerUseInternal Only 252 © 2006 SAP AG. All rights reserved. 2006/Q2
  • 261. BW360 Lesson: Missing Aggregates Indicators of Missing Aggregates InternalOnlyPartner Use Figure 181: Missing or Inappropriate Aggregates SAPSAP The main limited resources on the database server are: • CPU capacity • Memory capacity PartnerUse • Number of physical I/O operations that can be performed efficiently Expensive SQL statements read too much data from the database compared to what is needed in the query. Expensive SQL statements can have a negative impact on performance overall. Improper or unnecessary use of the database buffers results inInternal displacements of other data blocks, and affects overall system performance. A query from a poorly tuned aggregate, even one with adequate performance, may Only adversely affect the performance of other queries in the system. If aggregates are missing, the user who started the query must wait. Even if the user is satisfied with the runtime of a long-running query, it monopolizes resources that are shared between all users. A long-running query can have an impact on the overall performance of the system. 2006/Q2 © 2006 SAP AG. All rights reserved. 253
  • 262. Unit 7: Aggregates BW360 InternalOnlyPartner Use Figure 182: Query Design: General Recommendation When you create a query, keep the following in mind: SAPSAP • Building free characteristics into the query design is a good way to reduce the size of the result set of each drill-down step • Small result sets per query step can greatly improve performance. • Small result sets are also much easier to analyze. PartnerUse • If great detail is desired for a large amount of data, check if batch printing can be used instead of online reporting. • If you use a Web front end, check if you can use predefined Web queries from the reporting agent.Internal • The more selective your drill-down so much better is the usage of indexes on database level. Only 254 © 2006 SAP AG. All rights reserved. 2006/Q2
  • 263. BW360 Lesson: Missing Aggregates InternalOnlyPartner Figure 183: Indicators of Missing Aggregates Query performance may be poor due to missing aggregates: Use • If much data has to be selected on the database. • Much more data than necessary is selected on the database. • The database time is high. SAPSAP • The database time is responsible for a high percentage of the total runtime. The number of records read on the database is only the number of valid records, which fulfill the selection conditions. Only a high database time for a low number of records Partner read is a sign for a poor data access path.Use Aggregates also help to reduce the I/O load of the database, because less data must be read from disk. Indicators of Missing AggregatesInternal • Ratio records selected (DBSEL) / records transferred (DBRANS) > 10 Only • Records selected > 10,000 • Database time over 30 percent of total runtime • Database time higher than three seconds Query performance may be poor due to missing aggregates: If much data has to be selected on the database, much more data than necessary is selected on the database. The database time is high. The database time is responsible for a high percentage of the total runtime. 2006/Q2 © 2006 SAP AG. All rights reserved. 255
  • 264. Unit 7: Aggregates BW360 The number of records read on the database is only the number of valid records, which fulfill the selection conditions. Only a high database time for a low number of records read is a sign for a poor data access path. Aggregates also help to reduce the I/O load of the database, because less data must be read from disk. Detection of Missing Aggregrates To analyze query performance at InfoCube level, you can use the following tools: InternalOnly • Query Runtime Statistics in the Query Monitor (RSRT) • Queries of BI statistics InfoCubes • Database view: RSDDSTAT_DM. • Administrator CockpitPartner • Function module RSDDCVER_RFC_BW_STATISTICS • ST03 for BI Use SAPSAP PartnerUseInternal Only Figure 184: Indicators of Missing Aggregates 256 © 2006 SAP AG. All rights reserved. 2006/Q2
  • 265. BW360 Lesson: Missing Aggregates During BI query execution, sub-queries may be generated. For example, to: • access different InfoProviders in a MultiProvider. • separate requests to the OLAP processor (for example, with constant selections). • to divide requests because of varying selections in individual cells (for example, a key figure is restricted to a hierarchy node; another key figure is restricted to a value range). The Frontend/Calculation Layer tab for Query Runtime statistics displays the aggregated statistics for all the sub-queries. To view the detailed statistics for each InternalOnly sub-query go the Aggregation Layer tab.Partner Use SAPSAP PartnerUseInternal Figure 185: MultiProvider Runtime Statistics With MultiProviders, the aggregated data at the MultiProvider level is not meaningful. Only The statistics are just the sum of the statistics of the underlying InfoCubes. To accurately analyze Multiprovider statistics the details for each InfoCube must be analyzed separately. Note: Some small amounts of time may not be captured in the detailed statistics. For example, time for function calls or time to split up the subqueries. So, the sum of the details may be lower than the overall time for EVENTID 9000, Data Manager (DB time). 2006/Q2 © 2006 SAP AG. All rights reserved. 257
  • 266. Unit 7: Aggregates BW360 InternalOnlyPartner Use Figure 186: ST03 - Drill Down to Query This view gives us the InfoCubes that had the longest run time. SAPSAP All query run times are grouped and totaled by their respective InfoCube. Note: Before using BI statistics in ST03, make sure the Business Content process chains have been executed to update the Business Content InfoCubes. PartnerUse ST03 reads the InfoCubes and the Process Chains must be executed in order to load the most current data.Internal Only 258 © 2006 SAP AG. All rights reserved. 2006/Q2
  • 267. BW360 Lesson: Missing Aggregates InternalOnlyPartner Use Figure 187: Valuation of Aggregates The BI system offers an automatic evaluation of the aggregates. SAPSAP A bad evaluation is given to aggregates that are: • As large as the structure they are built from • Not used at all Partner • Not used for a long timeUse Column Records summarized (mean value) shows values that were valid when the aggregate was created. If further aggregates have been created, these values may no longer be valid.Internal Column Usage is reset only if you redefine an aggregate. To detect queries that are no longer in use, use column Last used. Only 2006/Q2 © 2006 SAP AG. All rights reserved. 259
  • 268. Unit 7: Aggregates BW360 InternalOnlyPartner Use Figure 188: How to Judge an Aggregate If you want to create aggregates to improve the performance of a specific query, you SAPSAP can create aggregates based on any of the following: • Analysis of BI statistics in general. • Statistics about specific queries from the query monitor (transaction RSRT). PartnerUse • Knowledge about your business, and the distribution of data in the InfoCube.Internal Only 260 © 2006 SAP AG. All rights reserved. 2006/Q2
  • 269. BW360 Lesson: Missing Aggregates Lesson Summary You should now be able to: • Discuss the impact of missing aggregates on query performance. • Define the indicators of missing indexes. • Describe tools that can be used to detect missing aggregates. InternalOnlyPartner Use SAPSAP PartnerUseInternal Only 2006/Q2 © 2006 SAP AG. All rights reserved. 261
  • 270. Unit 7: Aggregates BW360 Lesson: Analyzing Aggregate Usage Lesson Overview This lesson will help you understand the use of the query monitor to monitor query performance and the use of aggregates. Lesson Objectives After completing this lesson, you will be able to: InternalOnly • Discuss the use of the query monitor to monitor query performance. Business ExamplePartner As an employee of the Data Warehousing department of a chemical company, you need to analyze queries. To do this, you need to get aggregate suggestions. You can do this by using query analysis tools, such as query monitor and trace tools. Use Query Monitor - RSRT SAPSAP • Query Monitor – RSRT can be used to get aggregate suggestions for the first execution of a query. The dialog box that appears on screen is the same as that in the query monitor (transaction RSRT). However, here the dialog box appears for each navigation PartnerUse recorded in this trace.Internal Only Figure 189: Query Monitor: Execute + Debug 262 © 2006 SAP AG. All rights reserved. 2006/Q2
  • 271. BW360 Lesson: Analyzing Aggregate Usage To get suggestions for specific aggregates, you can use the query monitor (transaction RSRT). Choose Execute + Debug. A dialog box appears. Make sure that you set the following flags: • Display statistics data: This shows the statistical data written to the database. • Display aggregates found: This displays a dialog box which suggests the best aggregate possible for this query execution. The columns with S indicate system proposals. The columns with A indicate what was actually found if an aggregate was used. • No parallel processing: This switches off parallel processing. This makes InternalOnly analysis easier if you perform SQL traces. After you have set the flags, choose Continue or Enter. Only the first execution of a query can be performed, no navigations are possible.Partner Sub-queries are generated when: • separate requests to the OLAP processor are generated (ex: with constant Use selections). • individual InfoProviders in a MultiProvider are accessed. • there is a division of requests because of varying selections in individual cells (ex: one key figure is restricted to a hierarchy node, while another key figure is SAPSAP restricted to a value range). • different aggregates of an InfoCube are accessed. • there are separate requests to the E and F fact tables of an InfoCube. PartnerUse If dividing the query results in more than one sub-query, the read operation is performed in parallel by default. Note: In the IMG, RSCUSTV25, it is possible to split complex queries intoInternal multiple SQL statements. This function is activated with SQLstatements for BI queries that contain more than 50 tables, or for DataMart requests with more than 20 tables. See SAP Note 514907. Only Several dialog work processes are required in order to execute queries in parallel. The maximum degree of parallelism determines the maximum number of work processes that are used for each query. This value is restricted to 6 by default. The maximum value can be changed to a value between 1 and 100 in the QUERY_MAX_WP_DIAG entry in table RSADMIN. 2006/Q2 © 2006 SAP AG. All rights reserved. 263
  • 272. Unit 7: Aggregates BW360 The actual degree to which queries are executed in parallel depends on the load on the system at any given time. If the number of sub-queries is greater than the number of available workprocesses, then all existing sub-queries are divided between the work processes. The results of all sub-queries are collected at a synchronization point and collated to form an overall result. In sequential processing, the sub-queries are processed one after another. The interim result is immediately passed on to the analytic engine. InternalOnlyPartner Use SAPSAP PartnerUse Figure 190: RSRT: Split Queries When a query contains complex selections, such as calculated or restricted keyInternal figures, the requests are split and accessed individually. This is similar to the process performed when InfoCube aggregates are used. With a Split query, one logical query is split to several Database queries that run in Only parallel. In the above example, the system reads in parallel out of the fact table for one query and an aggregate for the other. Queries that contain a Constant selection, or queries that read MultiProviders, are managed by the OLAP processor, which performs separate reads. In these cases, there will be a separate Found Aggregates report displayed in RSRT, one for each of the reads. 264 © 2006 SAP AG. All rights reserved. 2006/Q2
  • 273. BW360 Lesson: Analyzing Aggregate Usage InternalOnlyPartner Use Figure 191: Best Possible Aggregate In the example above, the logical query is split into two database queries. For each SAPSAP database query, a different aggregate is used. For the first access, aggregate 100005 is used. This access requires information about 0CALMONTH and 0DMATERIAL in a disaggregated form (aggregation type *). The best possible aggregate would be an aggregate with only these two InfoObjects. The PartnerUse best existing aggregate includes additional information for InfoObject 0D_PLANT. BI chooses this aggregate. • If the aggregate size is increased by including InfoObject 0D_PLANT atInternal aggregation level *, performance may be improved by creating a new aggregate that only includes 0CALMONTH and 0MATERIAL. In some cases, the system may suggest many best possible aggregates. If many Only aggregates are suggested that differ only by one hierarchy level for one characteristic, you can summarize those aggregate definitions to get fewer aggregates. If an aggregate is already used, you will only see the aggregate name (number) at Structure to be read. You have to go to table RSDCUBE to link the aggregate name to the correct InfoCube. If you analyze a query to a MultiProvider, every access to a Basic Cube generates a new dialog box. 2006/Q2 © 2006 SAP AG. All rights reserved. 265
  • 274. Unit 7: Aggregates BW360 At Structure to be read, you only see the Basic Cube name or the aggregate name. To link the aggregate name to the correct InfoCube, go to table RSDCUBE. For a fixed value, you always get the SID of the characteristic value. Some suggested aggregation types are: • * : All values • F: fixed value • H: hierarchy level InternalOnly • %: navigational attributePartner Use SAPSAP PartnerUseInternal Figure 192: Converting SIDs for Fixed Values (1) Only If you use RSRT to detect missing aggregates you get only the SID value for fixed values. You have to convert SID = 3 manually to the right country for navigational attribute 0VC_CUS1__0VC_COUN. In transaction RSDCUBE, enter the InfoCube name 0BWVC_C09 and choose Display. Double-click on the selected characteristics 0VC_CUS1. 266 © 2006 SAP AG. All rights reserved. 2006/Q2
  • 275. BW360 Lesson: Analyzing Aggregate Usage If you do not use a navigational attribute but a normal characteristic, you can double-click directly on the SID table of the characteristic /BI0/SVC_CUS1. To get to the SID table of the attribute, select tab Attributes. This shows all navigational attributes. Double-click on the selected attribute 0VC_COUN. Double-click on the SID table of attribute /BI0/SVC_COUN. and choose the button Contents. Enter SID = 3 and choose Execute. Now you can see the value DE = GERMANY InternalOnly which is related to SID = 3. The query monitor (transaction RSRT) provides the statistics about the process you just started.Partner Automatic Proposals Use Choose transaction RSDDV or use the Data Warehousing Workbench to maintain your aggregates. In the aggregate maintenance screen, choose Propose in the menu to get the drop-down menu for proposals. SAP The are different ways to get suggestions are:SAP • Propose (statistics, usually query) • Propose from query • Propose from last navigation PartnerUse • Propose from BI statistics (table) • Propose from BI statistics (InfoCube)Internal Only 2006/Q2 © 2006 SAP AG. All rights reserved. 267
  • 276. Unit 7: Aggregates BW360 InternalOnlyPartner Figure 193: Proposal by BI Statistics (1) Use You can get automatic suggestions based on the BI statistics. The proposals can be restricted to a specific runtime or to a specific period or both. SAPSAP PartnerUseInternal Only 268 © 2006 SAP AG. All rights reserved. 2006/Q2
  • 277. BW360 Lesson: Analyzing Aggregate Usage Lesson Summary You should now be able to: • Discuss the use of the query monitor to monitor query performance. InternalOnlyPartner Use SAPSAP PartnerUseInternal Only 2006/Q2 © 2006 SAP AG. All rights reserved. 269
  • 278. Unit 7: Aggregates BW360 Lesson: Aggregate Maintenance Lesson Overview This lesson will help you understand the steps in aggregate rollup. You will understand the relationship between rollups and consistency. In addition, you will learn the steps in serial and parallel change runs. Lesson Objectives InternalOnly After completing this lesson, you will be able to: • Explain the steps in aggregate rollup. • Describe the relationship between rollups and consistency.Partner • Explain the purpose of change runs. • Identify the steps in serial change runs. • Identify the steps in parallel change runs. Use Business Example Aggregates are used to improve query performance. However, to use an aggregate for SAPSAP an InfoCube while executing a query, you first need to activate the aggregate and then fill it with data. As an employee of the Data Warehousing department of a chemical company, you need to activate the aggregate. PartnerUseInternal Only 270 © 2006 SAP AG. All rights reserved. 2006/Q2
  • 279. BW360 Lesson: Aggregate Maintenance Aggregate Rollup InternalOnlyPartner Use Figure 194: Rollup and Change Run (Master Data Activation) SAPSAP The system creates the tables required by the aggregate definition on the database. Aggregates are created according to the same schema as InfoCubes. New data packets or requests that are loaded into the InfoCube cannot be used at first for reporting if there are aggregates that are already filled. The new packets to be PartnerUse added must first be written to the aggregates by a rollup. Aggregates that contain navigational attributes and/or hierarchies are affected by any change in the master data.Internal A rollup affects only those aggregates belonging to the InfoCube the data is loaded in. A change run affects all aggregates over all InfoCubes that contain the changed Only attribute or hierarchy. 2006/Q2 © 2006 SAP AG. All rights reserved. 271
  • 280. Unit 7: Aggregates BW360 InternalOnlyPartner Use Figure 195: Why Are too Many Aggregates Harmful? Aggregates reduce query cost by reducing the amount of data used in the selection operation. However, a significant overhead in updating of aggregates is generated: SAPSAP • When new data is loaded, aggregate rollup must take place. • Changes to master data and hierarchies require that all dependent aggregates be recalculated by calculating the differences (delta) or by rebuilding. PartnerUse • Disk space is needed for the aggregates. The various factors involved are: • Frequency of changes that will cause recalculationInternal • Availability of time to run the recalculation: no master data update or hierarchy update can take place during a change run. • Changed aggregate data is not available by query until recalculation is complete. Only • Reporting on the old master data and hierarchies is possible during recalculation. You should delete unused aggregates. 272 © 2006 SAP AG. All rights reserved. 2006/Q2
  • 281. BW360 Lesson: Aggregate Maintenance InternalOnlyPartner Figure 196: Rollup Use Depending on whether the InfoCube is accessed or the aggregate is accessed, different results are obtained for a query which displays Country vs. Sales. To resolve this inconsistency, the newly uploaded data is not available for reporting until the aggregate provides the same information. SAPSAP PartnerUseInternal Only Figure 197: Aggregates after Rollup Now, the data between aggregate and InfoCube is consistent. After the rollup, the newly uploaded transactional data is available for reporting. It is now available in the InfoCube and in the aggregates. 2006/Q2 © 2006 SAP AG. All rights reserved. 273
  • 282. Unit 7: Aggregates BW360 During