While Query/400 (officially called Query for i5/OS) was a very popular product, most customers today use it only as a vehicle to get data out of DB2 for i and into some more sophisticated analytical tool, like a spreadsheet – or to dump data to another database for reporting purposes using graphical user interfaces. Today’s requirements are for direct access to data in DB2 for i that offer simplified reporting that the end user can execute to remove a dependency on IT. Through parameterized reports or OLAP functionality, the number of reports that have to be maintained can be SIGNIFICANTLY reduced. End users want simpler user interfaces, and IT wants to reduce the maintenance of software on fat client workstations. Web Based drag and drop interfaces reduce the learning curve and provide intuitive navigation, hiding complex processes and knowledge of the database from the end user. Different users require different ways to view data, be it dashboards, spreadsheets, static reports in “board room” quality, or with an ability to interact with the data as you analyze it through drill downs and slicing and dicing. Some customers may want to minimize the impact of running reports on production systems, and simplified creation of data warehouses, marts, or operational data stores (all of these are basically the creating of a reporting repository so queries run against this repository, typically on another server or partition) is a necessity to achieve this. Lastly, DB2 for i already has a GREAT reputation for KEEPING THE DATA SECURE. Why is it that reporting solutions so often are designed to TAKE THE DATA OUT of this highly secure environment and put it into what is arguably the LEAST secure environment. That makes no sense and adds to the exposure, and auditability.
On April 10, 2007, IBM announced plans to deliver a Web-based query and report writing product that replaces the IBM Query for iSeries (also commonly known as Query/400) product. The DB2 Web Query “base” product will provide capabilities to query or build reports against data stored in DB2 for i5/OS databases through the latest browser based user interface technologies. Build new reports with ease through Power Painter or Report and Graph Assistant components. Simplify the management of reports by leveraging parameterized reporting. Deliver data to end users in many different formats, including spreadsheets, PDF, HTML or through the Java based thin client interface browser support. Import Query/400 definitions and enhance their look and functionality with Power Painter or Graph Assistant. Interface to all data in i5/OS through either DB2 or Open Query File native adapters that automatically identify the files to be accessed and import the metadata into DB2 Web Query. Additional priced features can be added from IBM for OLAP analysis, or disconnected (but “active”) reporting. All users licensed to the “base” product will be able to use OLAP or Active Report features. Add a Developer’s Workbench to build more customized reports or enhance a meta data layer. This product is an OEM agreement with Information Builder’s WebFocus product. Add additional components from Information Builders such as ERP or other database adapters (to query Oracle, for instance, you’d add an Oracle Adapter). Grow into more complete BI solutions leveraging the product’s API support for SPSS’ Clementine (data mining) or ESSBASE/400 (cubing) technologies. Add a light weight ETL (Extract Transformation and Loading) tool for building data marts or data warehouses with Data Migrator.
The SQL Query Engine (SQE), first introduced way back in V5R2, has been one of IBM’s key initiatives for taking SQL performance on DB2 for i to new heights. This performance advancement for SQL workloads continues in V6R1 with the elimination of two major roadblocks that prevented the usage of the SQE: National Language Sort Sequences (NLSS) and functions relying on low-level translation function. The low-level translation capability is required by commonly used SQL-built in functions such as Upper and Lower. The NLSS support in i is frequently used in overseas markets where the application requires the character data be sorted in a manner that matches the local language instead of the default *HEX ordering which closely mirrors the sorting of the English alphabet. With these key functions now supported by SQE in V6R1, the most common items that prevent the usage of SQE in V6R1 are listed on this chart. While the SQE optimizer is still unable to utilize Select/Omit logical files when building query plans, IBM did change the default value for the Ignore_Derived_Index QAQQINI parameter to enable more SQE usage. This QAQQINI parameter was first added back in V5R3 to allow the SQL Query Engine to be used in environments where SQL statements are referencing DB2 objects created with DDS. Prior to V6R1, the default value for this parameter was *NO causing the SQE optimizer to reroute execution of any SQL request to the Classic Query Engine (CQE) anytime a derived logical file was encountered during the query optimization process. The default value in V6R1 for the Ignore_Derived_Index QAQQINI parameter has been changed to *YES. This values allows the SQE query optimizer to ignore any keyed logical files that contain select/omit criteria during optimization instead of rerouting execution to CQE. While eliminating barriers to SQE usage is important, the most interesting V6R1 enhancements are those new capabilities that leverage the extensible architecture of the SQL Query Engine. A prime example is DB2 for i’s first foray into self-learning query optimization. A self-learning query optimizer is able to analyze poor-performing query plans and dynamically adjust its internal algorithms based on feedback to select a better query plan on future executions. In V6R1, the SQE optimizer will automatically analyze poor performing query plans to examine the I/O characteristics as well as record retrieval patterns and compare them to the values used during optimization of the query. If the query optimizer detects significant mismatches, then the optimizer modifies its assumptions and algorithms during the next execution of that SQL request to generate a better plan. To complement the learning optimizer, the DB2 runtime engine is also equipped with adaptive technologies that allow the plan of a currently running query to be modified on the fly to improve the efficiency of the query for the remainder of the run. In addition to the self-learning enhancements, the SQE optimizer can also improve SQL performance with new indexing technologies. First, the SQE query optimizer can dramatically speed up the processing of the Sum and Average aggregate functions with Encoded Vector Indexes (EVI) whenever an EVI exists over the columns referenced on the function call. NOTE==========: Self-learning query examples (only applied when SQL has been run at least 7 times & estimated runtime > 1 sec) ColdIO to WarmIO Mitigation : Optimizer assumes a certain number of page faults for tables referenced in query, if the optimizer is noticing that the plan is not causing page faults during execution (ie, table) is staying in memory. The optimizer will reoptimize the query with the assumption that the table will not incur page faults. Plan may not be changed, but optimizer at least tried to change its default behavior. WarmIO assumption may cause table scans to look more attractive to optimizer since a table scan method would not need to worry about I/O. FirstIO to AllIO Mitigation : For access plans built with FirstIO optimization goal, the optimizer will attempt reoptimization if it sees the access plan continually reading all of the rows in the table to return result set. Since this is an ALLIO behavior the optimizer will reoptimize the query with the ALLIO optimization goal to determine if a better performing plan can be built. Self-learning considerations: These self-learning checks will only be done once by the optimizer (there’s an flag within the plan cache entry recording whether or not the optimizer has performed mitigation on the plan’s statement). New access plan rebuild reasons were added for these two new optimizer-initiated plan rebuilds. Self-adjusting query execution examples : Optimizer will build an index over a hash table to minimize page fault issues associated with a poorly sized hash table. Stats manager will update stale stats in small tables in real time.
What is different about DB2 Web Query? Most BI solutions on the market place require multiple windows (or other OS) servers/instances. Microsoft’s SQLServer, for instance, requires/recommends a separate server for their database, their OLAP server, their reporting server, their web server, and if ETL (Extract, Transformation, Load) is involved, a separate server for that too. Products like Cognos and Business Objects and Microstrategy require a Windows server for report governing, meta data, web serving, etc. DB2 Web Query is designed to leverage i. The BASE server product consists of the meta data engine, the reporting server, the web server, OLAP functionaliy (and obviously the database server is built into i). Single installation, simpler SW maintenance, simpler problem determination, simpler licensing, etc. DB2 for i is highly scalable, in a VERTICAL manner. What does this mean? The System i can grow to very large memory models, very large number of CPUs, and additional disk all within the single server footprint, without having to reconfigure the database and worry about partitioning data across multiple servers. Clustering, on System i, is usually an availability feature, not a scalability issue. However, with other architectures, clustering is required for scaling. I.e., growing horizontally refers to adding additional servers to the architecture, and spreading data from a single database across those servers to ensure a balanced configuration as you grow. This is much more complex to manage and tune. DB2 Web Query is based on Information Builder’s WebFOCUS product. It is a report writing solution designed for Small and Medium Business, but can be a foundation for larger, more complex BI implementations. But it is a subset of what WebFOCUS can bring to many companies looking for more customized or sophisticated BI solutions. For instance, WebFOCUS is based on the Focus 4GL programming language. This language is very powerful and is not part of DB2 Web Query. APIs to products such as SPSS’ Clementine data mining tool could be leveraged with WebFOCUS. Most customers will not want to get into these types of solutions, but if so, DB2 Web Query reports can be preserved should a move to WebFOCUS be warranted. A fundamental philosophy of DB2 Web Query is to leverage IBM i and DB2 for i. As you know, DB2 has a very strong reputation for security, and many auditing tools are also built in. Object Level security, for example, is highly recommended and DB2 Web Query will leverage that versus try to reinvent the wheel by putting a security layer in at the tool level that only users using that tool can abide by. Lastly, DB2 Web Query takes full advantage of IBM’s latest query optimization technology, including patented technologies like Encoded Vector Indexing (EVIs), and the SQL Query Engine function (both of these built into IBM i) first delivered in V5R2 and enhanced in subsequent releases.
This is here just for reference. I would point out the highlighted in green elements.
You can link reports and create a “drill down” affect very easily. In this case, we’ve created a drill down field, PRODUCT TYPE. When you click on product type you’ll see the linked Gross Profit report for that specific product type, in this case “Audio”
Note that the Excel with FORMULAs options will preserve some of the characteristics of the report. So for instance, if you have a SUM field in the report, that field comes down into Excel as a FORMULA….conditional styling is also preserved…
It dynamically (on the fly from live data) can create fully formatted spreadsheets with drilldowns, formulas , summations, and color-coding.
Similar to Report Assistant, an easy to use graphing assistant allows users to build powerful graphs/charts in over 100 different styles. You can also have drill downs, parameterized selection, headings/footers, etc.
DB2 Web Query offers an import function to “webify” query/400 reports. The first step is to import the query/400 definition into DB2 Web Query, and then if desired, the reports can be prettied up with Report Assistant. Add cross tabs, parameter driven selection criteria, style sheets or conditional styling, or burst it into multiple pages for easier navigation.
The Active Reports feature lets you analyze the data while disconnected. But this is NOT static reporting…users can interact with the reports as we’ll download the Java Scripts, HTML to provide “active” use of the reports.
End users can add visualization bars, sort / filter / highlight based on any column roll up data, convert to pivot tables and more All this simply by saying you want the output to go to an active report instead of the default HTML
One check box allows you to OLAP enable an existing report Assumes that you have previously told DB2 Web Query about the hierarchies in your tables Drill down on dimensions Add/remove columns chosen from hierarchies Drill down on the measures shows the details records that made up the summary measure Users can change which columns data is summed by, they can pivot and move columns from the sum by to sum across areas
Power Painter, a new AJAX based WYSIWYG Query design tool. An intuitive state of the art thin client tool for compound ad-hoc reports, graphs and page layouts.
Developer workbench is a fat client development environment for working with meta data, building more sophisticated reports (like compound reports) with an HTML report painter, and offering other features like Impact Analysis and data profiling that the developer may leverage. Federated queries is the idea of joining data from multiple databases into a single query. This function is built into db2 web query for db2 for i database, but for oracle, sqlserver, db2 on aix, you’ll need adapters from information Builders.
Slide Theme: One of the most important aspects of a BI tool is its capability for automated delivery. Many information workers don’t have hours of time to spend looking for data, so if you can have the data find them it makes their days much more productive. This function can automatically deliver reports. This is known in the industry as “pushing” or “casting” information. This can be setup by an administrator. Delivery can be based on a schedule or through a CL interface that could be called based on some event (e.g., a trigger program). E.g., a trigger could be placed on a transaction indicating if a stock price falls below a certain value to invoke the CL command to route a report out to a distribution list. This function can intelligently burst the pages of a single report out to the appropriate people. For example you might a a Profit and Loss statement that shows the P&L for each business unit on a different page and each page can automatically be burst out to the head of each business unit.
With the recently added Software Development Kit, ISVs can integrate reports into their web applications such as the screen shot shown here, that is a customized interface to run a query by passing parameters to it, and embedding the result set into the same HTML page. The SDK provides a set of Web Services to log in, retrieve parameters, execute a query, retrieve a list of domains or folders. There are also web services for interacting with DB2 Web Query report broker scheduled reports. The web services use SOAP and XML standards.
Customers that want to remove the impact of queries against production databases/servers may find the creation of a reporting repository to be an effective means to accomplish that. IBM offers some REPLICATION technologies that can create a repository (with minimal transformations to the data). Remote Journaling is built into IBM i and captures changed data in database logs (our term for this is a JOURNAL RECEIVER). These logs can be automatically shipped to a 2 nd partition/server, whereby products like Data Propagator or Transformation Server can pull the data out of the log on a scheduled basis and add it to the reporting repository. Information Builder’s offers a product called Data Migrator which is a GUI based tool to map source databases to target databases, define data transformation rules (e.g., create standard data formats, or parse out multiple fields from a combined field definition). Schedule the extractions and monitor the errors. Even pull in data from a variety of data sources other than DB2 for i.
OLAP, Report Writer, Web Server, Meta Data, Database
Grow without Disruption
Start Small, Grow Big
DB2 Web Query provides FOUNDATON
Add additional features per requirements
All the data in ONE PLACE
No Viruses, No “Patch Tuesday”
DB2 Query Optimization
Patented IBM Technology
SQE vs. CQE (Query/400)
Simplified Data Access
No Data Replication Required
Database Server OLAP Server Report Server Web Server ETL Server
Browser or Windows (Active Reports or output formats requiring Windows, e.g., Excel) 5250 UI for End User Yes (customization of meta data available with Dev. Workbench) No Meta Data Layer Yes (BASE), advanced parameterization controls with Dev. Workbench Prompting via 5250 RUNQRY Single Parameter Parameterized Reporting Yes (requires Developer Workbench) No Dashboarding Yes (Active Reports Feature) No Disconnected Analysis Yes (OLAP Feature) No OLAP Yes (BASE) No Linked Reports with Drill Down Yes (BASE) No Traffic Lighting YES. Basic function with BASE, advanced function (distribution) with Report Broker i5/OS Scheduler; Spoolfile, DB2 Table Batch Scheduling and Output PDF, XML, Spreadsheet, HTML, PC-File Formats, Spoolfile, DB2 Table, etc. (BASE) 5250, Spoofile, DB2 Table Interactive Output Via DRDA (BASE) No Remote DB2 for i5/OS Access Browser (BASE) or Windows (Developer Workbench) 5250 UI for Query Developer DB2 Web Query Base + IBM Features Query/400 Function
V5R4, V6R1 V5R2 and up i5/OS Support IT and Business Community IT Target Audience for developing reports Yes (Developer workbench) No Impact Analysis and Data Profiling SQE or CQE CQE Only DB2 for i5/OS Optimization Yes, Excel is an output option with report characteristics preserved No, must run Query, Download, then Import Excel Integration Yes (BASE), additional capabilities with Dev. Workbench) No Compound Reports Comes with Lightweight Web Server N/A Web Application Server Support As callable URL from Browser, or through Web Services with SDK Via RunQry API Integration with Operational application With SDK No Portal Integration Local report storage with Report Broker No Report Storage Named or Run Time (Concurrent, effectively unlimited) Yes Unlimited User Licensing DB2 Web Query Base + IBM Features Query/400 Function
DB2 Web Query vs. competition Y - Windows Y, with Report Broker Report Caching N Y Report Execution from 5250 app N Y Cross DB2 for i Joins Windows IBM i (Integrated) Web Server OS support N Y Import Query/400 Definitions Y – Windows Y, with Report Broker Report Distribution N Y, with Active Reports Disconnected Support Requires add’l OLAP engine in Windows DB2 based – no need to move data OLAP Windows IBM i (Integrated) Reporting Server OS support N Y Meta Data Layer N Y Multi record format files N Y Multiple Member File Support ODBC Native DRDA Interface to DB2 for i Y Y Access Data in DB2 for i Crystal Reports DB2 Web Query
Significantly reduced prices for scenarios with large set of run-time users
Assumes upgrade into QU2 from QU1
Named User Pricing New Run Time Pricing US list prices, subject to change without notice 150 80 40 Number of Named Users $57,600 P20 $32,000 P10 $15,200 P05 Price* Tier 20 10 5 Number of Run Time Groups 8 4 4 Number of “Authors” $23,200 P20 $10,400 P10 $6,000 P05 Price* Tier
Having trouble and don’t have SWMA? Send e-mail to [email_address]
Consider Getting Started Lab Services
This service offering will help you get started with developing and running reports using the New Web Query Tool. IBM experts will help you install the software, create and import meta data , help you import query/400 definitions and assist you in building reports using either the built in reporting features and/or the graphical assistance components allowing you to do drill downs.