SQL Server Denali: BI on Your Terms


Published on

Presentation given at CodeCamp NYC, October 1, 2011.

Published in: Technology
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

SQL Server Denali: BI on Your Terms

  1. 1. SQL Server Denali<br />BI on Your Terms<br />Code Camp NYC 2011<br />Andrew J. Brust, Founder/CEO Blue Badge Insights<br />
  2. 2. MARQUEE SPONSOR<br />
  3. 3. PLATINUM SPONSOR<br />
  4. 4. PLATINUM SPONSOR<br />
  5. 5. GOLD SPONSOR<br />
  6. 6. GOLD SPONSOR<br />
  7. 7. GOLD SPONSOR<br />
  8. 8. GOLD SPONSOR<br />
  9. 9. GOLD SPONSOR<br />
  10. 10. GOLD SPONSOR<br />
  11. 11. SILVER SPONSORS<br />
  12. 12. Who Am I?<br />Founder, CEO, Blue Badge Insights<br />Microsoft Regional Director, MVP<br />Organizing team, Code Camp NYC<br />Co-chair Visual Studio Live!Co-moderator, NYC .NET Developers Group<br />http://www.nycdotnetdev.com<br />Founder, MS BI User Group NYC<br />http://www.msbinyc.com<br />brustblog.com, Twitter: @andrewbrust<br />
  13. 13. Column and Blog<br />
  14. 14. Book<br />
  15. 15. Agenda<br />SQL Server BI – High Level<br />PowerPivot and Excel Services<br />Analysis Services Tabular Mode<br />Project “Crescent”<br />Overview: Master Data Services and Data Quality Services<br />
  16. 16. SQL Server BI Overview<br />
  17. 17. Microsoft Business Intelligence<br />Business User Experience<br />Familiar User Experience <br />Self-Service access & insight<br />Data exploration & analysis<br />Predictive analysis<br />Data visualization<br />Contextual visualization<br />Business Collaboration Platform<br />Dashboards & Scorecards<br />Excel Services<br />Web based forms & workflow<br />Collaboration<br />Search<br />Content Management<br />LOB data integration<br />Business Collaboration Platform<br />Data Infrastructure and BI Platform<br />Analysis Services<br />Reporting Services<br />Integration Services<br />Master Data Services<br />Data Mining<br />Data Warehousing<br />Information Platform<br />
  19. 19. But Wait, There’s More!<br />R2: PowerPivot<br />R2: Report Parts in SSRS<br />Denali: Analysis Services Tabular mode<br />And corresponding improvements in PowerPivot<br />Denali: “Crescent”<br />Denali: Data Quality Services<br />
  20. 20. PowerPivot and Excel Services<br />
  21. 21. Self-Service BI with PowerPivot<br />Excel + Analysis Services + SharePoint<br />Enables the working in Excel but mitigates the “spreadmart” pitfalls:<br />Use Analysis Services (AS) as a hidden engine<br />Instead of no engine<br />Share via SharePoint, accessible by all AS clients<br />Instead of “deploying” via email<br />Formal data refresh on server<br />So data doesn’t get stale, and users don’t have to make effort at updating<br />Allow IT to monitor<br />So it’s not all rogue<br />Provide path to more rigorous implementations<br />Can be upsized to Analysis Services<br />
  22. 22. Column-Oriented Stores<br /><ul><li>Imagine, instead of:
  23. 23. You have:
  24. 24. Perf: values you wish to aggregate are adjacent
  25. 25. Efficiency: great compression from identical or nearly-identical values in proximity
  26. 26. Fast aggregation and high compression means huge volumes of data can be stored and processed, in RAM</li></li></ul><li>Data Import<br />Relational databases<br />SQL Server (including SQL Azure!), Access<br />Oracle, DB2, Sybase, Informix<br />Teradata<br />“Others” (OLE DB, including OLE DB provider for ODBC)<br />OData feeds, incl. R2/Denali Reporting Services, Azure DataMarket, ADO.NET Data Services (Astoria)<br />Excel via clipboard, linked tables<br />Filter, preview, friendly names for tables/columns<br />
  27. 27. Calculated Columns and DAX<br /><ul><li>Formula-based columns may be created
  28. 28. Formula syntax is called DAX (Data Analysis eXpressions).</li></ul>Not to be confused with MDX or DMX. Or DACs.<br /><ul><li>DAX expressions are similar to Excel formulas</li></ul>Work with tables and columns; similar to, but distinct from, worksheets and their columns (and rows)<br /><ul><li>=FUNC('table name'[column name])
  29. 29. =FUNCX('table name', <filter expression>)
  30. 30. FILTER(Resellers,[ProductLine] = "Mountain")
  31. 31. RELATED(Products[EnglishProductName])
  32. 32. DAX expressions can be heavily nested</li></li></ul><li>PowerPivot Guidebook<br />Import data fromalmost anywhere<br />View data <br />in Excel<br />Sort one column by another<br />Calculatedcolumn<br />entry<br />Sort and filter<br />DAX formula bar<br />Relationship indicator<br />Table tabs<br />
  33. 33. What’s New?<br />Data and Diagram views<br />KPIs<br />Measures<br />Measuregrid<br />Measureformula<br />
  34. 34. Diagram View<br />Perspectives<br />Default Aggregations<br />Special Advanced Mode<br />Reporting Properties<br />Hierarchies<br />Hide specific <br />columns and tables<br />Create relationshipsvisually<br />Measures<br />KPIs<br />
  35. 35. PowerPivot Client<br />
  36. 36. Excel Services<br /><ul><li>A component of SharePoint Server 2007/2010; requires Enterprise CAL
  37. 37. Allows export of workbook, worksheet, or individual items to SharePoint report library</li></ul>Works great for PivotTables and Charts!<br />Also for sheets with CUBExxx formulas or conditional formatting-driven “scorecards”<br /><ul><li>Content can be viewed in browser</li></ul>Excel client not required<br />Drilldown interactivity maintained<br />Rendered in pure AJAX/HTML<br />Parameterization supported<br />
  38. 38. PowerPivot Server<br />Publish to Excel Services<br />Viewing and interacting<br />Data Refresh<br />Treating as SSAS cube<br />URL to .xlsx as server name<br />Db name is GUID-based; best to discover it<br />Use Excel, Reporting Services as clients<br />And now “Crescent” too…more later<br />
  39. 39. The IT Dashboard<br />Increase IT efficiency:<br />Familiar Technologies for Authoring, Sharing, Security, and Compliance<br />Customizable IT Dashboard<br />Visualize usage with animated charts<br />Simplify management of SSBI content usingIT Operations Dashboard for SharePoint<br />
  40. 40. PowerPivot Server<br />
  41. 41. Analysis Services Tabular Mode<br />
  42. 42. Analysis Services Tabular Mode<br />SSAS Tabular Mode is the enterprise/server implementation of PowerPivot<br />You must have a dedicated tabular mode SSAS instance<br />BI Developer Studio (BIDS) does PowerPivot<br />Implements equivalent tooling to PowerPivot Window<br />Can create an SSAS Tabular database project by importing an Excel workbook with PowerPivot model<br />SSAS tabular models support partitions and roles<br />
  43. 43. SSAS Tabular Project in BIDS <br />SSAS tabular project<br />menus and toolbar<br />Measure grid and <br />formula bar<br />Reporting properties <br />in Properties window<br />
  44. 44. DirectQuery Mode<br />In DQ mode, model defines schema, but is not used for data<br />Queries issued directly against source<br />Similar to ROLAP storage for conventional cubes<br />
  45. 45. SSAS Tabular Mode<br />
  46. 46. PROJECT “CRESCENT”<br />
  47. 47. What is Crescent?<br />Ad hoc reporting. Really!<br />Analysis, data Exploration<br />Data Visualization<br />In Silverlight, in the browser, in SharePoint<br />Feels a little like Excel BI<br />Is actually based on SSRS<br />Crescent makes a special RDL file<br />And wraps it in an RDLX<br />
  48. 48. Crescent Data Sources<br />Crescent works only against PowerPivot/SSAS Tabular models<br />DirectQuery mode supported, however<br />For PowerPivot, click “Create Crescent Report” button or option on workbook in SharePoint report gallery<br />For SSAS tabular model, create BISM data source, then click its “Create Crescent Report” button or option<br />BISM data sources can point to PowerPivot workbooks too, if you want.<br />
  49. 49. Crescent!<br />In the browser, <br />in Silverlight<br />Ribbon, like Excel<br />Variety of <br />visualizations<br />and data formats<br />Field list, like Excel<br />Data regions pane,like Excel<br />
  50. 50. Text and Viewing<br />Text boxes edited asif in Office<br />Maximize one chart, or put whole report in preview or full-screen<br />
  51. 51. Crescent Basics<br />
  52. 52. Constraining Your Data In Crescent<br />Tiles<br />A filtering mechanism within a visualization<br />Highlighting<br />Selection in one visualization affects the others<br />Slicers<br />Similar to Excel against PowerPivot<br />True Filters<br />Checked drop-down list; very Excel-like<br />Right Hand Filter Pane, similar to SSRS and Excel Services<br />
  53. 53. Crescent Filtering<br />
  54. 54. Scatter/Bubble Charts<br />Allow for 3 measures by up to 4 dimensions<br />One dimension is “playable” through a slider or animation<br />Excellent way to visualize trends over time<br />
  55. 55. Small Multipliers<br />Multiple charts within a chart, in columns, rows, or a matrix<br />Allows for visualizing an additional dimension<br />Think of it like a clustered chart with each series shown individually<br />
  56. 56. Advanced Properties<br />Setting the representative column and image tells Crescent how to summarize your data, and show stored images<br />Other properties tell it about key attribute, default aggregation and more<br />For SSAS tabular models, “Direct Query” mode tells Crescent to get data from relational data source instead of columnar cache<br />
  57. 57. Crescent Advanced Features<br />
  58. 58. Vocabulary<br />MOLAP: Multidimensional OLAP<br />UDM: Unified Dimensional Model<br />Cube: Unit of schema in a dimensional database<br />VertiPaq: PowerPivot/SSAS’ column store engine<br />BISM: BI Semantic Model<br />Tabular: A column store-based model<br />Because it uses tables, not cubes<br />
  59. 59. Apollo<br />Implementation of VertiPaq columnar storage engine for SQL Server relational databases<br />Use it by creating a column store index<br />CREATE COLUMNSTORE INDEX index ON table(col1, Col2, …)<br />Can ignore it too:<br />OPTION (IGNORE_NONCLUSTERED_COLUMNSTORE_INDEX)<br />Significantly increases performance of star join queries (i.e. aggregating queries with dimension lookups).<br />Not as good as SSAS, but better than plain old GROUP BY<br />
  60. 60. Overview:Master Data ServicesANDData Quality Services<br />
  61. 61. Microsoft’s Master Data Management (MDM) tool<br /><ul><li>Examples:</li></ul>Sales states, countries, currencies, customer types<br />Customers, products<br />Think of “lookup tables” or just think of dimensions!<br />Slowly changing non-transactional entities in your data<br /><ul><li>What gets stored:</li></ul>Schemas<br />Any hierarchies<br />The data!<br /><ul><li>Other features:</li></ul>Collections, business rules, security, workflows<br />Versioning<br />
  62. 62. Other Facts<br /><ul><li>Result of acquisition of Stratature
  63. 63. v1 was an ASP.NET application; UI is “different”
  64. 64. New in v2:
  65. 65. Now Silverlight-based; UI is still “different”
  66. 66. Excel add-in for data entry; creation of entities and attributes
  67. 67. Perform matching with DQS before loading
  68. 68. Includes .NET and Web Services APIs for reading/writing data and creating/editing models
  69. 69. Does not integrate with Analysis Services tools even though many of its features and concepts mirror those of dimension designer
  70. 70. Catalog kept in SQL Server database
  71. 71. Deployment packages can be created, shared and deployed</li></li></ul><li>Objects in MDS<br />Models<br />Entities (like tables or SSAS dimensions)<br />Attributes (like columns/fields or SSAS attributes)<br />Common attributes are Name and Code<br />Attribute Groups<br />Used to taxonomize attributes within tabs in UI <br />Members (like rows/records or SSAS members)<br />Hierarchies (like SSAS hierarchies)<br />Derived or Explicit<br />Collections (like SSAS named sets)<br />Versions<br />Business rules<br />Workflows<br />
  72. 72. Data Quality Services<br />Data Cleansing Tool<br />New to Denali<br />Result of Zoomix Acquisition<br />Uses Artificial Intelligence algorithms detect invalid data and perform matching (for de-duplication)<br />Allows manual intervention, too<br />Can integrate with MDS and SSIS<br />Cleaner data = better adoption of your BI project<br />
  73. 73. DQS Concepts<br />Knowledge Bases<br />Domains<br />“semantic representation[s] of a type of data in a data field…[contain] a list of trusted values, invalid values, and erroneous data.”<br />Mapping<br />Data Quality Projects<br />Cleansing (i.e. correcting)<br />Validate Using Reference Data Services and Use Azure DataMarket (or 3rd party providers)<br />Matching (i.e. de-duping)<br />Confidence<br />Profiling, Monitoring<br />
  74. 74. Questions?<br />Now?<br />Later?<br />Andrew.Brust@BlueBadgeInsights.com<br />@andrewbrust on Twitter<br />www.brustblog.com<br />Want to get the weekly Redmond Roundup Plus dispatch? Just text the word “bluebadge” to 22828<br />