Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Leveraging Azure Analysis Services Tabular Data Models with Power BI by Tim McAliley

1,125 views

Published on

We will take a look at an introduction and overview of Azure Analysis Services: Microsoft‘s cloud-based analytical engine and Platform as a Service (PaaS) offerings and how to leverage SQL Server Data Tools to build and deploy a tabular data model to Azure Analysis Services.
We will then connect with Power BI Desktop and the Power BI portal to build visualizations. We will discuss Azure Analysis Services features and capabilities, use cases, provisioning and deployment, managing and monitoring, tools, and report creation. Azure Analysis Service became Globally Available in April 2017, and Power
BI has released several major updates as well.

Published in: Technology
  • Be the first to comment

Leveraging Azure Analysis Services Tabular Data Models with Power BI by Tim McAliley

  1. 1. 5/16/2017 1
  2. 2. 5/16/2017 2 timothy.mcaliley@microsoft.com https://www.linkedin.com/in/timothymcaliley
  3. 3. 5/16/2017 3 1. Source: IDC “FutureScape: Worldwide Big Data and Analytics 2016 Predictions,” DOC #259835, November 2015 2. Source: IDC “FutureScape: Worldwide Cloud 2016 Predictions,” DOC #259840, November 2015 SAVINGS • CAPEX • Software costs • IT staff • Pay for what you need 3. Source: Gartner “Cost Optimization Secrets: Plan & Manage Cloud Services for 14% Savings, on Average,” ID: G00299793, March 2016 “By 2020, organizations able to analyze all relevant data and deliver actionable information will achieve an extra $430 billion in productivity benefits.”
  4. 4. 5/16/2017 4
  5. 5. 5/16/2017 5
  6. 6. 5/16/2017 6
  7. 7. 5/16/2017 7
  8. 8. 5/16/2017 8 Azure Analysis Services
  9. 9. 5/16/2017 9 SQL Server Analysis Services SQL Server Analysis Services SQL Server Other data sources On-prem data sources Visual Studio Authoring & Dev SSMS Power BI Visualizations & insights Other 3rd party tools & services SQL Server Reporting Services Azure Analysis Services Visual Studio Authoring & Dev Power BI Visualizations & insights SQL Server Other data sources On-prem data sources SSMS Other 3rd party tools & services Azure Analysis Services SQL Database SQL Data Warehouse Cloud data sources
  10. 10. 5/16/2017 10
  11. 11. 5/16/2017 11 BI semantic model Business logic & metrics Data modeling Security Azure Analysis Services Lifecycle management In-memory cache On-premises Cloud Data sources SQL Database SQL Data Warehouse Other data sources SQL Server Analytics platform system Other data sources On-premises Cloud Client tools Power BI Power BI Embedded Excel Third party BI tools Power BI Desktop Rich semantic modelling Data Sources KPIs Measures Perspectives Relationships Roles Tables Translations
  12. 12. 5/16/2017 12 Rich semantic modelling locatioidentifier identifiertype fieldidentifier numberofunitssoldtodate remaningnumber Numberofuntissoldtoday receivedback productidentifier descriptionline1 descriptionline2 qtyafterqtysales familyidentifier Source Data Complex raw data optimized for processing Semantic Model Rich, business user friendly semantic model Product Id Product Name Product Description Category Category Id Category Name Category Description Sub Category Product Id Shelf qty Return qty Order qty Sale qty Cloud Data On-Premises Data Azure Analysis Services locatioidentifier identifiertype fieldidentifier Lightning fast queries numberofunitssoldtodate remaningnumber Numberofuntissoldtoday receivedback productidentifier descriptionline1 descriptionline2 qtyafterqtysales familyidentifier Source Data Complex raw data optimized for processing Semantic Model Rich, business user friendly semantic model Cloud Data On-Premises Data Cache Semantic model Azure Analysis Services Insights at the speed of thought Queries
  13. 13. 5/16/2017 13 Who has the right version of the truth? Actual Revenue $50.33M Actual Revenue $50.85M Cloud Data On-Premises Data BI Analyst BI Analyst Each analyst creates their own model (Version of the truth) ? Single version of the truth with Azure Analysis Services Actual Revenue $50.75M Actual Revenue $50.75M Cloud Data On-Premises Data BI Analyst BI Analyst Azure Analysis Services IT Pro Life cycle management Source control Monitoring
  14. 14. 5/16/2017 14 ConsumeManage Key capabilities at a glance Create & Deploy AAD Auth/IdentityAAD Auth/IdentityAAD Auth/IdentityAAD Auth/Identity Full DAX & MDX support PowerBI.com Power BI Desktop Excel 3rd party tools Power BI Embedded (GA) Fully managed PlatformFully managed PlatformFully managed PlatformFully managed Platform----asasasas----aaaa----ServiceServiceServiceService Up to 100 GB memoryUp to 100 GB memoryUp to 100 GB memoryUp to 100 GB memory Deployment Wizard SSMS & XMLA Management APIs Azure Resource Manager & Portal integration Pause & Resume Elastic scale up/down (GA) 99.9% uptime SLA (GA) Service South Central US , West Europe Note:Note:Note:Note: Check Azure regions for latest information https://azure.microsoft.com/en-us/regions/services/ SQL Server Data Tools 2015 to create & deploy AAS models (SSDT) All SQL Server 2016 AS Tabular Enterprise Edition features • Tabular models (1400 compatibility) • Including Direct Query, Partitions, Perspectives, RLS, Translations • Cloud data sources • Gateway for on on-prem data sources • MD not supported at GA
  15. 15. 5/16/2017 15 Develop and deploy Connect to your data Hybrid connectivity – cloud and on-premises Azure SQL Database Azure SQL Data Warehouse Advanced modeling and analytics Tabular models (1400 compatibility) Powerful bi-directional relationship support KPIs, translations, perspectives, row level security Familiar developer tools Visual Studio & SQL Server Data Tools Performance and scalability Partitions for incremental data loading Parallel processing of partitions Create & manage Get started quickly Spin up a server in seconds Without managing infrastructure Configure to your needs Select your region (US or Europe at Preview) Choose your pricing tier (D1, B1, B2, S1, S2, S4) Easily manage Assign Analysis Services administrators Pause and resume Scale up or down with the tier (at GA) Backup and restore (at GA)
  16. 16. 5/16/2017 16 Consume Enable interactive, self-service tools for insights Analysis at the speed of thought Interactive query performance over billions of rows Simplified view over complex data Trusted data models Single model for one version of the truth Business user friendly Support for preferred visualization tool Full MDX and DAX support Power BI & Power BI Desktop Excel Major 3rd party tools Azure Portal Normal Azure Resource Manager Resource Provider Billed on a predictable hourly rate. Actual usage computed to the second and billed hourly Dev tier No charge when “paused” https://azure.microsoft.com/en-us/pricing/details/analysis-services/
  17. 17. 5/16/2017 17 Scale up and down to meet your needs Adapt to changing workloads Automate scale up and down with ARM API’s and PowerShell Backup & restore Lift & shift SQL Server Management Studio (SSMS) SQL Server Analysis Services Azure storage Azure Analysis Services Backup Easily move existing models to the cloud
  18. 18. 5/16/2017 18 Consume Enable interactive, self-service tools for insights Analysis at the speed of thought Interactive query performance over billions of rows Simplified view over complex data Trusted data models Single model for one version of the truth Business user friendly Support for preferred visualization tool Full MDX and DAX support Power BI & Power BI Desktop Excel Major 3rd party tools
  19. 19. 5/16/2017 19 Easy to connect Direct from reporting tools & services Microsoft Analysis Services client libraries ADOMD.NET Analysis Services Management Objects (AMO) Analysis Services OLE DB provider (MSOLAP DLL) Latest client libraries https://docs.microsoft.com/en-us/azure/analysis-services/analysis-services-data-providers Easy to consume with client reporting tools & services Power BI Other 3rd party tools & services Excel Azure Analysis Services
  20. 20. 5/16/2017 20 Scheduled refresh/Live connection/DirectQuery Group OneDrive Azure Analysis Services, cloud data sources, SQL Azure, etc. On-prem data sources Analysis Services, SQL Server, etc. PBI Desktop Scheduled refresh/ Live connection/ DirectQuery Consumption by end users On-Premises Gateway Dashboards Reports PBI Service Analysis Services (Hosted by Power BI)
  21. 21. 5/16/2017 21 Simple Moderate Complex time timetime Model size and complexity Data update frequency Active concurrent users Number of queries Query latency Scale up using Azure Analysis Services Target scenario Small to medium models with low to moderate concurrent usage Large models with high concurrent usage Query latency Low (Same DC) (In-Memory) Low (Same DC) (In-Memory) Data freshness Data updated up to 8 times/day Data updated on demandon demandon demandon demand Data model size limit 1GB Based on SKU sizeSKU sizeSKU sizeSKU size S0 = 10GB, S1 = 25GB, S2 = 50GB, S4 = 100GB100GB100GB100GB
  22. 22. 5/16/2017 22 Data Sources Supported A wide range of data sources supported Data Source In Memory Direct Query Cloud Azure SQL Data Warehouse .NET Framework Data Provider for SQL Server .NET Framework Data Provider for SQL Server Azure SQL Database .NET Framework Data Provider for SQL Server .NET Framework Data Provider for SQL Server On Premises (via Gateway) SQL Server SQL Server Native Client 11.0 .NET Framework Data Provider for SQL Server Microsoft OLE DB Provider for SQL Server .NET Framework Data Provider for SQL Server Oracle Microsoft OLE DB Provider for Oracle Oracle Data Provider for .NET Oracle Data Provider for .NET Teradata OLE DB Provider for Teradata Teradata Data Provider for .NET Teradata Data Provider for .NET Analytics Platform System .NET Framework Data Provider for SQL Server .NET Framework Data Provider for SQL Server For more information go to https://azure.microsoft.com/en-us/documentation/articles/analysis-services-overview/
  23. 23. 5/16/2017 23 Gateway Each Azure Analysis Service Server requires a dedicated gateway An Azure Analysis services on-prem gateway can only be used with a single Analysis Services Server Shares same technology as the Power BI gateway, but does not share the same admin or config For more information go to https://azure.microsoft.com/en-us/documentation/articles/analysis-services-gateway/ Azure Analysis Services Architecture Excel Third party BI tools Cloud data sources SQL Database SQL Data Warehouse Direct Query Cached Model Power BI Power BI Embedded (GA) SQL Server Oracle, Teradata Other data sources Power BI Desktop Visual Studio Authoring and development tools On-premises data sources Direct Query Cached Model Gateway Other data sources Cloud visualization tools On-premises visualization tools Azure Analysis Services
  24. 24. 5/16/2017 24 Common scenarios Move existing model Visual Studio Authoring and development tools Azure Analysis Services Gateway Power BI Cloud visualization tools SQL Server Other data sources On-premises data sources SSMSSQL Server Analysis Services Move existing model to reduce the burden up managing infrastructure Ensure models can scale up to handle spikes in demand with runtime elastic scale❶ ❶ Available at GA Common scenarios Leverage modern data sources Visual Studio Authoring and development tools Azure Analysis Services Power BI Cloud visualization tools SQL Server Other data sources On-premises data sources SSMS Scale your data models with your cloud data sources ❶ Gateway SQL Database SQL Data Warehouse Cloud data sources ❶ Available at GA
  25. 25. 5/16/2017 25 Proven DevOps experience Develop in the familiar Visual Studio environment Visual Studio Application Lifecycle Management DEVELOPMENT PRODUCTION Planning Analytics Continuous delivery Dev + test Life cycle management Easily deploy updates to dev, test, or production instances PowerShell scripts to create, delete, pause, and scale servers Source control Full Visual Studio source control support for team development and auditing Tabular Model Scripting Language (TMSL) Monitoring Runtime monitoring in Azure and tool such as SQL Profiler
  26. 26. 5/16/2017 26 Automation option Deploy resources with Resource Manager templates and Azure PowerShell Automation with scripts Easily add database user via scripts Users must be in your Azure Active Directory For more information: https://azure.microsoft.com/en-us/documentation/articles/analysis-services-manage/
  27. 27. 5/16/2017 27 Getting started with Azure Analysis Services Quick and easy to get started Quickly spin up a server without the burden of managing infrastructure.
  28. 28. 5/16/2017 28 Azure portal - Analysis Services Locations South Central US West Europe West Central US Check online for the latest support locations: https://azure.microsoft.com/en-us/regions/services/ Azure portal - Analysis Services Elastic scale Query Processing Units (QPU) plus cache sizes for your model workloads
  29. 29. 5/16/2017 29 Azure portal - Analysis Services Access Control Azure portal - Analysis Services Server Administrators
  30. 30. 5/16/2017 30 Azure portal - Analysis Services Connecting to Azure Analysis Services with SSMS Azure portal - Analysis Services Metrics SQL profile will work with the client redistributables xEvents will not at this time
  31. 31. 5/16/2017 31 Resources Azure Analysis Services Overview Get started in the Azure Portal Learn more Pricing Blog Some information relates to pre-released product which may be substantially modified before it’s commercially released. Microsoft makes no warranties, express or implied, with respect to the information provided here.
  32. 32. 5/16/2017 32 Client tools Tabular models DAX
  33. 33. 5/16/2017 33 Client tools The same tools you use today. Leverage your existing skill set Latest versions of SSDT & SSMS support Azure Analysis Services Connecting to Azure Analysis Services with familiar tools SSDT SSMS
  34. 34. 5/16/2017 34 SQL Server Tools SQL Server Management Studio SQL Server Management Studio (SSMS) is now a stand-alone install outside of the SQL Server release Available as a download, SSMS is free and does not require a SQL Server license to install and use The goal is to deliver frequent updates, to include: New functionality Fixes, and Support for the newest functionality in Azure Analysis Services, SQL Server and Azure SQL Database SQL Server Tools SQL Server Management Studio [Continued] SSMS now includes a tool to check for—and apply—updates
  35. 35. 5/16/2017 35 SQL Server Tools SQL Server Management Studio [Continued] SSMS now uses the Visual Studio 2015 Isolated Shell Supported SQL Server versions: SQL Server 2008 to SQL Server 2016, and Provides the greatest level of support for working with the latest cloud features in Azure SQL Database There is no explicit block for SQL Server 2000 or SQL Server 2005—but some features may not work properly Additionally, SSMS can be installed side-by-side with earlier released non-preview versions SQL Server Tools SQL Server Management Studio [Continued] Supported operating systems: Windows 10, Windows 8, Windows 8.1, Windows 7 (SP1) Windows Server 2012 (64-bit), Windows Server 2012 R2 (64-bit), Windows Server 2008 R2 (64-bit) Available languages: Chinese (People's Republic of China), Chinese (Taiwan), English (United States), French, German, Italian, Japanese, Korean, Portuguese (Brazil), Russian, and Spanish
  36. 36. 5/16/2017 36 SQL Server Tools SQL Server Data Tools SQL Server Data Tools (SSDT) is a stand-alone install outside of the SQL Server release SSDT is based on Visual Studio 2015, and is co- released with SQL Server as a free web download SQL Server Tools SQL Server Data Tools [Continued] This release supports SQL Server 2016 back to SQL Server 2005 SSDT is backwards compatible This means you can always use the newest SSDT to design and deploy databases, models, reports, and packages that run on older supported versions of SQL Server SSDT supports the same operating systems and languages as SSMS
  37. 37. 5/16/2017 37 SQL Server Tools SQL Server Data Tools [Continued] SSDT can be installed without Visual Studio 2015 The Visual Studio 2015 Isolated Shell will be installed, together with SQL Server project templates and designers As a standalone tool, SSDT does not include many of the features you would get with a full Visual Studio installation, such as project types and compilers for multiple languages, and extended Source Code Control integration If Visual Studio 2015 is already installed, the SSDT Setup simply adds the SQL Server-related templates If possible, install Visual Studio first, followed by SSDT—this helps avoid a situation where newer templates could be overwritten by older templates Analysis Services New Features ► Tabular • While most new tabular features are available only with SSAS 2016 models (compatibility level 1200), all tabular models can benefit from: • Parallel processing of multiple table partitions • Support for Visual Studio Configuration manager • Improved DAX formula editing (multi-line, tabs, and comments) • Formula fixup (renaming objects will automatically update formula references) • Saving incomplete measures (all measures must be complete to deploy the project) • New behaviors with DAX variables • New DAX functions (~60)
  38. 38. 5/16/2017 38 Analysis Services New Features ► Tabular ► New DAX Functions • DATEDIFF • PERCENTILE(X) • CONCATENATE(X) • MEDIAN(X) • GEOMEAN(X) • PRODUCT(X) • XIRR • XNPV • SELECTCOLUMNS • SUMMARIZECOLUMN S • GROUPBY • UNION • CROSSFILTER • And many more… Analysis Services New Features ► Tabular ► 1200 Models • New features for 1200 models only are grouped by sub-category: • Development • Management
  39. 39. 5/16/2017 39 Analysis Services New Features ► Tabular ► 1200 Models ► Development • New 1200 model development features: • Calculated tables • Bi-directional cross filters • Display folders, to organize model elements into logical groups • Translations, to view models in a preferred language • DirectQuery enhancements • Improved SSDT modeling performance, thanks to TMSL (introduced later) Analysis Services Tabular ► 1200 Models ► Development ► Calculated Tables • Define calculated tables to add new tables to the model, based on existing data from other tables • Defined by using DAX • Calculated tables are generally best for intermediate calculations of data stored in the model, rather than calculated on the fly • Calculated tables appear in Relationship view, and relationships can be defined with other tables • Calculated tables are recalculated in the same circumstances as calculated columns
  40. 40. 5/16/2017 40 Analysis Services Tabular ► 1200 Models ► Development ► Bi-Directional Cross Filters • Bi-directional cross filtering allows defining how filter context propagates between related tables • Bi-directional filters on both sides of a bridging table will solve the “many-to- many” problem—without the need to write DAX formulas Analysis Services Tabular ► 1200 Models ► Development ► DirectQuery • DirectQuery enhancements: • New data sources: • Microsoft Analytics Platform System (APS) • Oracle • Teradata • Support for common analytical tools with MDX queries • Improved query generation, resulting in faster performance MDX Query DAX Query Direct access without moving data
  41. 41. 5/16/2017 41 Analysis Services Tabular ► 1200 Models ► Development ► DirectQuery [Continued] • DirectQuery enhancements: • Row-level security, defined by model roles with DAX filters • Calculated columns (but not calculated tables), but note: • In certain known cases, the same formula can return different results from a cached model compared to a DirectQuery model • These differences are a consequence of the semantic differences between the in-memory analytics engine and the underlying data source, queried with SQL • Sample partitions, enabling in-memory data to be cached to facilitate model development Analysis Services Tabular ► 1200 Models ► Development ► DirectQuery [Continued] • DirectQuery benefits: • Data is up-to-date, with no extra management overhead of having to maintain a separate copy of the data • Data sets can be larger than the memory capacity of the SSAS server • DirectQuery can take advantage of provider-side query acceleration, such as that provided by memory-optimized column indexes • Security can be enforced by the back-end database, by using row-level security features from the back-end database—or alternatively, by using row-level security in the model • SSAS performs optimizations to ensure that the query plan for the query executed against the back-end database will be as efficient as possible
  42. 42. 5/16/2017 42 Analysis Services New Features ► Tabular ► 1200 Models ► Management • New 1200 model management features: • Tabular Object Model (TOM) • Tabular Model Scripting Language (TMSL) Analysis Services Tabular ► 1200 Models ► Management ► TOM • The Tabular Object Model (TOM) is part of AMO • With TOM, developers can now use familiar tabular concepts, rather than multidimensional concepts • This promotes simpler, and more readable, code when developing against tabular models Column(s)Column(s)Column(s)Column(s) Measure(s)Measure(s)Measure(s)Measure(s) Table(s)Table(s)Table(s)Table(s) ModelModelModelModel DatabaseDatabaseDatabaseDatabase
  43. 43. 5/16/2017 43 Analysis Services Tabular ► 1200 Models ► Management ► TOM ► Example public void ReprocessTable(string cnnString, string dbName, string tableName) { var server = new Server(); server.Connect(cnnString); // Connect to the server Database db = server.Databases[dbName]; // Connect to the database Model = db.Model; // Reprocess the table model.Tables[tableName].RequestRefresh(RefreshType.Full); model.SaveChanges(); // Commit the changes } Analysis Services Tabular ► 1200 Models ► Management ► TMSL • The Tabular Model Scripting Language (TMSL) allows for simplified scripting and development for tabular models • This new language describes and manages objects with JSON • SSMS database commands can produce TMSL scripts, including: • Create, Alter, Delete, Backup, Restore, Attach, and Detach • Changes to the model now only affect a single object instead of having to map everything to multidimensional objects—this makes metadata operations very fast • As metadata changes are now localized in the script, it allows for simple code merges
  44. 44. 5/16/2017 44 Analysis Services Tabular ► 1200 Models ► Management ► TMSL [Continued] • The script to represent the schema of the model was developed together with the Power BI team, with the goal to have feature parity with the Power BI APIs to allow reusability between products • SSAS PowerShell cmdlet Invoke-ASCmd accepts tabular model scripting language commands Analysis Services Tabular ► 1200 Models ► Management ► TMSL ► Examples { "refresh": { "type": "full", "objects": [ { "database": "Sales Analysis", "table": "Reseller Sales" } ] } } { "backup": { "database": "Sales Analysis", "file": "SalesAnalysis.abf", "password": "Pass@word1", "allowOverwrite": false, "applyCompression": true } }
  45. 45. 5/16/2017 45 Things not currently supported in preview Multidimension models xEvents Sync Backup and restore (Coming soon) Setting some Server properties (Check doc for latest info) Non AAD users or Anonymous users New DAX functions Over 50 new DAX functions DATEDIFF SELECTCOLUMNS PERCENTILE(X) SUMMARIZECOLUMNS CONCATENATE(X) GROUPBY MEDIAN(X) UNION GEOMEAN(X) CROSSFILTER PRODUCT(X) VARIABLE support XIRR and many more .. XNPV
  46. 46. 5/16/2017 46 Tabular Performance in 2016 (Super DAX) DAX Enhancements Non empty calculation Reduce the number of scans needed for non empty. Fusion Multiple measures from the same table will be combined into a single storage engine query. Grouping sets When a query asks for measures at multiple granularities (Total/Year/Month), a single query is sent at the lowest level and the rest of the granularities are derived from the lowest level. Redundant join elimination A single query to the storage engine returns both the dimension columns and the measure values. Strict evaluation of IF/SWITCH A branch whose condition is false will no longer result in storage engine queries. Previously, branches were eagerly evaluated but results discarded later on. Evaluate an expression once and use the results many times. Some information relates to pre-released product which may be substantially modified before it’s commercially released. Microsoft makes no warranties, express or implied, with respect to the information provided here.

×