DataOps for the Modern Data Warehouse on Microsoft Azure @ NDCOslo 2020 - Lac...Lace Lofranco
Talk Description:
The Modern Data Warehouse architecture is a response to the emergence of Big Data, Machine Learning and Advanced Analytics. DevOps is a key aspect of successfully operationalising a multi-source Modern Data Warehouse.
While there are many examples of how to build CI/CD pipelines for traditional applications, applying these concepts to Big Data Analytical Pipelines is a relatively new and emerging area. In this demo heavy session, we will see how to apply DevOps principles to an end-to-end Data Pipeline built on the Microsoft Azure Data Platform with technologies such as Data Factory, Databricks, Data Lake Gen2, Azure Synapse, and AzureDevOps.
Resources: https://aka.ms/mdw-dataops
Unable to attend Oracle OpenWorld to learn about the latest developments in Oracle EPM Cloud? It can be difficult to keep abreast of all the changes in this evolving landscape, but we’ve got you covered.
In our webinar, Perficient’s Oracle EPM leadership explored the current cloud offerings and what’s around the corner. Whether you are in IT or finance, your colleagues are driving digital transformation by including cloud in their performance management strategy.
Discussion covered:
-In-depth review of the Oracle EPM Cloud suite
-How the products can be integrated
-How SaaS products compare to on-premises editions
-Benefits of cloud strategies
Databricks is a Software-as-a-Service-like experience (or Spark-as-a-service) that is a tool for curating and processing massive amounts of data and developing, training and deploying models on that data, and managing the whole workflow process throughout the project. It is for those who are comfortable with Apache Spark as it is 100% based on Spark and is extensible with support for Scala, Java, R, and Python alongside Spark SQL, GraphX, Streaming and Machine Learning Library (Mllib). It has built-in integration with many data sources, has a workflow scheduler, allows for real-time workspace collaboration, and has performance improvements over traditional Apache Spark.
DataOps for the Modern Data Warehouse on Microsoft Azure @ NDCOslo 2020 - Lac...Lace Lofranco
Talk Description:
The Modern Data Warehouse architecture is a response to the emergence of Big Data, Machine Learning and Advanced Analytics. DevOps is a key aspect of successfully operationalising a multi-source Modern Data Warehouse.
While there are many examples of how to build CI/CD pipelines for traditional applications, applying these concepts to Big Data Analytical Pipelines is a relatively new and emerging area. In this demo heavy session, we will see how to apply DevOps principles to an end-to-end Data Pipeline built on the Microsoft Azure Data Platform with technologies such as Data Factory, Databricks, Data Lake Gen2, Azure Synapse, and AzureDevOps.
Resources: https://aka.ms/mdw-dataops
Unable to attend Oracle OpenWorld to learn about the latest developments in Oracle EPM Cloud? It can be difficult to keep abreast of all the changes in this evolving landscape, but we’ve got you covered.
In our webinar, Perficient’s Oracle EPM leadership explored the current cloud offerings and what’s around the corner. Whether you are in IT or finance, your colleagues are driving digital transformation by including cloud in their performance management strategy.
Discussion covered:
-In-depth review of the Oracle EPM Cloud suite
-How the products can be integrated
-How SaaS products compare to on-premises editions
-Benefits of cloud strategies
Databricks is a Software-as-a-Service-like experience (or Spark-as-a-service) that is a tool for curating and processing massive amounts of data and developing, training and deploying models on that data, and managing the whole workflow process throughout the project. It is for those who are comfortable with Apache Spark as it is 100% based on Spark and is extensible with support for Scala, Java, R, and Python alongside Spark SQL, GraphX, Streaming and Machine Learning Library (Mllib). It has built-in integration with many data sources, has a workflow scheduler, allows for real-time workspace collaboration, and has performance improvements over traditional Apache Spark.
This presentation is row of demos that introduce how to use Application Insights, how it works and how to build your own application telemetry on top of it. Two surprise demos show audience some case studies how to use Application Insights to plan hosting of global web site and how to support sales and logistics departments in real-time.
This presentation is for those of you who are interested in moving your on-prem SQL Server databases and servers to Azure virtual machines (VM’s) in the cloud so you can take advantage of all the benefits of being in the cloud. This is commonly referred to as a “lift and shift” as part of an Infrastructure-as-a-service (IaaS) solution. I will discuss the various Azure VM sizes and options, migration strategies, storage options, high availability (HA) and disaster recovery (DR) solutions, and best practices.
Microsoft Data Platform - What's includedJames Serra
The pace of Microsoft product innovation is so fast that even though I spend half my days learning, I struggle to keep up. And as I work with customers I find they are often in the dark about many of the products that we have since they are focused on just keeping what they have running and putting out fires. So, let me cover what products you might have missed in the Microsoft data platform world. Be prepared to discover all the various Microsoft technologies and products for collecting data, transforming it, storing it, and visualizing it. My goal is to help you not only understand each product but understand how they all fit together and there proper use case, allowing you to build the appropriate solution that can incorporate any data in the future no matter the size, frequency, or type. Along the way we will touch on technologies covering NoSQL, Hadoop, and open source.
Ground floor introduction to the tools and best practices surrounding SQL Server’s built-in web-based, enterprise-level reporting engine. We'll start with what SSRS is, what you'll use it for and give top tips to know when developing your first reports.
SAP Cloud Platform - Integration, Extensibility & ServicesAndrew Harding
SAP Cloud Platform enables businesses to extend their SAP solutions to create new applications, integrate with other SAP solutions and external third parties (applications, businesses & government) with the addition of cloud services bringing access to the latest technologies such as IoT, Machine Learning, Intelligent RPA, etc.
This is a brief technology introduction to Oracle Stream Analytics, and how to use the platform to develop streaming data pipelines that support a wide variety of industry use cases
Tag based policies using Apache Atlas and RangerVimal Sharma
With an ever increasing need to secure and limit access to sensitive data, enterprises today need an open source solution. Apache Atlas - which is the metadata and governance framework for Hadoop joins hands with Apache Ranger - security enforcement framework for Hadoop to address the need for compliance and security. Vimal will discuss the security and compliance requirements and demonstrate how the combination of Atlas and Ranger solves the problem. Vimal will focus on Tag based policy enforcement which is an elegant solution for large Hadoop clusters with wide variety of data
Your company is not-yet- ready for the cloud ?
How to refresh your BI solution by providing the beauty of Power BI reports on premises and the ability from the same place to consume your legacy reports or to share efficiently your data model through a unique place. Demo based session with an architecture introduction and a "from the field" real project feedback.
More and more, Organizations are considering off-premise hosting and cloud solutions for enterprise solutions. Other Organizations have strict policies to ensure critical and sensitive corporate systems stay within internal walls. This sessions explores what options are available for EPM solutions, including Oracle’s newly announced Planning and Budgeting on the Cloud Service.
Architect’s Open-Source Guide for a Data Mesh ArchitectureDatabricks
Data Mesh is an innovative concept addressing many data challenges from an architectural, cultural, and organizational perspective. But is the world ready to implement Data Mesh?
In this session, we will review the importance of core Data Mesh principles, what they can offer, and when it is a good idea to try a Data Mesh architecture. We will discuss common challenges with implementation of Data Mesh systems and focus on the role of open-source projects for it. Projects like Apache Spark can play a key part in standardized infrastructure platform implementation of Data Mesh. We will examine the landscape of useful data engineering open-source projects to utilize in several areas of a Data Mesh system in practice, along with an architectural example. We will touch on what work (culture, tools, mindset) needs to be done to ensure Data Mesh is more accessible for engineers in the industry.
The audience will leave with a good understanding of the benefits of Data Mesh architecture, common challenges, and the role of Apache Spark and other open-source projects for its implementation in real systems.
This session is targeted for architects, decision-makers, data-engineers, and system designers.
Data Quality Patterns in the Cloud with Azure Data FactoryMark Kromer
This is my slide presentation from Pragmatic Works' Azure Data Week 2019: Data Quality Patterns in the Cloud with Azure Data Factory using Mapping Data Flows
How One Company Offloaded Data Warehouse ETL To Hadoop and Saved $30 MillionDataWorks Summit
A Fortune 100 company recently introduced Hadoop into their data warehouse environment and ETL workflow to save $30 Million. This session examines the specific use case to illustrate the design considerations, as well as the economics behind ETL offload with Hadoop. Additional information about how the Hadoop platform was leveraged to support extended analytics will also be referenced.
An overview of the different sets of functionality of Tableau solution suite, and how it can address the many facets of a comprehensive data mining solution.
Full Stack Monitoring with Azure MonitorKnoldus Inc.
The full-stack monitoring solutions within Azure Monitor is a boon for DevOps & SRE professionals as they can achieve complete observability of all the applications at a centralized location. Be it troubleshooting issues within your application, infrastructure or network, a unified monitoring solution ensures that you can diagnose problems at one place and fix them within
This webinar talks about how Azure Monitor has eased the monitoring of complex modern applications, whether cloud-based or on-premise. It answers questions like -
~ How to quickly detect and diagnose issues across applications?
~ How to manage infrastructure concerns like those in VMs or containers?
~ How to gain insights from your monitoring data?
~ How to support operations at scale?
Azure Days 2019: Business Intelligence auf Azure (Marco Amhof & Yves Mauron)Trivadis
In dieser Session stellen wir ein Projekt vor, in welchem wir ein umfassendes BI-System mit Hilfe von Azure Blob Storage, Azure SQL, Azure Logic Apps und Azure Analysis Services für und in der Azure Cloud aufgebaut haben. Wir berichten über die Herausforderungen, wie wir diese gelöst haben und welche Learnings und Best Practices wir mitgenommen haben.
Azure SQL Database Managed Instance is a new flavor of Azure SQL Database that is a game changer. It offers near-complete SQL Server compatibility and network isolation to easily lift and shift databases to Azure (you can literally backup an on-premise database and restore it into a Azure SQL Database Managed Instance). Think of it as an enhancement to Azure SQL Database that is built on the same PaaS infrastructure and maintains all it's features (i.e. active geo-replication, high availability, automatic backups, database advisor, threat detection, intelligent insights, vulnerability assessment, etc) but adds support for databases up to 35TB, VNET, SQL Agent, cross-database querying, replication, etc. So, you can migrate your databases from on-prem to Azure with very little migration effort which is a big improvement from the current Singleton or Elastic Pool flavors which can require substantial changes.
This presentation is row of demos that introduce how to use Application Insights, how it works and how to build your own application telemetry on top of it. Two surprise demos show audience some case studies how to use Application Insights to plan hosting of global web site and how to support sales and logistics departments in real-time.
This presentation is for those of you who are interested in moving your on-prem SQL Server databases and servers to Azure virtual machines (VM’s) in the cloud so you can take advantage of all the benefits of being in the cloud. This is commonly referred to as a “lift and shift” as part of an Infrastructure-as-a-service (IaaS) solution. I will discuss the various Azure VM sizes and options, migration strategies, storage options, high availability (HA) and disaster recovery (DR) solutions, and best practices.
Microsoft Data Platform - What's includedJames Serra
The pace of Microsoft product innovation is so fast that even though I spend half my days learning, I struggle to keep up. And as I work with customers I find they are often in the dark about many of the products that we have since they are focused on just keeping what they have running and putting out fires. So, let me cover what products you might have missed in the Microsoft data platform world. Be prepared to discover all the various Microsoft technologies and products for collecting data, transforming it, storing it, and visualizing it. My goal is to help you not only understand each product but understand how they all fit together and there proper use case, allowing you to build the appropriate solution that can incorporate any data in the future no matter the size, frequency, or type. Along the way we will touch on technologies covering NoSQL, Hadoop, and open source.
Ground floor introduction to the tools and best practices surrounding SQL Server’s built-in web-based, enterprise-level reporting engine. We'll start with what SSRS is, what you'll use it for and give top tips to know when developing your first reports.
SAP Cloud Platform - Integration, Extensibility & ServicesAndrew Harding
SAP Cloud Platform enables businesses to extend their SAP solutions to create new applications, integrate with other SAP solutions and external third parties (applications, businesses & government) with the addition of cloud services bringing access to the latest technologies such as IoT, Machine Learning, Intelligent RPA, etc.
This is a brief technology introduction to Oracle Stream Analytics, and how to use the platform to develop streaming data pipelines that support a wide variety of industry use cases
Tag based policies using Apache Atlas and RangerVimal Sharma
With an ever increasing need to secure and limit access to sensitive data, enterprises today need an open source solution. Apache Atlas - which is the metadata and governance framework for Hadoop joins hands with Apache Ranger - security enforcement framework for Hadoop to address the need for compliance and security. Vimal will discuss the security and compliance requirements and demonstrate how the combination of Atlas and Ranger solves the problem. Vimal will focus on Tag based policy enforcement which is an elegant solution for large Hadoop clusters with wide variety of data
Your company is not-yet- ready for the cloud ?
How to refresh your BI solution by providing the beauty of Power BI reports on premises and the ability from the same place to consume your legacy reports or to share efficiently your data model through a unique place. Demo based session with an architecture introduction and a "from the field" real project feedback.
More and more, Organizations are considering off-premise hosting and cloud solutions for enterprise solutions. Other Organizations have strict policies to ensure critical and sensitive corporate systems stay within internal walls. This sessions explores what options are available for EPM solutions, including Oracle’s newly announced Planning and Budgeting on the Cloud Service.
Architect’s Open-Source Guide for a Data Mesh ArchitectureDatabricks
Data Mesh is an innovative concept addressing many data challenges from an architectural, cultural, and organizational perspective. But is the world ready to implement Data Mesh?
In this session, we will review the importance of core Data Mesh principles, what they can offer, and when it is a good idea to try a Data Mesh architecture. We will discuss common challenges with implementation of Data Mesh systems and focus on the role of open-source projects for it. Projects like Apache Spark can play a key part in standardized infrastructure platform implementation of Data Mesh. We will examine the landscape of useful data engineering open-source projects to utilize in several areas of a Data Mesh system in practice, along with an architectural example. We will touch on what work (culture, tools, mindset) needs to be done to ensure Data Mesh is more accessible for engineers in the industry.
The audience will leave with a good understanding of the benefits of Data Mesh architecture, common challenges, and the role of Apache Spark and other open-source projects for its implementation in real systems.
This session is targeted for architects, decision-makers, data-engineers, and system designers.
Data Quality Patterns in the Cloud with Azure Data FactoryMark Kromer
This is my slide presentation from Pragmatic Works' Azure Data Week 2019: Data Quality Patterns in the Cloud with Azure Data Factory using Mapping Data Flows
How One Company Offloaded Data Warehouse ETL To Hadoop and Saved $30 MillionDataWorks Summit
A Fortune 100 company recently introduced Hadoop into their data warehouse environment and ETL workflow to save $30 Million. This session examines the specific use case to illustrate the design considerations, as well as the economics behind ETL offload with Hadoop. Additional information about how the Hadoop platform was leveraged to support extended analytics will also be referenced.
An overview of the different sets of functionality of Tableau solution suite, and how it can address the many facets of a comprehensive data mining solution.
Full Stack Monitoring with Azure MonitorKnoldus Inc.
The full-stack monitoring solutions within Azure Monitor is a boon for DevOps & SRE professionals as they can achieve complete observability of all the applications at a centralized location. Be it troubleshooting issues within your application, infrastructure or network, a unified monitoring solution ensures that you can diagnose problems at one place and fix them within
This webinar talks about how Azure Monitor has eased the monitoring of complex modern applications, whether cloud-based or on-premise. It answers questions like -
~ How to quickly detect and diagnose issues across applications?
~ How to manage infrastructure concerns like those in VMs or containers?
~ How to gain insights from your monitoring data?
~ How to support operations at scale?
Azure Days 2019: Business Intelligence auf Azure (Marco Amhof & Yves Mauron)Trivadis
In dieser Session stellen wir ein Projekt vor, in welchem wir ein umfassendes BI-System mit Hilfe von Azure Blob Storage, Azure SQL, Azure Logic Apps und Azure Analysis Services für und in der Azure Cloud aufgebaut haben. Wir berichten über die Herausforderungen, wie wir diese gelöst haben und welche Learnings und Best Practices wir mitgenommen haben.
Azure SQL Database Managed Instance is a new flavor of Azure SQL Database that is a game changer. It offers near-complete SQL Server compatibility and network isolation to easily lift and shift databases to Azure (you can literally backup an on-premise database and restore it into a Azure SQL Database Managed Instance). Think of it as an enhancement to Azure SQL Database that is built on the same PaaS infrastructure and maintains all it's features (i.e. active geo-replication, high availability, automatic backups, database advisor, threat detection, intelligent insights, vulnerability assessment, etc) but adds support for databases up to 35TB, VNET, SQL Agent, cross-database querying, replication, etc. So, you can migrate your databases from on-prem to Azure with very little migration effort which is a big improvement from the current Singleton or Elastic Pool flavors which can require substantial changes.
Cloud and Analytics - From Platforms to an EcosystemDatabricks
Zurich North America is one of the largest providers of insurance solutions and services in the world with customers representing a wide range of industries from agriculture to construction and more than 90 percent of the Fortune 500.
Big Data and Data Warehousing Together with Azure Synapse Analytics (SQLBits ...Michael Rys
SQLBits 2020 presentation on how you can build solutions based on the modern data warehouse pattern with Azure Synapse Spark and SQL including demos of Azure Synapse.
Is your organization considering migrating an existing data center over to AWS to reduce cost, improve reliability, security, and operational performance of your IT operations? If so, join us for a webinar on how to plan and execute your migration to the cloud from classification of applications, assessing your application needs, identifying the target applications and other various migration strategies.
Migrating Enterprise Applications to AWS: Best Practices & Techniques (ENT303...Amazon Web Services
This session discusses strategies, tools, and techniques for migrating enterprise software systems to AWS. We consider applications like Oracle eBusiness Suite, SAP, PeopleSoft, JD Edwards, and Siebel. These applications are complex by themselves; they are frequently customized; they have many touch points on other systems in the enterprise; and they often have large associated databases. Nevertheless, running enterprise applications in the cloud affords powerful benefits. We identify success factors and best practices.
Understanding System Design and Architecture Blueprints of EfficiencyKnoldus Inc.
This exploration delves into the intricate world of system design and architecture, dissecting the fundamental principles and methodologies that underpin the creation of robust and scalable systems. From the conceptualization of software structures to the deployment of hardware components, this comprehensive study navigates through the critical decisions and considerations that engineers face when crafting efficient and reliable systems. Gain insights into best practices, design patterns, and emerging trends that shape the backbone of modern technology, empowering you to engineer solutions that stand the test of time. Whether you're a seasoned architect or an aspiring designer, embark on a journey to master the art and science of system design and architecture.
Azure SQL Database now has a Managed Instance, for near 100% compatibility for lifting-and-shifting applications running on Microsoft SQL Server to Azure. Contact me for more information.
This session provides an overview of how organizations can migrate workloads to the AWS cloud at scale. We will go through available migration frameworks and best practices with common use case examples during this session. After migrating the initial workloads, understand how to migrate at scale to the AWS cloud. Hear about real life experiences from the AWS Professional Services team and learn about common use case examples, frameworks, and best practices. Hear about what to avoid when migrating applications at scale to AWS and understand the tools and partner services that can assist you when migrating applications to AWS.
The breath and depth of Azure products that fall under the AI and ML umbrella can be difficult to follow. In this presentation I’ll first define exactly what AI, ML, and deep learning is, and then go over the various Microsoft AI and ML products and their use cases.
Database Week at the San Francisco Loft
What is Database Freedom?
How AWS can help you unshackle and achieve transformation. Are you operating with old world databases?Discover Database Freedom with AWS.
Speaker: Ben Willett - Solutions Architect, AWS
Adjusting primitives for graph : SHORT REPORT / NOTESSubhajit Sahu
Graph algorithms, like PageRank Compressed Sparse Row (CSR) is an adjacency-list based graph representation that is
Multiply with different modes (map)
1. Performance of sequential execution based vs OpenMP based vector multiply.
2. Comparing various launch configs for CUDA based vector multiply.
Sum with different storage types (reduce)
1. Performance of vector element sum using float vs bfloat16 as the storage type.
Sum with different modes (reduce)
1. Performance of sequential execution based vs OpenMP based vector element sum.
2. Performance of memcpy vs in-place based CUDA based vector element sum.
3. Comparing various launch configs for CUDA based vector element sum (memcpy).
4. Comparing various launch configs for CUDA based vector element sum (in-place).
Sum with in-place strategies of CUDA mode (reduce)
1. Comparing various launch configs for CUDA based vector element sum (in-place).
Show drafts
volume_up
Empowering the Data Analytics Ecosystem: A Laser Focus on Value
The data analytics ecosystem thrives when every component functions at its peak, unlocking the true potential of data. Here's a laser focus on key areas for an empowered ecosystem:
1. Democratize Access, Not Data:
Granular Access Controls: Provide users with self-service tools tailored to their specific needs, preventing data overload and misuse.
Data Catalogs: Implement robust data catalogs for easy discovery and understanding of available data sources.
2. Foster Collaboration with Clear Roles:
Data Mesh Architecture: Break down data silos by creating a distributed data ownership model with clear ownership and responsibilities.
Collaborative Workspaces: Utilize interactive platforms where data scientists, analysts, and domain experts can work seamlessly together.
3. Leverage Advanced Analytics Strategically:
AI-powered Automation: Automate repetitive tasks like data cleaning and feature engineering, freeing up data talent for higher-level analysis.
Right-Tool Selection: Strategically choose the most effective advanced analytics techniques (e.g., AI, ML) based on specific business problems.
4. Prioritize Data Quality with Automation:
Automated Data Validation: Implement automated data quality checks to identify and rectify errors at the source, minimizing downstream issues.
Data Lineage Tracking: Track the flow of data throughout the ecosystem, ensuring transparency and facilitating root cause analysis for errors.
5. Cultivate a Data-Driven Mindset:
Metrics-Driven Performance Management: Align KPIs and performance metrics with data-driven insights to ensure actionable decision making.
Data Storytelling Workshops: Equip stakeholders with the skills to translate complex data findings into compelling narratives that drive action.
Benefits of a Precise Ecosystem:
Sharpened Focus: Precise access and clear roles ensure everyone works with the most relevant data, maximizing efficiency.
Actionable Insights: Strategic analytics and automated quality checks lead to more reliable and actionable data insights.
Continuous Improvement: Data-driven performance management fosters a culture of learning and continuous improvement.
Sustainable Growth: Empowered by data, organizations can make informed decisions to drive sustainable growth and innovation.
By focusing on these precise actions, organizations can create an empowered data analytics ecosystem that delivers real value by driving data-driven decisions and maximizing the return on their data investment.
Techniques to optimize the pagerank algorithm usually fall in two categories. One is to try reducing the work per iteration, and the other is to try reducing the number of iterations. These goals are often at odds with one another. Skipping computation on vertices which have already converged has the potential to save iteration time. Skipping in-identical vertices, with the same in-links, helps reduce duplicate computations and thus could help reduce iteration time. Road networks often have chains which can be short-circuited before pagerank computation to improve performance. Final ranks of chain nodes can be easily calculated. This could reduce both the iteration time, and the number of iterations. If a graph has no dangling nodes, pagerank of each strongly connected component can be computed in topological order. This could help reduce the iteration time, no. of iterations, and also enable multi-iteration concurrency in pagerank computation. The combination of all of the above methods is the STICD algorithm. [sticd] For dynamic graphs, unchanged components whose ranks are unaffected can be skipped altogether.
2. Introduction
• Education
• B.S. in Computer Science from UW-Milwaukee
• M.S. in Science of Management from Cardinal
Stritch
• Professional
• Fiserv, DentaQuest, BMO Harris
• Focus on SQL, SSIS, and DevOps strategies
• Talavant
• Focus on larger BI strategies
Turner Kunkel
turner.Kunkel@talavant.com
Senior Consultant @ Talavant
• Hobbies
• Home brewing
• Bass playing (poorly)
• Sailing (light skipper license)
3. There is a better way to make data work for companies. Better resources, strategy, sustainability, inclusion of the
organization as a whole, understanding of client needs, tools, outcomes, better ROI.
• Accelerated planning, implementation and results
• Sustainable solutions
• Increased company-wide buy-in & usage
VALUE WE PROVIDE
By providing a holistic approach inclusive of a
client’s people, processes and technologies -
built on investment in our own employees and
company growth.
HOW WE DO IT
STRATEGY
ARCHITECTURE
IMPLEMENTATION
4. OVERVIEW
On-Premises SSAS
History of AS • ROI considerations
• Future research
• Questions/Comments
• Sources & References
Azure AS
Developing w/ AAS
Automation on AAS
5. END PRODUCT
Robust Power BI sample from Microsoft
2.04 billion NY Taxi trips
Refresh using 20 Azure cores in ~4 seconds
7. OLAP
Database theory
(X,Y,Z) → W (X,Y,Z are axis in a cube, W is value of
cell)
Need arose from tab report structure
from 1980’s database management
systems
Slice, Dice, Drill Down, Roll Up
11. BUSINESS USE
• Explore data from outside the organization
• Historical insight
• Trends and predictive analysis
• Self-service opportunities for business users
• Space saving
• Time saving
Reporting Tools
SSRS
SharePoint
Power Pivot
Power BI
Tableau (and other third parties)
19. MODEL DEVELOPMENT
Web interface (public preview)
Edit relationships, measures, hierarchies
Drag-and-drop query editor for data
Translates to DAX if needed
Interface to open model in several tools
SQL Server Data Tools/Visual Studio
(>=2016)
SQL Server Management Studio
(>=2016)
20. MIGRATION
Best practice
Incremental port of solution
Lean, Phased, Ramp up
Types
Full
Hybrid/piece-wise
Tools
On-premises Gateway
Visual Studio, SQL Management Studio
21. GATEWAY
Install
Download, install, register on-premises
Use
Connect gateway to AAS instance
Configure
Add gateway resource to Azure
*Performs slowly on wireless
*Communication is encrypted
22. DATA ACCESS
DirectQuery benefits
Up-to-date data
Stronger security
Optimized query plan
In-Memory Cache
Stronger query performance
Data refresh required
DirectQuery limitations
Sources
SQL server, Oracle, Teradata
No stored procedures
No calculated tables
Query language translations
23. “HIGH AVAILABILITY”
Backups
Azure allows backup storage
Configurable in the AAS instance
Can backup and restore to separate instances
Hint: Use deployment
scripts
Redundancy
Rarely, Azure servers go down
Ensure availability by deploying to another instance
in a different region
Process each instance in parallel
25. AUTOMATION
Automation Uses
• Model processing on schedule
• Full or incremental
• Start/Stop AAS on schedule
• Scale AAS automatically
• Backup AAS instance on schedule
How to Automate
• Azure Function Apps
• Azure RunBooks
• Classic SSMS/SSIS
• Custom .NET application
• Custom PowerShell
26. FUNCTION APPS
• Separate module in Azure
• Premade Functions
• Webhook/API, Timer (CRON), Data Processing
• Templates (C#, F#, JavaScript)
• Custom functions
• PowerShell, Python, and Windows Batch
• Analysis Server libraries, providers, and
connection string references needed
• Web application sitting in Azure
• Separate pricing model
• Can use Source Control/TFS
27. RUNBOOKS
• Requires Azure Automation account
• Uses ‘Run As’ account to connect to AAS
• RunBooks support PowerShell and Python scripts
• Can be run on a recurring schedule
29. ROI & CONSIDERATIONS
Advantages to movement to AAS
• No physical space needed for servers
• One cloud platform for development
• Azure is Microsoft’s future – AAS is beneficial to learn
• Scaling and automating use is quick
• Integration to already used tools
• Processing power is immense for large data sets
• Access administration is streamlined
• Redundancy built-in
• Simple and forecasted pricing model
Disadvantages to movement to AAS
• Learning curve
• (Possible!) replacement of SSAS resume skill
30. OTHER FEATURES
Query Replicas
Queries distributed among multiple replicas in
a pool
Up to 7 additional query streams (8 total)
Good for high-QPU usage times
Can be configured based on instance usage
Tags
Azure general feature – Name/Value pair for
resources
For example: Environment/Development
Locks
Azure general feature – Locks resource to
prevent actions on the resource
Diagnostics
Performance logs
Event logs
Error logs
Azure infrastructure logs
31. FUTURE FEATURES
AAS support in Power BI Embedded Azure Data Lake storage as data source
Multidimensional cube support
Schema compare
Excel Online connections
AAS data source in SSRS
32. FUTURE RESEARCH
Automation best practices
Deterministic scaling development
Best uses of RunBooks, Functions, Replicas, or
other techniques?
Migration strategy
Full approach to migration plan
Testing techniques while migrating leanly
True Implementation Cost
Production down time while migrating
Opportunity cost and monetary cost during
learning curve
Buy-in from business
On-boarding of business users, if necessary
Production monitoring
Best practices for maintenance
How many resources required
Improvements to implemented system and
development life cycle
33. RECAP
On-Premises SSAS
History of AS • ROI considerations
• Future research
• Questions/Comments
• Sources & References
Azure AS
Developing w/ AAS
Automation on AAS