TriggerDB Consulting SRL is an Argentina-based company founded by Maximiliano Accotto, an expert in Microsoft SQL Server and BI. They provide consulting, support, and training services for technologies like SQL Server, analytics, and PowerBI. Their 15+ years of experience and over 150 clients in the region have made them a leader in SQL Server, analytics, and PowerBI consulting services.
why an Opensea Clone Script might be your perfect match.pdf
Modernizando plataforma de bi
1. TriggerDB Consulting SRL | www.triggerdb.com | Info@triggerdb.com | Buenos Aires Argentina
Modernización de MS BI
Maximiliano Accotto
2. TriggerDB Consulting SRL | www.triggerdb.com | Info@triggerdb.com | Buenos Aires Argentina
¿Quienes somos?
TriggerDB Consulting SRL es una empresa Argentina fundada por Maximiliano Accotto ,
experto reconocido a nivel mundial en Microsoft SQL Server y BI.
Somos partner certificados de Microsoft en las competencias de SQL Server, Data Analytics y
Powerbi.
Nuestros mas de 15 años de trayectoria , alto expertice y mas de 150 clientes en toda la región,
nos hace lideres en servicios de consultoría y capacitación en tecnologías de SQL Server , analytics y
Powerbi.
3. TriggerDB Consulting SRL | www.triggerdb.com | Info@triggerdb.com | Buenos Aires Argentina
Servicios que ofrecemos
Consultoría Soporte Premier Training
4. TriggerDB Consulting SRL | www.triggerdb.com | Info@triggerdb.com | Buenos Aires Argentina
Productos en los que trabajamos
SQL Server Analysis Services Azure Data Factory HDInsight PowerBI
Azure
SQL Data Warehouse
Azure
Data Lake
Machine
Learning
Azure SQL Server
Power Pivot
Power Query
5. TriggerDB Consulting SRL | www.triggerdb.com | Info@triggerdb.com | Buenos Aires Argentina
Acerca de Maxi Accotto
Microsoft MVP Data Platform desde el 2005.
• Orador frecuente para eventos y comunidades.
Especialista en tecnologías de Data y BI con mas de 15 años de
trayectoria
https://twitter.com/maxiaccotto
https://ar.linkedin.com/in/maxiaccotto
maxi@triggerdb.com
https://www.triggerdb.com/blog/
11. Cualquier tipo y de diferentes orígenes
Social network
analysis
Weather
forecasting
Healthcare
outcomes
Predictive
maintenance
Targeted
advertising
Natural resource
exploration
Fraud
detection
Telemetry data
analysis
Buyer propensity
models
Churn analysis
Life sciences
research
Web app
optimization
Network intrusion
detection
Smart meter
monitoring
13. IT Científico de Datos Analista Usuario Final
BI en la era de AI
14. TriggerDB Consulting SRL | www.triggerdb.com | Info@triggerdb.com | Buenos Aires Argentina
Arquitectura de BI moderna clásica
15. TriggerDB Consulting SRL | www.triggerdb.com | Info@triggerdb.com | Buenos Aires Argentina
Big Data StoresInformation
Management
Data Warehouse Modern BI: Components
Action
People
Automated
Systems
Apps
Web
Mobile
Bots
Event Hubs
Data Catalog
Data Factory
HDInsight
(Hadoop and
Spark)
Stream Analytics
Intelligence
Data Lake
Analytics
Machine Learning
SQL Data
Warehouse
Data Lake Store
& Azure Storage
Data
Sources
Apps
Sensors
and
devices
Data
Intelligence
Dashboards &
Visualizations
Cortana
Bot
Service
Cognitive Services
Power BI
Azure Analysis
Services
IOT Hubs
Cosmos DB
Machine Learning
and Analytics
HDInsight
(Hadoop and
Spark)
Stream Analytics
Data Lake
Analytics
Machine
Learning
Azure Analysis
Services
Azure Databricks
16. TriggerDB Consulting SRL | www.triggerdb.com | Info@triggerdb.com | Buenos Aires Argentina
Azure SQL Database
17. TriggerDB Consulting SRL | www.triggerdb.com | Info@triggerdb.com | Buenos Aires Argentina
AZURE SQL DATABASE
Best for modernization at
scale with low cost and effort
Elastic PoolSingleSQL
Managed
Instance
SQL
Azure SQL Database
Instance-scoped programming
model with high compatibility
to SQL Server
Standalone managed
database for predictable and
stable workloads
Shared resource model for
greater efficiency through
multi-tenancy
18. TriggerDB Consulting SRL | www.triggerdb.com | Info@triggerdb.com | Buenos Aires Argentina
Seamless cloud integration
Easy lift-and-shift migration
Azure SQL Database Managed Instance
facilitates lift and shift migration from on-premises SQL
Server to cloud
Azure Hybrid Benefit for SQL Server
maximizes current on-premises license investments to
facilitate migration
Database Migration Service (DMS)
provides seamless and reliable migration at scale with
minimal downtime
Database Migration
Ser vice (DMS)
Azure SQL Database
Managed Instance
Azure Hybrid Benefit
(AHB) for SQL Ser ver
19. TriggerDB Consulting SRL | www.triggerdb.com | Info@triggerdb.com | Buenos Aires Argentina
Azure Data Factory
20. TriggerDB Consulting SRL | www.triggerdb.com | Info@triggerdb.com | Buenos Aires Argentina
Azure Data Factory
• Orchestrate, monitor & schedule - Compose data processing,
storage & movement services (on premises & cloud)
• Automatic infrastructure management
• Combine pipeline intent w/ resource allocation & mgmt
• Globally available data movement capability
• Single pane of glass - One place to manage and monitor
your data pipelines
21. TriggerDB Consulting SRL | www.triggerdb.com | Info@triggerdb.com | Buenos Aires Argentina
Azure Blob Storage
Game Log Files
Customer Table
On Premises
Data Mart
Game Logs
Customer Table
Azure DB
Customer
Game Usage
Visualize
Data Set
(Collection of files,
DB table, etc)
Activity: a processing step
(Hadoop job, custom code, ML model, etc)
Pipeline: a sequence of
activities (logical group)Data Factory
Concepts
…
Data Sources Ingest Transform & Analyze Publish
Customer
Table
Geocode
Transform,
Combine, etc Analyze Move
22. TriggerDB Consulting SRL | www.triggerdb.com | Info@triggerdb.com | Buenos Aires Argentina
Basic Concepts• ADF is Microsoft’s unified platform for ETL/ELT services in the cloud
• ADF allows you to build data pipelines and execute them/schedule their runs
• Data pipeline is a chain/group of activities to be performed on your data, e.g. data movements/transformations
• Some activities are powered by services with data store/compute resources allocated outside ADF, e.g.
HDI/ML/etc.
23. TriggerDB Consulting SRL | www.triggerdb.com | Info@triggerdb.com | Buenos Aires Argentina
Azure Data Factory v2 Service
Scheduling | Orchestration | Monitoring
UX & SDK
Authoring | Monitoring/Mgmt
Self Hosted
Integration
Runtime
Cloud Svcs, Apps
& Data
On Prem Apps
& Data
Azure
Integration
Runtime
Command and Control
Data
SSIS
Package
Data Factory
A data integration account.
Location of orchestration, service metadata
Integration Runtime (IR)
ADF’s execution engine
Three core capabilities:
• data movement
• pipeline activity execution
• SSIS package execution
To integrate data flow and control flow across
the enterprises’ hybrid cloud, customer can
instantiate multiple IR instances for different
network environments:
- On premises (similar to DMG in ADF V1)
- In public cloud
- Inside VNet
Pipeline
24. TriggerDB Consulting SRL | www.triggerdb.com | Info@triggerdb.com | Buenos Aires Argentina
Azure SQL Data Warehouse
25. TriggerDB Consulting SRL | www.triggerdb.com | Info@triggerdb.com | Buenos Aires Argentina
Azure SQL Data Warehouse
PB scale with MPP and independent compute/ storage scale
Grow, shrink, and pause in seconds
Compute-optimized for demanding workloads
Unlimited columnar storage
26. TriggerDB Consulting SRL | www.triggerdb.com | Info@triggerdb.com | Buenos Aires Argentina
Fully managed PaaS
27. TriggerDB Consulting SRL | www.triggerdb.com | Info@triggerdb.com | Buenos Aires Argentina
Target workload: Analytics
• Store large volumes of data
• Consolidate disparate data into a single location
• Shape, model, transform and aggregate data
• Perform query analysis across large datasets
• Ad-hoc reporting across large data volumes
• All using simple SQL constructs
“SQL on SQL”
28. TriggerDB Consulting SRL | www.triggerdb.com | Info@triggerdb.com | Buenos Aires Argentina
Unsuitable workloads
Operational workloads
High frequency reads & writes
Large numbers of singleton selects
High volume of single row inserts
Data Preparation
Row by row processing needs
Incompatible formats (JSON, XML)
30. TriggerDB Consulting SRL | www.triggerdb.com | Info@triggerdb.com | Buenos Aires Argentina
Distributing Data
CREATE TABLE [build].[FactOnlineSales]
(
[OnlineSalesKey] int NOT NULL
, [DateKey] datetime NOT NULL
, [StoreKey] int NOT NULL
, [ProductKey] int NOT NULL
, [PromotionKey] int NOT NULL
, [CurrencyKey] int NOT NULL
, [CustomerKey] int NOT NULL
, [SalesOrderNumber] nvarchar(20) NOT NULL
, [SalesOrderLineNumber] int NULL
, [SalesQuantity] int NOT NULL
, [SalesAmount] money NOT NULL
)
WITH
(
DISTRIBUTION = HASH([ProductKey])
)
;
32. TriggerDB Consulting SRL | www.triggerdb.com | Info@triggerdb.com | Buenos Aires Argentina
Azure Analysis Services
33. TriggerDB Consulting SRL | www.triggerdb.com | Info@triggerdb.com | Buenos Aires Argentina
Azure Analysis Services
Azure Analysis Services
34. TriggerDB Consulting SRL | www.triggerdb.com | Info@triggerdb.com | Buenos Aires Argentina
Azure Analysis Services
BI semantic model
Business logic & metrics
Data modeling
Security
Analysis Services
Lifecycle management
In-memory
cache
On-premises
Cloud
Data sources
SQL Database
SQL Data Warehouse
Other data sources
SQL Server
Analytics platform
system
Other data sources
On-premises
Cloud
Client tools
Power BI
Power BI Embedded
Excel
Third party BI tools
Power BI Desktop
35. TriggerDB Consulting SRL | www.triggerdb.com | Info@triggerdb.com | Buenos Aires Argentina
SQL Server Analysis Services
SQL Server
Analysis Services
SQL Server
Other
data sources
On-prem data sources
Visual Studio
Authoring & Dev
SSMS
Power BI
Visualizations & insights
Other 3rd party tools & services
SQL Server
Reporting Services
36. TriggerDB Consulting SRL | www.triggerdb.com | Info@triggerdb.com | Buenos Aires Argentina
Azure Analysis Services
Visual Studio
Authoring & Dev
Power BI
Visualizations & insights
SQL Server
Other
data sources
On-prem data sources
SSMS
Other 3rd party tools & services
Azure
Analysis Services
SQL Database
SQL
Data Warehouse
Cloud data sources
37. TriggerDB Consulting SRL | www.triggerdb.com | Info@triggerdb.com | Buenos Aires Argentina
Big Data Store
38. TriggerDB Consulting SRL | www.triggerdb.com | Info@triggerdb.com | Buenos Aires Argentina
Big Data Stores
Data
Information
Management
Event Hubs
Data Catalog
Data Factory
Big Data Stores
SQL Data
Warehouse
Data Lake Store
Data
Sources
Apps
Sensors
and
devices
IOT Hubs
Cosmos DB
39. TriggerDB Consulting SRL | www.triggerdb.com | Info@triggerdb.com | Buenos Aires Argentina
Big Data Stores
A hyper-scale repository for big data analytics workloads
• A Hadoop Distributed File System for the cloud
• No fixed limits on file size
• No fixed limits on account size
• Unstructured and structured data in their native format
• Massive throughput to increase analytic performance
• High durability, availability, and reliability
• Azure Active Directory access control
LOB
Applications
SocialDevices
Clickstream
Sensors
Video
Web
Relational
HDInsight
ADL Analytics
Machine Learning
Spark
R
ADL StoreSQL Data
Warehouse
Data Lake Store
Cosmos DB
40. TriggerDB Consulting SRL | www.triggerdb.com | Info@triggerdb.com | Buenos Aires Argentina
Data lake is the center of a big data solution
A storage repository, usually Hadoop, that holds a vast amount of raw data in its native
format until it is needed.
• Inexpensively store unlimited data
• Collect all data “just in case”
• Store data with no modeling – “Schema on read”
• Complements EDW
• Frees up expensive EDW resources
• Quick user access to data
• ETL Hadoop tools
• Easily scalable
• With Hadoop, high availability built in
41. TriggerDB Consulting SRL | www.triggerdb.com | Info@triggerdb.com | Buenos Aires Argentina
Azure Data Lake Store Azure Blob Storage
Purpose Optimized for big data analytics General purpose bulk storage
Use Cases Batch, Interactive, Streaming App backend, backup data, media storage for
streaming
Units of Storage Accounts / Folders / Files Accounts / Containers / Blobs
Structure Hierarchical File System Flat namespace
WebHDFS Implements WebHDFS No (WASB)
Security AD SAS keys
Storage Auto Shared/Files chunked Manually manage expansion/Files intact
Size Limits No limits on account size, file size, # files 500TB account, 4.75TB file
Service State Generally Available Generally Available
Billing Pay for data stored and for I/O Pay for data stored and for I/O
Region Availability Two US regions (East, Central) & North Europe
(Other regions coming soon)
All Azure Regions
ADL Store vs Blob Store
42. TriggerDB Consulting SRL | www.triggerdb.com | Info@triggerdb.com | Buenos Aires Argentina
Big Data
43. TriggerDB Consulting SRL | www.triggerdb.com | Info@triggerdb.com | Buenos Aires Argentina
CONTROL EASE OF USE
Azure Data Lake
Analytics
Azure Data Lake Store
Azure Storage
Any Hadoop technology,
any distribution
Workload optimized,
managed clusters
Data Engineering in a
Job-as-a-service model
Azure Marketplace
HDP | CDH | MapR
Azure Data Lake
Analytics
IaaS Clusters Managed Clusters Big Data as-a-service
Azure HDInsight
Frictionless & Optimized
Spark clusters
Azure Databricks
BIGDATA
STORAGE
BIGDATA
ANALYTICS
ReducedAdministration
K N O W I N G T H E V A R I O U S B I G D A T A S O L U T I O N S
44. TriggerDB Consulting SRL | www.triggerdb.com | Info@triggerdb.com | Buenos Aires Argentina
Azure
HDInsight
Hadoop and Spark
as a Service on Azure
Fully-managed Hadoop and Spark
for the cloud
100% Open Source Hortonworks
data platform
Clusters up and running in minutes
Managed, monitored and supported
by Microsoft with the industry’s best SLA
Familiar BI tools for analysis, or open source
notebooks for interactive data science
63% lower TCO than deploy your own
Hadoop on-premises*
*IDC study “The Business Value and TCO Advantage of Apache Hadoop in the Cloud with Microsoft Azure HDInsight”
45. TriggerDB Consulting SRL | www.triggerdb.com | Info@triggerdb.com | Buenos Aires Argentina
Microsoft + Hortonworks
Promoting Open Hadoop
Engineering alignment
Corporate alignment
Field alignment
46. TriggerDB Consulting SRL | www.triggerdb.com | Info@triggerdb.com | Buenos Aires Argentina
Spark for Azure HDInsight
In Memory Processing on Multiple Workloads
Azure
HDInsight
In Memory
Spark
• Single execution model for multiple
tasks
• Processing up to 100x faster
performance than Map Reduce
• Developer friendly (Java, Python, Scala)
• BI tool of choice (Power BI, Tabelau,
Qlik, SAP)
• Notebook experience (Jupyter/iPython,
Zeppelin)
47. TriggerDB Consulting SRL | www.triggerdb.com | Info@triggerdb.com | Buenos Aires Argentina
HDInsight Supports Hive
SQL-like queries on Hadoop data in HDInsight
HDInsight provides easy-to-use graphical query interface for Hive
HiveQL is a SQL-like language (subset of SQL)
Hive structures include well-understood database concepts such as tables, rows, columns, partitions
Compiled into MapReduce jobs that are executed on Hadoop
Dramatic performance gains with Stinger/Tez
Stinger is a Microsoft, Hortonworks and OSS driven initiative to bring interactive queries with Hive
Brings query execution engine technology from Microsoft SQL Server to Hive
Performance gains up to 100x
Hadoop 2.0
1400s
44.3s
35.1s
Sample Query
Hive
1.0
HDP
1.3 /
Hive
11
HDP
2.0
32x
Speedup 40X
Speedup
HDP
2.1
15s
100x
Speedup
48. TriggerDB Consulting SRL | www.triggerdb.com | Info@triggerdb.com | Buenos Aires Argentina
HDInsight Supports HBase
• NoSQL database on data in HDInsight
Data Node Data Node Data Node Data Node
Task Tracker Task Tracker Task Tracker Task Tracker
Name Node
Job Tracker
HMaster
Coordination
Region Server Region Server Region Server Region Server
49. TriggerDB Consulting SRL | www.triggerdb.com | Info@triggerdb.com | Buenos Aires Argentina
HDInsight Supports Storm
Stream analytics for Near-Real Time Processing
Consumes millions of real-time events from a scalable event broker (ie. Apache Kafka, Azure Event Hub)
Performs time-sensitive computation
Output to persistent stores, dashboards or devices
Customizable with Java + .NET
Deeply integrated to Visual Studio
Event Queuing
System
Collection Presentation and
action
Event producers Transformation Long-term storage
Event Hubs
Storage
adapters
Stream
processing
Cloud gateways
(web APIs)
Field
gateways
Applications
Search and query
Data analytics (Excel)
Web/thick client
dashboards
Live Dashboards
Apache Storm
on
HDInsight
Devices to take action
Kafka /
RabbitMQ /
ActiveMQ
Web and Social
Devices
Sensors
Azure Stream
Analytics
HDFS
Azure DBs
Azure storage
HBase
50. TriggerDB Consulting SRL | www.triggerdb.com | Info@triggerdb.com | Buenos Aires Argentina
Machine Learning Server for HDInsight
• Familiarity of R o Python
• Scalability of Hadoop and Spark
• Up to 7x faster using Spark engine
• Train and run ML models on datasets of any size
• Cloud managed solution (easy setup, elastic, SLA)
Only managed, cloud solution for doing R
or Python
51. TriggerDB Consulting SRL | www.triggerdb.com | Info@triggerdb.com | Buenos Aires Argentina
Real-time stream processing in the cloud
• Perform real-time analytics for your Internet of Things solutions
• Stream millions of events per second
• Get mission-critical reliability and performance with predictable
results
• Create real-time dashboards and alerts over data from
devices and applications
• Correlate across multiple streams of data
• Use familiar SQL-based language for rapid development
Event Hubs
Blob Storage
Stream
Analytics
SQL Database
Event Hubs
Power BI
Blob Storage
Table Storage
10
10
10
10
Machine Learning
and Analytics
HDInsight
(Hadoop and
Spark)
Stream Analytics
Data Lake
Analytics
Machine
Learning
Azure Analysis
Services
52. TriggerDB Consulting SRL | www.triggerdb.com | Info@triggerdb.com | Buenos Aires Argentina
Maximum
CONTROL
Greatest
SIMPLICITY.
Lowest TCO
Azure Stream Analytics
Fully
managed -
3 9s of
availability
SQL like
query
language
Machine
Learning
integrations
Integrations for
real-time
dashboarding
Custom
code
extensions
Spark Streaming & Storm (HD Insight)
SLA with 3
9s of
availability
Virtual Machines
Full stack
control
Integrationswith
Kafka,ipython
notebookR-
Server
Managed
Spark
Streaming &
Storm
Streaming Options with Microsoft
Cluster ServiceJob Service
53. TriggerDB Consulting SRL | www.triggerdb.com | Info@triggerdb.com | Buenos Aires Argentina
Real-time analytics
54. TriggerDB Consulting SRL | www.triggerdb.com | Info@triggerdb.com | Buenos Aires Argentina
PowerBI
55. TriggerDB Consulting SRL | www.triggerdb.com | Info@triggerdb.com | Buenos Aires Argentina
Any data, any way, anywhere
56. TriggerDB Consulting SRL | www.triggerdb.com | Info@triggerdb.com | Buenos Aires Argentina
Power BI product portfolio
57. TriggerDB Consulting SRL | www.triggerdb.com | Info@triggerdb.com | Buenos Aires Argentina
Get Data
Easily connect, clean, and mashup data
Free companion authoring tool for the Power BI service
Analyze
Build powerful models and flexible measures
Visualize
Create stunning interactive reports
Publish
Share insights with others
Collaborate
Empower your organization with self-service analytics
58. TriggerDB Consulting SRL | www.triggerdb.com | Info@triggerdb.com | Buenos Aires Argentina
Power BI service
Cloud-based SaaS solutions
Get started quickly
Secure, live connection to your data sources,
on-premises and in the cloud
Auto insights and intuitive data exploration using
natural language query
Deliver insights through other services such as
SharePoint, PowerApps and Teams
Pre-built dashboards and reports for popular SaaS
solutions
Live, real-time dashboard updates
59. TriggerDB Consulting SRL | www.triggerdb.com | Info@triggerdb.com | Buenos Aires Argentina
Power BI Premium
Dedicated resources in the cloud
Flexibility to license by capacity
Greater scale and performance
Extending on-premises capabilities
Premium capacity – P3
Premium capacity – P2
Premium capacity – P1
My workspace
User 2
My workspace
User 3
App workspace
Marketing
App workspace
Sales
My workspace
User 1
APIs
Custom app
Power BI service – Contoso organization
Power BI Premium
60. TriggerDB Consulting SRL | www.triggerdb.com | Info@triggerdb.com | Buenos Aires Argentina
On-premise reporting solution
Power BI reports and SSRS report on-premises
Connect to data
Over 70+. Data can be imported, queried directly or live connection to SSAS
Power BI reports
Fully interactive reports on-premises to visualize your data and gain insights
SSRS reports
Precisely formatted operational reports
Consume in multiple ways
Mobile Apps, Web Portal, Embedded in your organizational apps
Power BI Report Server
61. TriggerDB Consulting SRL | www.triggerdb.com | Info@triggerdb.com | Buenos Aires Argentina
Spend time focusing on your product instead of
building visual analytic features from scratch
Connect to countless data sources so that you can
expose insights to your customers
Take advantage of our existing Custom Visual library
or dream up the right visual for your customers
Embed consistent, easy-to-navigate visualization
experiences across any device
Leverage other familiar services like Visual Studio,
Azure Web Aps and other Azure services
Power BI Embedded
Embed stunning, fully interactive reports and visuals into your applications
62. TriggerDB Consulting SRL | www.triggerdb.com | Info@triggerdb.com | Buenos Aires Argentina
Muchas gracias
https://www.linkedin.com/company/triggerdbsrl
info@triggerdb.com
https://www.triggerdb.com
Editor's Notes
Cortana Intelligence delivers an end-to-end platform with an integrated and comprehensive set of tools and services to help you build intelligent applications that let you easily take advantage of Advanced Analytics and intelligence capabilities.
First, Cortana Intelligence provides services to bring data in, so that you can analyze it. It provides information management capabilities like Azure Data Factory so that you can pull data from any source (relational DB like SQL or non-relational ones like your Hadoop cluster) in an automated and scheduled way, while performing the necessary data transforms (like setting certain data columns as dates vs. currency etc). Think ETL (Extract, Transform, Load) in the cloud. Event Hubs does the same for IoT type ingestion of data that streams in from lots of end points.
The data brought in then can be persisted in flexible big data storage services like Data Lake Store and Azure SQL Data Warehouse.
You can then use a wide range of analytics services from Machine Learning to Azure Data Lake Analytics to Azure HDInsight to Azure Stream Analytics to analyze the data stored in the big data storage. This means you can create analytics services and models specific to your business need (say real time demand forecasting).
The resultant analytics services and models created by taking these steps can then be surfaced as interactive dashboards and visualizations via Power BI.
These same analytics services and models created can also be integrated into various different UI (web apps or mobile apps or rich client apps), or with Cortana, so end users can naturally interact with them via speech etc., and so that end users can get proactively be notified by Cortana if the analytics model finds a new anomaly (unusual growth in certain product purchases- in the case of real time demand forecasting example given above) or whatever deserves the attention of the business users. Similar integration can occur with Cognitive Services or Bot Framework based applications.
At a high level though, Cortana Intelligence capabilities are in three main areas: data, analytics and intelligence.
<Transition>: We’re going to dive into each one.
Introducing Azure SQL Database Managed Instance
SQL Database Managed Instance is an expansion of the existing SQL Database service designed to enable database migration to a fully-managed database service, without re-designing the application.
It’s important to note that Managed Instance isn’t a new service – it is a third deployment option within Azure SQL Database, sitting alongside single databases and elastic pools. As part of Azure SQL Database, Microsoft’s fully managed cloud database service, it inherits all its built-in features.
A SQL Database Managed Instance offers a simplified instance-scoped programming model that is like an on-premises SQL Server instance. The databases in a SQL Database Managed Instance share the resources allocated to the Managed Instance, and the Managed Instance also represents the management grouping for these databases. The emphasis of this offering is on high compatibility with the programming model of an on-premises SQL Server and out-of-box support for a large majority of SQL Server features and accompanying tools/services.
Each standalone database is assigned a certain amount of resources via performance tiers: Basic, Standard, and Premium. The emphasis of this offering focuses on a simplified database-scoped programming model and applications with a predictable pattern and relatively stable workload.
An elastic database pool is a shared resource model that enables higher resource utilization efficiency, and all the databases within an elastic pool share predefined resources within the same pool. The emphasis of this offering is on a simplified database-scoped programming model for multi-tenant SaaS apps. The workload pattern is well-defined and is highly cost-effective in multi-tenant scenarios. For ISVs with SaaS apps, the savings can be significant, in the hundreds of thousands of dollars or more.
External resources are represented by linked services that provide connection info for ADF to orchestrate the related activities.
Billing for SSISDB will be under your Azure SQL DB/MI subscription, while billing for SSIS Everest compute will be under your ADF subscription.
At the heart of big data analytics sits Azure SQL Data Warehouse. It’s our SQL-based, enterprise-class cloud data warehouse.
With the new ’optimized for compute’ tier, customers can tailor their fully managed DW to meet workload performance needs.
It scales further than ever before!
The Azure SQL Data Warehouse team has optimized the experience for customers who require the most intensive compute power for their mission critical analytics
Fully managed
Petabyte scale
Petabyte scale
De-coupled storage from compute & control
Completely elastic
Pay for the data you store and the compute you provision
Scale-out distributed query engine
Key points: Summarize key benefits for Azure Analysis Services
Azure Analysis Services helps you transform complex data from different data sources into a BI semantic model, so users in your organization can easily gain insights by connecting to the data models using tools like Excel, Power BI, and others to create reports and perform ad hoc data analysis
Talk Track
Transform Complex Data into rich BI semantic models: Azure Analysis Services Analysis Services helps you transform complex data into a single business user friendly data model making it easy for business users to understand and analyze data across different data sources.
Gain instant insights with in-memory cache using your preferred visualization tools : Not only can business users get insights from data easily using their preferred data visualization tool, whether it is Power BI, Excel or other major data visualization tools, but with the in-memory cache capabilities of Azure Analysis Services, users can gain insights over billions of rows of data at the speed of thought
Proven Technology: Azure Analysis Services is based on the proven analytics engine in SQL Server 2016 Analysis Services, that has helped organizations turn complex data into a trusted, single source of truth for years. This means that BI professionals who are familiar with SQL Server Analysis Services, tabular models can get started quickly and do not need to learn new tools or skills.
Analytics engine as-a-service (provision fast, scale faster): The same proven enterprise grade BI platform is now available as a fully managed service in Azure. With the power of the trusted Microsoft Cloud, you do not need to manage infrastructure on-premises and can benefit from the scalability of the cloud. Additionally you can use Azure Resource Manager to create and deploy an Azure Analysis Services instance within seconds, and use backup restore to quickly move your existing models to Azure Analysis Services and take advantage of the scale, flexibility and management benefits of the cloud. Scale up, scale down, or pause the service and pay only for what you use.
Azure Analysis Services is built for Hybrid BI - Organizations store data in the cloud and on-premises. Azure Analysis Services is built for hybrid data. Data can be access in the cloud, on-premises or a combination of both, enabling a hybrid solution. So - you do not have to move on-premises data to the cloud.
To summarize, Azure Analysis Services is simple to use – it is easy to get started, you can use your existing skills to create BI semantic models, and your favorite data visualizations tools to analyze your data.
Key points:
Azure Analysis Services works just like SQL Server Analysis services (for tabular models), supports data sources on-premises and in the cloud, and consume using your preferred data visualization tools.
Talk track:
Azure Analysis Services is based on SQL Server 2016 Analysis Services technology. In preview, we are focusing on tabular models only. Data can be accessed on-premises and in the cloud and business users and BI professionals can consume the data on-mobile devices, on the web and in custom apps.
This means that BI professionals who are familiar with SQL Server Analysis Services, tabular models can get started quickly and do not need to learn new tools or skills.
Develop in the familiar Visual Studio environment and take advantage of Visual Studio Application Lifecycle Management
Key points
SQL Server Analysis Services is typically used by organization today for their on premises BI solution. It allows them to connect to their on-prem data sources a
turn complex data into a trusted, single source of truth. Then visualize and gain insights from their data with SSRS or Power BI or other 3rd party visualization & reporting tools.
There are many new capabilities in SQL Server 2016 Analysis Services including:
Enhance performance with parallel partition processing
Over 50 new DAX functions
Ability to view semantic models in a preferred language
Role level security
Support for more data sources like Oracle and Teradata
Etc.,
Key points
Azure Analysis Services provides the same enterprise-grade data modeling in the cloud. Fully managed PaaS. It’s built on the proven analytics engine in Microsoft SQL Server 2016 Analysis Services.
So all that same functionality and capability now applies to the cloud service
Additionally it enables new scenarios like:
Freeing up managing infrastructure & scale your data models with your cloud data sources. Whether new or existing models.
Take advantage of Azure to easily scale up to handle increased demand from your BI user
Leverage the same tools like SQL Server Management Studio (SSMS) to mange your BI models in the cloud.
…
T: Azure provides a variety of storage options, including Blobs. Here we’ll dive into the two main storage options with Cortana Intelligence, starting with Data Lake Store.
Your data are valuable assets to your organization and have both present and future value. Because of this, all data should be stored for future analysis. Today this is often not done because of the restrictions of traditional analytics infrastructure, like the pre-definition of schemas, the cost of storing large datasets, and the propagation of different data silos.
To address this challenge, the data lake concept was introduced as an enterprise-wide repository to store every type of data collected in a single place. For the purpose of operational and exploratory analytics, data of all types can be stored in a data lake prior to defining requirements or schema.
Microsoft Azure Data Lake Store is a Hadoop file system that’s compatible with Hadoop Distributed File System (HDFS) and works with the Hadoop ecosystem. Data Lake Store is integrated with Azure Data Lake Analytics and Azure HDInsight and will be integrated with Microsoft offerings like Revolution-R Enterprise; industry-standard distributions like Hortonworks, Cloudera, and MapR; and individual Hadoop projects like Spark, Storm, Flume, Sqoop, and Kafka.
Data Lake Store is a very open, massive scale data store designed for extremely high throughput and low latency for analytics workloads.
At the same time, its built-in security ensures you can include even your most critical workloads on it.
T: You may have noticed we are emphasizing these services are all built to scale, and SQL Data Warehouse is no different.
Also called bit bucket, staging area, landing zone or enterprise data hub (Cloudera)
http://www.jamesserra.com/archive/2014/05/hadoop-and-data-warehouses/
http://www.jamesserra.com/archive/2014/12/the-modern-data-warehouse/
http://adtmag.com/articles/2014/07/28/gartner-warns-on-data-lakes.aspx
http://intellyx.com/2015/01/30/make-sure-your-data-lake-is-both-just-in-case-and-just-in-time/
http://www.blue-granite.com/blog/bid/402596/Top-Five-Differences-between-Data-Lakes-and-Data-Warehouses
http://www.martinsights.com/?p=1088
http://data-informed.com/hadoop-vs-data-warehouse-comparing-apples-oranges/
http://www.martinsights.com/?p=1082
http://www.martinsights.com/?p=1094
http://www.martinsights.com/?p=1102
https://docs.microsoft.com/en-us/azure/data-lake-store/data-lake-store-comparison-with-blob-storage
Mention no new improvements to ADLS in 2018, only adding more regions, so recommend blob storage until ADLS is in your region.
Mention: GDPR
Joint engineering teams – joint people on each other’s teams. They report to Hortonworks and Hortonworks report to us. We share resources and have a joint engineering team. This is not just a re-licensing of a distro. We are one engineering team.
Milestone cadence – we do a 12-month calendar year roadmap with each other. That roadmap is how each company lock to. They revisit it every 3 months. When Cloudera released Impala, Microsoft and Hortonworks changed their resources to start working on Stinger project. In CY13, this became a joint commitment to build Tez (vectorized query executor) and Stinger
Joint planning and execution –
Joint signoffs (Linux, Windows Azure) – Microsoft and Hortonworks signedoff on Linux, on Windows, and on Azure.
Aligned offerings – HDP on Windows
HDI Appliance, HDI Service
Joint roadmap
CY planning/ roadmap development
Quarterly adjustments
Hortonworks is making Hadoop an enterprise viable platform, by the end of 2015 more than half the worlds data will be processed by Hadoop
<look at partner page on Hortonworks site>
Engineering alignment
Microsoft worked with Hortonworks to develop Hortonworks Data Platform for Windows
HDInsight on Windows Azure and HDInsight on PDW is based on HDP
Microsoft and Hortonworks are working on various Hadoop sub-projects together (Stinger, Tez) contributed over 6k engineering hours, 25k+ lines of code
Corporate alignment
Microsoft is one of Hortonworks strategic partners (Q for Audrey)
Joint Marketing alignment (webinars, events strategy, Press, etc.)
Joint Support alignment (Microsoft provides Lvl1 support and pays Hortownworks for Lvl2 and Lvl3)
Field Alignment
Hortonworks field reps will get quota compensation or relief if HDP for Windows OR if PDW, Azure is sold
Microsoft field reps work with Hortonworks reps to get credibility from Hadoop story
Slide 1 – it’s supposed to represent a zoom in of the right most box with certain new details.
Slide 2 and 3 are the same slide. I received feedback from my original slide 2 below and I made slide 3. They still didn’t like it. Here is the feedback:
Original feedback:
Interactive experience conceptual picture
Make the top and bottom be proportional
Drop the boxes and headings (OSS notebooks, 3rd party BI tools)
Power BI should stand out but the current image is not adding new value, can we just use Power BI logo and also some additional image to have it stand out
Secondary feedback about slide 3
In the image that shows Power BI, we do still want to make Power BI pop above the others and have it featured, but Ranga was a little unsure about having the images of the devices included. Let’s get a version that shows it both ways, (with the devices, and making the Power BI section stand out in another way, ie size, but not including the devices,) and then let him decide which one he prefers.
Sldie 4 General cleanup
Azure Stream Analytics lets you rapidly develop and deploy low-cost solutions to gain real-time insights from streaming data from devices, sensors, infrastructure, and applications. Use it for Internet of Things (IoT) scenarios, such as real-time remote management and monitoring or gaining insights from devices like mobile phones and connected cars.
With every device, service, and process becoming a data point, the problem can often be analyzing and acting on data fast enough. Stream Analytics is easy to deploy, and simple to develop for, a low cost end to end event stream processing solution, that scales on demand.
Use fewer lines of code with built in analysis processes. Combine Stream Analytics with Event Hubs to analyze millions of points of data in a reliable environment.
T: Next, we’ll take a look at a demo that illustrates the value of the intelligence capabilities that Cortana Intelligence offers.
Slide Objective: Highlight the unique features that make Power BI a market-leading Modern BI platform.
Power BI has in its DNA the goal of enabling everybody to experience their data any way, anywhere—in seconds and at global scale.
Power BI offers a set of capabilities that are uniquely enabled by its global and cloud nature:
The ability to harness data from Excel spreadsheets, on-premises data sources through the data gateway, big data, streaming data, and cloud services. It doesn’t matter what type of data you want or where it lives, Power BI allows you to connect to hundreds of data sources.
Out-of-the box SaaS content packs that deliver a curated experience with pre-built dashboards to get you up and running quickly. We have hundreds of ISVs building content packs to cater to the needs of millions of Power BI users.
Unmatched, unique ways for users to experience their data with speed and agility:
Live dashboards that maintain a real-time pulse on the business and provide critical insights.
Natural language query that enables users to simply and intuitively ask questions of their data, including through Cortana.
Custom visuals that bring data to life and surface intelligence hidden in the sea of data, with our community leveraging the Power BI visualization stack to create new ways to visualize data in a way that makes more sense. (Now available in the Office store.)
Integration of Power BI with the Microsoft stack. Power BI is part of larger ecosystem that integrates with services like Microsoft Teams, Office 365, and Dynamics 365. These services are aware of Power BI, are wired to Power BI, and enable you to use Power BI in the context of your work.
Anywhere access to insights. Whether in the office or on-the-go, Power BI provides anywhere access to insights with dashboards accessible via the desktop, on the web, or across mobile devices. Inside Excel, embedded—we have hundreds of ISVs embedding Power BI in their offerings.
Power BI Desktop - A free tool to download to create and author modern and interactive reports
Power BI - A SaaS service to share, collaborate reports & Dashboard and gain insight from your data
Power BI Premium - Add-on dedicated capacity to the Power BI SaaS service
Power BI Report Server – On-premises report server
Power BI Embedded - Power BI Embedded analytics APIs for ISVs and Developers
Power BI Desktop is a visual data exploration and interactive reporting tool, providing a free-form canvas for drag and drop exploration of your data, an extensive library of interactive visualizations, and an authoring experience for ease of report creation for the Power BI service.
To get started, you must download the free 32/64bit desktop application optimized for use with the Power BI Service.
You can expect an update monthly with a variety of new features.
Once you have Power BI Desktop downloaded, you can take your data through the full analytical cycle by:
Connecting and prepping your data.
Modeling and analyzing your data with the combined Excel engine of Power Query, Power Pivot and Power View.
Creating custom interactive visuals to highlight the insights you discover.
Authoring interactive Reports.
Publishing interactive visual reports to users of Power BI.
Starting with a pre-built solution to make using Power BI a breeze.
Power BI Desktop also connects with Excel Microsoft’s premier business analyst tool which includes rich business intelligence features (Power Query, Power Pivot, Power View, Power Map) fully integrated with the powerful ad hoc analysis capabilities and familiar features of Excel – like Pivot Tables and Excel Charting. With Excel, analysts can publish Excel Workbooks to Power BI, sharing data, analysis and reports with users of Power BI.
T: However, for customers who don’t have access to Excel 2013, the new Power BI Desktop can be used to import and model data, then author and publish Power BI Reports to the Power BI service. While lacking the rich analytical features of Excel, it does provide a simple solution expressly designed for Power BI content creation. <Click>
Key message:
Industry-leading SaaS service with large Power BI ecosystem (ISVs, SIs and partners creating unparalleled momentum)
Customers benefit from integration with other Microsoft products
There are several capabilities that differentiate Power BI from other offerings in the market.
Talking points:
Power BI is a cloud-based analytics service that provides faster time to insight. It is used for visualizing, exploring and extracting insights from data. It brings together data from diverse sources to deliver rich, comprehensive views of business operations.
What that means is, with Power BI, you can see all of your data through a single pane of glass, and create an analytics environment in minutes to monitor data and share reports. Live dashboards and reports show visualizations and KPIs from data that reside both on-premises and in the cloud, providing a consolidated view across your business regardless of where your data lives.
There are several capabilities that are unique to Power BI – these include:
Pre-built dashboards and reports for popular SaaS solutions such as Marketo, Salesforce, GitHub, ZenDesk, Dynamics CRM, and many others.
Dashboards and visualizations that support real-time updates.
Secure, live connection to on-premises data sources to drive insight across the entire range of organizational information (Analysis Services, Azure SQL Database, SQL Database Auditing, Azure SQL Data Warehouse).
Automated scheduled refresh to keep Power BI data in sync with on premises data sources.
Native mobile applications for keeping users connected on the go.
Easy integration with other Microsoft offerings, from Excel to Azure services.
Intuitive data exploration, which enables users to ask questions of their data in plain English.
The Power BI Service, sometimes referred to as powerbi.com, is what allows you to:
Create beautiful visualizations to tell compelling data stories.
Build rich, live dashboards that turn BI into business insights.
Create reports & datasets from which you can create visualizations and dashboards.
Enjoy the benefits of up-to-date data with real-time, automatic and scheduled refreshes.
Share dashboards easily with other people in your organization.
Ask questions of data in plain English with Natural Language Query.
Stay connected to data all the time with mobile applications.
To summarize, Power BI is a SaaS offering that enables anyone to easily connect to their data, create live operational dashboards and explore data via interactive visualizations.
Greater scale and performance
Dedicated hardware for exclusive use by the tenant fully managed by Power BI
Easily scale up as your usage grows
Large user quota up to 100 TB of storage and large dataset support up to 10 GB in size
More frequent data refresh up to 48 times a day and faster refresh with incremental data refresh support
Flexibility to license by capacity
Extend read access to a broader set of people without requiring per user license
Tailor performance to the unique needs of your organization, department, or team
Scale as your requirements change.
Extending on-premises capabilities
Flexibility to keep reports on-premises (Assign v-cores of Premium to Power BI Report Server)
Concurrently deploy an equivalent number of cores on-premises
Compatibility with SSRS reports
Reference link
https://docs.microsoft.com/en-us/power-bi/service-premium
An on-premises solution for self-service BI as well as operational reporting
SSRS reports
Precisely formatted operational reports
Optimized for document generation and printing
Page headers, footers, and page breaks
Modern chart and gauge styles
Visual Studio 2015 and 2017 support
Supported data sources
https://docs.microsoft.com/en-us/power-bi/report-server/data-sources
So first and foremost you get a true Platform-as-a-Service with complete cloud development and deployment. We are designed for ISVs with complete cloud development and deployment.
Next, I mentioned the countless data sources. You want to expose insights to your customers, and you can’t do that if you don’t have the data plugged in. We’ll get you plugged in.
What about visuals? Oh, we have plenty of those for you to use, but what if you want something that we don’t offer? Well, our community has been making custom visuals for a while now, and many of them are available for download and if you *still* can’t find what you need, dream up your own and we’ll support it.
Data is no good if it’s out of date, so enjoy real-time data.
And you want to deploy across all the device types too, I’d imagine. You can do that too. Consistent rendering in HTML5 across all devices
And you don’t have to leave your comfort zone because Power BI Embedded seamlessly integrates with tools you’re using every day like Visual Studio.
Now that you know why you should choose Power BI Embedded, let’s talk about how it all works -