Its
All
About
Data
Who Am I
• 7+ years of experience in Microsoft BI Stack
• Currently working in The Most Group as MS BI Solution Architect
• Microsoft BI Head in EnterprizeERP Consultancy Service
• Consulting partner of Right Click Info , Mentalist Network
• NMUG Community Leader , Speaker
• Microsoft Community Representative for Navi Mumbai region.
• SQL PASS Chapter Lead for Mumbai & Navi Mumbai
• MCTS: Microsoft® Certified Technology Specialist -SQL Server 2008,Business Intelligence
Development and Maintenance
• MCSA: Microsoft® Certified Solutions Associate -SQL Server 2012 – Querying Microsoft
SQL Server 2012,Administering Microsoft SQL Server 2012 Databases, Implementing a
Data Warehouse with Microsoft SQL Server 2012
• CSM: Certified Scrum Master from Scrum Alliance, US
• BE-Computer Science & Engineering , RAIT , Mumbai
• MBA-Information System, Arianth College , Pune
• Blogging @ http://bimentalist.com , http://sqlmentalist.com
• BI Articles :: 200+ , SQL Article – 200 +, SQL Script – 200+
Why
Data Is
Important !!
Gartner Magic Quadrant DWH & BI 2013
Gartner delivers technology research to global technology business leaders to
make informed decisions on key initiatives.
6
SQL Server Release timelines
1996
SQL Server 6.5
1990 1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012
2005
SQL Server 2005
Unicode Support
Native XML
SQLCLR
Service Broker
Integration Services
1993
SQL Server 4.21
(NT)
1995
SQL Server 6.0
1989
SQL Server 1.0
(OS/2)
2000
SQL Server 2000
Reporting Services
2010
SQL Server 2008 R2
Data-tier Apps
StreamInsight
PowerPivot
Master Data Services
2008
SQL Server 2008
Sparse Columns
Spatial Types
FILESTREAM
1998
SQL Server 7.0
Dynamic Locking
Auto-Tuning
Full-text search
Replication
Analysis Services
1991
SQL Server 1.1
(OS/2)
2012
SQL Server 2012
AlwaysOn
Columnstore
FileTable
Semantic Search
Power View
& Now Introducing ..
Business Intelligence
Agile BI Out of The Box, Forrester Research, Inc., April 22, 2010
32% of Excel users are
comfortable with using it
for “Advanced Analysis”
e.g., PivotTables
Only 28% of potential
BI users have any
meaningful access
31% of all BI initiatives
partially meet or don’t
meet the business
goals originally set
Knowledge workers spend
too much time looking for,
not enough time analyzing
the information.
Lines between producers
and consumers of
information have largely
disappeared.
Requirements change way
too fast for IT and
traditional technology to
keep up.
– Excel Segmentation Research – BI Survey 9, BARC
– Business Intelligence Purchase
Drivers and Adoption Rates, Gartner
Evolution of BI
IT has been busy keeping up with the growing demand for BI…
Self-Service BITraditional BI
IT Pro
End Users
Existing
Data
LOB
Applications FilesData Marts
ETL/Data
Quality
Analysis Reports Dashboards &
Scorecards
Provision
Analysis
Cubes Data Warehouse
Data
Staging
Spreadsheets
Specialized Tools
Harvesting existing data, cleaning it and staging it …To provision reports, dashboards & scorecards to meet End User demand.But business is outpacing IT and End Users are demanding Agility …Rapidly breaking down the barriers of Control with Self-Service BI!Is it possible to balance Control with Agility?
Analysis
Services
Reporting
Services
Integration
Services
Master Data
Services
SharePoint
Scorecards
Excel
Workbooks
PowerPivot
Applications
SharePoint
Search &
Dashboards
Microsoft Business Intelligence
Most widely deployed EIM
& BI Platform
Most Broadly adopted
Productivity &
Collaboration Tools
Odata
Feeds
LOB
Apps
SQL Server 2012 Highlights
High
Availability
SQL Server
AlwaysOn
Security &
Manageability
User-Defined
Server Roles
Web &
Breadth
PHP Driver
Scalability &
Performance
Distributed
Replay
Application
Centric Failover
Contained
Database
Authentication
Local DB
Runtime
Fast FILESTEAM UTF-16
Readable
Secondaries
Fast Full-Text
Search
Reliable,
Integrated Failover
Detection
Multiple
Secondaries
Audit
Enhancements
Paging for result
sets
Column Store
Index, “Apollo”
HA for
StreamInsight
Management
Pack for High
Availability
Backup
Secondaries
JDBC 4.0 driver
Support for ARM
processors
Active Directory
w/ SharePoint for
SSRS
Default Schema
for Windows
Group
Windows Server
Core Support
Beyond
Relational
FileTable
SQL Server Data
Tools
Online
Operations
Win32 Access
to Database files
Semantic Search
Platform
Full Globe Spatial
Support
DAC
Enhancements
ODBC for Linux
Fast Spatial
Performance
Hybrid
Applications with
SQL Azure
Up to 15K
Partitions/Table
EIM
SSIS Server
Business
Intelligence
Unified Semantic
Model
Power View Data Quality
Alerting
Enhanced MDSIn-memory BI for
corporate
Sysprep for AS
PowerPivot
Enhancements
Reporting as
SharePoint Shared
Service
DBC & OLTP
Appliances/RAs
MDS add-in for
Excel
AlwaysOn Reliable Secondaries
FileTable
ColumnStore Index
15k Partitions
SQL Server Data Tools
Power View
BI Semantic Model
Data Quality Services
Full-Text Search Performance
Reporting Alerts
ODBC Driver for Linux
Statistical Semantic SearchWindows Server Core Support
Multiple Secondaries
Availability Groups
Default Scheme for Windows Groups
T-SQL Enhancements
Full Globe Spatial
SSMS to Windows Azure Platform
PowerPivot Enhancements
Master Data Management Excel Add-in
PowerShell 2.0 Support
PHP & Java Connectivity
SQL Audit for All Editions
CDC Support for SSIS
New SSIS Design Surface
Online Operation Enhancements
Multi-site Clustering
Unstructured Data Performance
Resource Governor Enhancements
Database Recovery Advisor
HA for StreamInsight
Flexible Failover Policy
Extended Events Enhancements
Contained Database Authentication
SharePoint Active Directory Support
SQL Server Express LocalDB
User-defined Audit
Audit Filtering
Audit Resilience
FTS Support for Czech and Greek
AlwaysOn Connection Director
Ad Hoc Reporting
SSIS Troubleshooting
SSIS Package Management
T-SQL Debugger Enhancements
Spatial 2D Support
Unstructured Data Performance
11
Organizational
Compliance
Optimized
Productivity
11
Extend Any
Data, Anywhere
Fast Time
to Solution
Scalable
Analytics & DW
8
Credible,
Consistent Data
Peace of
Mind
Managed Self-
Service BI
4
Rapid Data
Exploration
3
Blazing-Fast
Performance
2
Required 9s
& Protection
1
Scale on
Demand
5 76
12109
II-BREAKTHROUGHINSIGHT
III-CLOUDONYOURTERMS
I-MISSION CRITICAL CONFIDENCE
12
12 Important Feature's of SQL Server 2012
Secure
Complete & Consistent Data Platform
In-Memory Built-In
Seamless Transition of the BI Semantic Model
Across BI Spectrum
TeamBIPersonalBI CorporateBI
PowerPivot for Excel PowerPivot for SharePoint Analysis Services
My Context
BI solution created by user.
Context is only for user &
exists as document.
Our Context
BI Solution created by
power user. Context is for a
small team & it’s managed
on a server.
The Org’s Context
BI Solution created by IT,
Established corporate context
& is reusable, scalable and
backed up.
AlignedEmpowered
Who have accepted SQL Server 2012
Why do customers chose SQL Server?
18
Increased productivity and decistion Makting
thru Self Service BI + IT Management
„Analysis take now seconds instead of days“
High Availability, Scalability & Security
Up to 100%, SQL is leading in lowest sec. patches rate
Reduce IT costs through IT Management
69% reduced DB costs, 50% lower Admin. efforts
Lower Purchase Costs
All BI features in one BI Edition, SQL & Office are already in use
Enabling Integration & Sharing
SQL Server 2012 Master Data Services
Versioning
Validation
Authoring business rules to
ensure data correctness
Modeling
Entities, Attributes, Hierarchies
MDS
Role-based Security and
Transaction Annotation
Master Data
Stewardship
External
(CRM, ..)Excel DWH
Loading batched
data through
Staging Tables
Consuming data
through Views
Registering to
changes through
APIs
Excel Add-In Web UI Data Matching
Workflow /
Notifications
SQL Server 2012 Data Quality Services
Knowledge-Driven
Based on a Data Quality Knowledge Base (DQKB) that is reusable for a variety of data
quality improvements
Knowledge Discovery
Acquire additional knowledge through data samples and user feedback
Open and Extendible
Support use of user-generated knowledge and IP by 3rd party reference data providers
Easy to Use
Compelling user experience
designed for increased
productivity
Integrated into SSIS
How Should I Study Now !
Build the necessary skills to grow in your IT role
and career with Microsoft Cloud technologies.
• Learn for free and at your own pace.
• Make a name for yourself within the
community of IT Pros worldwide, by climbing
the ladder all the way to Platinum Level.
• Benefit from promotions and initiatives
published at the MVA
Virtual Labs enable you to quickly
evaluate and test Microsoft's newest
products and technologies through a
series of guided, hands-on labs that you
can complete in 90 minutes or less.
There is no complex setup or installation
required, and you can use Virtual Labs
online immediately, free.
www.microsoftvirtualacademy.com
http://www.microsoft.com/en-
in/sqlserver/learning-center/virtual-labs.aspx
Showcase your expert skills at developing and deploying rich business intelligence solutions.
Demonstrate your expertise at building enterprise-scale data solutions.
Lets Learn SQL Server & Big Data
The World of Data HAS CHANGED
Type of data generated by sector
The Transition to Big Data
BIG DATA on Axis
Variety - Data is present in
various form
Volume – Data size is Huge
Velocity – Increasing Data
day by Day
Interpreting Big Data
Requires New Approaches: Hadoop
27
velocity
variety complexity
volume
Data
INTERPRETINGPREPARING
Big Data: Big and Getting Bigger Fast!
Machine-generated data is a key driver in data growth
1 ZB = 1,000,000,000,000 GBs!
Projected to increase 15x by 2020
*2012 IDC Digital Universe Study
 950 million users
 2.5 billion content items shared per day (status updates +
wall posts + photos + videos + comments)
 2.7 billion Likes per day
 300 million photos uploaded per day
 100+ petabytes of disk space in one of FB’s largest Hadoop
(HDFS) clusters
 105 terabytes of data scanned via Hive, Facebook’s Hadoop
query language, every 30 minutes
 70,000 queries executed on these databases per day
 500+terabytes of new data ingested into the databases
every day
What Makes Up Big Data?
Megabytes
Gigabytes
Terabytes
Petabytes
Purchase detail
Purchase record
Payment record
ERP
CRM
WEB
BIG DATA
Offer details
Support Contacts
Customer Touches
Segmentation
Web logs
Offer history
A/B testing
Dynamic Pricing
Affiliate Networks
Search Marketing
Behavioral Targeting
Dynamic Funnels
User Generated Content
Mobile Web
SMS/MMSSentiment
External Demographics
HD Video, Audio, Images
Speech to Text
Product/Service Logs
Social Interactions & Feeds
Business Data Feeds
User Click Stream
Sensors / RFID / Devices
Spatial & GPS Coordinates
Increasing Data Variety and Complexity
Transactions +
Interactions +
Observations
= BIG DATA
Microsoft End-To-End Big Data Platform
Right Tool for the Right Usage
Traditional
Database
SCALE (storage & processing)
Hadoop
Platform
NoSQL
MPP
Analytics
EDW
schema
speed
governance
best fit use
processing
Required on write Required on read
Reads are fast Writes are fast
Standards and structured Loosely structured
Limited, no data processing Processing coupled with data
data typesStructured Multi and unstructured
Interactive OLAP Analytics
Complex ACID Transactions
Operational Data Store
Data Discovery
Processing unstructured data
Massive Storage/Processing
Track Resources
mva
SQL Server

Introduction To SQL Server 2014

  • 1.
  • 2.
    Who Am I •7+ years of experience in Microsoft BI Stack • Currently working in The Most Group as MS BI Solution Architect • Microsoft BI Head in EnterprizeERP Consultancy Service • Consulting partner of Right Click Info , Mentalist Network • NMUG Community Leader , Speaker • Microsoft Community Representative for Navi Mumbai region. • SQL PASS Chapter Lead for Mumbai & Navi Mumbai • MCTS: Microsoft® Certified Technology Specialist -SQL Server 2008,Business Intelligence Development and Maintenance • MCSA: Microsoft® Certified Solutions Associate -SQL Server 2012 – Querying Microsoft SQL Server 2012,Administering Microsoft SQL Server 2012 Databases, Implementing a Data Warehouse with Microsoft SQL Server 2012 • CSM: Certified Scrum Master from Scrum Alliance, US • BE-Computer Science & Engineering , RAIT , Mumbai • MBA-Information System, Arianth College , Pune • Blogging @ http://bimentalist.com , http://sqlmentalist.com • BI Articles :: 200+ , SQL Article – 200 +, SQL Script – 200+
  • 4.
  • 5.
    Gartner Magic QuadrantDWH & BI 2013 Gartner delivers technology research to global technology business leaders to make informed decisions on key initiatives.
  • 6.
    6 SQL Server Releasetimelines 1996 SQL Server 6.5 1990 1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2005 SQL Server 2005 Unicode Support Native XML SQLCLR Service Broker Integration Services 1993 SQL Server 4.21 (NT) 1995 SQL Server 6.0 1989 SQL Server 1.0 (OS/2) 2000 SQL Server 2000 Reporting Services 2010 SQL Server 2008 R2 Data-tier Apps StreamInsight PowerPivot Master Data Services 2008 SQL Server 2008 Sparse Columns Spatial Types FILESTREAM 1998 SQL Server 7.0 Dynamic Locking Auto-Tuning Full-text search Replication Analysis Services 1991 SQL Server 1.1 (OS/2) 2012 SQL Server 2012 AlwaysOn Columnstore FileTable Semantic Search Power View & Now Introducing ..
  • 7.
    Business Intelligence Agile BIOut of The Box, Forrester Research, Inc., April 22, 2010 32% of Excel users are comfortable with using it for “Advanced Analysis” e.g., PivotTables Only 28% of potential BI users have any meaningful access 31% of all BI initiatives partially meet or don’t meet the business goals originally set Knowledge workers spend too much time looking for, not enough time analyzing the information. Lines between producers and consumers of information have largely disappeared. Requirements change way too fast for IT and traditional technology to keep up. – Excel Segmentation Research – BI Survey 9, BARC – Business Intelligence Purchase Drivers and Adoption Rates, Gartner
  • 8.
    Evolution of BI IThas been busy keeping up with the growing demand for BI… Self-Service BITraditional BI IT Pro End Users Existing Data LOB Applications FilesData Marts ETL/Data Quality Analysis Reports Dashboards & Scorecards Provision Analysis Cubes Data Warehouse Data Staging Spreadsheets Specialized Tools Harvesting existing data, cleaning it and staging it …To provision reports, dashboards & scorecards to meet End User demand.But business is outpacing IT and End Users are demanding Agility …Rapidly breaking down the barriers of Control with Self-Service BI!Is it possible to balance Control with Agility?
  • 9.
    Analysis Services Reporting Services Integration Services Master Data Services SharePoint Scorecards Excel Workbooks PowerPivot Applications SharePoint Search & Dashboards MicrosoftBusiness Intelligence Most widely deployed EIM & BI Platform Most Broadly adopted Productivity & Collaboration Tools Odata Feeds LOB Apps
  • 10.
    SQL Server 2012Highlights High Availability SQL Server AlwaysOn Security & Manageability User-Defined Server Roles Web & Breadth PHP Driver Scalability & Performance Distributed Replay Application Centric Failover Contained Database Authentication Local DB Runtime Fast FILESTEAM UTF-16 Readable Secondaries Fast Full-Text Search Reliable, Integrated Failover Detection Multiple Secondaries Audit Enhancements Paging for result sets Column Store Index, “Apollo” HA for StreamInsight Management Pack for High Availability Backup Secondaries JDBC 4.0 driver Support for ARM processors Active Directory w/ SharePoint for SSRS Default Schema for Windows Group Windows Server Core Support Beyond Relational FileTable SQL Server Data Tools Online Operations Win32 Access to Database files Semantic Search Platform Full Globe Spatial Support DAC Enhancements ODBC for Linux Fast Spatial Performance Hybrid Applications with SQL Azure Up to 15K Partitions/Table EIM SSIS Server Business Intelligence Unified Semantic Model Power View Data Quality Alerting Enhanced MDSIn-memory BI for corporate Sysprep for AS PowerPivot Enhancements Reporting as SharePoint Shared Service DBC & OLTP Appliances/RAs MDS add-in for Excel
  • 11.
    AlwaysOn Reliable Secondaries FileTable ColumnStoreIndex 15k Partitions SQL Server Data Tools Power View BI Semantic Model Data Quality Services Full-Text Search Performance Reporting Alerts ODBC Driver for Linux Statistical Semantic SearchWindows Server Core Support Multiple Secondaries Availability Groups Default Scheme for Windows Groups T-SQL Enhancements Full Globe Spatial SSMS to Windows Azure Platform PowerPivot Enhancements Master Data Management Excel Add-in PowerShell 2.0 Support PHP & Java Connectivity SQL Audit for All Editions CDC Support for SSIS New SSIS Design Surface Online Operation Enhancements Multi-site Clustering Unstructured Data Performance Resource Governor Enhancements Database Recovery Advisor HA for StreamInsight Flexible Failover Policy Extended Events Enhancements Contained Database Authentication SharePoint Active Directory Support SQL Server Express LocalDB User-defined Audit Audit Filtering Audit Resilience FTS Support for Czech and Greek AlwaysOn Connection Director Ad Hoc Reporting SSIS Troubleshooting SSIS Package Management T-SQL Debugger Enhancements Spatial 2D Support Unstructured Data Performance 11
  • 12.
    Organizational Compliance Optimized Productivity 11 Extend Any Data, Anywhere FastTime to Solution Scalable Analytics & DW 8 Credible, Consistent Data Peace of Mind Managed Self- Service BI 4 Rapid Data Exploration 3 Blazing-Fast Performance 2 Required 9s & Protection 1 Scale on Demand 5 76 12109 II-BREAKTHROUGHINSIGHT III-CLOUDONYOURTERMS I-MISSION CRITICAL CONFIDENCE 12 12 Important Feature's of SQL Server 2012
  • 13.
  • 14.
    Complete & ConsistentData Platform
  • 15.
  • 16.
    Seamless Transition ofthe BI Semantic Model Across BI Spectrum TeamBIPersonalBI CorporateBI PowerPivot for Excel PowerPivot for SharePoint Analysis Services My Context BI solution created by user. Context is only for user & exists as document. Our Context BI Solution created by power user. Context is for a small team & it’s managed on a server. The Org’s Context BI Solution created by IT, Established corporate context & is reusable, scalable and backed up. AlignedEmpowered
  • 17.
    Who have acceptedSQL Server 2012
  • 18.
    Why do customerschose SQL Server? 18 Increased productivity and decistion Makting thru Self Service BI + IT Management „Analysis take now seconds instead of days“ High Availability, Scalability & Security Up to 100%, SQL is leading in lowest sec. patches rate Reduce IT costs through IT Management 69% reduced DB costs, 50% lower Admin. efforts Lower Purchase Costs All BI features in one BI Edition, SQL & Office are already in use
  • 19.
    Enabling Integration &Sharing SQL Server 2012 Master Data Services Versioning Validation Authoring business rules to ensure data correctness Modeling Entities, Attributes, Hierarchies MDS Role-based Security and Transaction Annotation Master Data Stewardship External (CRM, ..)Excel DWH Loading batched data through Staging Tables Consuming data through Views Registering to changes through APIs Excel Add-In Web UI Data Matching Workflow / Notifications
  • 20.
    SQL Server 2012Data Quality Services Knowledge-Driven Based on a Data Quality Knowledge Base (DQKB) that is reusable for a variety of data quality improvements Knowledge Discovery Acquire additional knowledge through data samples and user feedback Open and Extendible Support use of user-generated knowledge and IP by 3rd party reference data providers Easy to Use Compelling user experience designed for increased productivity Integrated into SSIS
  • 21.
    How Should IStudy Now ! Build the necessary skills to grow in your IT role and career with Microsoft Cloud technologies. • Learn for free and at your own pace. • Make a name for yourself within the community of IT Pros worldwide, by climbing the ladder all the way to Platinum Level. • Benefit from promotions and initiatives published at the MVA Virtual Labs enable you to quickly evaluate and test Microsoft's newest products and technologies through a series of guided, hands-on labs that you can complete in 90 minutes or less. There is no complex setup or installation required, and you can use Virtual Labs online immediately, free. www.microsoftvirtualacademy.com http://www.microsoft.com/en- in/sqlserver/learning-center/virtual-labs.aspx
  • 22.
    Showcase your expertskills at developing and deploying rich business intelligence solutions. Demonstrate your expertise at building enterprise-scale data solutions.
  • 23.
    Lets Learn SQLServer & Big Data
  • 24.
    The World ofData HAS CHANGED
  • 25.
    Type of datagenerated by sector
  • 26.
    The Transition toBig Data BIG DATA on Axis Variety - Data is present in various form Volume – Data size is Huge Velocity – Increasing Data day by Day
  • 27.
    Interpreting Big Data RequiresNew Approaches: Hadoop 27 velocity variety complexity volume Data INTERPRETINGPREPARING
  • 28.
    Big Data: Bigand Getting Bigger Fast! Machine-generated data is a key driver in data growth 1 ZB = 1,000,000,000,000 GBs! Projected to increase 15x by 2020 *2012 IDC Digital Universe Study  950 million users  2.5 billion content items shared per day (status updates + wall posts + photos + videos + comments)  2.7 billion Likes per day  300 million photos uploaded per day  100+ petabytes of disk space in one of FB’s largest Hadoop (HDFS) clusters  105 terabytes of data scanned via Hive, Facebook’s Hadoop query language, every 30 minutes  70,000 queries executed on these databases per day  500+terabytes of new data ingested into the databases every day
  • 29.
    What Makes UpBig Data? Megabytes Gigabytes Terabytes Petabytes Purchase detail Purchase record Payment record ERP CRM WEB BIG DATA Offer details Support Contacts Customer Touches Segmentation Web logs Offer history A/B testing Dynamic Pricing Affiliate Networks Search Marketing Behavioral Targeting Dynamic Funnels User Generated Content Mobile Web SMS/MMSSentiment External Demographics HD Video, Audio, Images Speech to Text Product/Service Logs Social Interactions & Feeds Business Data Feeds User Click Stream Sensors / RFID / Devices Spatial & GPS Coordinates Increasing Data Variety and Complexity Transactions + Interactions + Observations = BIG DATA
  • 30.
  • 31.
    Right Tool forthe Right Usage Traditional Database SCALE (storage & processing) Hadoop Platform NoSQL MPP Analytics EDW schema speed governance best fit use processing Required on write Required on read Reads are fast Writes are fast Standards and structured Loosely structured Limited, no data processing Processing coupled with data data typesStructured Multi and unstructured Interactive OLAP Analytics Complex ACID Transactions Operational Data Store Data Discovery Processing unstructured data Massive Storage/Processing
  • 34.