SlideShare a Scribd company logo
1 of 34
Tony Rogerson
Microsoft Data Platform MVP
tonyrogerson@sqlserverfaq.com
@tonyrogerson
 Professional
◦ 29 years of Database experience – (6 on DB2, 1 on Oracle
and 23 on SQL Server)
◦ Freelance SQL Server and Data Platform specialist
◦ Fellow BCS, Masters in BI, PGCert in Data Science
◦ I also do F# (and the less relevant cousin C#)
 Community
◦ Founder member of UK SQL User Group,
SQLServerFAQ.com, DataIdol.com, DDD, SQLBits and SQL
Relay
◦ Microsoft SQL Server MVP since 1997, and now a Data
Platform MVP
◦ Technical blog:
http://sqlblogcasts.com/blogs/tonyrogerson (legacy)
http://dataidol.com/tonyrogerson (General DP blog)
http://sqlserverfaq.com/tonyrogerson (MS DP blog)
Group discussion – I can only discuss from
what I’ve seen myself over the past few years
and recent while looking for work
 What’s a Data Platform?
 Define the traditional Database Administrator
◦ Logical and Physical Modelling
◦ Data Governance
◦ HADR
 The importance of a play area
 The expanding skillset
◦ Beyond Relational – alternative Databases
◦ Polyglot Database Environment
◦ The Distributed Database and understanding CAP
◦ Alternate architectures - LAMBDA
◦ ETL
◦ Business Intelligence, Data Science, Data Platform Engineer
◦ What else? Audience please….
Types
Structured
Un-structured
Semi-structured
Applications
Fat client, Web
Intranet, Mobile
Storage
Database Type
SQL
NoSQL
NewSQL
Business Intelligence
Standard Reporting from
standard process metrics
from the Data Warehouse/
Reporting database
Business Analytics
Investigative Reporting
over past data.
Management Science
Data Science
Investigative {Data
Analytics, Business
Analytics}
over structured, semi,
unstructured data for
possible patterns – use of
Machine Learning and
Pattern Matching
algorithms.
Data Creators,
Data Contributors,
Data Consumers
Business
Intelligence
SSRS, Crystal,
Business Objects,
PowerPivot, Excel,
QlikView, Tableau,
Reporting apps….
Types
Structured – Normal Form, JSON, XML
Un-structured – {developers think all data is like this }
Semi-structured – JSON, XML, Key/Value Pair
Applications
C#, F#, Java etc.
[Data sourcing]
Storage
Database Type
SQL – Oracle, DB2, Sybase, SQL Server, MySQL etc.
NoSQL – CouchDB, Raven, Cassandra, Hadoop, MongoDB, Neo4j
NewSQL – Postgres-XL, Postgres-XC, Volt-DB, NuoDB
Business
Analytics
SAS, SPSS,
Statistica, MatLab
etc..
Data Science
BI + BA + ‘R’, Pyphon,
Machine Learning
packages, SQL, MapR,
Data Extraction, ML,
Visualisations, Story
Boarding
SQL, MapR, U-SQL..Data Creators,
Data Contributors,
Data Consumers
 SSIS
◦ pull RSS feed and store in SQL Server
◦ ODATA source example
 Azure File Share
◦ Storing archive data
Modelling
Data Governance
HADR
Releasing Stuff
 Data is an Asset – Security Guard
 Data Custodian – Compliance, ???
 Liaison between Business and Devs
 Liaison between Business and Infrastructure
 What else?
 Custodian of the Business Taxonomy
◦ Data Dictionary
 Logical / Physical
◦ Normal Form
◦ Logical Model (relationships) V Physical Model
(vender dependent schema)
 Relational V Dimensional
◦ Entity Relationship modelling (tables and
relationships between)
◦ Dimensional Modelling (facts and dimensions) –
models to usability and performance
 ICO Principals
 Data Protection Laws – Security, Retention
 Your responsibilities – vary within the Org
 High Availability
◦ Understanding Latency
◦ Mirroring
◦ Availability Groups
◦ Log Shipping (?)
 Disaster Recovery
◦ Practiced Procedures
◦ DR Resource misalignment
◦ Implementing contingency
◦ Dealing with Data corruption or Accidents (if I only
have AG’s – what’s the issue?)
 Applying Database releases
◦ Which Databases? SQL / NoSQL etc.
 Supportability (level of reqd knowledge)
 Patching Servers
 You protect the Integrity and Availability of
the “Database Platform”
 Not limited to SQL Server
◦ NoSQL products
◦ Relational “SQL” products
◦ NewSQL
Play Areas
Knowing what to learn
 Align with your company
◦ Talk to developers, see what they are using, take a
lead with Data Technology – nurture their use of
Data.
◦ Data is an Asset, without data your company won’t
exist – make your company realise your importance
and you need to be right up there in the decision
making for technology direction
 Align with the industry
◦ Job boards, trends
 Be one (ok – a couple of) steps ahead!
 You can’t play in live!
 Decent laptop – 16GiB+ RAM, SSD / M2 Flash
 VirtualBox
◦ Multiple Windows Server, build a domain, build a
cluster etc.
◦ Multiple Linux
◦ Etc.
Beyond Relational – alternative
Databases
Polyglot Database Environment
The Distributed Database and
CAP
LAMBDA
ETL
MDM
Cloud
 Business environment is “Polyglot”
 Require understanding of
◦ NoSQL
◦ CAP Theorem
◦ LAMBDA (edge case)
◦ Big Data – what it really is
◦ CEP (is this a Database related tech?)
◦ ETL
◦ Data Science – what it really is
◦ BI
◦ Kimball, Inmon
◦ Data Vault
 Really means – No NF
 Key Value Stores (Riak, CouchDB)
 Column (Cassandra)
 Document (MongoDB)
 Graph (Neo4J)
 Object (Bit niche )
 Ironically – most have a SQL like interface
now or in development!
 Consistency
◦ All nodes show the same value
◦ Eventual Consistency
 Availability
◦ Node will return data
 Partition Tolerance
◦ Islands form when network fails – clients connect to
local nodes so when isolated you lose consistency.
 You can only have two of the 3 and never all
three.
1 2
3 4
5 6
Insert
Update
Delete
DatCtr A
Insert
Update
Delete
DatCtr B
Insert
Update
Delete
DatCtr C
 No – it’s not just Hadoop
 Velocity, Variety, Volume
 BD can be done in anything.
◦ Velocity – CEP, In-Memory, distributed computing
◦ Variety – varied types of data, structured / un.
◦ Volume – size of the data
 BD is not definitive – depends on your
budget, ability etc.
 Processing a data stream in flight
 Window over the stream and determine
trends
 Read the stream rather than poll the database
 If you aren’t using Machine Learning / Data
Mining algol’s you aren’t doing Data Science
 If you know what you are looking for – you
aren’t doing DS.
 DS isn’t just R, you can do DS in numerous
tools, R has a large library of packages to use
against your data
 DS is where you are looking for patterns in
your data and trying to understand them to
then formulate standard process flows to take
advantage.
 Scale out – distributed – data processing
architecture
 Batch, Speed, Service layers
 For low latency, high updates
 Robust
 Kimball
◦ Dimensional modelling with star schema
◦ Dimensions and Facts
◦ Bottom up – data marts to EDW
◦ Aspires to Single Version of the Truth
 Inmon
◦ Normal Form
◦ Can also use star schema
◦ Form the EDW and then use data marts
◦ Stronger approach to Single Version of the Truth
 Modelling method
 Pull all your uncleansed data and store it in
one place
 Buffer between Operational Databases and
the Conformed Data Warehouse
 Are you really on the Cloud or just managed
remotely located server environment?
 Real cloud has immediate elasticity, hides
infrastructure, easy to spawn up new
resource and near immediate.
 Market’d cloud is really managed servers – no
immediate elasticity, servers are provisioned
and that takes time.
 True cloud offers elasticity for Distributed
Database capabilities – proper scale out.
◦ Azure Elastic Database (Sharding)
◦ SQL 2016 Stretch Feature
 Remember CAP? Yep – you need to understand
that.
 On-Prem tends to be scale up, single box –
single database
 Cloud – some of your tasks will disappear
because it’s done for you. But your role is a Data
Centric role and not Infrastructure Centric.
Evolution of the DBA to Data Platform Administrator/Specialist

More Related Content

What's hot

NoSQL-Database-Concepts
NoSQL-Database-ConceptsNoSQL-Database-Concepts
NoSQL-Database-Concepts
Bhaskar Gunda
 

What's hot (20)

NOSQL- Presentation on NoSQL
NOSQL- Presentation on NoSQLNOSQL- Presentation on NoSQL
NOSQL- Presentation on NoSQL
 
RDBMS vs NoSQL
RDBMS vs NoSQLRDBMS vs NoSQL
RDBMS vs NoSQL
 
SQL & NoSQL
SQL & NoSQLSQL & NoSQL
SQL & NoSQL
 
NoSQL databases
NoSQL databasesNoSQL databases
NoSQL databases
 
Selecting best NoSQL
Selecting best NoSQL Selecting best NoSQL
Selecting best NoSQL
 
NoSQL Slideshare Presentation
NoSQL Slideshare Presentation NoSQL Slideshare Presentation
NoSQL Slideshare Presentation
 
NOSQL Databases types and Uses
NOSQL Databases types and UsesNOSQL Databases types and Uses
NOSQL Databases types and Uses
 
Introduction to NoSQL
Introduction to NoSQLIntroduction to NoSQL
Introduction to NoSQL
 
Non Relational Databases
Non Relational DatabasesNon Relational Databases
Non Relational Databases
 
introduction to NOSQL Database
introduction to NOSQL Databaseintroduction to NOSQL Database
introduction to NOSQL Database
 
SQL vs NoSQL: Big Data Adoption & Success in the Enterprise
SQL vs NoSQL: Big Data Adoption & Success in the EnterpriseSQL vs NoSQL: Big Data Adoption & Success in the Enterprise
SQL vs NoSQL: Big Data Adoption & Success in the Enterprise
 
Nosql seminar
Nosql seminarNosql seminar
Nosql seminar
 
SQL/NoSQL How to choose ?
SQL/NoSQL How to choose ?SQL/NoSQL How to choose ?
SQL/NoSQL How to choose ?
 
Introduction to NoSQL
Introduction to NoSQLIntroduction to NoSQL
Introduction to NoSQL
 
NoSQL-Database-Concepts
NoSQL-Database-ConceptsNoSQL-Database-Concepts
NoSQL-Database-Concepts
 
A Seminar on NoSQL Databases.
A Seminar on NoSQL Databases.A Seminar on NoSQL Databases.
A Seminar on NoSQL Databases.
 
No SQL- The Future Of Data Storage
No SQL- The Future Of Data StorageNo SQL- The Future Of Data Storage
No SQL- The Future Of Data Storage
 
Sql vs NoSQL
Sql vs NoSQLSql vs NoSQL
Sql vs NoSQL
 
Making Sense of Schema on Read
Making Sense of Schema on ReadMaking Sense of Schema on Read
Making Sense of Schema on Read
 
Sql vs NO-SQL database differences explained
Sql vs NO-SQL database differences explainedSql vs NO-SQL database differences explained
Sql vs NO-SQL database differences explained
 

Similar to Evolution of the DBA to Data Platform Administrator/Specialist

NoSQLDatabases
NoSQLDatabasesNoSQLDatabases
NoSQLDatabases
Adi Challa
 
Pass chapter meeting dec 2013 - compression a hidden gem for io heavy databas...
Pass chapter meeting dec 2013 - compression a hidden gem for io heavy databas...Pass chapter meeting dec 2013 - compression a hidden gem for io heavy databas...
Pass chapter meeting dec 2013 - compression a hidden gem for io heavy databas...
Charley Hanania
 
NoSQL Data Stores in Research and Practice - ICDE 2016 Tutorial - Extended Ve...
NoSQL Data Stores in Research and Practice - ICDE 2016 Tutorial - Extended Ve...NoSQL Data Stores in Research and Practice - ICDE 2016 Tutorial - Extended Ve...
NoSQL Data Stores in Research and Practice - ICDE 2016 Tutorial - Extended Ve...
Felix Gessert
 

Similar to Evolution of the DBA to Data Platform Administrator/Specialist (20)

Relational databases vs Non-relational databases
Relational databases vs Non-relational databasesRelational databases vs Non-relational databases
Relational databases vs Non-relational databases
 
Prague data management meetup 2018-03-27
Prague data management meetup 2018-03-27Prague data management meetup 2018-03-27
Prague data management meetup 2018-03-27
 
Building a modern data warehouse
Building a modern data warehouseBuilding a modern data warehouse
Building a modern data warehouse
 
NoSQL for great good [hanoi.rb talk]
NoSQL for great good [hanoi.rb talk]NoSQL for great good [hanoi.rb talk]
NoSQL for great good [hanoi.rb talk]
 
NoSQL
NoSQLNoSQL
NoSQL
 
NoSQL
NoSQLNoSQL
NoSQL
 
Jump Start on Apache Spark 2.2 with Databricks
Jump Start on Apache Spark 2.2 with DatabricksJump Start on Apache Spark 2.2 with Databricks
Jump Start on Apache Spark 2.2 with Databricks
 
How to Survive as a Data Architect in a Polyglot Database World
How to Survive as a Data Architect in a Polyglot Database WorldHow to Survive as a Data Architect in a Polyglot Database World
How to Survive as a Data Architect in a Polyglot Database World
 
NoSQLDatabases
NoSQLDatabasesNoSQLDatabases
NoSQLDatabases
 
NoSQL.pptx
NoSQL.pptxNoSQL.pptx
NoSQL.pptx
 
Changing the game with cloud dw
Changing the game with cloud dwChanging the game with cloud dw
Changing the game with cloud dw
 
Big Data technology Landscape
Big Data technology LandscapeBig Data technology Landscape
Big Data technology Landscape
 
NoSQL Basics - a quick tour
NoSQL Basics - a quick tourNoSQL Basics - a quick tour
NoSQL Basics - a quick tour
 
Druid Adoption Tips and Tricks
Druid Adoption Tips and TricksDruid Adoption Tips and Tricks
Druid Adoption Tips and Tricks
 
Azure data platform overview
Azure data platform overviewAzure data platform overview
Azure data platform overview
 
Transform your DBMS to drive engagement innovation with Big Data
Transform your DBMS to drive engagement innovation with Big DataTransform your DBMS to drive engagement innovation with Big Data
Transform your DBMS to drive engagement innovation with Big Data
 
Pass chapter meeting dec 2013 - compression a hidden gem for io heavy databas...
Pass chapter meeting dec 2013 - compression a hidden gem for io heavy databas...Pass chapter meeting dec 2013 - compression a hidden gem for io heavy databas...
Pass chapter meeting dec 2013 - compression a hidden gem for io heavy databas...
 
NoSQL Data Stores in Research and Practice - ICDE 2016 Tutorial - Extended Ve...
NoSQL Data Stores in Research and Practice - ICDE 2016 Tutorial - Extended Ve...NoSQL Data Stores in Research and Practice - ICDE 2016 Tutorial - Extended Ve...
NoSQL Data Stores in Research and Practice - ICDE 2016 Tutorial - Extended Ve...
 
Myth Busters II: BI Tools and Data Virtualization are Interchangeable
Myth Busters II: BI Tools and Data Virtualization are InterchangeableMyth Busters II: BI Tools and Data Virtualization are Interchangeable
Myth Busters II: BI Tools and Data Virtualization are Interchangeable
 
Some NoSQL
Some NoSQLSome NoSQL
Some NoSQL
 

Recently uploaded

Revolutionizing SAP® Processes with Automation and Artificial Intelligence
Revolutionizing SAP® Processes with Automation and Artificial IntelligenceRevolutionizing SAP® Processes with Automation and Artificial Intelligence
Revolutionizing SAP® Processes with Automation and Artificial Intelligence
Precisely
 
Easier, Faster, and More Powerful – Alles Neu macht der Mai -Wir durchleuchte...
Easier, Faster, and More Powerful – Alles Neu macht der Mai -Wir durchleuchte...Easier, Faster, and More Powerful – Alles Neu macht der Mai -Wir durchleuchte...
Easier, Faster, and More Powerful – Alles Neu macht der Mai -Wir durchleuchte...
panagenda
 

Recently uploaded (20)

Observability Concepts EVERY Developer Should Know (DevOpsDays Seattle)
Observability Concepts EVERY Developer Should Know (DevOpsDays Seattle)Observability Concepts EVERY Developer Should Know (DevOpsDays Seattle)
Observability Concepts EVERY Developer Should Know (DevOpsDays Seattle)
 
Revolutionizing SAP® Processes with Automation and Artificial Intelligence
Revolutionizing SAP® Processes with Automation and Artificial IntelligenceRevolutionizing SAP® Processes with Automation and Artificial Intelligence
Revolutionizing SAP® Processes with Automation and Artificial Intelligence
 
UiPath manufacturing technology benefits and AI overview
UiPath manufacturing technology benefits and AI overviewUiPath manufacturing technology benefits and AI overview
UiPath manufacturing technology benefits and AI overview
 
Design and Development of a Provenance Capture Platform for Data Science
Design and Development of a Provenance Capture Platform for Data ScienceDesign and Development of a Provenance Capture Platform for Data Science
Design and Development of a Provenance Capture Platform for Data Science
 
Event-Driven Architecture Masterclass: Integrating Distributed Data Stores Ac...
Event-Driven Architecture Masterclass: Integrating Distributed Data Stores Ac...Event-Driven Architecture Masterclass: Integrating Distributed Data Stores Ac...
Event-Driven Architecture Masterclass: Integrating Distributed Data Stores Ac...
 
TopCryptoSupers 12thReport OrionX May2024
TopCryptoSupers 12thReport OrionX May2024TopCryptoSupers 12thReport OrionX May2024
TopCryptoSupers 12thReport OrionX May2024
 
Human Expert Website Manual WCAG 2.0 2.1 2.2 Audit - Digital Accessibility Au...
Human Expert Website Manual WCAG 2.0 2.1 2.2 Audit - Digital Accessibility Au...Human Expert Website Manual WCAG 2.0 2.1 2.2 Audit - Digital Accessibility Au...
Human Expert Website Manual WCAG 2.0 2.1 2.2 Audit - Digital Accessibility Au...
 
Easier, Faster, and More Powerful – Alles Neu macht der Mai -Wir durchleuchte...
Easier, Faster, and More Powerful – Alles Neu macht der Mai -Wir durchleuchte...Easier, Faster, and More Powerful – Alles Neu macht der Mai -Wir durchleuchte...
Easier, Faster, and More Powerful – Alles Neu macht der Mai -Wir durchleuchte...
 
(Explainable) Data-Centric AI: what are you explaininhg, and to whom?
(Explainable) Data-Centric AI: what are you explaininhg, and to whom?(Explainable) Data-Centric AI: what are you explaininhg, and to whom?
(Explainable) Data-Centric AI: what are you explaininhg, and to whom?
 
State of the Smart Building Startup Landscape 2024!
State of the Smart Building Startup Landscape 2024!State of the Smart Building Startup Landscape 2024!
State of the Smart Building Startup Landscape 2024!
 
Oauth 2.0 Introduction and Flows with MuleSoft
Oauth 2.0 Introduction and Flows with MuleSoftOauth 2.0 Introduction and Flows with MuleSoft
Oauth 2.0 Introduction and Flows with MuleSoft
 
JavaScript Usage Statistics 2024 - The Ultimate Guide
JavaScript Usage Statistics 2024 - The Ultimate GuideJavaScript Usage Statistics 2024 - The Ultimate Guide
JavaScript Usage Statistics 2024 - The Ultimate Guide
 
Frisco Automating Purchase Orders with MuleSoft IDP- May 10th, 2024.pptx.pdf
Frisco Automating Purchase Orders with MuleSoft IDP- May 10th, 2024.pptx.pdfFrisco Automating Purchase Orders with MuleSoft IDP- May 10th, 2024.pptx.pdf
Frisco Automating Purchase Orders with MuleSoft IDP- May 10th, 2024.pptx.pdf
 
TEST BANK For, Information Technology Project Management 9th Edition Kathy Sc...
TEST BANK For, Information Technology Project Management 9th Edition Kathy Sc...TEST BANK For, Information Technology Project Management 9th Edition Kathy Sc...
TEST BANK For, Information Technology Project Management 9th Edition Kathy Sc...
 
Event-Driven Architecture Masterclass: Challenges in Stream Processing
Event-Driven Architecture Masterclass: Challenges in Stream ProcessingEvent-Driven Architecture Masterclass: Challenges in Stream Processing
Event-Driven Architecture Masterclass: Challenges in Stream Processing
 
Microsoft CSP Briefing Pre-Engagement - Questionnaire
Microsoft CSP Briefing Pre-Engagement - QuestionnaireMicrosoft CSP Briefing Pre-Engagement - Questionnaire
Microsoft CSP Briefing Pre-Engagement - Questionnaire
 
Generative AI Use Cases and Applications.pdf
Generative AI Use Cases and Applications.pdfGenerative AI Use Cases and Applications.pdf
Generative AI Use Cases and Applications.pdf
 
Intro to Passkeys and the State of Passwordless.pptx
Intro to Passkeys and the State of Passwordless.pptxIntro to Passkeys and the State of Passwordless.pptx
Intro to Passkeys and the State of Passwordless.pptx
 
Google I/O Extended 2024 Warsaw
Google I/O Extended 2024 WarsawGoogle I/O Extended 2024 Warsaw
Google I/O Extended 2024 Warsaw
 
Extensible Python: Robustness through Addition - PyCon 2024
Extensible Python: Robustness through Addition - PyCon 2024Extensible Python: Robustness through Addition - PyCon 2024
Extensible Python: Robustness through Addition - PyCon 2024
 

Evolution of the DBA to Data Platform Administrator/Specialist

  • 1. Tony Rogerson Microsoft Data Platform MVP tonyrogerson@sqlserverfaq.com @tonyrogerson
  • 2.  Professional ◦ 29 years of Database experience – (6 on DB2, 1 on Oracle and 23 on SQL Server) ◦ Freelance SQL Server and Data Platform specialist ◦ Fellow BCS, Masters in BI, PGCert in Data Science ◦ I also do F# (and the less relevant cousin C#)  Community ◦ Founder member of UK SQL User Group, SQLServerFAQ.com, DataIdol.com, DDD, SQLBits and SQL Relay ◦ Microsoft SQL Server MVP since 1997, and now a Data Platform MVP ◦ Technical blog: http://sqlblogcasts.com/blogs/tonyrogerson (legacy) http://dataidol.com/tonyrogerson (General DP blog) http://sqlserverfaq.com/tonyrogerson (MS DP blog)
  • 3. Group discussion – I can only discuss from what I’ve seen myself over the past few years and recent while looking for work
  • 4.  What’s a Data Platform?  Define the traditional Database Administrator ◦ Logical and Physical Modelling ◦ Data Governance ◦ HADR  The importance of a play area  The expanding skillset ◦ Beyond Relational – alternative Databases ◦ Polyglot Database Environment ◦ The Distributed Database and understanding CAP ◦ Alternate architectures - LAMBDA ◦ ETL ◦ Business Intelligence, Data Science, Data Platform Engineer ◦ What else? Audience please….
  • 5.
  • 6. Types Structured Un-structured Semi-structured Applications Fat client, Web Intranet, Mobile Storage Database Type SQL NoSQL NewSQL Business Intelligence Standard Reporting from standard process metrics from the Data Warehouse/ Reporting database Business Analytics Investigative Reporting over past data. Management Science Data Science Investigative {Data Analytics, Business Analytics} over structured, semi, unstructured data for possible patterns – use of Machine Learning and Pattern Matching algorithms. Data Creators, Data Contributors, Data Consumers
  • 7. Business Intelligence SSRS, Crystal, Business Objects, PowerPivot, Excel, QlikView, Tableau, Reporting apps…. Types Structured – Normal Form, JSON, XML Un-structured – {developers think all data is like this } Semi-structured – JSON, XML, Key/Value Pair Applications C#, F#, Java etc. [Data sourcing] Storage Database Type SQL – Oracle, DB2, Sybase, SQL Server, MySQL etc. NoSQL – CouchDB, Raven, Cassandra, Hadoop, MongoDB, Neo4j NewSQL – Postgres-XL, Postgres-XC, Volt-DB, NuoDB Business Analytics SAS, SPSS, Statistica, MatLab etc.. Data Science BI + BA + ‘R’, Pyphon, Machine Learning packages, SQL, MapR, Data Extraction, ML, Visualisations, Story Boarding SQL, MapR, U-SQL..Data Creators, Data Contributors, Data Consumers
  • 8.  SSIS ◦ pull RSS feed and store in SQL Server ◦ ODATA source example  Azure File Share ◦ Storing archive data
  • 10.  Data is an Asset – Security Guard  Data Custodian – Compliance, ???  Liaison between Business and Devs  Liaison between Business and Infrastructure  What else?
  • 11.  Custodian of the Business Taxonomy ◦ Data Dictionary  Logical / Physical ◦ Normal Form ◦ Logical Model (relationships) V Physical Model (vender dependent schema)  Relational V Dimensional ◦ Entity Relationship modelling (tables and relationships between) ◦ Dimensional Modelling (facts and dimensions) – models to usability and performance
  • 12.  ICO Principals  Data Protection Laws – Security, Retention  Your responsibilities – vary within the Org
  • 13.  High Availability ◦ Understanding Latency ◦ Mirroring ◦ Availability Groups ◦ Log Shipping (?)  Disaster Recovery ◦ Practiced Procedures ◦ DR Resource misalignment ◦ Implementing contingency ◦ Dealing with Data corruption or Accidents (if I only have AG’s – what’s the issue?)
  • 14.  Applying Database releases ◦ Which Databases? SQL / NoSQL etc.  Supportability (level of reqd knowledge)  Patching Servers
  • 15.
  • 16.  You protect the Integrity and Availability of the “Database Platform”  Not limited to SQL Server ◦ NoSQL products ◦ Relational “SQL” products ◦ NewSQL
  • 18.  Align with your company ◦ Talk to developers, see what they are using, take a lead with Data Technology – nurture their use of Data. ◦ Data is an Asset, without data your company won’t exist – make your company realise your importance and you need to be right up there in the decision making for technology direction  Align with the industry ◦ Job boards, trends  Be one (ok – a couple of) steps ahead!
  • 19.  You can’t play in live!  Decent laptop – 16GiB+ RAM, SSD / M2 Flash  VirtualBox ◦ Multiple Windows Server, build a domain, build a cluster etc. ◦ Multiple Linux ◦ Etc.
  • 20. Beyond Relational – alternative Databases Polyglot Database Environment The Distributed Database and CAP LAMBDA ETL MDM Cloud
  • 21.
  • 22.  Business environment is “Polyglot”  Require understanding of ◦ NoSQL ◦ CAP Theorem ◦ LAMBDA (edge case) ◦ Big Data – what it really is ◦ CEP (is this a Database related tech?) ◦ ETL ◦ Data Science – what it really is ◦ BI ◦ Kimball, Inmon ◦ Data Vault
  • 23.  Really means – No NF  Key Value Stores (Riak, CouchDB)  Column (Cassandra)  Document (MongoDB)  Graph (Neo4J)  Object (Bit niche )  Ironically – most have a SQL like interface now or in development!
  • 24.  Consistency ◦ All nodes show the same value ◦ Eventual Consistency  Availability ◦ Node will return data  Partition Tolerance ◦ Islands form when network fails – clients connect to local nodes so when isolated you lose consistency.  You can only have two of the 3 and never all three.
  • 25. 1 2 3 4 5 6 Insert Update Delete DatCtr A Insert Update Delete DatCtr B Insert Update Delete DatCtr C
  • 26.  No – it’s not just Hadoop  Velocity, Variety, Volume  BD can be done in anything. ◦ Velocity – CEP, In-Memory, distributed computing ◦ Variety – varied types of data, structured / un. ◦ Volume – size of the data  BD is not definitive – depends on your budget, ability etc.
  • 27.  Processing a data stream in flight  Window over the stream and determine trends  Read the stream rather than poll the database
  • 28.  If you aren’t using Machine Learning / Data Mining algol’s you aren’t doing Data Science  If you know what you are looking for – you aren’t doing DS.  DS isn’t just R, you can do DS in numerous tools, R has a large library of packages to use against your data  DS is where you are looking for patterns in your data and trying to understand them to then formulate standard process flows to take advantage.
  • 29.  Scale out – distributed – data processing architecture  Batch, Speed, Service layers  For low latency, high updates  Robust
  • 30.  Kimball ◦ Dimensional modelling with star schema ◦ Dimensions and Facts ◦ Bottom up – data marts to EDW ◦ Aspires to Single Version of the Truth  Inmon ◦ Normal Form ◦ Can also use star schema ◦ Form the EDW and then use data marts ◦ Stronger approach to Single Version of the Truth
  • 31.  Modelling method  Pull all your uncleansed data and store it in one place  Buffer between Operational Databases and the Conformed Data Warehouse
  • 32.  Are you really on the Cloud or just managed remotely located server environment?  Real cloud has immediate elasticity, hides infrastructure, easy to spawn up new resource and near immediate.  Market’d cloud is really managed servers – no immediate elasticity, servers are provisioned and that takes time.
  • 33.  True cloud offers elasticity for Distributed Database capabilities – proper scale out. ◦ Azure Elastic Database (Sharding) ◦ SQL 2016 Stretch Feature  Remember CAP? Yep – you need to understand that.  On-Prem tends to be scale up, single box – single database  Cloud – some of your tasks will disappear because it’s done for you. But your role is a Data Centric role and not Infrastructure Centric.

Editor's Notes

  1. 20:00 – 21:00 Tony Rogerson - SQL Server Data Platform specialist” who used to be known as “Database Administrator" The year was 1995 and I was a SQL Developer/Database Administrator designing schema, writing and optimising SQL, managing log shipping and backups. The year is now 2016 and that relatively small skill set has exploded dramatically with ETL (SSIS plus some C#), MDM, Business Intelligence (Kimball, Inmon, Lambda, hybrid), Data Science (Statistics, Business Skills, R, F#, HDInsight, Hadoop), Cloud (AWS, Azure, Thirdparty on/off prem), Data Governance (ICO principles/rules, Security, International DP rules).  In this session we will look at today’s SQL Server Data Platform specialists, you know who they are because even though you are still called “DBA” you are actually one of them!  We will cover off introductions with demos into the following technology areas: ETL, BI, DS and Azure with examples on using them within a Data Platform setting.