SlideShare a Scribd company logo
POWER QUERY
HARVESTING XML DATA
JULIE SMITH AND ROB VOLK
• DATA SERVICES CONSULTANT AT INNOVATIVE ARCHITECTS IN ATLANTA, GA
• ONE HALF OF THE DATACHIX WITH AUDREY HAMMONDS
• SQL SERVER MVP
• ROB VOLK – NEEDS NO INTRODUCTION, SINCE HE’S NOT GONNA DO MUCH
• EXCEPT COMPLAIN
• AND XML STUFF
• WHICH HE’S COME TO FIND OUT IS A DEATH SENTENCE FOR SESSION SUBMISSIONS
JULIE SMITH AND ROB VOLK
• ROB VOLK IS A DATABASE ADMINISTRATOR FROM THE ATLANTA, GEORGIA
AREA, HAVING WORKED IN THE WIRELESS TELECOM, UTILITY BILLING, AND
CREDIT CARD PROCESSING INDUSTRIES FOR THE PAST 10 YEARS. HE STARTED
USING SQL SERVER WITH VERSION 6.5 AND IS VERY GLAD TO HAVE UPGRADED
SINCE THEN. HE HAS PRESENTED AT SEVERAL SQL SATURDAYS AND USER
GROUPS. IN HIS SPARE TIME HE ACCOMPLISHES MORE THAN HE DOES ON THE
JOB, JUST DON'T TELL HIS BOSS.
TODAY’S AGENDA
• STORY OF HOW ANNOYED ROB WAS WITH PLANNING SQL SATURDAYS
• HAS TO BE A BETTER WAY
• FORAGING FOR DATA—ON SPEAKERS, VENUES, STRANGE COINCIDENCES
• POWER QUERY –FIRST CAME OUT AS DATA EXPLORER
• CURL
• QUERYING XML WITH SQL (TRANSACT SQL TO BE EXACT)
WHAT WOULD WE LIKE TO KNOW?
• HOW MANY EVENTS?
• WHERE ARE THEY?
• SPEAKER DETAILS (MOST SESSIONS/EVENTS/TRAVELED)
• SPONSOR DETAILS
• DATES
• UPCOMING EVENTS
• IT’S NOT JUST FOR SATURDAYS!
INTRODUCING
POWER QUERY
• RELEASED JULY 2013
• EXCEL ADD-IN FOR 2010 OR 2013
• SELF SERVICE ETL!!
POWER BI SUITE
• POWERPIVOT – IN MEMORY EMBEDDED DATA MODELS
• POWERMAP
• POWER VIEW – PRESENTATION READY VISUALIZATIONS
• POWER QUERY
• ALL IN EXCEL !!!!
POWER BI
• OFFICE 365
• DATA MANAGEMENT GATEWAY
• COLLABORATE WITH OFFICE 365 SHAREPOINT
HISTORY
• M LANGUAGE FOR BUSINESS ANALYSTS—XAML
• PROJECT OSLO
• DOMAIN SPECIFIC LANGUAGE
• M HAS MORPHED INTO THE POWER QUERY FOR EXCEL FORMULA LANGUAGE
• DOWNLOAD A PDF HERE:
HTTP://GO.MICROSOFT.COM/FWLINK/?LINKID=235475&CLCID=0X409 TABLE.
PAGE 122
JULIE WHEN SHE FIRST MET DATA EXPLORER…
IT CAN HARVEST THE FOLLOWING:
• “OTHER”
• SHAREPOINT LIST,
• WINDOWS AZURE MARKETPLACE,
• ANY HADOOP FILE (HDFS),
• HDINSIGHT,
• ACTIVE DIRECTORY,
• FACEBOOK.
• RDBMS: SQL SERVER, AZURE SQL SERVER, ACCESS (OOPS), ORACLE, DB2, MYSQL,
POSTGRESS, TERADATA
• FILE:
• EXCEL, CSV, TXT, , XML, JSON, FOLDER (METADATA)
• FROM THE INTARWEBZ!
• WRITE AN M OR POWER QUERY FORMULA LANGUAGE QUERY YOURSELF FROM
SOURCES
• REGULAR SOURCES—MOST DATABASE PLATFORMS INCLUDING HADOOP
• FILES—ALL TYPES
• ODATA
• WEB SITE TABLES!
• ACTIVE DIRECTORY!
FUNCTIONS
• TRANSFORMATION!
• MERGE (JOIN ON KEY)
• APPEND—COMBINE SOURCES TO NEW DESTINATION
RESOURCES
• HTTP://OFFICE.MICROSOFT.COM/EN-US/EXCEL-HELP/LEARN-ABOUT-DATA-
EXPLORER-FORMULAS-HA104003958.ASPX FORMULA CHEAT SHEET
• GREAT POST BY DATAPIG
HTTP://DATAPIGTECHNOLOGIES.COM/BLOG/INDEX.PHP/COOL-THINGS-YOU-
CAN-DO-WITH-DATA-EXPLORER/
• ON M LANGUAGE:
HTTP://BLOGS.MSDN.COM/B/MODELCITIZEN/ARCHIVE/2010/09/22/UPDATE-
ON-SQL-SERVER-MODELING-CTP-REPOSITORY-MODELING-SERVICES-QUOT-
QUADRANT-QUOT-AND-QUOT-M-QUOT.ASPX
• M LANGUAGE SPECS FROM MICROSOFT: HTTP://MSDN.MICROSOFT.COM/EN-
US/LIBRARY/DD285271.ASPX
• HTTP://BLOGS.MSDN.COM/B/MLANGUAGE/
• DOWNLOADABLE 200 PAGE DOC ON FORMULA LANGUAGE (M)
HTTP://GO.MICROSOFT.COM/FWLINK/?LINKID=235474&CLCID=0X409

More Related Content

Similar to HarvestingWebData-PowerQuery.pptx

ITIL Foundation V3
ITIL Foundation V3ITIL Foundation V3
ITIL Foundation V3
Carlos Oliveira
 
Real-time SQL Access for Your Salesforce.com Data
Real-time SQL Access for Your Salesforce.com DataReal-time SQL Access for Your Salesforce.com Data
Real-time SQL Access for Your Salesforce.com DataSalesforce Developers
 
So you think you can scale
So you think you can scaleSo you think you can scale
So you think you can scale
Lori MacVittie
 
From SQL to NoSQL: Structured Querying for JSON
From SQL to NoSQL: Structured Querying for JSONFrom SQL to NoSQL: Structured Querying for JSON
From SQL to NoSQL: Structured Querying for JSON
Keshav Murthy
 
Accelerate your business with flow
Accelerate your business with flowAccelerate your business with flow
Accelerate your business with flow
JoAnna Cheshire
 
Big Data & Oracle Technologies
Big Data & Oracle TechnologiesBig Data & Oracle Technologies
Big Data & Oracle TechnologiesOleksii Movchaniuk
 
Integration intervention: Get your apps and data up to speed
Integration intervention: Get your apps and data up to speedIntegration intervention: Get your apps and data up to speed
Integration intervention: Get your apps and data up to speed
Kenneth Peeples
 
20140116 Tim Willoughby and James Fogarty FOSS in Local Government
20140116   Tim Willoughby and James Fogarty FOSS in Local Government20140116   Tim Willoughby and James Fogarty FOSS in Local Government
20140116 Tim Willoughby and James Fogarty FOSS in Local Government
Tim Willoughby
 
Chapter deck january 2015
Chapter deck january 2015Chapter deck january 2015
Chapter deck january 2015
Christopher Schmidt
 
Denodo DataFest 2017: Business Needs for a Fast Data Strategy
Denodo DataFest 2017: Business Needs for a Fast Data StrategyDenodo DataFest 2017: Business Needs for a Fast Data Strategy
Denodo DataFest 2017: Business Needs for a Fast Data Strategy
Denodo
 
KoprowskiT_SQLRelay2014#8_Birmingham_FromPlanToBackupToCloud
KoprowskiT_SQLRelay2014#8_Birmingham_FromPlanToBackupToCloudKoprowskiT_SQLRelay2014#8_Birmingham_FromPlanToBackupToCloud
KoprowskiT_SQLRelay2014#8_Birmingham_FromPlanToBackupToCloud
Tobias Koprowski
 
Don't simply deploy, transform! Build your digital workplace in Office 365
Don't simply deploy, transform! Build your digital workplace in Office 365Don't simply deploy, transform! Build your digital workplace in Office 365
Don't simply deploy, transform! Build your digital workplace in Office 365
BIWUG
 
Microsoft Flow advanced: tips, pitfalls, problems and warnings to be known be...
Microsoft Flow advanced: tips, pitfalls, problems and warnings to be known be...Microsoft Flow advanced: tips, pitfalls, problems and warnings to be known be...
Microsoft Flow advanced: tips, pitfalls, problems and warnings to be known be...
serge luca
 
Ibm integration bus
Ibm integration busIbm integration bus
Ibm integration bus
FuturePoint Technologies
 
SharePoint Online vs. On-Premise
SharePoint Online vs. On-PremiseSharePoint Online vs. On-Premise
SharePoint Online vs. On-Premise
Evan Hodges
 
Microsoft Flow best practices European Collaboration Summit 2018
Microsoft Flow best practices European Collaboration Summit 2018Microsoft Flow best practices European Collaboration Summit 2018
Microsoft Flow best practices European Collaboration Summit 2018
serge luca
 
[Luca/Van Campenhoudt] Microsoft Flow Beyone the Limits: Tips, Pitfalls, Patt...
[Luca/Van Campenhoudt] Microsoft Flow Beyone the Limits: Tips, Pitfalls, Patt...[Luca/Van Campenhoudt] Microsoft Flow Beyone the Limits: Tips, Pitfalls, Patt...
[Luca/Van Campenhoudt] Microsoft Flow Beyone the Limits: Tips, Pitfalls, Patt...
European Collaboration Summit
 
KoprowskiT_SQLRelay2014#3_Bristol_FromPlanToBackupToCloud
KoprowskiT_SQLRelay2014#3_Bristol_FromPlanToBackupToCloudKoprowskiT_SQLRelay2014#3_Bristol_FromPlanToBackupToCloud
KoprowskiT_SQLRelay2014#3_Bristol_FromPlanToBackupToCloud
Tobias Koprowski
 
Hadoop @ LifeWay
Hadoop @ LifeWayHadoop @ LifeWay
Hadoop @ LifeWay
jimforrester11
 
Xtending nintex workflow cloud w azure functions - xchange conference
Xtending nintex workflow cloud w azure functions - xchange conferenceXtending nintex workflow cloud w azure functions - xchange conference
Xtending nintex workflow cloud w azure functions - xchange conference
Michael Oryszak
 

Similar to HarvestingWebData-PowerQuery.pptx (20)

ITIL Foundation V3
ITIL Foundation V3ITIL Foundation V3
ITIL Foundation V3
 
Real-time SQL Access for Your Salesforce.com Data
Real-time SQL Access for Your Salesforce.com DataReal-time SQL Access for Your Salesforce.com Data
Real-time SQL Access for Your Salesforce.com Data
 
So you think you can scale
So you think you can scaleSo you think you can scale
So you think you can scale
 
From SQL to NoSQL: Structured Querying for JSON
From SQL to NoSQL: Structured Querying for JSONFrom SQL to NoSQL: Structured Querying for JSON
From SQL to NoSQL: Structured Querying for JSON
 
Accelerate your business with flow
Accelerate your business with flowAccelerate your business with flow
Accelerate your business with flow
 
Big Data & Oracle Technologies
Big Data & Oracle TechnologiesBig Data & Oracle Technologies
Big Data & Oracle Technologies
 
Integration intervention: Get your apps and data up to speed
Integration intervention: Get your apps and data up to speedIntegration intervention: Get your apps and data up to speed
Integration intervention: Get your apps and data up to speed
 
20140116 Tim Willoughby and James Fogarty FOSS in Local Government
20140116   Tim Willoughby and James Fogarty FOSS in Local Government20140116   Tim Willoughby and James Fogarty FOSS in Local Government
20140116 Tim Willoughby and James Fogarty FOSS in Local Government
 
Chapter deck january 2015
Chapter deck january 2015Chapter deck january 2015
Chapter deck january 2015
 
Denodo DataFest 2017: Business Needs for a Fast Data Strategy
Denodo DataFest 2017: Business Needs for a Fast Data StrategyDenodo DataFest 2017: Business Needs for a Fast Data Strategy
Denodo DataFest 2017: Business Needs for a Fast Data Strategy
 
KoprowskiT_SQLRelay2014#8_Birmingham_FromPlanToBackupToCloud
KoprowskiT_SQLRelay2014#8_Birmingham_FromPlanToBackupToCloudKoprowskiT_SQLRelay2014#8_Birmingham_FromPlanToBackupToCloud
KoprowskiT_SQLRelay2014#8_Birmingham_FromPlanToBackupToCloud
 
Don't simply deploy, transform! Build your digital workplace in Office 365
Don't simply deploy, transform! Build your digital workplace in Office 365Don't simply deploy, transform! Build your digital workplace in Office 365
Don't simply deploy, transform! Build your digital workplace in Office 365
 
Microsoft Flow advanced: tips, pitfalls, problems and warnings to be known be...
Microsoft Flow advanced: tips, pitfalls, problems and warnings to be known be...Microsoft Flow advanced: tips, pitfalls, problems and warnings to be known be...
Microsoft Flow advanced: tips, pitfalls, problems and warnings to be known be...
 
Ibm integration bus
Ibm integration busIbm integration bus
Ibm integration bus
 
SharePoint Online vs. On-Premise
SharePoint Online vs. On-PremiseSharePoint Online vs. On-Premise
SharePoint Online vs. On-Premise
 
Microsoft Flow best practices European Collaboration Summit 2018
Microsoft Flow best practices European Collaboration Summit 2018Microsoft Flow best practices European Collaboration Summit 2018
Microsoft Flow best practices European Collaboration Summit 2018
 
[Luca/Van Campenhoudt] Microsoft Flow Beyone the Limits: Tips, Pitfalls, Patt...
[Luca/Van Campenhoudt] Microsoft Flow Beyone the Limits: Tips, Pitfalls, Patt...[Luca/Van Campenhoudt] Microsoft Flow Beyone the Limits: Tips, Pitfalls, Patt...
[Luca/Van Campenhoudt] Microsoft Flow Beyone the Limits: Tips, Pitfalls, Patt...
 
KoprowskiT_SQLRelay2014#3_Bristol_FromPlanToBackupToCloud
KoprowskiT_SQLRelay2014#3_Bristol_FromPlanToBackupToCloudKoprowskiT_SQLRelay2014#3_Bristol_FromPlanToBackupToCloud
KoprowskiT_SQLRelay2014#3_Bristol_FromPlanToBackupToCloud
 
Hadoop @ LifeWay
Hadoop @ LifeWayHadoop @ LifeWay
Hadoop @ LifeWay
 
Xtending nintex workflow cloud w azure functions - xchange conference
Xtending nintex workflow cloud w azure functions - xchange conferenceXtending nintex workflow cloud w azure functions - xchange conference
Xtending nintex workflow cloud w azure functions - xchange conference
 

Recently uploaded

UiPath Test Automation using UiPath Test Suite series, part 3
UiPath Test Automation using UiPath Test Suite series, part 3UiPath Test Automation using UiPath Test Suite series, part 3
UiPath Test Automation using UiPath Test Suite series, part 3
DianaGray10
 
JMeter webinar - integration with InfluxDB and Grafana
JMeter webinar - integration with InfluxDB and GrafanaJMeter webinar - integration with InfluxDB and Grafana
JMeter webinar - integration with InfluxDB and Grafana
RTTS
 
Search and Society: Reimagining Information Access for Radical Futures
Search and Society: Reimagining Information Access for Radical FuturesSearch and Society: Reimagining Information Access for Radical Futures
Search and Society: Reimagining Information Access for Radical Futures
Bhaskar Mitra
 
When stars align: studies in data quality, knowledge graphs, and machine lear...
When stars align: studies in data quality, knowledge graphs, and machine lear...When stars align: studies in data quality, knowledge graphs, and machine lear...
When stars align: studies in data quality, knowledge graphs, and machine lear...
Elena Simperl
 
GraphRAG is All You need? LLM & Knowledge Graph
GraphRAG is All You need? LLM & Knowledge GraphGraphRAG is All You need? LLM & Knowledge Graph
GraphRAG is All You need? LLM & Knowledge Graph
Guy Korland
 
Securing your Kubernetes cluster: a step-by-step guide to success!
Securing your Kubernetes cluster: a step-by-step guide to success!Securing your Kubernetes cluster: a step-by-step guide to success!
Securing your Kubernetes cluster: a step-by-step guide to success!
KatiaHIMEUR1
 
From Siloed Products to Connected Ecosystem: Building a Sustainable and Scala...
From Siloed Products to Connected Ecosystem: Building a Sustainable and Scala...From Siloed Products to Connected Ecosystem: Building a Sustainable and Scala...
From Siloed Products to Connected Ecosystem: Building a Sustainable and Scala...
Product School
 
The Art of the Pitch: WordPress Relationships and Sales
The Art of the Pitch: WordPress Relationships and SalesThe Art of the Pitch: WordPress Relationships and Sales
The Art of the Pitch: WordPress Relationships and Sales
Laura Byrne
 
PHP Frameworks: I want to break free (IPC Berlin 2024)
PHP Frameworks: I want to break free (IPC Berlin 2024)PHP Frameworks: I want to break free (IPC Berlin 2024)
PHP Frameworks: I want to break free (IPC Berlin 2024)
Ralf Eggert
 
Kubernetes & AI - Beauty and the Beast !?! @KCD Istanbul 2024
Kubernetes & AI - Beauty and the Beast !?! @KCD Istanbul 2024Kubernetes & AI - Beauty and the Beast !?! @KCD Istanbul 2024
Kubernetes & AI - Beauty and the Beast !?! @KCD Istanbul 2024
Tobias Schneck
 
Connector Corner: Automate dynamic content and events by pushing a button
Connector Corner: Automate dynamic content and events by pushing a buttonConnector Corner: Automate dynamic content and events by pushing a button
Connector Corner: Automate dynamic content and events by pushing a button
DianaGray10
 
Transcript: Selling digital books in 2024: Insights from industry leaders - T...
Transcript: Selling digital books in 2024: Insights from industry leaders - T...Transcript: Selling digital books in 2024: Insights from industry leaders - T...
Transcript: Selling digital books in 2024: Insights from industry leaders - T...
BookNet Canada
 
Newntide latest company Introduction.pdf
Newntide latest company Introduction.pdfNewntide latest company Introduction.pdf
Newntide latest company Introduction.pdf
LucyLuo36
 
Bits & Pixels using AI for Good.........
Bits & Pixels using AI for Good.........Bits & Pixels using AI for Good.........
Bits & Pixels using AI for Good.........
Alison B. Lowndes
 
AI for Every Business: Unlocking Your Product's Universal Potential by VP of ...
AI for Every Business: Unlocking Your Product's Universal Potential by VP of ...AI for Every Business: Unlocking Your Product's Universal Potential by VP of ...
AI for Every Business: Unlocking Your Product's Universal Potential by VP of ...
Product School
 
Mission to Decommission: Importance of Decommissioning Products to Increase E...
Mission to Decommission: Importance of Decommissioning Products to Increase E...Mission to Decommission: Importance of Decommissioning Products to Increase E...
Mission to Decommission: Importance of Decommissioning Products to Increase E...
Product School
 
GenAISummit 2024 May 28 Sri Ambati Keynote: AGI Belongs to The Community in O...
GenAISummit 2024 May 28 Sri Ambati Keynote: AGI Belongs to The Community in O...GenAISummit 2024 May 28 Sri Ambati Keynote: AGI Belongs to The Community in O...
GenAISummit 2024 May 28 Sri Ambati Keynote: AGI Belongs to The Community in O...
Sri Ambati
 
UiPath Test Automation using UiPath Test Suite series, part 4
UiPath Test Automation using UiPath Test Suite series, part 4UiPath Test Automation using UiPath Test Suite series, part 4
UiPath Test Automation using UiPath Test Suite series, part 4
DianaGray10
 
How world-class product teams are winning in the AI era by CEO and Founder, P...
How world-class product teams are winning in the AI era by CEO and Founder, P...How world-class product teams are winning in the AI era by CEO and Founder, P...
How world-class product teams are winning in the AI era by CEO and Founder, P...
Product School
 
Free Complete Python - A step towards Data Science
Free Complete Python - A step towards Data ScienceFree Complete Python - A step towards Data Science
Free Complete Python - A step towards Data Science
RinaMondal9
 

Recently uploaded (20)

UiPath Test Automation using UiPath Test Suite series, part 3
UiPath Test Automation using UiPath Test Suite series, part 3UiPath Test Automation using UiPath Test Suite series, part 3
UiPath Test Automation using UiPath Test Suite series, part 3
 
JMeter webinar - integration with InfluxDB and Grafana
JMeter webinar - integration with InfluxDB and GrafanaJMeter webinar - integration with InfluxDB and Grafana
JMeter webinar - integration with InfluxDB and Grafana
 
Search and Society: Reimagining Information Access for Radical Futures
Search and Society: Reimagining Information Access for Radical FuturesSearch and Society: Reimagining Information Access for Radical Futures
Search and Society: Reimagining Information Access for Radical Futures
 
When stars align: studies in data quality, knowledge graphs, and machine lear...
When stars align: studies in data quality, knowledge graphs, and machine lear...When stars align: studies in data quality, knowledge graphs, and machine lear...
When stars align: studies in data quality, knowledge graphs, and machine lear...
 
GraphRAG is All You need? LLM & Knowledge Graph
GraphRAG is All You need? LLM & Knowledge GraphGraphRAG is All You need? LLM & Knowledge Graph
GraphRAG is All You need? LLM & Knowledge Graph
 
Securing your Kubernetes cluster: a step-by-step guide to success!
Securing your Kubernetes cluster: a step-by-step guide to success!Securing your Kubernetes cluster: a step-by-step guide to success!
Securing your Kubernetes cluster: a step-by-step guide to success!
 
From Siloed Products to Connected Ecosystem: Building a Sustainable and Scala...
From Siloed Products to Connected Ecosystem: Building a Sustainable and Scala...From Siloed Products to Connected Ecosystem: Building a Sustainable and Scala...
From Siloed Products to Connected Ecosystem: Building a Sustainable and Scala...
 
The Art of the Pitch: WordPress Relationships and Sales
The Art of the Pitch: WordPress Relationships and SalesThe Art of the Pitch: WordPress Relationships and Sales
The Art of the Pitch: WordPress Relationships and Sales
 
PHP Frameworks: I want to break free (IPC Berlin 2024)
PHP Frameworks: I want to break free (IPC Berlin 2024)PHP Frameworks: I want to break free (IPC Berlin 2024)
PHP Frameworks: I want to break free (IPC Berlin 2024)
 
Kubernetes & AI - Beauty and the Beast !?! @KCD Istanbul 2024
Kubernetes & AI - Beauty and the Beast !?! @KCD Istanbul 2024Kubernetes & AI - Beauty and the Beast !?! @KCD Istanbul 2024
Kubernetes & AI - Beauty and the Beast !?! @KCD Istanbul 2024
 
Connector Corner: Automate dynamic content and events by pushing a button
Connector Corner: Automate dynamic content and events by pushing a buttonConnector Corner: Automate dynamic content and events by pushing a button
Connector Corner: Automate dynamic content and events by pushing a button
 
Transcript: Selling digital books in 2024: Insights from industry leaders - T...
Transcript: Selling digital books in 2024: Insights from industry leaders - T...Transcript: Selling digital books in 2024: Insights from industry leaders - T...
Transcript: Selling digital books in 2024: Insights from industry leaders - T...
 
Newntide latest company Introduction.pdf
Newntide latest company Introduction.pdfNewntide latest company Introduction.pdf
Newntide latest company Introduction.pdf
 
Bits & Pixels using AI for Good.........
Bits & Pixels using AI for Good.........Bits & Pixels using AI for Good.........
Bits & Pixels using AI for Good.........
 
AI for Every Business: Unlocking Your Product's Universal Potential by VP of ...
AI for Every Business: Unlocking Your Product's Universal Potential by VP of ...AI for Every Business: Unlocking Your Product's Universal Potential by VP of ...
AI for Every Business: Unlocking Your Product's Universal Potential by VP of ...
 
Mission to Decommission: Importance of Decommissioning Products to Increase E...
Mission to Decommission: Importance of Decommissioning Products to Increase E...Mission to Decommission: Importance of Decommissioning Products to Increase E...
Mission to Decommission: Importance of Decommissioning Products to Increase E...
 
GenAISummit 2024 May 28 Sri Ambati Keynote: AGI Belongs to The Community in O...
GenAISummit 2024 May 28 Sri Ambati Keynote: AGI Belongs to The Community in O...GenAISummit 2024 May 28 Sri Ambati Keynote: AGI Belongs to The Community in O...
GenAISummit 2024 May 28 Sri Ambati Keynote: AGI Belongs to The Community in O...
 
UiPath Test Automation using UiPath Test Suite series, part 4
UiPath Test Automation using UiPath Test Suite series, part 4UiPath Test Automation using UiPath Test Suite series, part 4
UiPath Test Automation using UiPath Test Suite series, part 4
 
How world-class product teams are winning in the AI era by CEO and Founder, P...
How world-class product teams are winning in the AI era by CEO and Founder, P...How world-class product teams are winning in the AI era by CEO and Founder, P...
How world-class product teams are winning in the AI era by CEO and Founder, P...
 
Free Complete Python - A step towards Data Science
Free Complete Python - A step towards Data ScienceFree Complete Python - A step towards Data Science
Free Complete Python - A step towards Data Science
 

HarvestingWebData-PowerQuery.pptx

  • 2. JULIE SMITH AND ROB VOLK • DATA SERVICES CONSULTANT AT INNOVATIVE ARCHITECTS IN ATLANTA, GA • ONE HALF OF THE DATACHIX WITH AUDREY HAMMONDS • SQL SERVER MVP • ROB VOLK – NEEDS NO INTRODUCTION, SINCE HE’S NOT GONNA DO MUCH • EXCEPT COMPLAIN • AND XML STUFF • WHICH HE’S COME TO FIND OUT IS A DEATH SENTENCE FOR SESSION SUBMISSIONS
  • 3. JULIE SMITH AND ROB VOLK • ROB VOLK IS A DATABASE ADMINISTRATOR FROM THE ATLANTA, GEORGIA AREA, HAVING WORKED IN THE WIRELESS TELECOM, UTILITY BILLING, AND CREDIT CARD PROCESSING INDUSTRIES FOR THE PAST 10 YEARS. HE STARTED USING SQL SERVER WITH VERSION 6.5 AND IS VERY GLAD TO HAVE UPGRADED SINCE THEN. HE HAS PRESENTED AT SEVERAL SQL SATURDAYS AND USER GROUPS. IN HIS SPARE TIME HE ACCOMPLISHES MORE THAN HE DOES ON THE JOB, JUST DON'T TELL HIS BOSS.
  • 4. TODAY’S AGENDA • STORY OF HOW ANNOYED ROB WAS WITH PLANNING SQL SATURDAYS • HAS TO BE A BETTER WAY • FORAGING FOR DATA—ON SPEAKERS, VENUES, STRANGE COINCIDENCES • POWER QUERY –FIRST CAME OUT AS DATA EXPLORER • CURL • QUERYING XML WITH SQL (TRANSACT SQL TO BE EXACT)
  • 5. WHAT WOULD WE LIKE TO KNOW? • HOW MANY EVENTS? • WHERE ARE THEY? • SPEAKER DETAILS (MOST SESSIONS/EVENTS/TRAVELED) • SPONSOR DETAILS • DATES • UPCOMING EVENTS • IT’S NOT JUST FOR SATURDAYS!
  • 6. INTRODUCING POWER QUERY • RELEASED JULY 2013 • EXCEL ADD-IN FOR 2010 OR 2013 • SELF SERVICE ETL!!
  • 7. POWER BI SUITE • POWERPIVOT – IN MEMORY EMBEDDED DATA MODELS • POWERMAP • POWER VIEW – PRESENTATION READY VISUALIZATIONS • POWER QUERY • ALL IN EXCEL !!!!
  • 8. POWER BI • OFFICE 365 • DATA MANAGEMENT GATEWAY • COLLABORATE WITH OFFICE 365 SHAREPOINT
  • 9. HISTORY • M LANGUAGE FOR BUSINESS ANALYSTS—XAML • PROJECT OSLO • DOMAIN SPECIFIC LANGUAGE • M HAS MORPHED INTO THE POWER QUERY FOR EXCEL FORMULA LANGUAGE • DOWNLOAD A PDF HERE: HTTP://GO.MICROSOFT.COM/FWLINK/?LINKID=235475&CLCID=0X409 TABLE. PAGE 122
  • 10. JULIE WHEN SHE FIRST MET DATA EXPLORER…
  • 11. IT CAN HARVEST THE FOLLOWING: • “OTHER” • SHAREPOINT LIST, • WINDOWS AZURE MARKETPLACE, • ANY HADOOP FILE (HDFS), • HDINSIGHT, • ACTIVE DIRECTORY, • FACEBOOK. • RDBMS: SQL SERVER, AZURE SQL SERVER, ACCESS (OOPS), ORACLE, DB2, MYSQL, POSTGRESS, TERADATA • FILE: • EXCEL, CSV, TXT, , XML, JSON, FOLDER (METADATA) • FROM THE INTARWEBZ! • WRITE AN M OR POWER QUERY FORMULA LANGUAGE QUERY YOURSELF FROM
  • 12. SOURCES • REGULAR SOURCES—MOST DATABASE PLATFORMS INCLUDING HADOOP • FILES—ALL TYPES • ODATA • WEB SITE TABLES! • ACTIVE DIRECTORY!
  • 13. FUNCTIONS • TRANSFORMATION! • MERGE (JOIN ON KEY) • APPEND—COMBINE SOURCES TO NEW DESTINATION
  • 14. RESOURCES • HTTP://OFFICE.MICROSOFT.COM/EN-US/EXCEL-HELP/LEARN-ABOUT-DATA- EXPLORER-FORMULAS-HA104003958.ASPX FORMULA CHEAT SHEET • GREAT POST BY DATAPIG HTTP://DATAPIGTECHNOLOGIES.COM/BLOG/INDEX.PHP/COOL-THINGS-YOU- CAN-DO-WITH-DATA-EXPLORER/ • ON M LANGUAGE: HTTP://BLOGS.MSDN.COM/B/MODELCITIZEN/ARCHIVE/2010/09/22/UPDATE- ON-SQL-SERVER-MODELING-CTP-REPOSITORY-MODELING-SERVICES-QUOT- QUADRANT-QUOT-AND-QUOT-M-QUOT.ASPX • M LANGUAGE SPECS FROM MICROSOFT: HTTP://MSDN.MICROSOFT.COM/EN- US/LIBRARY/DD285271.ASPX • HTTP://BLOGS.MSDN.COM/B/MLANGUAGE/ • DOWNLOADABLE 200 PAGE DOC ON FORMULA LANGUAGE (M) HTTP://GO.MICROSOFT.COM/FWLINK/?LINKID=235474&CLCID=0X409