Successfully reported this slideshow.
Your SlideShare is downloading. ×

Pipelines and Data Flows: Introduction to Data Integration in Azure Synapse Analytics (Global Azure Norway 2021)

Pipelines and Data Flows: Introduction to Data Integration in Azure Synapse Analytics (Global Azure Norway 2021)

Download to read offline

Pipelines and Data Flows: Introduction to Data Integration in Azure Synapse Analytics (Presented at Global Azure Norway on April 16th, 2021)

Pipelines and Data Flows: Introduction to Data Integration in Azure Synapse Analytics (Presented at Global Azure Norway on April 16th, 2021)

More Related Content

More from Cathrine Wilhelmsen

Pipelines and Data Flows: Introduction to Data Integration in Azure Synapse Analytics (Global Azure Norway 2021)

  1. 1. © 2021 Cathrine Wilhelmsen (hi@cathrinew.net) Pipelines & Data Flows: Introduction to Data Integration in Azure Synapse Analytics Cathrine Wilhelmsen Global Azure Norway · April 16th, 2021
  2. 2. © 2021 Cathrine Wilhelmsen (hi@cathrinew.net) Session Abstract Do you regularly need to get data for your projects? Yep! 🙋‍♀️ Data is at the core of every Business Intelligence, Data Science, and Machine Learning project. You need data to understand what has happened in the past, to predict what may happen in the future, to discover patterns and anomalies, and to gain the insight necessary for making faster and better decisions. But before you can do any of those things, you need to ingest, store, transform, integrate, and prepare your data. Guess what? You can do all of those things in Azure Synapse Analytics – without having to write any code! In this session, we will cover the fundamentals of data integration in Azure Synapse Analytics. First, we will discuss when Azure Synapse Analytics is the right tool of choice. Then, we will go through what Pipelines and Data Flows are, and when to use them. Finally, we will see how easy it is to ingest and transform data both on-premises and in the cloud.
  3. 3. © 2021 Cathrine Wilhelmsen (hi@cathrinew.net) @cathrinew cathrinew.net
  4. 4. © 2020 Cathrine Wilhelmsen (hi@cathrinew.net) Data Warehousing Business Intelligence Artificial Intelligence Big Data and Analytics Machine Learning Data Science
  5. 5. © 2020 Cathrine Wilhelmsen (hi@cathrinew.net) Data Warehousing Business Intelligence Artificial Intelligence Big Data and Analytics Machine Learning Data Science
  6. 6. © 2020 Cathrine Wilhelmsen (hi@cathrinew.net) What? When? Why?
  7. 7. © 2020 Cathrine Wilhelmsen (hi@cathrinew.net) Ingest Store Transform Integrate Prepare
  8. 8. © 2020 Cathrine Wilhelmsen (hi@cathrinew.net)
  9. 9. © 2021 Cathrine Wilhelmsen (hi@cathrinew.net) Azure Synapse Analytics What? Pipelines & Data Flows How? Data Integration When? …the next 45 minutes…
  10. 10. Azure Synapse Analytics
  11. 11. © 2021 Cathrine Wilhelmsen (hi@cathrinew.net) What is Azure Synapse Analytics? All-in-one platform for analytical projects: • Data Lake (All Data) • Data Warehouse (Relational Data) • Data Analytics (SQL, Spark) • Data Integration (Ingest, Transform)
  12. 12. © 2021 Cathrine Wilhelmsen (hi@cathrinew.net) What is Azure Synapse Analytics? Deeply integrated with other services: • Azure Purview • Azure Machine Learning • Azure Cosmos DB • Power BI
  13. 13. © 2021 Cathrine Wilhelmsen (hi@cathrinew.net) Who can use Azure Synapse Analytics? Built for collaboration between: • Data Engineers • Data Analysts • Data Scientists • Data Consumers
  14. 14. © 2021 Cathrine Wilhelmsen (hi@cathrinew.net) Integration Data
  15. 15. © 2021 Cathrine Wilhelmsen (hi@cathrinew.net) Data Integration in Azure Synapse Analytics Code-First Scripts, Notebooks Designer-First Pipelines, Data Flows
  16. 16. © 2021 Cathrine Wilhelmsen (hi@cathrinew.net) Data Integration in Azure Synapse Analytics Copy Data Transform Data
  17. 17. Pipelines & Data Flows
  18. 18. © 2021 Cathrine Wilhelmsen (hi@cathrinew.net) What are Pipelines?
  19. 19. © 2021 Cathrine Wilhelmsen (hi@cathrinew.net) What are Activities?
  20. 20. © 2021 Cathrine Wilhelmsen (hi@cathrinew.net) What are Datasets?
  21. 21. © 2021 Cathrine Wilhelmsen (hi@cathrinew.net) What are Linked Services?
  22. 22. © 2021 Cathrine Wilhelmsen (hi@cathrinew.net) What are Data Flows?
  23. 23. © 2021 Cathrine Wilhelmsen (hi@cathrinew.net) What are Triggers?
  24. 24. DEMO DEMO Pipelines & Data Flows
  25. 25. © 2021 Cathrine Wilhelmsen (hi@cathrinew.net)
  26. 26. © 2021 Cathrine Wilhelmsen (hi@cathrinew.net) Code-First Scripts, Notebooks Designer-First Pipelines, Data Flows
  27. 27. ? ?
  28. 28. © 2021 Cathrine Wilhelmsen (hi@cathrinew.net) Azure Synapse or Azure Data Factory?
  29. 29. © 2021 Cathrine Wilhelmsen (hi@cathrinew.net) @cathrinew cathrinew.net hi@cathrinew.net cathrinew.net/adf

×