Azure Synapse is the evolution of Azure SQL Data Warehouse, combining big data, data storage and data integration into a single service for end-to-end cloud scale analytics. It provides unlimited analytics with unparalleled speed to gain insights. Azure Synapse brings together enterprise data warehousing and big data analytics to give a unified experience with the advantages of both worlds.
Azure Synapse Analytics is Azure SQL Data Warehouse evolved: a limitless analytics service, that brings together enterprise data warehousing and Big Data analytics into a single service. It gives you the freedom to query data on your terms, using either serverless on-demand or provisioned resources, at scale. Azure Synapse brings these two worlds together with a unified experience to ingest, prepare, manage, and serve data for immediate business intelligence and machine learning needs. This is a huge deck with lots of screenshots so you can see exactly how it works.
Azure Synapse Analytics is Azure SQL Data Warehouse evolved: a limitless analytics service, that brings together enterprise data warehousing and Big Data analytics into a single service. It gives you the freedom to query data on your terms, using either serverless on-demand or provisioned resources, at scale. Azure Synapse brings these two worlds together with a unified experience to ingest, prepare, manage, and serve data for immediate business intelligence and machine learning needs. This is a huge deck with lots of screenshots so you can see exactly how it works.
Databricks is a Software-as-a-Service-like experience (or Spark-as-a-service) that is a tool for curating and processing massive amounts of data and developing, training and deploying models on that data, and managing the whole workflow process throughout the project. It is for those who are comfortable with Apache Spark as it is 100% based on Spark and is extensible with support for Scala, Java, R, and Python alongside Spark SQL, GraphX, Streaming and Machine Learning Library (Mllib). It has built-in integration with many data sources, has a workflow scheduler, allows for real-time workspace collaboration, and has performance improvements over traditional Apache Spark.
The new Microsoft Azure SQL Data Warehouse (SQL DW) is an elastic data warehouse-as-a-service and is a Massively Parallel Processing (MPP) solution for "big data" with true enterprise class features. The SQL DW service is built for data warehouse workloads from a few hundred gigabytes to petabytes of data with truly unique features like disaggregated compute and storage allowing for customers to be able to utilize the service to match their needs. In this presentation, we take an in-depth look at implementing a SQL DW, elastic scale (grow, shrink, and pause), and hybrid data clouds with Hadoop integration via Polybase allowing for a true SQL experience across structured and unstructured data.
In this presentation, we will do assess the on-premises environment and determining what workloads and databases are ready to make the move and what can you do to improve their Azure readiness while reducing downtime during the migration. Planning and assessment plays a critical role in moving to the cloud. We would see wide range of resources and tools to get an assessment completed with ease while identifying workload dependencies with practical tips and tricks focusing on sizing and costs. And finally, we’ll assess the SQL instances and identify their readiness for Azure as well.
Azure Synapse Analytics is Azure SQL Data Warehouse evolved: a limitless analytics service, that brings together enterprise data warehousing and Big Data analytics into a single service. It gives you the freedom to query data on your terms, using either serverless on-demand or provisioned resources, at scale. Azure Synapse brings these two worlds together with a unified experience to ingest, prepare, manage, and serve data for immediate business intelligence and machine learning needs. This is a huge deck with lots of screenshots so you can see exactly how it works.
Azure Synapse Analytics is Azure SQL Data Warehouse evolved: a limitless analytics service, that brings together enterprise data warehousing and Big Data analytics into a single service. It gives you the freedom to query data on your terms, using either serverless on-demand or provisioned resources, at scale. Azure Synapse brings these two worlds together with a unified experience to ingest, prepare, manage, and serve data for immediate business intelligence and machine learning needs. This is a huge deck with lots of screenshots so you can see exactly how it works.
Databricks is a Software-as-a-Service-like experience (or Spark-as-a-service) that is a tool for curating and processing massive amounts of data and developing, training and deploying models on that data, and managing the whole workflow process throughout the project. It is for those who are comfortable with Apache Spark as it is 100% based on Spark and is extensible with support for Scala, Java, R, and Python alongside Spark SQL, GraphX, Streaming and Machine Learning Library (Mllib). It has built-in integration with many data sources, has a workflow scheduler, allows for real-time workspace collaboration, and has performance improvements over traditional Apache Spark.
The new Microsoft Azure SQL Data Warehouse (SQL DW) is an elastic data warehouse-as-a-service and is a Massively Parallel Processing (MPP) solution for "big data" with true enterprise class features. The SQL DW service is built for data warehouse workloads from a few hundred gigabytes to petabytes of data with truly unique features like disaggregated compute and storage allowing for customers to be able to utilize the service to match their needs. In this presentation, we take an in-depth look at implementing a SQL DW, elastic scale (grow, shrink, and pause), and hybrid data clouds with Hadoop integration via Polybase allowing for a true SQL experience across structured and unstructured data.
In this presentation, we will do assess the on-premises environment and determining what workloads and databases are ready to make the move and what can you do to improve their Azure readiness while reducing downtime during the migration. Planning and assessment plays a critical role in moving to the cloud. We would see wide range of resources and tools to get an assessment completed with ease while identifying workload dependencies with practical tips and tricks focusing on sizing and costs. And finally, we’ll assess the SQL instances and identify their readiness for Azure as well.
Part 3 - Modern Data Warehouse with Azure SynapseNilesh Gule
Slide deck of the third part of building Modern Data Warehouse using Azure. This session covered Azure Synapse, formerly SQL Data Warehouse. We look at the Azure Synapse Architecture, external files, integration with Azuer Data Factory.
The recording of the session is available on YouTube
https://www.youtube.com/watch?v=LZlu6_rFzm8&WT.mc_id=DP-MVP-5003170
Achieving Lakehouse Models with Spark 3.0Databricks
It’s very easy to be distracted by the latest and greatest approaches with technology, but sometimes there’s a reason old approaches stand the test of time. Star Schemas & Kimball is one of those things that isn’t going anywhere, but as we move towards the “Data Lakehouse” paradigm – how appropriate is this modelling technique, and how can we harness the Delta Engine & Spark 3.0 to maximise it’s performance?
Want to see a high-level overview of the products in the Microsoft data platform portfolio in Azure? I’ll cover products in the categories of OLTP, OLAP, data warehouse, storage, data transport, data prep, data lake, IaaS, PaaS, SMP/MPP, NoSQL, Hadoop, open source, reporting, machine learning, and AI. It’s a lot to digest but I’ll categorize the products and discuss their use cases to help you narrow down the best products for the solution you want to build.
[DSC Europe 22] Lakehouse architecture with Delta Lake and Databricks - Draga...DataScienceConferenc1
Dragan Berić will take a deep dive into Lakehouse architecture, a game-changing concept bridging the best elements of data lake and data warehouse. The presentation will focus on the Delta Lake format as the foundation of the Lakehouse philosophy, and Databricks as the primary platform for its implementation.
This one-hour presentation covers the tools and techniques for migrating SQL Server databases and data to Azure SQL DB or SQL Server on VM. Includes SSMA, DMA, DMS, and more.
This migration plan aims to explore the potential of migrating from on-premises Hadoop to Azure Databricks. By leveraging Databricks' scalability, performance, collaboration, and advanced analytics capabilities, organizations can unlock faster insights and facilitate data-driven decision-making.
Large Scale Lakehouse Implementation Using Structured StreamingDatabricks
Business leads, executives, analysts, and data scientists rely on up-to-date information to make business decision, adjust to the market, meet needs of their customers or run effective supply chain operations.
Come hear how Asurion used Delta, Structured Streaming, AutoLoader and SQL Analytics to improve production data latency from day-minus-one to near real time Asurion’s technical team will share battle tested tips and tricks you only get with certain scale. Asurion data lake executes 4000+ streaming jobs and hosts over 4000 tables in production Data Lake on AWS.
Lambda Architecture in the Cloud with Azure Databricks with Andrei VaranovichDatabricks
The term “Lambda Architecture” stands for a generic, scalable and fault-tolerant data processing architecture. As the hyper-scale now offers a various PaaS services for data ingestion, storage and processing, the need for a revised, cloud-native implementation of the lambda architecture is arising.
In this talk we demonstrate the blueprint for such an implementation in Microsoft Azure, with Azure Databricks — a PaaS Spark offering – as a key component. We go back to some core principles of functional programming and link them to the capabilities of Apache Spark for various end-to-end big data analytics scenarios.
We also illustrate the “Lambda architecture in use” and the associated tread-offs using the real customer scenario – Rijksmuseum in Amsterdam – a terabyte-scale Azure-based data platform handles data from 2.500.000 visitors per year.
Big Data and Data Warehousing Together with Azure Synapse Analytics (SQLBits ...Michael Rys
SQLBits 2020 presentation on how you can build solutions based on the modern data warehouse pattern with Azure Synapse Spark and SQL including demos of Azure Synapse.
This presentation is for those of you who are interested in moving your on-prem SQL Server databases and servers to Azure virtual machines (VM’s) in the cloud so you can take advantage of all the benefits of being in the cloud. This is commonly referred to as a “lift and shift” as part of an Infrastructure-as-a-service (IaaS) solution. I will discuss the various Azure VM sizes and options, migration strategies, storage options, high availability (HA) and disaster recovery (DR) solutions, and best practices.
Delta Lake OSS: Create reliable and performant Data Lake by Quentin AmbardParis Data Engineers !
Delta Lake is an open source framework living on top of parquet in your data lake to provide Reliability and performances. It has been open-sourced by Databricks this year and is gaining traction to become the defacto delta lake format.
We’ll see all the goods Delta Lake can do to your data with ACID transactions, DDL operations, Schema enforcement, batch and stream support etc !
Part 3 - Modern Data Warehouse with Azure SynapseNilesh Gule
Slide deck of the third part of building Modern Data Warehouse using Azure. This session covered Azure Synapse, formerly SQL Data Warehouse. We look at the Azure Synapse Architecture, external files, integration with Azuer Data Factory.
The recording of the session is available on YouTube
https://www.youtube.com/watch?v=LZlu6_rFzm8&WT.mc_id=DP-MVP-5003170
Achieving Lakehouse Models with Spark 3.0Databricks
It’s very easy to be distracted by the latest and greatest approaches with technology, but sometimes there’s a reason old approaches stand the test of time. Star Schemas & Kimball is one of those things that isn’t going anywhere, but as we move towards the “Data Lakehouse” paradigm – how appropriate is this modelling technique, and how can we harness the Delta Engine & Spark 3.0 to maximise it’s performance?
Want to see a high-level overview of the products in the Microsoft data platform portfolio in Azure? I’ll cover products in the categories of OLTP, OLAP, data warehouse, storage, data transport, data prep, data lake, IaaS, PaaS, SMP/MPP, NoSQL, Hadoop, open source, reporting, machine learning, and AI. It’s a lot to digest but I’ll categorize the products and discuss their use cases to help you narrow down the best products for the solution you want to build.
[DSC Europe 22] Lakehouse architecture with Delta Lake and Databricks - Draga...DataScienceConferenc1
Dragan Berić will take a deep dive into Lakehouse architecture, a game-changing concept bridging the best elements of data lake and data warehouse. The presentation will focus on the Delta Lake format as the foundation of the Lakehouse philosophy, and Databricks as the primary platform for its implementation.
This one-hour presentation covers the tools and techniques for migrating SQL Server databases and data to Azure SQL DB or SQL Server on VM. Includes SSMA, DMA, DMS, and more.
This migration plan aims to explore the potential of migrating from on-premises Hadoop to Azure Databricks. By leveraging Databricks' scalability, performance, collaboration, and advanced analytics capabilities, organizations can unlock faster insights and facilitate data-driven decision-making.
Large Scale Lakehouse Implementation Using Structured StreamingDatabricks
Business leads, executives, analysts, and data scientists rely on up-to-date information to make business decision, adjust to the market, meet needs of their customers or run effective supply chain operations.
Come hear how Asurion used Delta, Structured Streaming, AutoLoader and SQL Analytics to improve production data latency from day-minus-one to near real time Asurion’s technical team will share battle tested tips and tricks you only get with certain scale. Asurion data lake executes 4000+ streaming jobs and hosts over 4000 tables in production Data Lake on AWS.
Lambda Architecture in the Cloud with Azure Databricks with Andrei VaranovichDatabricks
The term “Lambda Architecture” stands for a generic, scalable and fault-tolerant data processing architecture. As the hyper-scale now offers a various PaaS services for data ingestion, storage and processing, the need for a revised, cloud-native implementation of the lambda architecture is arising.
In this talk we demonstrate the blueprint for such an implementation in Microsoft Azure, with Azure Databricks — a PaaS Spark offering – as a key component. We go back to some core principles of functional programming and link them to the capabilities of Apache Spark for various end-to-end big data analytics scenarios.
We also illustrate the “Lambda architecture in use” and the associated tread-offs using the real customer scenario – Rijksmuseum in Amsterdam – a terabyte-scale Azure-based data platform handles data from 2.500.000 visitors per year.
Big Data and Data Warehousing Together with Azure Synapse Analytics (SQLBits ...Michael Rys
SQLBits 2020 presentation on how you can build solutions based on the modern data warehouse pattern with Azure Synapse Spark and SQL including demos of Azure Synapse.
This presentation is for those of you who are interested in moving your on-prem SQL Server databases and servers to Azure virtual machines (VM’s) in the cloud so you can take advantage of all the benefits of being in the cloud. This is commonly referred to as a “lift and shift” as part of an Infrastructure-as-a-service (IaaS) solution. I will discuss the various Azure VM sizes and options, migration strategies, storage options, high availability (HA) and disaster recovery (DR) solutions, and best practices.
Delta Lake OSS: Create reliable and performant Data Lake by Quentin AmbardParis Data Engineers !
Delta Lake is an open source framework living on top of parquet in your data lake to provide Reliability and performances. It has been open-sourced by Databricks this year and is gaining traction to become the defacto delta lake format.
We’ll see all the goods Delta Lake can do to your data with ACID transactions, DDL operations, Schema enforcement, batch and stream support etc !
At our March Data Analytics Meetup, Dan Rodriguez and Cherian Mathew demonstrated the variations in Microsoft Azure programs and how they are impacting digital transformation.
This webinar by Volodymyr Trishyn (Senior Software Engineer, Consultant, GlobalLogic) was delivered at On Air webinar #15 on July 31, 2020.
Webinar agenda:
- SQL Database
- Azure SQL Data Warehouse
- Azure SQL Elastic Database Pool
- Geo-replication
- Distributed Transactions
- Transaction Isolation Level
- Table Partitioning
- Materialized View Pattern
More details and presentation: https://www.globallogic.com/ua/about/events/webinar-azure-sql/
Self-serve analytics journey at Celtra: Snowflake, Spark, and DatabricksGrega Kespret
Celtra provides a platform for streamlined ad creation and campaign management used by customers including Porsche, Taco Bell, and Fox to create, track, and analyze their digital display advertising. Celtra’s platform processes billions of ad events daily to give analysts fast and easy access to reports and ad hoc analytics. Celtra’s Grega Kešpret leads a technical dive into Celtra’s data-pipeline challenges and explains how it solved them by combining Snowflake’s cloud data warehouse with Spark to get the best of both.
Topics include:
- Why Celtra changed its pipeline, materializing session representations to eliminate the need to rerun its pipeline
- How and why it decided to use Snowflake rather than an alternative data warehouse or a home-grown custom solution
- How Snowflake complemented the existing Spark environment with the ability to store and analyze deeply nested data with full consistency
- How Snowflake + Spark enables production and ad hoc analytics on a single repository of data
Ai big dataconference_eugene_polonichko_azure data lake Olga Zinkevych
Topic of presentation: Azure Data Lake: what is it? why is it? where is it?
The main points of the presentation:
What is Azure Data Lake? Why does this technology call Microsoft Big Data? Azure Data Lake includes all the capabilities required to make it easy for developers, data scientists, and analysts to store data of any size, shape, and speed, and do all types of processing and analytics across platforms and languages. It removes the complexities of ingesting and storing all of your data while making it faster to get up and running with batch, streaming, and interactive analytics.
http://dataconf.com.ua/index.php#agenda
#dataconf
#AIBDConference
Slides: Proven Strategies for Hybrid Cloud Computing with Mainframes — From A...DATAVERSITY
Mainframes continue to perform mission-critical transaction processing and contain massive amounts of core business data. But digital transformation initiatives and cloud computing have created both opportunities and challenges for unlocking and utilizing this data. Qlik and AWS will share some of the proven strategies from successful customer deployments across a range of different mainframe to cloud use cases, including legacy application modernization, data analytics, and data migrations.
In this presentation, you will learn how to:
• Replicate very large volumes of mainframe data in real-time to the cloud
• Automate the creation of analytics-ready data lakes and data warehouses
• Achieve a 30% reduction in cost of compute
This was a very interesting conference, TIC students oriented where I take him to the azure ecosystem for data warehousing architecture and best practices to reach powerful Business Intelligence Solutions according to the new era
Oracle Business Intelligence is a product of Oracle Corporation. It is a Data Warehousing BI tool. It is very user friendly. OBIEE is one of the most emerging reporting tools ever since Oracle has taken over Siebel. In the coming days there are going to be many existing and new projects that will be migrated to Obiee from their existing reporting tools. And above all Obiee is very easy to learn and fun to learn, you do not need coding skills, but just a little logic and familiarity with the tool. And we are training on version 11g (11.1.1.6) which is a new release.
Streaming Real-time Data to Azure Data Lake Storage Gen 2Carole Gunst
Check out this presentation to learn the basics of using Attunity Replicate to stream real-time data to Azure Data Lake Storage Gen2 for analytics projects.
Dans cette session nous vous présenterons les différentes manières d'utiliser SQL Server dans une infrastructure Cloud (Microsoft Azure). Seront présentés des scénarios hybrides, de migration, de backup, et d'hébergement de bases de données SQL Server en mode IaaS ou PaaS.
How to Use a Semantic Layer on Big Data to Drive AI & BI ImpactDATAVERSITY
Learn about using a semantic layer to make data accessible and how to accelerate the business impact of AI and BI at your organization.
This session will offer practical advice on how to drive AI & BI business outcomes with an effective data strategy that leverages a semantic layer.
You will learn how to achieve quantifiable results by modernizing your data and analytics stack with a semantic layer that delivers an order of magnitude better query performance, increased data team productivity, lower query compute costs, and improved Speed-to-Insights.
Attend this session to learn about:
- Gaining business alignment and reducing data prep for your AI and BI teams.
- Making a consistent set of business metrics “analytics-ready” and accessible.
- Accelerating end-to-end query performance while optimizing cloud resources.
- Treating “data as a product” and how to drive business value for all consumers.
Machine Learning con Azure Managed InstanceEduardo Castro
En esta presentación mostramos las opciones para implementar Machine Learning dentro de Azure, así como las formas de configurar y utilizar Python dentro de Azure Managed Instance
Software Delivery At the Speed of AI: Inflectra Invests In AI-Powered QualityInflectra
In this insightful webinar, Inflectra explores how artificial intelligence (AI) is transforming software development and testing. Discover how AI-powered tools are revolutionizing every stage of the software development lifecycle (SDLC), from design and prototyping to testing, deployment, and monitoring.
Learn about:
• The Future of Testing: How AI is shifting testing towards verification, analysis, and higher-level skills, while reducing repetitive tasks.
• Test Automation: How AI-powered test case generation, optimization, and self-healing tests are making testing more efficient and effective.
• Visual Testing: Explore the emerging capabilities of AI in visual testing and how it's set to revolutionize UI verification.
• Inflectra's AI Solutions: See demonstrations of Inflectra's cutting-edge AI tools like the ChatGPT plugin and Azure Open AI platform, designed to streamline your testing process.
Whether you're a developer, tester, or QA professional, this webinar will give you valuable insights into how AI is shaping the future of software delivery.
JMeter webinar - integration with InfluxDB and GrafanaRTTS
Watch this recorded webinar about real-time monitoring of application performance. See how to integrate Apache JMeter, the open-source leader in performance testing, with InfluxDB, the open-source time-series database, and Grafana, the open-source analytics and visualization application.
In this webinar, we will review the benefits of leveraging InfluxDB and Grafana when executing load tests and demonstrate how these tools are used to visualize performance metrics.
Length: 30 minutes
Session Overview
-------------------------------------------
During this webinar, we will cover the following topics while demonstrating the integrations of JMeter, InfluxDB and Grafana:
- What out-of-the-box solutions are available for real-time monitoring JMeter tests?
- What are the benefits of integrating InfluxDB and Grafana into the load testing stack?
- Which features are provided by Grafana?
- Demonstration of InfluxDB and Grafana using a practice web application
To view the webinar recording, go to:
https://www.rttsweb.com/jmeter-integration-webinar
Search and Society: Reimagining Information Access for Radical FuturesBhaskar Mitra
The field of Information retrieval (IR) is currently undergoing a transformative shift, at least partly due to the emerging applications of generative AI to information access. In this talk, we will deliberate on the sociotechnical implications of generative AI for information access. We will argue that there is both a critical necessity and an exciting opportunity for the IR community to re-center our research agendas on societal needs while dismantling the artificial separation between the work on fairness, accountability, transparency, and ethics in IR and the rest of IR research. Instead of adopting a reactionary strategy of trying to mitigate potential social harms from emerging technologies, the community should aim to proactively set the research agenda for the kinds of systems we should build inspired by diverse explicitly stated sociotechnical imaginaries. The sociotechnical imaginaries that underpin the design and development of information access technologies needs to be explicitly articulated, and we need to develop theories of change in context of these diverse perspectives. Our guiding future imaginaries must be informed by other academic fields, such as democratic theory and critical theory, and should be co-developed with social science scholars, legal scholars, civil rights and social justice activists, and artists, among others.
Kubernetes & AI - Beauty and the Beast !?! @KCD Istanbul 2024Tobias Schneck
As AI technology is pushing into IT I was wondering myself, as an “infrastructure container kubernetes guy”, how get this fancy AI technology get managed from an infrastructure operational view? Is it possible to apply our lovely cloud native principals as well? What benefit’s both technologies could bring to each other?
Let me take this questions and provide you a short journey through existing deployment models and use cases for AI software. On practical examples, we discuss what cloud/on-premise strategy we may need for applying it to our own infrastructure to get it to work from an enterprise perspective. I want to give an overview about infrastructure requirements and technologies, what could be beneficial or limiting your AI use cases in an enterprise environment. An interactive Demo will give you some insides, what approaches I got already working for real.
GraphRAG is All You need? LLM & Knowledge GraphGuy Korland
Guy Korland, CEO and Co-founder of FalkorDB, will review two articles on the integration of language models with knowledge graphs.
1. Unifying Large Language Models and Knowledge Graphs: A Roadmap.
https://arxiv.org/abs/2306.08302
2. Microsoft Research's GraphRAG paper and a review paper on various uses of knowledge graphs:
https://www.microsoft.com/en-us/research/blog/graphrag-unlocking-llm-discovery-on-narrative-private-data/
Connector Corner: Automate dynamic content and events by pushing a buttonDianaGray10
Here is something new! In our next Connector Corner webinar, we will demonstrate how you can use a single workflow to:
Create a campaign using Mailchimp with merge tags/fields
Send an interactive Slack channel message (using buttons)
Have the message received by managers and peers along with a test email for review
But there’s more:
In a second workflow supporting the same use case, you’ll see:
Your campaign sent to target colleagues for approval
If the “Approve” button is clicked, a Jira/Zendesk ticket is created for the marketing design team
But—if the “Reject” button is pushed, colleagues will be alerted via Slack message
Join us to learn more about this new, human-in-the-loop capability, brought to you by Integration Service connectors.
And...
Speakers:
Akshay Agnihotri, Product Manager
Charlie Greenberg, Host
Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...UiPathCommunity
💥 Speed, accuracy, and scaling – discover the superpowers of GenAI in action with UiPath Document Understanding and Communications Mining™:
See how to accelerate model training and optimize model performance with active learning
Learn about the latest enhancements to out-of-the-box document processing – with little to no training required
Get an exclusive demo of the new family of UiPath LLMs – GenAI models specialized for processing different types of documents and messages
This is a hands-on session specifically designed for automation developers and AI enthusiasts seeking to enhance their knowledge in leveraging the latest intelligent document processing capabilities offered by UiPath.
Speakers:
👨🏫 Andras Palfi, Senior Product Manager, UiPath
👩🏫 Lenka Dulovicova, Product Program Manager, UiPath
Transcript: Selling digital books in 2024: Insights from industry leaders - T...BookNet Canada
The publishing industry has been selling digital audiobooks and ebooks for over a decade and has found its groove. What’s changed? What has stayed the same? Where do we go from here? Join a group of leading sales peers from across the industry for a conversation about the lessons learned since the popularization of digital books, best practices, digital book supply chain management, and more.
Link to video recording: https://bnctechforum.ca/sessions/selling-digital-books-in-2024-insights-from-industry-leaders/
Presented by BookNet Canada on May 28, 2024, with support from the Department of Canadian Heritage.
Builder.ai Founder Sachin Dev Duggal's Strategic Approach to Create an Innova...Ramesh Iyer
In today's fast-changing business world, Companies that adapt and embrace new ideas often need help to keep up with the competition. However, fostering a culture of innovation takes much work. It takes vision, leadership and willingness to take risks in the right proportion. Sachin Dev Duggal, co-founder of Builder.ai, has perfected the art of this balance, creating a company culture where creativity and growth are nurtured at each stage.
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...DanBrown980551
Do you want to learn how to model and simulate an electrical network from scratch in under an hour?
Then welcome to this PowSyBl workshop, hosted by Rte, the French Transmission System Operator (TSO)!
During the webinar, you will discover the PowSyBl ecosystem as well as handle and study an electrical network through an interactive Python notebook.
PowSyBl is an open source project hosted by LF Energy, which offers a comprehensive set of features for electrical grid modelling and simulation. Among other advanced features, PowSyBl provides:
- A fully editable and extendable library for grid component modelling;
- Visualization tools to display your network;
- Grid simulation tools, such as power flows, security analyses (with or without remedial actions) and sensitivity analyses;
The framework is mostly written in Java, with a Python binding so that Python developers can access PowSyBl functionalities as well.
What you will learn during the webinar:
- For beginners: discover PowSyBl's functionalities through a quick general presentation and the notebook, without needing any expert coding skills;
- For advanced developers: master the skills to efficiently apply PowSyBl functionalities to your real-world scenarios.
Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...Jeffrey Haguewood
Sidekick Solutions uses Bonterra Impact Management (fka Social Solutions Apricot) and automation solutions to integrate data for business workflows.
We believe integration and automation are essential to user experience and the promise of efficient work through technology. Automation is the critical ingredient to realizing that full vision. We develop integration products and services for Bonterra Case Management software to support the deployment of automations for a variety of use cases.
This video focuses on the notifications, alerts, and approval requests using Slack for Bonterra Impact Management. The solutions covered in this webinar can also be deployed for Microsoft Teams.
Interested in deploying notification automations for Bonterra Impact Management? Contact us at sales@sidekicksolutionsllc.com to discuss next steps.
FIDO Alliance Osaka Seminar: The WebAuthn API and Discoverable Credentials.pdf
Data warehouse con azure synapse analytics
1. 08-May-20 7:12 AM
1
Azure Synapse es la evolución de Azure SQL Data Warehouse,
combinando big data, almacenamiento de datos e integración de datos
en un único servicio para análisis de extremo a extremo a escala de nube.
Azure Synapse Analytics
Servicio de análisis ilimitado con un tiempo inigualable para obtener información
2. 08-May-20 7:12 AM
2
INGEST
Data warehouse moderno
PREPARE TRANSFORM
& ENRICH
SERVE
STORE
VISUALIZE
On-premises data
Cloud data
SaaS data
Integrated data platform for BI, AI and continuous intelligence
Platform
Azure
Data Lake Storage
Common Data Model
Enterprise Security
Optimized for Analytics
METASTORE
SECURITY
MANAGEMENT
MONITORING
DATA INTEGRATION
Analytics Runtimes
PROVISIONED ON-DEMAND
Form Factors
SQL
Languages
Python .NET Java Scala R
Experience Synapse Analytics Studio
Artificial Intelligence / Machine Learning / Internet of Things
Intelligent Apps / Business Intelligence
3. 08-May-20 7:12 AM
3
Plataforma de datos integrada para BI, IA e inteligencia continua
Platform
Azure
Data Lake Storage
Common Data Model
Enterprise Security
Optimized for Analytics
METASTORE
SECURITY
MANAGEMENT
MONITORING
DATA INTEGRATION
Analytics Runtimes
PROVISIONED ON-DEMAND
Form Factors
SQL
Languages
Python .NET Java Scala R
Experience Synapse Analytics Studio
Inteligencia Artificial / Aprendizaje Automático / Internet de las
cosas/ Aplicaciones inteligentes / Inteligencia empresarial
Servicios conectados
Azure Data Catalog
Azure Data Lake Storage
Azure Data Share
Azure Databricks
Azure HDInsight
Azure Machine Learning
Power BI
3rd Party Integration
Arquitecturas elásticas
Híbrido
Analizar todos los datosComputación
optimizada para cargas
de trabajo
Autoservicio gobernadoSin silos de datos
4. 08-May-20 7:12 AM
4
Tiempo Costo Riesgo
Plataforma: Rendimiento
• Azure Synapse aprovecha el ecosistema de Azure y las
mejoras principales del motor de SQL Server para producir
mejoras masivas en el rendimiento.
• Estos beneficios no requieren ninguna configuración del
cliente y se proporcionan de fábrica para cada almacén de
datos
• Gen2 adaptive caching – utilizando unidades de estado
sólido (NVMe) de memoria no volátil para aumentar el
ancho de banda de E/S disponible para las consultas.
• Azure FPGA-accelerated networking enhancements – para
mover datos a velocidades de hasta 1 GB/s por nodo para
mejorar las consultas
• Instant data movement – aprovecha el paralelismo
multinúcleo en los servidores SQL Server subyacentes para
mover datos de forma eficiente entre nodos de proceso.
• Query Optimization –optimización de consultas
distribuidas
7. 08-May-20 7:12 AM
7
Gestión de la
carga de
trabajo
Scale-In Isolation
Coste predecible
Elasticidaden línea
Eficiente paracargasde trabajo impredecibles
Intra Cluster Workload Isolation
(Scale In)
Marketing
CREATE WORKLOAD GROUP Sales
WITH
(
[ MIN_PERCENTAGE_RESOURCE = 60 ]
[ CAP_PERCENTAGE_RESOURCE = 100 ]
[ MAX_CONCURRENCY = 6 ] )
40%
Compute
1000c DWU
60%
Sales
60%
100%
Seguridad integral
Category Feature
Data Protection
Data in Transit
Data Encryption at Rest
Data Discovery and Classification
Access Control
Object Level Security (Tables/Views)
Row Level Security
Column Level Security
Dynamic Data Masking
SQL Login
Authentication Azure Active Directory
Multi-Factor Authentication
Virtual Networks
Network Security Firewall
Azure ExpressRoute
Thread Detection
Threat Protection Auditing
Vulnerability Assessment
8. 08-May-20 7:12 AM
8
Integración de
datos
Data Warehouse Reporting
Integración de datos de Synapse
Más de 90 conectores listos para usar
Sin servidor, sin infraestructura que
administrar
Ingestión sostenida de 4 GB/s
CSV, AVRO, ORC, Parquet, JSON support
9. 08-May-20 7:12 AM
9
Integración de datos de Synapse
Code First
Code Free
GUI based
+ many more
Power BI Azure Machine Learning
Azure Data Share Ecosystem
Azure Synapse Analytics
10. 08-May-20 7:12 AM
10
Data Integration Data Warehouse Reporting
Almacenamiento optimizado para el rendimiento
Elastic Architecture Columnar Storage Columnar Ordering Table Partitioning
Nonclustered Indexes Hash Distribution Materialized Views Resultset Cache
11. 08-May-20 7:12 AM
11
Migración de tablas de base de datos
CREATE TABLE StoreSales (
[sales_city] varchar(60),
[sales_year] int,
[sales_state] char(2),
[item_sk] int,
[sales_zip] char(10),
[sales_date] date,
[customer_sk] int)
WITH(
CLUSTERED COLUMNSTORE INDEX ORDER ([customer_sk]),
DISTRIBUTION = HASH([sales_zip],[item_sk]),
PARTITION ([sales_year] RANGE RIGHT FOR VALUES (1998,1999,2000,2001,2002,2003)))
Vista de base de
datos
Migración Materialized Views
Views
12. 08-May-20 7:12 AM
12
Migración de vista de base de
datos
Vista Vista materializada
Abstrae estructura a los usuarios YES YES
Requiere una referencia explícita YES No
Mejora el rendimiento No YES
Se requiere almacenamiento adicional No YES
Asegurable YES YES
Soporte completo de SQL
YES No
Migración de vista de base de datos
CREATE VIEW vw_TopSalesState
AS
SELECT
SubQ.StateAbbrev,
SubQ.FirstSoldDate,
(SubQ.SalesPrice / sum(SubQ.SalesPrice) OVER (order by (select null)))*100,
(1- (SalesPrice/ListPrice))*100 AS Discount,
RANK() OVER (order by (1- (SalesPrice/ListPrice))) AS StateDiscRank
FROM (
SELECT
s_state AS StateAbbrev,
MIN(d_date) AS FirstSoldDate,
SUM([ss_list_price]) AS ListPrice,
SUM([ss_sales_price]) AS SalesPrice
FROM [tpcds10TB].[store_sales2] ss
INNER JOIN [tpcds10TB].store s on s.[s_store_sk] = ss.[ss_store_sk]
INNER JOIN [tpcds10TB].[date_dim] d on d.[d_date_sk] = ss.ss_sold_date_sk
GROUP BY
s_state) AS SubQ
13. 08-May-20 7:12 AM
13
Migración de la vista materializada de la base de datos
CREATE MATERIALIZED VIEW [dbo].[mvw_StoreSalesSummary]
WITH (DISTRIBUTION = HASH(ss_store_sk))
AS
SELECT
s_state,
c_birth_country,
ss_store_sk AS ss_store_sk,
ss_sold_date_sk AS ss_sold_date_sk,
SUM([ss_list_price]) AS [ss_list_price],
SUM([ss_sales_price]) AS [ss_sales_price],
count_big(*) AS cb
FROM [tpcds10TB].[store_sales2] ss
INNER JOIN [tpcds10TB].customer c ON c.[c_customer_sk] = ss.[ss_customer_sk]
INNER JOIN [tpcds10TB].store s on s.[s_store_sk] = ss.[ss_store_sk]
GROUP BY
s_state,c_birth_country,ss_store_sk, ss_sold_date_sk
Customer
65
Million
Rows
Store
1500
Rows
Store Sales
26
Billion
Rows
Materialized View
287
Million
Rows
Data Integration Data Warehouse Informes
14. 08-May-20 7:12 AM
14
Synapse Connected Service: Power BI
Experiencia integrada de
creación de Power BI
Publicar en Power BI
Escalado a
Petabytes
Materialized Views
Transactionalconsistentlyto datamodification
AutomaticQueryOptimizermatching
CREATE MATERIALZIED VIEW vw_ProductSales
WITH (DISTRIBUTION = HASH(ProductKey))
AS
SELECT
ProductName
ProductKey,
SUM(Amount) AS TotalSales
FROM
FactSales fs
INNER JOIN DimProduct dp ON fs.prodkey = dp.prodkey
GROUP BY
ProductName,
ProductKey
15. 08-May-20 7:12 AM
15
Escalado a
Petabytes
Materialized Views
Transactionalconsistentlyto datamodification
AutomaticQueryOptimizermatching
ProductName ProductKey TotalSales
Product A 5453 784,943.00
Product B 763 48,723.00
… … …
FactSales Table
10B Records
DimProduct Table
1,000 Records
FactSales
DimProduct
FactInventory
Table
mvw_ProductSales
1,000 Records
SELECT
ProductName
ProductKey,
SUM(Amount) AS TotalSales
FROM
FactSales fs
INNER JOIN DimProduct dp
GROUP BY
ProductName,
ProductKey
FactInventory
Escalado a
Petabytes
Result set Cache
Automaticquery matching
Implicitcreatingfrom queryactivity
Resilient to cluster elasticity
Execution2
Cache Hit
~.2 seconds
Execution1
Cache Miss
Regular Execution
16. 08-May-20 7:12 AM
16
Escalado a
Petabytes
Materialized Views
Transactionalconsistentlyto datamodification
AutomaticQueryOptimizermatching
CREATE MATERIALZIED VIEW vw_ProductSales
WITH (DISTRIBUTION = HASH(ProductKey))
AS
SELECT
ProductName
ProductKey,
SUM(Amount) AS TotalSales
FROM
FactSales fs
INNER JOIN DimProduct dp ON fs.prodkey = dp.prodkey
GROUP BY
ProductName,
ProductKey
ProductName ProductKey TotalSales
Product A 5453 784,943.00
Product B 763 48,723.00
… … …
FactSales Table
10B Records
DimProduct Table
1,000 Records
Escalado a
Petabytes
Materialized Views
Transactionalconsistentlyto datamodification
AutomaticQueryOptimizermatching
FactSales
DimProduct
FactInventory
Table
mvw_ProductSales
1,000 Records
SELECT
ProductName
ProductKey,
SUM(Amount) AS TotalSales
FROM
FactSales fs
INNER JOIN DimProduct dp
GROUP BY
ProductName,
ProductKey
FactInventory
17. 08-May-20 7:12 AM
17
Escalado a
Petabytes
Materialized Views
Transactionalconsistentlyto datamodification
AutomaticQueryOptimizermatching
SELECT
c_customerkey,
c_nationkey,
SUM(l_quantity),
SUM(l_extendedprice)
FROM [dbo].[lineitem_MonthPartition] l
INNER JOIN [dbo].[orders] o on o.o_orderkey = l.l_orderkey
INNER JOIN [dbo].[customer] c on c.c_customerkey = o.o_customerkey
GROUP BY
c_customerkey,
c_nationkey
[dbo].[lineitem_MonthPartition] HASH(l_orderkey)
[dbo].[orders] HASH(o_orderkey)
[dbo].[customer] HASH(c_customerkey)
Table Distributions
Escalado a
Petabytes
Materialized Views
Transactionalconsistentlyto datamodification
AutomaticQueryOptimizermatching
LineItem Orders
Collocated Join (DistributionAligned)
Customer
Non-collocatedJoin (Shuffle Required)
FROM [dbo].[lineitem_MonthPartition] l
INNER JOIN [dbo].[orders] o on o.o_orderkey = l.l_orderkey
INNER JOIN [dbo].[customer] c on c.c_customerkey = o.o_customerkey
18. 08-May-20 7:12 AM
18
Escalado a
Petabytes
Materialized Views
Transactionalconsistentlyto datamodification
AutomaticQueryOptimizermatching
(Shuffle Required)
LineItem Orders
Collocated Join (DistributionAligned)
Stage 1
Customer
Stage 2
#temp (Orders + Lineitem)
Nation
Collocated Join (Replicate Aligned)
Collocated Join (DistributionAligned)
Escalado a
Petabytes
Materialized Views
Transactionalconsistentlyto datamodification
AutomaticQueryOptimizermatching
CREATE MATERIALIZED VIEW mvw_CustomerSales
WITH (DISTRIBUTION = HASH(o_custkey))
AS
SELECT
o_custkey,
l_shipdate,
SUM(l_quantity) AS l_quantity,
SUM(l_extendedprice) AS l_extendedprice
FROM [dbo].[lineitem_MonthPartition] l
INNER JOIN [dbo].[orders] o on o.o_orderkey = l.l_orderkey
WHERE
l_shipdate >= CONVERT(DATETIME, '1998-11-01', 103)
GROUP BY
o_custkey,
l_shipdate
19. 08-May-20 7:12 AM
19
Escalado a
Petabytes
Materialized Views
Transactionalconsistentlyto datamodification
AutomaticQueryOptimizermatching
Legend
mvw_CustomerSales
Nation
Customer
<replicated table>
Collocated Join (DistributionAligned)
Collocated Join (Replicate Aligned)
Escalado a
Petabytes
Materialized Views
Transactionalconsistentlyto datamodification
AutomaticQueryOptimizermatching
275
5
0
50
100
150
200
250
300
No MaterializedView WithMaterializedView
Seconds
Query Execution Time
20. 08-May-20 7:12 AM
20
Power BI
Materialized Views
Tables
Escalado a
Petabytes
Power BI
DirectQuery
Composite Models
Aggregation Tables