Project_Plan-Datalake_v1.0_26-10-2022.pptx
•Download as PPTX, PDF•
0 likes•1 view
The document outlines a project timeline for migrating databases to AWS and developing analytics and customer experience reports. It involves migrating multiple databases to AWS from September to December, optimizing performance, and renewing contracts. From November to March, the project focuses on data modeling, aggregate generation, identifying reports for EMR delivery, and developing reports in batches for testing and production go-live.
Report
Share
Report
Share
Recommended
SQL Analytics Powering Telemetry Analysis at Comcast
Comcast is one of the leading providers of communications, entertainment, and cable products and services. At the heart of it is Comcast RDK providing the backbone of telemetry to the industry. RDK (Reference Design Kit) is pre-bundled opensource firmware for a complete home platform covering video, broadband and IoT devices. RDK team at Comcast analyzes petabytes of data, collected every 15 minutes from 70 million devices (video and broadband and IoT devices) installed in customer homes. They run ETL and aggregation pipelines and publish analytical dashboards on a daily basis to reduce customer calls and firmware rollout. The analysis is also used to calculate WIFI happiness index which is a critical KPI for Comcast customer experience.
In addition to this, RDK team also does release tracking by analyzing the RDK firmware quality. SQL Analytics allows customers to operate a lakehouse architecture that provides data warehousing performance at data lake economics for up to 4x better price/performance for SQL workloads than traditional cloud data warehouses.
We present the results of the “Test and Learn” with SQL Analytics and the delta engine that we worked in partnership with the Databricks team. We present a quick demo introducing the SQL native interface, the challenges we faced with migration, The results of the execution and our journey of productionizing this at scale.
CCI2017 - Considerations for Migrating Databases to Azure - Gianluca Sartori
The document discusses considerations for migrating databases to Microsoft Azure SQL Database. It covers cloud options like Infrastructure as a Service (IaaS) using SQL Server on Azure VMs and Platform as a Service (PaaS) options like Azure SQL Database. It also discusses analyzing database compatibility, different migration methods like using BACPAC files or the Data Migration Assistant, and ways to optimize the migration process like monitoring tempdb usage.
Migrating Big Data Workloads to the Cloud
Cloud Services have become an integral part of many organizations’ workloads. With the ease of spinning up instances to react to changing or increased business demands, it’s certainly not hard to see why. Many organizations that once relied on internal data centers are even considering migrating to the cloud to take advantage of not just the ability to spin up instances rapidly, but also to leverage all the services that are available with these offerings. Such as Elastic Map Reduce from Amazon and HD Insight from Microsoft. In this talk, we will go over the various methods and strategies you can employ to migrate your data and workloads to the cloud and what all options this opens up for your business and IT.
Building End-to-End Delta Pipelines on GCP
Delta has been powering many production pipelines at scale in the Data and AI space since it has been introduced for the past few years.
Built on open standards, Delta provides data reliability, enhances storage and query performance to support big data use cases (both batch and streaming), fast interactive queries for BI and enabling machine learning. Delta has matured over the past couple of years in both AWS and AZURE and has become the de-facto standard for organizations building their Data and AI pipelines.
In today’s talk, we will explore building end-to-end pipelines on the Google Cloud Platform (GCP). Through presentation, code examples and notebooks, we will build the Delta Pipeline from ingest to consumption using our Delta Bronze-Silver-Gold architecture pattern and show examples of Consuming the delta files using the Big Query Connector.
Cloud-native Semantic Layer on Data Lake
With larger volume and more real-time data stored in data lake, it becomes more complex to manage these data and serve analytics and applications. With different service interfaces, data caliber, performance bias on different scenarios, the business users begin to suffer low confidence on quality and efficiency to get insight from data.
Migrating to Amazon RDS with Database Migration Service
The document provides an overview of database migration options using AWS Database Migration Service (DMS) and AWS Schema Conversion Tool (SCT). It discusses how DMS can be used to migrate databases across different database platforms with minimal downtime. It also outlines how SCT can be used to convert schemas from commercial databases to open-source databases like PostgreSQL. The document shares customer examples and benefits of using DMS and SCT for heterogeneous, scale-up, and split migrations. It also lists available resources for customers on DMS and SCT.
Tarun poladi resume
Tarun Poladi is a data engineer with over 7 years of experience in business intelligence and data engineering. He has extensive experience with Microsoft BI tools like SSIS, SSRS, and Power BI. He is proficient in SQL, Python, and R. Tarun has worked on various cloud platforms including AWS, Azure, and Databricks. His experience includes data modeling, ETL processes, building reports and dashboards, and developing analytics solutions. He holds certifications in Azure Data Engineering and Microsoft BI Reporting.
Weathering the Data Storm – How SnapLogic and AWS Deliver Analytics in the Cl...
In this webinar, learn how SnapLogic and Amazon Web Services helped Earth Networks create a responsive, self-service cloud for data integration, preparation and analytics.
We also discuss how Earth Networks gained faster data insights using SnapLogic’s Amazon Redshift data integration and other connectors to quickly integrate, transfer and analyze data from multiple applications.
To learn more, visit: www.snaplogic.com/redshift
Recommended
SQL Analytics Powering Telemetry Analysis at Comcast
Comcast is one of the leading providers of communications, entertainment, and cable products and services. At the heart of it is Comcast RDK providing the backbone of telemetry to the industry. RDK (Reference Design Kit) is pre-bundled opensource firmware for a complete home platform covering video, broadband and IoT devices. RDK team at Comcast analyzes petabytes of data, collected every 15 minutes from 70 million devices (video and broadband and IoT devices) installed in customer homes. They run ETL and aggregation pipelines and publish analytical dashboards on a daily basis to reduce customer calls and firmware rollout. The analysis is also used to calculate WIFI happiness index which is a critical KPI for Comcast customer experience.
In addition to this, RDK team also does release tracking by analyzing the RDK firmware quality. SQL Analytics allows customers to operate a lakehouse architecture that provides data warehousing performance at data lake economics for up to 4x better price/performance for SQL workloads than traditional cloud data warehouses.
We present the results of the “Test and Learn” with SQL Analytics and the delta engine that we worked in partnership with the Databricks team. We present a quick demo introducing the SQL native interface, the challenges we faced with migration, The results of the execution and our journey of productionizing this at scale.
CCI2017 - Considerations for Migrating Databases to Azure - Gianluca Sartori
The document discusses considerations for migrating databases to Microsoft Azure SQL Database. It covers cloud options like Infrastructure as a Service (IaaS) using SQL Server on Azure VMs and Platform as a Service (PaaS) options like Azure SQL Database. It also discusses analyzing database compatibility, different migration methods like using BACPAC files or the Data Migration Assistant, and ways to optimize the migration process like monitoring tempdb usage.
Migrating Big Data Workloads to the Cloud
Cloud Services have become an integral part of many organizations’ workloads. With the ease of spinning up instances to react to changing or increased business demands, it’s certainly not hard to see why. Many organizations that once relied on internal data centers are even considering migrating to the cloud to take advantage of not just the ability to spin up instances rapidly, but also to leverage all the services that are available with these offerings. Such as Elastic Map Reduce from Amazon and HD Insight from Microsoft. In this talk, we will go over the various methods and strategies you can employ to migrate your data and workloads to the cloud and what all options this opens up for your business and IT.
Building End-to-End Delta Pipelines on GCP
Delta has been powering many production pipelines at scale in the Data and AI space since it has been introduced for the past few years.
Built on open standards, Delta provides data reliability, enhances storage and query performance to support big data use cases (both batch and streaming), fast interactive queries for BI and enabling machine learning. Delta has matured over the past couple of years in both AWS and AZURE and has become the de-facto standard for organizations building their Data and AI pipelines.
In today’s talk, we will explore building end-to-end pipelines on the Google Cloud Platform (GCP). Through presentation, code examples and notebooks, we will build the Delta Pipeline from ingest to consumption using our Delta Bronze-Silver-Gold architecture pattern and show examples of Consuming the delta files using the Big Query Connector.
Cloud-native Semantic Layer on Data Lake
With larger volume and more real-time data stored in data lake, it becomes more complex to manage these data and serve analytics and applications. With different service interfaces, data caliber, performance bias on different scenarios, the business users begin to suffer low confidence on quality and efficiency to get insight from data.
Migrating to Amazon RDS with Database Migration Service
The document provides an overview of database migration options using AWS Database Migration Service (DMS) and AWS Schema Conversion Tool (SCT). It discusses how DMS can be used to migrate databases across different database platforms with minimal downtime. It also outlines how SCT can be used to convert schemas from commercial databases to open-source databases like PostgreSQL. The document shares customer examples and benefits of using DMS and SCT for heterogeneous, scale-up, and split migrations. It also lists available resources for customers on DMS and SCT.
Tarun poladi resume
Tarun Poladi is a data engineer with over 7 years of experience in business intelligence and data engineering. He has extensive experience with Microsoft BI tools like SSIS, SSRS, and Power BI. He is proficient in SQL, Python, and R. Tarun has worked on various cloud platforms including AWS, Azure, and Databricks. His experience includes data modeling, ETL processes, building reports and dashboards, and developing analytics solutions. He holds certifications in Azure Data Engineering and Microsoft BI Reporting.
Weathering the Data Storm – How SnapLogic and AWS Deliver Analytics in the Cl...
In this webinar, learn how SnapLogic and Amazon Web Services helped Earth Networks create a responsive, self-service cloud for data integration, preparation and analytics.
We also discuss how Earth Networks gained faster data insights using SnapLogic’s Amazon Redshift data integration and other connectors to quickly integrate, transfer and analyze data from multiple applications.
To learn more, visit: www.snaplogic.com/redshift
Bi and AI updates in the Microsoft Data Platform stack
This document provides an overview and agenda for a presentation on new features in the Microsoft data platform universe including SQL Server, Azure, and Power BI, with a focus on business intelligence (BI) and artificial intelligence (AI).
The presentation covers new features in SQL Server 2019 for BI like calculation groups and dynamic formatting in Analysis Services tabular mode. It also highlights important updates in Azure including new serverless and hyperscale options for Azure SQL Database, updates to data ingestion and storage with Azure Data Factory v2 and Azure Data Lake Gen 2, and changes to data modeling, preparation, and serving capabilities in Azure SQL Data Warehouse.
The document concludes by discussing options for incorporating AI and machine learning into BI solutions using either
Integration Monday - Analysing StackExchange data with Azure Data Lake
Big data is the new big thing where storing the data is the easy part. Gaining insights in your pile of data is something different.
Based on a data dump of the well-known StackExchange websites, we will store & analyse 150+ GB of data with Azure Data Lake Store & Analytics to gain some insights about their users. After that we will use Power BI to give an at a glance overview of our learnings.
If you are a developer that is interested in big data, this is your time to shine! We will use our existing SQL & C# skills to analyse everything without having to worry about running clusters.
Analyzing StackExchange Data with Azure Data Lake (Tom Kerkhove @ Integration...
Why should we care about Big Data? Tom Kerkhove, Integration Professional at Codit, explains it all in this presentation.
DesignMind SQL Server 2008 Migration
This presentation discusses SQL Server 2008 Migration tools, planning and execution. You will learn about the SQL Server Featuer Pack, the SQL Server Migration Assistant, and Performance Benchmarks of SQL Server 2005 vs. 2008.
DesignMind is located in Emeryville, California.
www.designmind.com
Big data at AWS Chicago User Group - 2014
Big data at AWS Chicago User Group
Most of the slides from the Sept 23rd 2014 AWS User Group in Chicago.
Talks:
"AWS Storage Options" Ben Blair, CTO at MarkITx @stochastic_code
"APIs and Big Data in AWS" - Kin Lane, API Evangelist @kinlane
[coming soon] "Democratizing Data Analysis with Amazon Redshift" - Bill Wanjohi @billwanjohi and Michelangelo D'Agostino @MichelangeloDA, Civis Analytics
Sponsored by Cohesive and CivisAnalytics.
Amazon Redshift with Full 360 Inc.
Full 360 is a cloud consulting firm that provides big data, API/UX, and cloud operations services. They helped a customer migrate their data from Netezza to Redshift, building a structured data lake and optimizing queries for equivalent or better performance. Lessons from the project included data standardization, tuning techniques like encoding and sort keys, and creating reusable ingestion processes. The migration reduced license costs and improved operational flexibility.
Journey to the Cloud: Database Modernization Best Practices
In this presentation from the AWS Dallas workshop, Datavail's migration team discusses the different decision paths to the cloud, how to decide when to migrate to AWS, and 3 case studies examples of migrations to AWS
Power BI 2
A walkthrough fo the possebilities with Power BI 2.0 which is currently available in public preview. I will go through the latest functions and give and overview of the tools included in the application combined with a demo of the designer and Power Query tool.
These slides do also cover Datazen - Microfts latest Business Intelligence Acquisition which now enables them to deliver on-premise mobile bi.
Modernize Legacy and Enterprise Application Through Implementation of Cloud N...
Many Federal agencies are taking on initiatives to consolidate datacenters, modernize legacy and enterprise applications, and transform the digital portfolio to enable agility and other advantageous through cloud based delivery models. Deloitte provides a breadth of services to help federal government agencies select the right cloud solutions to accelerate their missions and derive value while helping agencies to be at the forefront of technology and innovation. Join us as we demonstrate specific client use cases where we have successfully assisted clients in selecting a cloud service model for migrating and transforming current data center to the cloud, enabling agility by taking advantage of Agile, DevOps, and continuous delivery, and modernized legacy and enterprise application through the implementation of cloud native solutions on AWS infrastructure to deliver value at the speed of your mission. Learn More: https://aws.amazon.com/government-education/
Tah 03302015 withendclient
Terry Hendrickson has over 15 years of experience as a software/database developer with expertise in SQL Server, Oracle, Visual Basic, VB.NET, T-SQL, PL/SQL, ETL, Crystal Reports, and other technologies. He has worked on projects involving database development, reporting, ETL, and application development for companies across various industries. Hendrickson holds an Associate's Degree in Computer Information Systems and Business Management.
Automating Data Quality Processes at Reckitt
Reckitt is a fast-moving consumer goods company with a portfolio of famous brands and over 30k employees worldwide. With that scale small projects can quickly grow into big datasets, and processing and cleaning all that data can become a challenge. To solve that challenge we have created a metadata driven ETL framework for orchestrating data transformations through parametrised SQL scripts. It allows us to create various paths for our data as well as easily version control them. The approach of standardising incoming datasets and creating reusable SQL processes has proven to be a winning formula. It has helped simplify complicated landing/stage/merge processes and allowed them to be self-documenting.
But this is only half the battle, we also want to create data products. Documented, quality assured data sets that are intuitive to use. As we move to a CI/CD approach, increasing the frequency of deployments, the demand of keeping documentation and data quality assessments up to date becomes increasingly challenging. To solve this problem, we have expanded our ETL framework to include SQL processes that automate data quality activities. Using the Hive metastore as a starting point, we have leveraged this framework to automate the maintenance of a data dictionary and reduce documenting, model refinement, testing data quality and filtering out bad data to a box filling exercise. In this talk we discuss our approach to maintaining high quality data products and share examples of how we automate data quality processes.
Bi Resume Ejd
The document is a resume for Elmer Donavan, a Business Intelligence Professional with over 20 years of experience in SQL Server and Microsoft technologies including SQL Server 2005, Integration Services, Analysis Services, and Reporting Services. He has extensive experience designing, developing and implementing BI solutions for various industries, and has led projects involving data warehousing, ETL, reporting, analytics and dashboarding. His technical skills include databases, query languages, software, and programming languages.
[DSC DACH 23] The Modern Data Stack - Bogdan Pirvu
After this tutorial you will be equipped with the knowledge necessary to make informed decisions about the design and setup of a cloud-based Big Data platform using the so-called Modern Data Stack (MDS). Bogdan will introduce the main concepts of the MDS and explain its benefits and why it is such a compelling choice for companies of any size. He will also provide advice to help you navigate the ever increasing number of vendors in this space. You will learn what you need to take into consideration when implementing the MDS in your company, and see how it works in a live demo.
Sql connections germany - migration considerations when migrating your on pre...
This document discusses considerations for migrating an on-premise SQL Server database system to SQL Azure. It provides an overview of the existing on-premise system and SQL Azure capabilities. Key challenges of the migration include determining which data to store where based on cost and performance, implementing connectivity retry logic to handle throttling and faults, and designing a cost-driven architecture in the cloud. The document recommends migrating sensor data and databases to SQL Azure while using Azure services like Data Sync and Analysis Services.
Webinar - Macy’s: Why Your Database Decision Directly Impacts Customer Experi...
According to a recent Harvard Business Review study, there’s only a 43% chance that customers who have a poor experience will stick with you for the next 12 months. Contrast that to the 74% that will remain your customer if they have a great experience. Learn how Macy’s, a leading American department store chain founded in 1858 with over 750 stores in North America, is transforming their customer experience with DataStax Enterprise.
Webinar recording: https://youtu.be/CiUVxh6Ov_E
View current and past DataStax webinars: http://www.datastax.com/resources/webinars
Supercharging Smart Meter BIG DATA Analytics with Microsoft Azure Cloud- SRP ...
Explosive growth of Smart Meter (SM) deployments has presented key infrastructure challenges across the utility industry. The huge volumes of smart meter data has led the industry to a tipping point which requires investments in modernizing existing data warehouses. Typical modernization efforts lead to huge capital expenditures for DW appliances and storage. Sizing this new infrastructure is tricky and can lead to underutilized or poorly performing hardware.
The Cloud is the catalyst to solving these Big Data challenges.
Utilizing a Cloud architecture delivers huge benefits by:
Maximizing use of existing architecture
Minimizing new CapEx expenditures
Lowering overall storage costs
Enabling scale on demand
AWS Partner Webcast - Analyze Big Data for Consumer Applications with Looker ...
Analyze Big Data for Consumer Applications with Looker BI and Amazon Redshift Customizing the customer experience based on user behavior is a constant challenge for today’s consumer apps. Business intelligence helps analyze and model large amounts of data. Looker offers a modern approach to BI leveraging AWS that’s fast, agile, and easy to manage. Join this webinar to learn how MessageMe, which provides emotionally engaging messaging apps to consumers, leverages Looker business intelligence software and the Amazon Redshift data warehouse service to analyze billions of rows of customer data in seconds.
Webinar topics include:
• How MessageMe turns billions of rows of customer data stored in Amazon Redshift into actionable insights
• How Looker connects directly to Amazon Redshift in just a few clicks, enabling MessageMe to build a modern, big data analytics in the cloud. Who should attend
• Information or Solution Architects, Data Analysts, BI Directors, DBAs, Development Leads, Developers, or Technical IT Leaders.
Presenters:
• Justin Rosenthal, CTO, MessageMe
• Keenan Rice, VP, Marketing & Alliances, Looker
• Tina Adams, Senior Product Manager, AWS
Bi developer gary thompson
The document provides a summary of Gary A Thompson's experience and qualifications as a Business Intelligence Professional. It outlines his technical skills using Microsoft technologies including SQL Server and Analysis Services. It also details his professional experience over 20 years manipulating and reporting data to support business decisions. Positions included roles as a Business Intelligence Developer, Data Analyst, and Programmer Analyst working with clients in insurance, healthcare, and other industries.
Bi developer gary thompson
The document provides a summary of Gary A Thompson's experience and skills as a Business Intelligence Professional. It outlines his technical skills using Microsoft technologies including SQL Server and Microsoft Office products. It also details his professional experience over 20 years manipulating and reporting data to support business decisions. Positions included roles as a Data Analyst, Business Intelligence Developer, and Programmer Analyst working with clients in insurance, healthcare, and other industries.
Evolutionary database design
A shortened summary of the original Article by "Martin Fowler" and "Pramod Sadalage"
https://martinfowler.com/articles/evodb.html
The Genesis of BriansClub.cm Famous Dark WEb Platform
BriansClub.cm, a famous platform on the dark web, has become one of the most infamous carding marketplaces, specializing in the sale of stolen credit card data.
Cover Story - China's Investment Leader - Dr. Alyce SU
In World Expo 2010 Shanghai – the most visited Expo in the World History
https://www.britannica.com/event/Expo-Shanghai-2010
China’s official organizer of the Expo, CCPIT (China Council for the Promotion of International Trade https://en.ccpit.org/) has chosen Dr. Alyce Su as the Cover Person with Cover Story, in the Expo’s official magazine distributed throughout the Expo, showcasing China’s New Generation of Leaders to the World.
More Related Content
Similar to Project_Plan-Datalake_v1.0_26-10-2022.pptx
Bi and AI updates in the Microsoft Data Platform stack
This document provides an overview and agenda for a presentation on new features in the Microsoft data platform universe including SQL Server, Azure, and Power BI, with a focus on business intelligence (BI) and artificial intelligence (AI).
The presentation covers new features in SQL Server 2019 for BI like calculation groups and dynamic formatting in Analysis Services tabular mode. It also highlights important updates in Azure including new serverless and hyperscale options for Azure SQL Database, updates to data ingestion and storage with Azure Data Factory v2 and Azure Data Lake Gen 2, and changes to data modeling, preparation, and serving capabilities in Azure SQL Data Warehouse.
The document concludes by discussing options for incorporating AI and machine learning into BI solutions using either
Integration Monday - Analysing StackExchange data with Azure Data Lake
Big data is the new big thing where storing the data is the easy part. Gaining insights in your pile of data is something different.
Based on a data dump of the well-known StackExchange websites, we will store & analyse 150+ GB of data with Azure Data Lake Store & Analytics to gain some insights about their users. After that we will use Power BI to give an at a glance overview of our learnings.
If you are a developer that is interested in big data, this is your time to shine! We will use our existing SQL & C# skills to analyse everything without having to worry about running clusters.
Analyzing StackExchange Data with Azure Data Lake (Tom Kerkhove @ Integration...
Why should we care about Big Data? Tom Kerkhove, Integration Professional at Codit, explains it all in this presentation.
DesignMind SQL Server 2008 Migration
This presentation discusses SQL Server 2008 Migration tools, planning and execution. You will learn about the SQL Server Featuer Pack, the SQL Server Migration Assistant, and Performance Benchmarks of SQL Server 2005 vs. 2008.
DesignMind is located in Emeryville, California.
www.designmind.com
Big data at AWS Chicago User Group - 2014
Big data at AWS Chicago User Group
Most of the slides from the Sept 23rd 2014 AWS User Group in Chicago.
Talks:
"AWS Storage Options" Ben Blair, CTO at MarkITx @stochastic_code
"APIs and Big Data in AWS" - Kin Lane, API Evangelist @kinlane
[coming soon] "Democratizing Data Analysis with Amazon Redshift" - Bill Wanjohi @billwanjohi and Michelangelo D'Agostino @MichelangeloDA, Civis Analytics
Sponsored by Cohesive and CivisAnalytics.
Amazon Redshift with Full 360 Inc.
Full 360 is a cloud consulting firm that provides big data, API/UX, and cloud operations services. They helped a customer migrate their data from Netezza to Redshift, building a structured data lake and optimizing queries for equivalent or better performance. Lessons from the project included data standardization, tuning techniques like encoding and sort keys, and creating reusable ingestion processes. The migration reduced license costs and improved operational flexibility.
Journey to the Cloud: Database Modernization Best Practices
In this presentation from the AWS Dallas workshop, Datavail's migration team discusses the different decision paths to the cloud, how to decide when to migrate to AWS, and 3 case studies examples of migrations to AWS
Power BI 2
A walkthrough fo the possebilities with Power BI 2.0 which is currently available in public preview. I will go through the latest functions and give and overview of the tools included in the application combined with a demo of the designer and Power Query tool.
These slides do also cover Datazen - Microfts latest Business Intelligence Acquisition which now enables them to deliver on-premise mobile bi.
Modernize Legacy and Enterprise Application Through Implementation of Cloud N...
Many Federal agencies are taking on initiatives to consolidate datacenters, modernize legacy and enterprise applications, and transform the digital portfolio to enable agility and other advantageous through cloud based delivery models. Deloitte provides a breadth of services to help federal government agencies select the right cloud solutions to accelerate their missions and derive value while helping agencies to be at the forefront of technology and innovation. Join us as we demonstrate specific client use cases where we have successfully assisted clients in selecting a cloud service model for migrating and transforming current data center to the cloud, enabling agility by taking advantage of Agile, DevOps, and continuous delivery, and modernized legacy and enterprise application through the implementation of cloud native solutions on AWS infrastructure to deliver value at the speed of your mission. Learn More: https://aws.amazon.com/government-education/
Tah 03302015 withendclient
Terry Hendrickson has over 15 years of experience as a software/database developer with expertise in SQL Server, Oracle, Visual Basic, VB.NET, T-SQL, PL/SQL, ETL, Crystal Reports, and other technologies. He has worked on projects involving database development, reporting, ETL, and application development for companies across various industries. Hendrickson holds an Associate's Degree in Computer Information Systems and Business Management.
Automating Data Quality Processes at Reckitt
Reckitt is a fast-moving consumer goods company with a portfolio of famous brands and over 30k employees worldwide. With that scale small projects can quickly grow into big datasets, and processing and cleaning all that data can become a challenge. To solve that challenge we have created a metadata driven ETL framework for orchestrating data transformations through parametrised SQL scripts. It allows us to create various paths for our data as well as easily version control them. The approach of standardising incoming datasets and creating reusable SQL processes has proven to be a winning formula. It has helped simplify complicated landing/stage/merge processes and allowed them to be self-documenting.
But this is only half the battle, we also want to create data products. Documented, quality assured data sets that are intuitive to use. As we move to a CI/CD approach, increasing the frequency of deployments, the demand of keeping documentation and data quality assessments up to date becomes increasingly challenging. To solve this problem, we have expanded our ETL framework to include SQL processes that automate data quality activities. Using the Hive metastore as a starting point, we have leveraged this framework to automate the maintenance of a data dictionary and reduce documenting, model refinement, testing data quality and filtering out bad data to a box filling exercise. In this talk we discuss our approach to maintaining high quality data products and share examples of how we automate data quality processes.
Bi Resume Ejd
The document is a resume for Elmer Donavan, a Business Intelligence Professional with over 20 years of experience in SQL Server and Microsoft technologies including SQL Server 2005, Integration Services, Analysis Services, and Reporting Services. He has extensive experience designing, developing and implementing BI solutions for various industries, and has led projects involving data warehousing, ETL, reporting, analytics and dashboarding. His technical skills include databases, query languages, software, and programming languages.
[DSC DACH 23] The Modern Data Stack - Bogdan Pirvu
After this tutorial you will be equipped with the knowledge necessary to make informed decisions about the design and setup of a cloud-based Big Data platform using the so-called Modern Data Stack (MDS). Bogdan will introduce the main concepts of the MDS and explain its benefits and why it is such a compelling choice for companies of any size. He will also provide advice to help you navigate the ever increasing number of vendors in this space. You will learn what you need to take into consideration when implementing the MDS in your company, and see how it works in a live demo.
Sql connections germany - migration considerations when migrating your on pre...
This document discusses considerations for migrating an on-premise SQL Server database system to SQL Azure. It provides an overview of the existing on-premise system and SQL Azure capabilities. Key challenges of the migration include determining which data to store where based on cost and performance, implementing connectivity retry logic to handle throttling and faults, and designing a cost-driven architecture in the cloud. The document recommends migrating sensor data and databases to SQL Azure while using Azure services like Data Sync and Analysis Services.
Webinar - Macy’s: Why Your Database Decision Directly Impacts Customer Experi...
According to a recent Harvard Business Review study, there’s only a 43% chance that customers who have a poor experience will stick with you for the next 12 months. Contrast that to the 74% that will remain your customer if they have a great experience. Learn how Macy’s, a leading American department store chain founded in 1858 with over 750 stores in North America, is transforming their customer experience with DataStax Enterprise.
Webinar recording: https://youtu.be/CiUVxh6Ov_E
View current and past DataStax webinars: http://www.datastax.com/resources/webinars
Supercharging Smart Meter BIG DATA Analytics with Microsoft Azure Cloud- SRP ...
Explosive growth of Smart Meter (SM) deployments has presented key infrastructure challenges across the utility industry. The huge volumes of smart meter data has led the industry to a tipping point which requires investments in modernizing existing data warehouses. Typical modernization efforts lead to huge capital expenditures for DW appliances and storage. Sizing this new infrastructure is tricky and can lead to underutilized or poorly performing hardware.
The Cloud is the catalyst to solving these Big Data challenges.
Utilizing a Cloud architecture delivers huge benefits by:
Maximizing use of existing architecture
Minimizing new CapEx expenditures
Lowering overall storage costs
Enabling scale on demand
AWS Partner Webcast - Analyze Big Data for Consumer Applications with Looker ...
Analyze Big Data for Consumer Applications with Looker BI and Amazon Redshift Customizing the customer experience based on user behavior is a constant challenge for today’s consumer apps. Business intelligence helps analyze and model large amounts of data. Looker offers a modern approach to BI leveraging AWS that’s fast, agile, and easy to manage. Join this webinar to learn how MessageMe, which provides emotionally engaging messaging apps to consumers, leverages Looker business intelligence software and the Amazon Redshift data warehouse service to analyze billions of rows of customer data in seconds.
Webinar topics include:
• How MessageMe turns billions of rows of customer data stored in Amazon Redshift into actionable insights
• How Looker connects directly to Amazon Redshift in just a few clicks, enabling MessageMe to build a modern, big data analytics in the cloud. Who should attend
• Information or Solution Architects, Data Analysts, BI Directors, DBAs, Development Leads, Developers, or Technical IT Leaders.
Presenters:
• Justin Rosenthal, CTO, MessageMe
• Keenan Rice, VP, Marketing & Alliances, Looker
• Tina Adams, Senior Product Manager, AWS
Bi developer gary thompson
The document provides a summary of Gary A Thompson's experience and qualifications as a Business Intelligence Professional. It outlines his technical skills using Microsoft technologies including SQL Server and Analysis Services. It also details his professional experience over 20 years manipulating and reporting data to support business decisions. Positions included roles as a Business Intelligence Developer, Data Analyst, and Programmer Analyst working with clients in insurance, healthcare, and other industries.
Bi developer gary thompson
The document provides a summary of Gary A Thompson's experience and skills as a Business Intelligence Professional. It outlines his technical skills using Microsoft technologies including SQL Server and Microsoft Office products. It also details his professional experience over 20 years manipulating and reporting data to support business decisions. Positions included roles as a Data Analyst, Business Intelligence Developer, and Programmer Analyst working with clients in insurance, healthcare, and other industries.
Evolutionary database design
A shortened summary of the original Article by "Martin Fowler" and "Pramod Sadalage"
https://martinfowler.com/articles/evodb.html
Similar to Project_Plan-Datalake_v1.0_26-10-2022.pptx (20)
Bi and AI updates in the Microsoft Data Platform stack
Bi and AI updates in the Microsoft Data Platform stack
Integration Monday - Analysing StackExchange data with Azure Data Lake
Integration Monday - Analysing StackExchange data with Azure Data Lake
Analyzing StackExchange Data with Azure Data Lake (Tom Kerkhove @ Integration...
Analyzing StackExchange Data with Azure Data Lake (Tom Kerkhove @ Integration...
Journey to the Cloud: Database Modernization Best Practices
Journey to the Cloud: Database Modernization Best Practices
Modernize Legacy and Enterprise Application Through Implementation of Cloud N...
Modernize Legacy and Enterprise Application Through Implementation of Cloud N...
[DSC DACH 23] The Modern Data Stack - Bogdan Pirvu
[DSC DACH 23] The Modern Data Stack - Bogdan Pirvu
Sql connections germany - migration considerations when migrating your on pre...
Sql connections germany - migration considerations when migrating your on pre...
Webinar - Macy’s: Why Your Database Decision Directly Impacts Customer Experi...
Webinar - Macy’s: Why Your Database Decision Directly Impacts Customer Experi...
Supercharging Smart Meter BIG DATA Analytics with Microsoft Azure Cloud- SRP ...
Supercharging Smart Meter BIG DATA Analytics with Microsoft Azure Cloud- SRP ...
AWS Partner Webcast - Analyze Big Data for Consumer Applications with Looker ...
AWS Partner Webcast - Analyze Big Data for Consumer Applications with Looker ...
Recently uploaded
The Genesis of BriansClub.cm Famous Dark WEb Platform
BriansClub.cm, a famous platform on the dark web, has become one of the most infamous carding marketplaces, specializing in the sale of stolen credit card data.
Cover Story - China's Investment Leader - Dr. Alyce SU
In World Expo 2010 Shanghai – the most visited Expo in the World History
https://www.britannica.com/event/Expo-Shanghai-2010
China’s official organizer of the Expo, CCPIT (China Council for the Promotion of International Trade https://en.ccpit.org/) has chosen Dr. Alyce Su as the Cover Person with Cover Story, in the Expo’s official magazine distributed throughout the Expo, showcasing China’s New Generation of Leaders to the World.
The APCO Geopolitical Radar - Q3 2024 The Global Operating Environment for Bu...
The Radar reflects input from APCO’s teams located around the world. It distils a host of interconnected events and trends into insights to inform operational and strategic decisions. Issues covered in this edition include:
TIMES BPO: Business Plan For Startup Industry
Starting a business is like embarking on an unpredictable adventure. It’s a journey filled with highs and lows, victories and defeats. But what if I told you that those setbacks and failures could be the very stepping stones that lead you to fortune? Let’s explore how resilience, adaptability, and strategic thinking can transform adversity into opportunity.
Ellen Burstyn: From Detroit Dreamer to Hollywood Legend | CIO Women Magazine
In this article, we will dive into the extraordinary life of Ellen Burstyn, where the curtains rise on a story that's far more attractive than any script.
Best practices for project execution and delivery
A select set of project management best practices to keep your project on-track, on-cost and aligned to scope. Many firms have don't have the necessary skills, diligence, methods and oversight of their projects; this leads to slippage, higher costs and longer timeframes. Often firms have a history of projects that simply failed to move the needle. These best practices will help your firm avoid these pitfalls but they require fortitude to apply.
Industrial Tech SW: Category Renewal and Creation
Every industrial revolution has created a new set of categories and a new set of players.
Multiple new technologies have emerged, but Samsara and C3.ai are only two companies which have gone public so far.
Manufacturing startups constitute the largest pipeline share of unicorns and IPO candidates in the SF Bay Area, and software startups dominate in Germany.
The Steadfast and Reliable Bull: Taurus Zodiac Sign
Explore the steadfast and reliable nature of the Taurus Zodiac Sign. Discover the personality traits, key dates, and horoscope insights that define the determined and practical Taurus, and learn how their grounded nature makes them the anchor of the zodiac.
Garments ERP Software in Bangladesh _ Pridesys IT Ltd.pdf
Pridesys Garments ERP is one of the leading ERP solution provider, especially for Garments industries which is integrated with
different modules that cover all the aspects of your Garments Business. This solution supports multi-currency and multi-location
based operations. It aims at keeping track of all the activities including receiving an order from buyer, costing of order, resource
planning, procurement of raw materials, production management, inventory management, import-export process, order
reconciliation process etc. It’s also integrated with other modules of Pridesys ERP including finance, accounts, HR, supply-chain etc.
With this automated solution you can easily track your business activities and entire operations of your garments manufacturing
proces
Best Competitive Marble Pricing in Dubai - ☎ 9928909666
Stone Art Hub offers the best competitive Marble Pricing in Dubai, ensuring affordability without compromising quality. With a wide range of exquisite marble options to choose from, you can enhance your spaces with elegance and sophistication. For inquiries or orders, contact us at ☎ 9928909666. Experience luxury at unbeatable prices.
Innovative Uses of Revit in Urban Planning and Design
Discover innovative uses of Revit in urban planning and design, enhancing city landscapes with advanced architectural solutions. Understand how architectural firms are using Revit to transform how processes and outcomes within urban planning and design fields look. They are supplementing work and putting in value through speed and imagination that the architects and planners are placing into composing progressive urban areas that are not only colorful but also pragmatic.
How HR Search Helps in Company Success.pdf
HR search is critical to a company's success because it ensures the correct people are in place. HR search integrates workforce capabilities with company goals by painstakingly identifying, screening, and employing qualified candidates, supporting innovation, productivity, and growth. Efficient talent acquisition improves teamwork while encouraging collaboration. Also, it reduces turnover, saves money, and ensures consistency. Furthermore, HR search discovers and develops leadership potential, resulting in a strong pipeline of future leaders. Finally, this strategic approach to recruitment enables businesses to respond to market changes, beat competitors, and achieve long-term success.
Part 2 Deep Dive: Navigating the 2024 Slowdown
Introduction
The global retail industry has weathered numerous storms, with the financial crisis of 2008 serving as a poignant reminder of the sector's resilience and adaptability. However, as we navigate the complex landscape of 2024, retailers face a unique set of challenges that demand innovative strategies and a fundamental shift in mindset. This white paper contrasts the impact of the 2008 recession on the retail sector with the current headwinds retailers are grappling with, while offering a comprehensive roadmap for success in this new paradigm.
Zodiac Signs and Food Preferences_ What Your Sign Says About Your Taste
Know what your zodiac sign says about your taste in food! Explore how the 12 zodiac signs influence your culinary preferences with insights from MyPandit. Dive into astrology and flavors!
The Most Inspiring Entrepreneurs to Follow in 2024.pdf
In a world where the potential of youth innovation remains vastly untouched, there emerges a guiding light in the form of Norm Goldstein, the Founder and CEO of EduNetwork Partners. His dedication to this cause has earned him recognition as a Congressional Leadership Award recipient.
Brian Fitzsimmons on the Business Strategy and Content Flywheel of Barstool S...
On episode 272 of the Digital and Social Media Sports Podcast, Neil chatted with Brian Fitzsimmons, Director of Licensing and Business Development for Barstool Sports.
What follows is a collection of snippets from the podcast. To hear the full interview and more, check out the podcast on all podcast platforms and at www.dsmsports.net
一比一原版(QMUE毕业证书)英国爱丁堡玛格丽特女王大学毕业证文凭如何办理
永久可查学历认证【微信：A575476】【(QMUE毕业证书)英国爱丁堡玛格丽特女王大学毕业证成绩单Offer】【微信：A575476】（留信学历认证永久存档查询）采用学校原版纸张、特殊工艺完全按照原版一比一制作（包括：隐形水印，阴影底纹，钢印LOGO烫金烫银，LOGO烫金烫银复合重叠，文字图案浮雕，激光镭射，紫外荧光，温感，复印防伪）行业标杆！精益求精，诚心合作，真诚制作！多年品质 ,按需精细制作，24小时接单,全套进口原装设备，十五年致力于帮助留学生解决难题，业务范围有加拿大、英国、澳洲、韩国、美国、新加坡，新西兰等学历材料，包您满意。
【业务选择办理准则】
一、工作未确定，回国需先给父母、亲戚朋友看下文凭的情况，办理一份就读学校的毕业证【微信：A575476】文凭即可
二、回国进私企、外企、自己做生意的情况，这些单位是不查询毕业证真伪的，而且国内没有渠道去查询国外文凭的真假，也不需要提供真实教育部认证。鉴于此，办理一份毕业证【微信：A575476】即可
三、进国企，银行，事业单位，考公务员等等，这些单位是必需要提供真实教育部认证的，办理教育部认证所需资料众多且烦琐，所有材料您都必须提供原件，我们凭借丰富的经验，快捷的绿色通道帮您快速整合材料，让您少走弯路。
留信网认证的作用:
1:该专业认证可证明留学生真实身份
2:同时对留学生所学专业登记给予评定
3:国家专业人才认证中心颁发入库证书
4:这个认证书并且可以归档倒地方
5:凡事获得留信网入网的信息将会逐步更新到个人身份内，将在公安局网内查询个人身份证信息后，同步读取人才网入库信息
6:个人职称评审加20分
7:个人信誉贷款加10分
8:在国家人才网主办的国家网络招聘大会中纳入资料，供国家高端企业选择人才
→ 【关于价格问题（保证一手价格）
我们所定的价格是非常合理的，而且我们现在做得单子大多数都是代理和回头客户介绍的所以一般现在有新的单子 我给客户的都是第一手的代理价格，因为我想坦诚对待大家 不想跟大家在价格方面浪费时间
对于老客户或者被老客户介绍过来的朋友，我们都会适当给一些优惠。
选择实体注册公司办理，更放心，更安全！我们的承诺：可来公司面谈，可签订合同，会陪同客户一起到教育部认证窗口递交认证材料，客户在教育部官方认证查询网站查询到认证通过结果后付款，不成功不收费！
Recently uploaded (20)
The Genesis of BriansClub.cm Famous Dark WEb Platform
The Genesis of BriansClub.cm Famous Dark WEb Platform
Cover Story - China's Investment Leader - Dr. Alyce SU
Cover Story - China's Investment Leader - Dr. Alyce SU
The APCO Geopolitical Radar - Q3 2024 The Global Operating Environment for Bu...
The APCO Geopolitical Radar - Q3 2024 The Global Operating Environment for Bu...
Ellen Burstyn: From Detroit Dreamer to Hollywood Legend | CIO Women Magazine
Ellen Burstyn: From Detroit Dreamer to Hollywood Legend | CIO Women Magazine
The Steadfast and Reliable Bull: Taurus Zodiac Sign
The Steadfast and Reliable Bull: Taurus Zodiac Sign
Garments ERP Software in Bangladesh _ Pridesys IT Ltd.pdf
Garments ERP Software in Bangladesh _ Pridesys IT Ltd.pdf
Best Competitive Marble Pricing in Dubai - ☎ 9928909666
Best Competitive Marble Pricing in Dubai - ☎ 9928909666
Innovative Uses of Revit in Urban Planning and Design
Innovative Uses of Revit in Urban Planning and Design
Zodiac Signs and Food Preferences_ What Your Sign Says About Your Taste
Zodiac Signs and Food Preferences_ What Your Sign Says About Your Taste
The Most Inspiring Entrepreneurs to Follow in 2024.pdf
The Most Inspiring Entrepreneurs to Follow in 2024.pdf
Brian Fitzsimmons on the Business Strategy and Content Flywheel of Barstool S...
Brian Fitzsimmons on the Business Strategy and Content Flywheel of Barstool S...
Project_Plan-Datalake_v1.0_26-10-2022.pptx
- 1. DB Migration to AWS Data lake PROJECT TIMELINE Sep 16 Sep 23 Sep 30 Oct 7 Oct 14 Oct 21 Oct 28 Nov 4 Nov 11 Nov 18 Nov 25 Dec 2 Dec 9 Dec 16 Dec 23 Dec 30 Completed In Progress Leaves New Joiner Milestones/Deliverabl es To be started DB Migration • Finance DB • ArchiveDB • Marketing DB DB Migration • Dailer • Faq Service • Rewards • Collection DB Migration • Collections- external • finance_logs DB Migration • smartcredit- roach2 • Smartcredit 1. DB Migration • Analytics DB • Auth Left over table migration in all the DB’s with issues Timescale DB optimization Tuning Performance Improvement DB Migration • mpokketlive • mpokket • mpokket – common • Marketing Dataease contract renewal CE report migration • 15 report migration to Athena • Report Validation • Scheduling over jenkins CE report go live for 15 reports
- 2. Data Modeling Aggregate generation and Analytics and CE Report Storage Saving Performance optimization PROJECT TIMELINE Nov 15 Nov 30 Dec 15 Dec 30 Jan 15 Jan 30 Feb 15 Feb 28 Mar 15 Mar 30 Completed In Progress Leaves New Joiner Milestones/Deliverabl es To be started • Base data model • Master Creation • Source system study • Future scope Identification of reports which can be delivered via EMR Dataease contract renewal • Aggregate development • Testing • Deployment in production Batch 1 of 15 CE reports • Development • Testing Batch 1 • Productio n go live Batch 2 of 15 CE reports • Development • Testing Batch 1 • Productio n go live Batch 3 of 15 CE reports • Development • Testing Batch 1 • Productio n go live