Survey on NoSQL Database

The data lake has become extremely popular, but there is still confusion on how it should be used. In this presentation I will cover common big data architectures that use the data lake, the characteristics and benefits of a data lake, and how it works in conjunction with a relational data warehouse. Then I’ll go into details on using Azure Data Lake Store Gen2 as your data lake, and various typical use cases of the data lake. As a bonus I’ll talk about how to organize a data lake and discuss the various products that can be used in a modern data warehouse.

Chug building a data lake in azure with spark and databricks

Brandon Berlinrut

Varadarajan CVVaradarajan Sourirajan

Modern business intelligence

Michael Stephenson

Automating Data Quality Processes at Reckitt

Sandeep Sharma IIMK Smart City,IoT,Bigdata,Cloud,BI,DW

Reckitt is a fast-moving consumer goods company with a portfolio of famous brands and over 30k employees worldwide. With that scale small projects can quickly grow into big datasets, and processing and cleaning all that data can become a challenge. To solve that challenge we have created a metadata driven ETL framework for orchestrating data transformations through parametrised SQL scripts. It allows us to create various paths for our data as well as easily version control them. The approach of standardising incoming datasets and creating reusable SQL processes has proven to be a winning formula. It has helped simplify complicated landing/stage/merge processes and allowed them to be self-documenting. But this is only half the battle, we also want to create data products. Documented, quality assured data sets that are intuitive to use. As we move to a CI/CD approach, increasing the frequency of deployments, the demand of keeping documentation and data quality assessments up to date becomes increasingly challenging. To solve this problem, we have expanded our ETL framework to include SQL processes that automate data quality activities. Using the Hive metastore as a starting point, we have leveraged this framework to automate the maintenance of a data dictionary and reduce documenting, model refinement, testing data quality and filtering out bad data to a box filling exercise. In this talk we discuss our approach to maintaining high quality data products and share examples of how we automate data quality processes.

Democratizing data science Using spark, hive and druid

DataWorks Summit

MZ is re-inventing how the entire world experiences data via our mobile games division MZ Games Studios, our digital marketing division Cognant, and our live data platform division Satori. Growing need of data science capabilities across the organization requires an architecture that can democratize building these applications and disseminating insight from the outcome of data science applications to the wider organization. Attend this session to learn about how we built a platform for data science using spark, hive, and druid specifically for our performance marketing division cognant.This platform powers several data science application like fraud detection and bid optimization at large scale. We will be sharing lessons learned over past 3 years in building this platform by also walking through some of the actual data science applications built on top of this platform. Attendees from ML engineering and data science background can gain deep insight from our experience of building this platform. Speakers Pushkar Priyadarshi, Director of Engineer, Michaine Zone Inc Igor Yurinok, Staff Software Engineer, MZ

Solution architecture Amazon web services

IBM Cloud Native Day April 2021: Serverless Data Lake

Torsten Steinbach

Formulating Power BI Enterprise StrategyTeo Lachev

Is the traditional data warehouse dead?

With new technologies such as Hive LLAP or Spark SQL, do I still need a data warehouse or can I just put everything in a data lake and report off of that? No! In the presentation I’ll discuss why you still need a relational data warehouse and how to use a data lake and a RDBMS data warehouse to get the best of both worlds. I will go into detail on the characteristics of a data lake and its benefits and why you still need data governance tasks in a data lake. I’ll also discuss using Hadoop as the data lake, data virtualization, and the need for OLAP in a big data solution. And I’ll put it all together by showing common big data architectures.

Snowflakes in the Cloud Real world experience on a new approach for Big Data

DevFest DC

Building End-to-End Delta Pipelines on GCP

Delta has been powering many production pipelines at scale in the Data and AI space since it has been introduced for the past few years. Built on open standards, Delta provides data reliability, enhances storage and query performance to support big data use cases (both batch and streaming), fast interactive queries for BI and enabling machine learning. Delta has matured over the past couple of years in both AWS and AZURE and has become the de-facto standard for organizations building their Data and AI pipelines. In today’s talk, we will explore building end-to-end pipelines on the Google Cloud Platform (GCP). Through presentation, code examples and notebooks, we will build the Delta Pipeline from ingest to consumption using our Delta Bronze-Silver-Gold architecture pattern and show examples of Consuming the delta files using the Big Query Connector.

Running cost effective big data workloads with Azure Synapse and ADLS (MS Ign...

Michael Rys

Presentation by James Baker and myself on Running cost effective big data workloads with Azure Synapse and Azure Datalake Storage (ADLS) at Microsoft Ignite 2020. Covers Modern Data warehouse architecture supported by Azure Synapse, integration benefits with ADLS and some features that reduce cost such as Query Acceleration, integration of Spark and SQL processing with integrated meta data and .NET For Apache Spark support.

Data Mesh

Piethein Strengholt

Power BI for Big Data and the New Look of Big Data Solutions

New features in Power BI give it enterprise tools, but that does not mean it automatically creates an enterprise solution. In this talk we will cover these new features (composite models, aggregations tables, dataflow) as well as Azure Data Lake Store Gen2, and describe the use cases and products of an individual, departmental, and enterprise big data solution. We will also talk about why a data warehouse and cubes still should be part of an enterprise solution, and how a data lake should be organized.

Data Lake

Anitha Krishnappa

Power bi ea content pack v0.1

Luca Mauri

Tarun poladi resume

Tarun P

Lesson Design and Development for 21st Century Learners (in Thai) การออกแบบแล...

การใช้พื้นที่และการทำงานร่วมกันบนคลาวด์ Cloud storage and collaboration (ภาษา...

What's hot

Leveraging Azure Analysis Services Tabular Data Models with Power BI by Tim M...

KTL Solutions

Data pipeline and data lake for autonomous driving

Yu Huang

Data Lake Overview

Chug building a data lake in azure with spark and databricks

Brandon Berlinrut

Varadarajan CVVaradarajan Sourirajan

Modern business intelligence

Michael Stephenson

Automating Data Quality Processes at Reckitt

Sandeep Sharma IIMK Smart City,IoT,Bigdata,Cloud,BI,DW

Democratizing data science Using spark, hive and druid

DataWorks Summit

Solution architecture Amazon web services

IBM Cloud Native Day April 2021: Serverless Data Lake

Torsten Steinbach

Formulating Power BI Enterprise StrategyTeo Lachev

Is the traditional data warehouse dead?

Snowflakes in the Cloud Real world experience on a new approach for Big Data

DevFest DC

Building End-to-End Delta Pipelines on GCP

Running cost effective big data workloads with Azure Synapse and ADLS (MS Ign...

Michael Rys

Data Mesh

Piethein Strengholt

Power BI for Big Data and the New Look of Big Data Solutions

Data Lake

Anitha Krishnappa

Power bi ea content pack v0.1

Luca Mauri

Tarun poladi resume

Tarun P

What's hot (20)

Leveraging Azure Analysis Services Tabular Data Models with Power BI by Tim M...

Data pipeline and data lake for autonomous driving

Data Lake Overview

Chug building a data lake in azure with spark and databricks

Varadarajan CV

Modern business intelligence

Automating Data Quality Processes at Reckitt

Democratizing data science Using spark, hive and druid

Solution architecture Amazon web services

IBM Cloud Native Day April 2021: Serverless Data Lake

Formulating Power BI Enterprise Strategy

Is the traditional data warehouse dead?

Snowflakes in the Cloud Real world experience on a new approach for Big Data

Building End-to-End Delta Pipelines on GCP

Running cost effective big data workloads with Azure Synapse and ADLS (MS Ign...

Data Mesh

Power BI for Big Data and the New Look of Big Data Solutions

Data Lake

Power bi ea content pack v0.1

Tarun poladi resume

More from Mayuree Srikulwong

Lesson Design and Development for 21st Century Learners (in Thai) การออกแบบแล...

การใช้พื้นที่และการทำงานร่วมกันบนคลาวด์ Cloud storage and collaboration (ภาษา...

Introduction to Computer and Software คอมพิวเตอร์เบื้องต้นและซอฟแวร์ (ภาษาไทย)

Privacy and security ความเป็นส่วนตัวและความปลอดภัยในการใช้คอมพิวเตอร์ (ภาษาไทย)

กฎหมายและ พรบ คอมพิวเตอร์ การป้องกันข้อมูลและความเป็นส่วนตัว ความปลอดภัยในการใช้อินเตอร์เน็ท การป้องกันภัยคุกคามอื่น ๆ ประเด็นทางจริยธรรมในการใช้เครือข่ายสังคมออนไลน์ ไวรัสและการป้องกัน มัลแวร์

เทคนิคการทำงานนำเสนอ Presentation Design Techniques (2019) - In Thai and English

(ปรับปรุงจากฉบับเดิม) The ultimate guides to prepare presentation which contains three major elements: structure, content and techniques. I have combined knowledge and knowhow from several world renowned experts adding tips I have learned by practising and teaching for years. หลักการจัดทำงานนำเสนอที่รวบรวมเคล็ดลับจากกูรูระดับโลก นำเสนอพร้อมตัวอย่างจากการฝึกใช้งานจริง และ การให้การอบรมแก่นักศึกษาและประชาชนทั่วไป

การพัฒนานวัตกรรมและเทคโนโลยีการศึกษา Development of Educational Innovation an...

เอกสารประกอบการบรรยายเรื่อง การพัฒนานวัตกรรมและเทคโนโลยีการศึกษา Development of Educational Innovation and Technology ในงานประชุมวิชาการ NEPCon 2019 ณ โรงเรียนสุรวิวัฒน์ มหาวิทยาล้ยเทคโนโลยีราชสีมา

Design thinking การคิดเชิงออกแบบ (Thai - English)

17 เรื่องต้องรู้ก่อนขายของออนไลน์เพื่อการส่งออก Introduction to E-commerce B2...

***Content is in Thai.*** ความรู้เบื้องต้นเกี่ยวกับ E-commerce 17 เรื่องต้องรู้ก่อนขายของออนไลน์ เพื่อการส่งออก สถิติ E-commerce ในประเทศไทย และ ระดับโลก เทคนิคการหา Keywords เทคนิคการตั้งชื่อสินค้าเพื่อขายของออนไลน์ เทคนิคการถ่ายภาพสินค้า Algorithm การคิดคะแนนของแพลตฟอร์มต่าง ๆ และ อื่น ๆ

Understanding Gender Stereotype Lesson Plan

Understanding Gender Stereotype Lesson Plan. It gives the objectives and procedures (activities) of a 3-hour session. It aims to help students understand the following: 1. Assumptions can lead to stereotypes and unfair judgments about individuals and groups. 2. Stereotypes and biases affect our lives. Using this lesson plan, students will achieve deep learning. They are not only understand the matter but also share intimate experience on the matter. Feedbacks from students prove that this method yield better results compared to traditional lesson (lecture-based). The same lesson plan can be adapted for race and age stereotypes.

Business model canvas in Thai

Innovative Learning Classroom Workshop for ThaiPOD2018

Slides used to demonstrate a case study of teaching digital innovation creation at the University of the Thai Chamber of Commerce, presented at the 13th annual conference for ThaiPOD on Mar 30, 2018. The slides contain UTCC's journey history to 21st century learning and role-play workshop scripts which direct participants to use Swift Playgrounds and Marvel App to learn computational thinking skill and create a mock-up app to solve a real-world problem. The slides end with a recap of TPACK and OBDA structure created on iTunesU for students to follow and a summary of innovative learning classroom benefits.

Airbnb Pitch Deck Case Study (in Thai)

Digital Innovative Thinking and Coding

Using a WHY-WHAT-HOW model, this material provides a summary of processes required for digital innovation creation which starts with a reason why we should be studying this subject; briefly explains what Computational Thinking (CT), User eXperience (UX) and User Interface (UI) are; and describes step-by-step how to achieve the goal of designing digital innovation through CT, UX and UI design processes.

Building Computational Thinking Skill with Swift Playgrounds

User Experience UX Design (Thai mixed with English)

The Power of Visual Content

Presentation Design Techniques (in Thai) 2017

Fan culture (case from Thailand, 2017)

Anger in Social Media (Thailand landscape, 2017)