Customer Use Cases & Pain Points of (Big) Metadata

•

0 likes•209 views

Customer Use Cases & Pain Points of (Big) Metadata talk from May 2018 Hive User Group Meeting @Cloudera - https://www.meetup.com/Hive-User-Group-Meeting/events/249641278/

Data & Analytics

1© Cloudera, Inc. All rights reserved.
Metadata
Rituparna Agrawal
May 08, 2018

2 © Cloudera, Inc. All rights reserved.
Workflows
OLTP/DW
OLTP/DW
OLTP/DW
Raw Base Tables
Datalake1
Datalake2
Datalaken
App 1
App 2
App n
Hive/Spark IMPALASqoop
ETL/BI Handoff

3 © Cloudera, Inc. All rights reserved.
Workflows
OLTP/DW
OLTP/DW
OLTP/DW
Raw Base Tables
Datalake1
Datalake2
Datalaken
App 1
App 2
App n
Hive/Spark IMPALASqoop
Impala Crash
NameNode Crash
HMS Crash/Hung ETL/BI Handoff

4 © Cloudera, Inc. All rights reserved.
LARGE METADATA
Databases
Tables Columns
Non-temp files and directories
Partitions
Queries

5 © Cloudera, Inc. All rights reserved.
Small Files
Non-temp files and directories < 100 Kb

6 © Cloudera, Inc. All rights reserved.
Handoff
Web_sales
Store_sales
Logical Checkpoints
in ingest
Ingest activity
Impala Checkpoint
(“Refresh <table>”)

7 © Cloudera, Inc. All rights reserved.
Eco System
HMS
NameNode
Sentry
Ranger
Metadata

Recently uploaded

Anomaly detection and data imputation within time seriesParis Women in Machine Learning and Data Science

Halmar dropshipping via API with DroFxolyaivanovalion

April 2024 - Crypto Market Report's Analysismanisha194592

5CL-ADBA,5cladba, Chinese supplier, safety is guaranteedamy56318795

Call me @ 9892124323 Cheap Rate Call Girls in Vashi with Real Photo 100% SecurePooja Nehwal

Call Girls Bommasandra Just Call 👗 7737669865 👗 Top Class Call Girl Service B...amitlee9823

Escorts Service Kumaraswamy Layout ☎ 7737669865☎ Book Your One night Stand (B...amitlee9823

Call Girls Hsr Layout Just Call 👗 7737669865 👗 Top Class Call Girl Service Ba...amitlee9823

Thane Call Girls 7091864438 Call Girls in Thane Escort service book now -Pooja Nehwal

Week-01-2.ppt BBB human Computer interactionfulawalesam

Mature dropshipping via API with DroFx.pptxolyaivanovalion

BigBuy dropshipping via API with DroFx.pptxolyaivanovalion

Predicting Loan Approval: A Data Science ProjectBoston Institute of Analytics

Call Girls Indiranagar Just Call 👗 7737669865 👗 Top Class Call Girl Service B...amitlee9823

Call Girls In Bellandur ☎ 7737669865 🥵 Book Your One night Standamitlee9823

Chintamani Call Girls: 🍓 7737669865 🍓 High Profile Model Escorts | Bangalore ...amitlee9823

Sampling (random) method and Non random.pptDr. Soumendra Kumar Patra

Al Barsha Escorts $#$ O565212860 $#$ Escort Service In Al BarshaAroojKhan71

Midocean dropshipping via API with DroFxolyaivanovalion

Accredited-Transport-Cooperatives-Jan-2021-Web.pdfadriantubila

Recently uploaded (20)

Anomaly detection and data imputation within time series

Halmar dropshipping via API with DroFx

April 2024 - Crypto Market Report's Analysis

5CL-ADBA,5cladba, Chinese supplier, safety is guaranteed

Call me @ 9892124323 Cheap Rate Call Girls in Vashi with Real Photo 100% Secure

Call Girls Bommasandra Just Call 👗 7737669865 👗 Top Class Call Girl Service B...

Escorts Service Kumaraswamy Layout ☎ 7737669865☎ Book Your One night Stand (B...

Call Girls Hsr Layout Just Call 👗 7737669865 👗 Top Class Call Girl Service Ba...

Thane Call Girls 7091864438 Call Girls in Thane Escort service book now -

Week-01-2.ppt BBB human Computer interaction

Mature dropshipping via API with DroFx.pptx

BigBuy dropshipping via API with DroFx.pptx

Predicting Loan Approval: A Data Science Project

Call Girls Indiranagar Just Call 👗 7737669865 👗 Top Class Call Girl Service B...

Call Girls In Bellandur ☎ 7737669865 🥵 Book Your One night Stand

Chintamani Call Girls: 🍓 7737669865 🍓 High Profile Model Escorts | Bangalore ...

Sampling (random) method and Non random.ppt

Al Barsha Escorts $#$ O565212860 $#$ Escort Service In Al Barsha

Midocean dropshipping via API with DroFx

Accredited-Transport-Cooperatives-Jan-2021-Web.pdf

Featured

Skeleton Culture CodeSkeleton Technologies

PEPSICO Presentation to CAGNY Conference Feb 2024Neil Kimberley

Content Methodology: A Best Practices Report (Webinar)contently

How to Prepare For a Successful Job Search for 2024Albert Qian

Social Media Marketing Trends 2024 // The Global Indie InsightsKurio // The Social Media Age(ncy)

Trends In Paid Search: Navigating The Digital Landscape In 2024Search Engine Journal

5 Public speaking tips from TED - Visualized summarySpeakerHub

ChatGPT and the Future of Work - Clark Boyd Clark Boyd

Getting into the tech field. what next Tessa Mero

Google's Just Not That Into You: Understanding Core Updates & Search IntentLily Ray

How to have difficult conversations Rajiv Jayarajah, MAppComm, ACC

Introduction to Data ScienceChristy Abraham Joy

Time Management & Productivity - Best PracticesVit Horky

The six step guide to practical project managementMindGenius

Beginners Guide to TikTok for Search - Rachel Pearson - We are Tilt __ Bright...RachelPearson36

Unlocking the Power of ChatGPT and AI in Testing - A Real-World Look, present...Applitools

12 Ways to Increase Your Influence at WorkGetSmarter

ChatGPT webinar slidesAlireza Esmikhani

More than Just Lines on a Map: Best Practices for U.S Bike RoutesProject for Public Spaces & National Center for Biking and Walking

Ride the Storm: Navigating Through Unstable Periods / Katerina Rudko (Belka G...DevGAMM Conference

Featured (20)

Skeleton Culture Code

PEPSICO Presentation to CAGNY Conference Feb 2024

Content Methodology: A Best Practices Report (Webinar)

How to Prepare For a Successful Job Search for 2024

Social Media Marketing Trends 2024 // The Global Indie Insights

Trends In Paid Search: Navigating The Digital Landscape In 2024

5 Public speaking tips from TED - Visualized summary

ChatGPT and the Future of Work - Clark Boyd

Getting into the tech field. what next

Google's Just Not That Into You: Understanding Core Updates & Search Intent

How to have difficult conversations

Introduction to Data Science

Time Management & Productivity - Best Practices

The six step guide to practical project management

Beginners Guide to TikTok for Search - Rachel Pearson - We are Tilt __ Bright...

Unlocking the Power of ChatGPT and AI in Testing - A Real-World Look, present...

12 Ways to Increase Your Influence at Work

ChatGPT webinar slides

More than Just Lines on a Map: Best Practices for U.S Bike Routes

Ride the Storm: Navigating Through Unstable Periods / Katerina Rudko (Belka G...