Skip to main content
UploadLanguage (EN)Support
BusinessMobileSocial MediaMarketingTechnologyArt & PhotosCareerDesignEducationPresentations & Public SpeakingGovernment & NonprofitHealthcareInternetLawLeadership & ManagementAutomotiveEngineeringSoftwareRecruiting & HRRetailSalesServicesScienceSmall Business & EntrepreneurshipFoodEnvironmentEconomy & FinanceData & AnalyticsInvestor RelationsSportsSpiritualNews & PoliticsTravelSelf ImprovementReal EstateEntertainment & HumorHealth & MedicineDevices & HardwareLifestyle
Till Rohrmann, profile picture
Uploaded byTill Rohrmann
PDF, PPTX5,759 views

Interactive Data Analysis with Apache Flink @ Flink Meetup in Berlin

The document discusses interactive data analysis using Apache Flink and Apache Zeppelin, highlighting their features such as exploratory data analysis, a read-evaluate-print loop, and web-based notebooks. It provides examples of word count analysis and linear regression to predict sales based on advertisement spending, as well as a classifier for insult detection from social commentary. The conclusion emphasizes the ease of interactive data analysis with these tools, especially in machine learning tasks.

Technology◦
Related topics:
Apache Flink•Data Analysis Insights•
Download as PDF, PPTX
Till Rohrmann Flink PMC member trohrmann@apache.org @stsffap Interactive Data Analysis with Apache Flink
Data Analysis 1
Exploratory Data Analysis §  Visualize data §  Calculate main characteristics §  Understand data and find possibly new hypothesis 2
Data Analysts 3
Read-Evaluate-Print Loop §  New Scala shell offers REPL §  Interactive queries §  Let’s you explore data quickly 4
Scala Shell 5
Simple Scala Shell Example 6
Problems §  No visualization §  No saving or replaying of written code §  No assistance à Bad IDE 7
Notebooks §  Web-based interactive computation environment §  Combines rich text, execution code, plots and rich media §  Storytelling 8
Apache Zeppelin §  Web-based REPL with pluggable interpreters §  Since 2014 in the Apache Incubator §  Supported interpreters: •  Flink •  Spark •  Python •  Markdown •  Many more … 9
Word Count with Zeppelin §  Find the 10 most frequent words with more than 4 letters in the King James version of the bible. 10
11
12
13
14
Linear regression §  Let’s predict the influence of advertisement spending on sales §  Input data set: http://www-bcf.usc.edu/~gareth/ISL/ Advertising.csv §  Features: •  TV advertisement money •  Radio advertisement money •  Newspaper advertisement money §  Response: •  Sales 15
16
17
18
19
20
21
22
23
24
Classification §  Let’s build a classifier for insult detection §  Kaggle challenge https://www.kaggle.com/c/detecting- insults-in-social-commentary §  Label: 1 – Insult, 0 – No insult §  Feature: Comment text 25
26
27
Conclusion §  Interactive data analysis is really easy with Apache Flink §  Apache Zeppelin is great interactive notebook §  Zeppelin and Flink play well together to solve machine learning tasks and more 28
29
flink.apache.org @ApacheFlink