SparkR 이해 및 분석 사례 소개

1. SparkR 을 이용한 Fraud Detection
빅데이터본부 | FEA팀 남종환
SparkR 을 이해하고, Credit Card Fraud Detection 을 구현 해봅니다.

2. 1. Spark Overview
2. SparkR Motivation / Creator / History
3. SparkRArchitecture
4. SparkR Usage
5. Demo – Credit Card Fraud Detection
6. Future Work
목차

3. Spark Overview

4. SparkR Motivation

5. SparkR Creator
http://shivaram.org/#projects https://www.linkedin.com/in/zonghengyang/

6. SparkR History
SparkR
DataFrame
https://en.wikipedia.org/wiki/Apache_Spark
SparkR Start

7. SparkR Architecture
https://www.slideshare.net/databricks/recent-
developments-in-sparkr-for-advanced-analytics

8. SparkR Usage – How to Run
• Run from Script
• Run from R Studio

9. SparkR Usage - API
http://spark.apache.org/docs/latest/api/R/index.html

10. SparkR Usage - API
http://spark.apache.org/docs/latest/api/R/index.html

11. SparkR Usage - Data Conversion
https://www.slideshare.net/databricks/recent-
developments-in-sparkr-for-advanced-analytics

12. SparkR Usage – Descriptive
https://www.slideshare.net/databricks/recent-
developments-in-sparkr-for-advanced-analytics

13. SparkR Usage – Descriptive
https://www.slideshare.net/databricks/recent-
developments-in-sparkr-for-advanced-analytics

14. SparkR Usage – Descriptive
https://www.slideshare.net/databricks/recent-
developments-in-sparkr-for-advanced-analytics

15. SparkR Usage – Predictive
https://www.slideshare.net/databricks/recent-
developments-in-sparkr-for-advanced-analytics

16. SparkR Usage – Predictive
https://www.slideshare.net/databricks/recent-
developments-in-sparkr-for-advanced-analytics

17. SparkR Usage – 1.6 to 2.0

18. SparkR Future Directions
https://www.slideshare.net/databricks/recent-
developments-in-sparkr-for-advanced-analytics

19. • http://spark.apache.org/docs/latest/api/R/index.html
• http://hoondongkim.blogspot.kr/search/label/SparkML
• https://www.youtube.com/watch?v=cSnkb7HYdc0&feature=youtu.be
• https://www.slideshare.net/databricks/recent-developments-in-sparkr-for-
advanced-analytics
• https://github.com/hoxo-m/SparkRext
참고링크

20. • Credit Card Fraud Detection
• From R To SparkR
Demo
https://www.kaggle.com/dalpozz/creditcardfraud

21. • Dimension : 284,807 rows x 31 columns
• Description
• Time : 시간
• V1 - V29 : 카드 사용에 관한 데이터, 변수명으로 추측불가
• Class : 1 = Fraud, 0 = normal
Demo

22. Demo
• Work Flow ( Logistic Regression )
• Data collection
• Data Sampling
• Predictive Modelling
• Split data 70 : 30
• Convert Rdata to SparkDataFrame
• Logistic Regression
• Check output class distribution

23. 감사합니다
빅데이터본부 | FEA팀 남종환