Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

빅데이터 구축 사례

2,698 views

Published on

유통업에서 구축한 빅데이터 구축 사례 입니다

Published in: Software
  • If you want to download or read this book, copy link or url below in the New tab ......................................................................................................................... DOWNLOAD FULL PDF EBOOK here { http://bit.ly/2m77EgH } ......................................................................................................................... Download EPUB Ebook here { http://bit.ly/2m77EgH } .........................................................................................................................
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • accessibility Books Library allowing access to top content, including thousands of title from favorite author, plus the ability to read or download a huge selection of books for your pc or smartphone within minutes DOWNLOAD THIS BOOKS INTO AVAILABLE FORMAT ......................................................................................................................... ......................................................................................................................... Download Full PDF EBOOK here { https://urlzs.com/UABbn } ......................................................................................................................... Download Full EPUB Ebook here { https://urlzs.com/UABbn } ......................................................................................................................... ...................................ALL FOR EBOOKS................................................. Cookbooks, Manga, Memoir, Music, Mystery, Non Fiction, Paranormal, Philosophy, Poetry, Psychology, Religion, Art, Biography, Business, Chick Lit, Children's, Christian, Classics, Comics, Contemporary, Romance, Science, Science Fiction, Self Help, Suspense, Spirituality, Sports, Thriller, Travel, Young Adult, Crime, Ebooks, Fantasy, Fiction, Graphic Novels, Historical Fiction, History, Horror, Humor And Comedy,
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • DOWNLOAD THIS BOOKS INTO AVAILABLE FORMAT (Unlimited) ......................................................................................................................... ......................................................................................................................... Download Full PDF EBOOK here { https://tinyurl.com/y6a5rkg5 } ......................................................................................................................... Download Full EPUB Ebook here { https://tinyurl.com/y6a5rkg5 } ......................................................................................................................... ACCESS WEBSITE for All Ebooks ......................................................................................................................... Download Full PDF EBOOK here { https://tinyurl.com/y6a5rkg5 } ......................................................................................................................... Download EPUB Ebook here { https://tinyurl.com/y6a5rkg5 } ......................................................................................................................... Download doc Ebook here { https://tinyurl.com/y6a5rkg5 } ......................................................................................................................... ......................................................................................................................... ......................................................................................................................... .............. Browse by Genre Available eBooks ......................................................................................................................... Art, Biography, Business, Chick Lit, Children's, Christian, Classics, Comics, Contemporary, Cookbooks, Crime, Ebooks, Fantasy, Fiction, Graphic Novels, Historical Fiction, History, Horror, Humor And Comedy, Manga, Memoir, Music, Mystery, Non Fiction, Paranormal, Philosophy, Poetry, Psychology, Religion, Romance, Science, Science Fiction, Self Help, Suspense, Spirituality, Sports, Thriller, Travel, Young Adult,
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here

빅데이터 구축 사례

  1. 1. 빅데이터 구축 사례 오태현 paranmul@gmail.com paranmul@inbrein.com
  2. 2. 개요 빅데이터 구축 사례 소개 주요 진행 내용 하드웨어 구성 소프트웨어 아키텍쳐 오픈소스 목록 기타 구성에 관한 내용
  3. 3. 하드웨어 구성 X 3 Master server Hadoop Hbase X 7 Kafka X 2 Storm X 4 X 2 Backup node
  4. 4. 소프트웨어 아키텍쳐 Collection Request Handler Collection Queue 원천 시스템 Data Store HDFS HBase Memcached OrientDB Read time Storm Beatch Data Workflow Hadoop M/R Campaign Data DW Management Mornitoring 이벤트 원천 Service View
  5. 5. 사용된 오픈소스 목록 • Sqoop * • Hbase • Azkaban * • Graphite * • Ganglia * • Memcached • MariaDB • Verte.x • Flume • Kafka * • Camus • Storm * • Hive • Hadoop • jCascalog *
  6. 6. Kafka • 데이터를 분산, 파티션, 복제 커밋 로그서버 • Topic • 초당 18000개, 로그 하나 크기 1k, • 1초에 17M(한 서버당),1분에 840M, 1시간에 50G • 두대의 Kafka server를 사용
  7. 7. Storm • Realtime • Nimbus • Topology • Spout • Bolt • -> Hbase
  8. 8. jCascalog • Hadoop 의 mapreduce 코딩을 쉽게 하는 오픈소스 • Cascading을 java로 사용할 수 있게함
  9. 9. Sqoop • RDB 와 HDFS사이에 데이터를 전달 하는 오픈소스 • pwd, mssql, mariaDB, Oracle, netezza, mysql, hbase
  10. 10. Azkaban • 스케줄러
  11. 11. Azkaban • 스케줄러
  12. 12. Graphite • Python 기반 • Java 프로세스 모니터링
  13. 13. Ganglia • 서버 자원 상황 모니터링 • 네트워크, 메모리, CPU
  14. 14. 환경 구성에 관해서 • 기간, 인원 • 구성 중 문제점들 • Hortonworks • Oozie 관리 페이지 로딩 • Zookeeper 사용하는 오픈소스 증가 – 분리 • 한 서버에 다양한 오픈소스 공생 • 많은 수의 포트 오픈 • MR job 증가 • 개발서버 부재 • 스톰 트라이던트 & 카프카 • Hbase 키 설계
  15. 15. 감사합니다

×