2013-09-28 Hadoop in Taiwan 2013
Three New Trends of Big Data
即時‧安全‧易用
王耀聰 / 國家高速網路與計算中心
Jazz Yao-Tsung Wang / NCHC
<jazz@...
2013-09-28 Hadoop in Taiwan 2013 2
教師節快樂!謝謝各位蒞臨!教師節快樂!謝謝各位蒞臨!
感謝主辦單位
與贊助廠商
祝台下的老師們教師節快樂!
Happy Teacher's Day !!
2013-09-28 Hadoop in Taiwan 2013 3
3 Vs of Big Data3 Vs of Big Data
3
巨量資料的挑戰在於如何管理「數量」、「增加率」與「多樣性」
Volume 資料數量
(amount of...
2013-09-28 Hadoop in Taiwan 2013 4
Life of Big DataLife of Big Data :蒐、存、取、析、用:蒐、存、取、析、用
2013-09-28 Hadoop in Taiwan 2013 5
Big Data is the Answer - What was the Question?Big Data is the Answer - What was the Qu...
2013-09-28 Hadoop in Taiwan 2013 6
Big Data at Rest – MapReduce FrameworkBig Data at Rest – MapReduce Framework
6
Volume
V...
2013-09-28 Hadoop in Taiwan 2013 7
Big Data in Motion –Big Data in Motion –
In-Memory ProcessingIn-Memory Processing 、、 Pr...
2013-09-28 Hadoop in Taiwan 2013 8
Big Data in Motion –Big Data in Motion –
Streaming Data Collection / Data CleaningStrea...
2013-09-28 Hadoop in Taiwan 2013 9
NoHadoop ?! Not Only Hadoop !!NoHadoop ?! Not Only Hadoop !!
Source: Lambda Architectur...
2013-09-28 Hadoop in Taiwan 2013 10
Next Step : Big Data SecurityNext Step : Big Data Security
當我們緊密相連 .....
世界政經:歐盟想分 Twe...
2013-09-28 Hadoop in Taiwan 2013 11
To Find the Value of Big Data
We need Data Scientist Team !
電機
資訊
數學數學
統計統計
商商 做決策
資
料...
2013-09-28 Hadoop in Taiwan 2013 12
議程安排 ( 上午場次 )
即時‧安全‧易用
2013-09-28 Hadoop in Taiwan 2013 13
議程安排 ( 下午場次 )
Upcoming SlideShare
Loading in …5
×

13 09-28 hadoop-in_taiwan_2013_opening

1,027 views
976 views

Published on

Published in: Travel, Technology
0 Comments
3 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
1,027
On SlideShare
0
From Embeds
0
Number of Embeds
15
Actions
Shares
0
Downloads
36
Comments
0
Likes
3
Embeds 0
No embeds

No notes for slide

13 09-28 hadoop-in_taiwan_2013_opening

  1. 1. 2013-09-28 Hadoop in Taiwan 2013 Three New Trends of Big Data 即時‧安全‧易用 王耀聰 / 國家高速網路與計算中心 Jazz Yao-Tsung Wang / NCHC <jazz@nchc.narl.org.tw>
  2. 2. 2013-09-28 Hadoop in Taiwan 2013 2 教師節快樂!謝謝各位蒞臨!教師節快樂!謝謝各位蒞臨! 感謝主辦單位 與贊助廠商 祝台下的老師們教師節快樂! Happy Teacher's Day !!
  3. 3. 2013-09-28 Hadoop in Taiwan 2013 3 3 Vs of Big Data3 Vs of Big Data 3 巨量資料的挑戰在於如何管理「數量」、「增加率」與「多樣性」 Volume 資料數量 (amount of data) Velocity 資料增加率 (speed of data in/out) Variety 資料多樣性 (data types, sources) Batch ( 批次作業 ) Realtime ( 即時資料 ) TB EB Unstructured 非結構化資料 Semi-structured 半結構化資料 Structured 結構化資料 PB 參考來源: [1] Laney, Douglas. "3D Data Management: Controlling Data Volume, Velocity and Variety" (6 February 2001) [2] Gartner Says Solving 'Big Data' Challenge Involves More Than Just Managing Volumes of Data, June 2011
  4. 4. 2013-09-28 Hadoop in Taiwan 2013 4 Life of Big DataLife of Big Data :蒐、存、取、析、用:蒐、存、取、析、用
  5. 5. 2013-09-28 Hadoop in Taiwan 2013 5 Big Data is the Answer - What was the Question?Big Data is the Answer - What was the Question? 參考來源: Big Data is the Answer - What was the Question? http://www.saama.com/blog/bid/76211/Big-Data-is-the-Answer-What-was-the-Question
  6. 6. 2013-09-28 Hadoop in Taiwan 2013 6 Big Data at Rest – MapReduce FrameworkBig Data at Rest – MapReduce Framework 6 Volume VelocityVariety TB EB PB Realtime Batch Structured Unstructured M apReduce Fram ework PetabyteFileSystem HadoopHadoop HPCCHPCC 存、取、析
  7. 7. 2013-09-28 Hadoop in Taiwan 2013 7 Big Data in Motion –Big Data in Motion – In-Memory ProcessingIn-Memory Processing 、、 Predictive AnalyticsPredictive Analytics Volume VelocityVariety TB EB PB Realtime Batch Structured Unstructured HBase / DrillHBase / Drill Impala / SparkImpala / Spark 取、析、用
  8. 8. 2013-09-28 Hadoop in Taiwan 2013 8 Big Data in Motion –Big Data in Motion – Streaming Data Collection / Data CleaningStreaming Data Collection / Data Cleaning 8 Volume VelocityVariety TB EB PB Realtime Batch Structured Unstructured Message QueueMessage Queue ( AMQP , RabbitMQ )( AMQP , RabbitMQ ) Storm / KafkaStorm / Kafka 蒐、存 ( 前處理 )
  9. 9. 2013-09-28 Hadoop in Taiwan 2013 9 NoHadoop ?! Not Only Hadoop !!NoHadoop ?! Not Only Hadoop !! Source: Lambda Architecture, 8. March 2013 http://www.ymc.ch/en/lambda-architecture-part-1 HBase Storm ElephantDB Or Voldemort Hadoop
  10. 10. 2013-09-28 Hadoop in Taiwan 2013 10 Next Step : Big Data SecurityNext Step : Big Data Security 當我們緊密相連 ..... 世界政經:歐盟想分 Tweeter 找出經濟、政治的脈動 國家安全:美國 PRISM 計劃 ( 網軍 ! 終極警探 4.0 ) 組織如何因應 APT ? Big Data 平台本身的安全性 ? 有太多安全的問題等待解決! Source: Gartner (March 2011), 'Big Data' Is Only the Beginning of Extreme Information Management, 7 April 2011, http://www.gartner.com/id=1622715 權限管控 品質管控 數量管控
  11. 11. 2013-09-28 Hadoop in Taiwan 2013 11 To Find the Value of Big Data We need Data Scientist Team ! 電機 資訊 數學數學 統計統計 商商 做決策 資 料 科 學 家 分 析 軟 體 重點在找到價值 Value
  12. 12. 2013-09-28 Hadoop in Taiwan 2013 12 議程安排 ( 上午場次 ) 即時‧安全‧易用
  13. 13. 2013-09-28 Hadoop in Taiwan 2013 13 議程安排 ( 下午場次 )

×