Week 15 - Scaling & Security_Jim Huang

1,358 views

Published on

Week 15

軟體效能分析與安全性議題

黃敬群 Jim Huang
(聯發科技顧問)

(2013-05-30)

Published in: Education, Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
1,358
On SlideShare
0
From Embeds
0
Number of Embeds
7
Actions
Shares
0
Downloads
0
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Week 15 - Scaling & Security_Jim Huang

  1. 1. App創業與實作本授權條款允許使用者重製、散布、傳輸著作,但不得為商業目的之使用,亦不得修改該著作。使用時必須按照著作人指定的方式表彰其姓名。App Entrepreneurship and implementationWeek 15軟體效能分析與安全性議題黃敬群 Jim Huang(聯發科技顧問)(2013-05-30)
  2. 2. 姓名標示─非商業性─禁止改作本授權條款允許使用者重製、散布、傳輸著作,但不得為商業目的之使用,亦不得修改該著作。使用時必須按照著作人指定的方式表彰其姓名。CC (Creative Commons)
  3. 3. 淺談系統效能與安全議題Jim Huang ( 黃敬群 ) <jserv@0xlab.org>May 30, 2013 / NTHU, Taiwan
  4. 4. About me• 台灣創新行動服務推廣協會 / 理事長• 成功大學資訊工程系 / 兼任教師• 聯發科技 / 顧問• 杏昕科技 / 技術長• 晶心科技 / 幕僚工程師• 開放原始碼開發– 新酷音輸入法 , GCC / GNU Classpath, LXDE,Android Open Source Project• Others: http://about.me/jserv
  5. 5. About this presentation• 我不是專家,只是碼農• 今天不是以經營者的觀點去解讀,而以 * 低層次 *工程師的角度去分享
  6. 6. Think of the possibility萬一你的期末計畫或者玩票性質的App 獲得上萬次下載,接著會如何?
  7. 7. Think of the possibilitysource: http://www.cw.com.tw/article/article.action?id=5033139部落客化身食譜網站創辦人,平均年齡三十的大男生,如何改寫日本最大食譜網站的成功模式,六個月攻陷萬名粉絲?三千多道菜,每天有二十多道新菜色上線。每個月有近三百萬瀏覽量,熱烈討論哪道人氣菜色怎麼煮,食材可以在哪裡買。這是上線才半年,食譜分享社群網站「 iCook 愛料理」的成績。
  8. 8. The traffic3M visit per month;10K membersin 6 month
  9. 9. The impact• operation cost– 當瀏覽次數從每月 3K 到每月 3M ,成本即以指數性成長• membership variance– 當會員數量超過 1K 時,很可能面臨質變,考驗經營者的智慧• performance impact– 原本的雛型設計勢必得改寫,無論是 web 前端、後端、資料庫,抑或整體的運作模型• security issue– 樹大招風
  10. 10. The key point“Scalable”
  11. 11. Agenda (1) Concepts about Scaling(2) Performance
  12. 12. Concepts about Scaling
  13. 13. Scaling• 以 web 為例,有以下考量點– web server– database– cachingone serverloadbalanced
  14. 14. Database ScalingEveryone starts with just one server:Multiple steps to take as you move forward
  15. 15. Step 1: Master/Slave
  16. 16. Step 1: Master/Slave - PreparationEven with one server:Make code write to master and read from slaveDont write code that would fail with slave lag:$master->query(update users set comments += 1);$slave->query(select comments from users);
  17. 17. Step 2: Multiple Slaves
  18. 18. One slave per server?Not as flexible
  19. 19. Better solution: Random
  20. 20. Code to select Random Slaveclass DB {private static $cfg = array(write =>array(mysql:dbname=MyDB;host=10.1.2.3),read =>array(mysql:dbname=MyDB;host=10.1.2.7,mysql:dbname=MyDB;host=10.1.2.8,mysql:dbname=MyDB;host=10.1.2.9););public static function getConnection($pool) {$max = count(self::$cfg[$pool]) - 1;$dsn = self::$cfg[$pool][mt_rand(0, $max)];return new PDO($dsn, USER, PASS);}}$db = DB::getConnection(read);
  21. 21. Step 3: Slave PoolVirtually divide your slaves into poolsUse this to isolation high database loadPotentially enhance query caches
  22. 22. Possible Pool Layout
  23. 23. Step 4: PartitioningSimplest Definition:Break your tables or databases into smaller ones
  24. 24. Cost of Partitioning• Loss of direct SQL support• Increased Web Server / Application Load• More complicated programming
  25. 25. Caching in Brief• Often considered performance• Can absolutely be a scalability factor, especiallywhen combined with smaller discrete DB queries• Allows you to get around DB scalability byignoring the DB
  26. 26. Type of Caching• Single server memory caches– For PHP: APC or Zend Server Data Cache– Limited due to lack of syncd cache• Distributed– Memcached (Generic) or Zend Platform (PHP)– Required for true scalability enhancement
  27. 27. Performance
  28. 28. Synchronization PrimitivesWorkPartitioningInteractingWith HardwareParallelAccess ControlResourcePartitioning& ReplicationDo this first!Job #1 is not selecting primitives!
  29. 29. Atomic Increment of global variableCPU CPU CPU CPU$ $ $ $InterconnectCPU CPU CPU CPU$ $ $ $InterconnectCPU CPU CPU CPU$ $ $ $InterconnectCPU CPU CPU CPU$ $ $ $InterconnectInterconnect MemoryMemoryStoreBufferStoreBufferStoreBufferStoreBufferStoreBufferStoreBufferStoreBufferStoreBufferStoreBufferStoreBufferStoreBufferStoreBufferStoreBufferStoreBufferStoreBufferStoreBufferLots and Lots of Latency!!!
  30. 30. Security
  31. 31. Security Groups…DBSecurityGroupWebSecurityGroup… …WebWebVMVMWebWebVMVMWebWebVMVMWebWebVMVMWebWebVMVMWebWebVMVMWebWebVMVMWebWebVMVMDBVMDBVMWebWebVMVMWebWebVMVMDBVMDBVMWebWebVMVMWebWebVMVM
  32. 32. …DBSecurityGroupWebSecurityGroupUnderstanding securitygroups… …WebWebVMVMWebWebVMVMWebWebVMVMWebWebVMVMWebWebVMVMWebWebVMVMWebWebVMVMWebWebVMVMDBVMDBVMWebWebVMVMWebWebVMVMDBVMDBVMWebWebVMVMWebWebVMVMIngress Rule: Allow VMs in Web Security Group access to VMs in DB Security Group on Port 3306
  33. 33. Atomic Increment of per CPU variableCPU CPU CPU CPU$ $ $ $InterconnectCPU CPU CPU CPU$ $ $ $InterconnectCPU CPU CPU CPU$ $ $ $InterconnectCPU CPU CPU CPU$ $ $ $InterconnectInterconnect MemoryMemoryStoreBufferStoreBufferStoreBufferStoreBufferStoreBufferStoreBufferStoreBufferStoreBufferStoreBufferStoreBufferStoreBufferStoreBufferStoreBufferStoreBufferStoreBufferStoreBufferLittle Latency, Lots of Increments at Core Clock RateLittle Latency, Lots of Increments at Core Clock Rate
  34. 34. Cloud at scale• Host management• Capacity management• What host to use to deploy a new VM• Failure handling• Security group propagation
  35. 35. Reference• Highly Scalable Web Applications, Eli White (2009)• 雲端運算 - 商業模式、核心技術與架構 , Jazz Wang(2012)
  36. 36. http://0xlab.org

×