Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Salesforce API シリーズ: バルクAPIを使った高速並列データロードWebinar

2,241 views

Published on

9月12日に行われた、「Salesforce API シリーズ: バルクAPIを使った高速並列データロードWebinar」で使用されたスライドです。

Published in: Technology
  • Be the first to comment

Salesforce API シリーズ: バルクAPIを使った高速並列データロードWebinar

  1. 1. Salesforce API シリーズ Bulk APIを使った高速並列データロード
  2. 2. Safe Harbor Safe harbor statement under the Private Securities Litigation Reform Act of 1995: This presentation may contain forward-looking statements that involve risks, uncertainties, and assumptions. If any such uncertainties materialize or if any of the assumptions proves incorrect, the results of salesforce.com, inc. could differ materially from the results expressed or implied by the forward-looking statements we make. All statements other than statements of historical fact could be deemed forward-looking, including any projections of product or service availability, subscriber growth, earnings, revenues, or other financial items and any statements regarding strategies or plans of management for future operations, statements of belief, any statements concerning new, planned, or upgraded services or technology developments and customer contracts or use of our services. The risks and uncertainties referred to above include – but are not limited to – risks associated with developing and delivering new functionality for our service, new products and services, our new business model, our past operating losses, possible fluctuations in our operating results and rate of growth, interruptions or delays in our Web hosting, breach of our security measures, the outcome of intellectual property and other litigation, risks associated with possible mergers and acquisitions, the immature market in which we operate, our relatively limited operating history, our ability to expand, retain, and motivate our employees and manage our growth, new releases of our service and successful customer deployment, our limited history reselling non-salesforce. com products, and utilization and selling to larger enterprise customers. Further information on potential factors that could affect the financial results of salesforce.com, inc. is included in our annual report on Form 10-Q for the most recent fiscal quarter ended July 31, 2012. This documents and others containing important disclosures are available on the SEC Filings section of the Investor Information section of our Web site. Any unreleased services or features referenced in this or other presentations, press releases or public statements are not currently available and may not be delivered on time or at all. Customers who purchase our services should make the purchase decisions based upon features that are currently available. Salesforce.com, inc. assumes no obligation and does not intend to update these forward-looking statements. #salesforcedevjp
  3. 3. #salesforcedevjp Speakers Mitsuhiro Okamoto Developer Program Manager @mitsuhiro
  4. 4. 䝋䞊䝅䝱䝹䜰䜹䜴䞁䝖䜢䝣䜷䝻䞊䛟䛰䛥䛔 @salesforcedevjp / #salesforcedevjp Salesforce Developers Japan +Salesforce Developers Japan Developer Force Japan Developer Force Group #salesforcedevjp 本Webinarは録画されています 録画ビデオはYoutube及びWebinar 詳細ページにて公開されます (参加登録URLと同じです)
  5. 5. ㉁ၥ䛜䛒䜛ሙྜ䛻䛿? § ㉁ၥ䛿᭱ᚋ䜎䛷ᚅ䛴ᚲせ䛿䛒䜚䜎䛫䜣! – Question䛾ᶵ⬟䜢౑䛳䛶㉁ၥ䜢䛧䛶ୗ䛥䛔䚹 ྍ⬟䛺ሙྜ䛿ู䛾䝇䝍䝑䝣䛜㉁ၥ䛻䛴䛔䛶ゎ⟅䛧䜎䛩䚹 䛩䛠䛻ゎ⟅ฟ᮶䛺䛔ሙྜ䛿䚸ᚋ䜋䛹䝇䝢䞊䜹䞊䛜ゎ⟅䛧䜎䛩 § 䝷䜲䝤Q&A䜢᭱ᚋ䛻⾜䛔䜎䛩 – ᫬㛫䛾チ䛩㝈䜚䚸Webinar䛾᭱ᚋ䛻Q & A䛾᫬㛫䜢タ䛡䜎䛩 § 㛤Ⓨ⪅䝣䜷䞊䝷䝮䜢ά⏝䛧䜎䛧䜗䛖 – 䜘䜚䛯䛟䛥䜣䛾㉁ၥ䛜䛒䜛ሙྜ䛿?䛣䛱䜙䜢䛤฼⏝ୗ䛥䛔 http://developer.salesforce.com/jp/forums #salesforcedevjp
  6. 6. Salesforce䜈䛾䝕䞊䝍ᢞධ䜢 㧗㏿䛻䛩䜛䛻䛿䛹䛖䛩䜛䛛?
  7. 7. 䛒䛺䛯䛿 1᫬㛫䛷䛹䜜䛠䜙䛔䛾䝕䞊䝍䜢 Salesforce䛻Ⓩ㘓䛷䛝䜎䛩䛛?
  8. 8. 䝕䞊䝍䝻䞊䝗䛾䝇䝹䞊䝥䝑䝖 25,000,000 20,000,000 15,000,000 10,000,000 5,000,000 - #salesforcedevjp レコード件数 / 1時間 OK なかなか 素晴らしい
  9. 9. ୪⾜ฎ⌮
  10. 10. ୪⾜ฎ⌮䛸㢮ఝ䛩䜛䜒䛾: ⁁᥀䜚 #salesforcedevjp
  11. 11. 㡰ḟ(㏲ḟ)ฎ⌮ #salesforcedevjp
  12. 12. ୪⾜ฎ⌮ #salesforcedevjp
  13. 13. 䝥䝻䝉䝑䝃䜔䝇䝺䝑䝗䛜 䜂䛸䛴䛾ฎ⌮䛻㈝䜔䛥䜜䜛ᩘ
  14. 14. 䜒䛳䛸䜒᭱㐺䛺୪⾜ฎ⌮ 並列 順次 #salesforcedevjp 2,000୓㻌䝺䝁䞊䝗 500୓㻌䝺䝁䞊䝗 500୓㻌䝺䝁䞊䝗 500୓㻌䝺䝁䞊䝗 500୓㻌䝺䝁䞊䝗 時間
  15. 15. ᭱㐺໬䛥䜜䛶䛔䛺䛔୪⾜ฎ⌮ 並列 順次 #salesforcedevjp 500୓㻌䝺䝁䞊䝗 500୓㻌䝺䝁䞊䝗 500୓㻌䝺䝁䞊䝗 500୓㻌䝺䝁䞊䝗 2,000୓㻌䝺䝁䞊䝗 Time
  16. 16. 䝻䝑䜽, ౛እ, 䝖䝸䜺, 㛵㐃, …䛺䛹䛺䛹 並列 順次 #salesforcedevjp 500୓㻌䝺䝁䞊䝗 500୓㻌䝺䝁䞊䝗 500୓㻌䝺䝁䞊䝗 500୓㻌䝺䝁䞊䝗 2,000୓㻌䝺䝁䞊䝗 Time 䝇䝹䞊䝥䝑䝖 䜢㜼ᐖ
  17. 17. 䝕䞊䝍䝻䞊䝗䜿䞊䝇䝇䝍䝕䜱 § Salesforce Bulk API 䜢ᐇ㝿䛻ヨ䛧䛶䜏䜛 § 㡰ḟ䝕䞊䝍ฎ⌮㻌䛸㻌୪ิ䝕䞊䝍ฎ⌮㻌䜢ẚ䜉䜛 § ୪ิᗘ㻌ཬ䜃㻌䝇䝹䞊䝥䝑䝖䜢⤫ィ䛩䜛 § 䝇䝹䞊䝥䝑䝖㜼ᐖせᅉ䜢≉ᐃ䛧ᅇ㑊䛩䜛 § ᭱኱䛾䝇䝹䞊䝥䝑䝖䜢㐩ᡂ #salesforcedevjp
  18. 18. ๓ᥦ᮲௳
  19. 19. Salesforce Bulk API § 㠀ྠᮇ㻌䛾䝕䞊䝍䝻䞊䝗 § ኱㔞䝕䞊䝍䝉䝑䝖䛻᭱㐺໬ § REST API § ᵝ䚻䛺䝒䞊䝹䛷฼⏝䛥䜜䛶䛔䜛 § ௵ព䛾䝥䝻䜾䝷䝭䞁䜾ゝㄒ㻌(Java➼)䛛䜙 ࿧䜃ฟ䛩䛣䛸䛜䛷䛝䜛 #salesforcedevjp
  20. 20. Force.com Web Service Connector (WSC) § Salesforce API䜢⡆༢䛻౑䛖䛯䜑䛾Java䝒䞊䝹䜻䝑䝖 – Web Services/SOAP – Asynchronous/BULK § ౫Ꮡ䝷䜲䝤䝷䝸䛜↓䛟䝅䞁䝥䝹䛻฼⏝ྍ⬟ #salesforcedevjp
  21. 21. 䝕䝰䝇䜻䞊䝬 #salesforcedevjp
  22. 22. Bulk API䛾䝻䞊䝗䛻䜎䛴䜟䜛䜻䞊䝽䞊䝗 … ealize, nvestigate, lan ᢕᥱ䚸ㄪᰝ䚸ィ⏬
  23. 23. 䜿䞊䝇䝇䝍䝕䜱
  24. 24. 㡰ḟ䝕䞊䝍䝻䞊䝗
  25. 25. 㡰ḟ䝻䞊䝗: ᝿ᐃ䝥䝷䞁 Thread Thread Thread Thread Thread Thread Thread Thread Thread Thread Thread Thread Thread Thread Thread Thread #salesforcedevjp 時間 • 単一ジョブ • 100䝞䝑䝏 • 10,000䝺䝁䞊䝗 / 䝞䝑䝏 • 䝖䞊䝍䝹100୓䝺䝁䞊䝗
  26. 26. 㡰ḟ䝻䞊䝗: Jobタᐃ #salesforcedevjp
  27. 27. 㡰ḟ䝻䞊䝗: 䝞䝑䝏సᡂ #salesforcedevjp
  28. 28. 㡰ḟ䝻䞊䝗: 䝞䝑䝏ᐇ⾜ #salesforcedevjp
  29. 29. Demo 㡰ḟ䝻䞊䝗
  30. 30. 㡰ḟ䝻䞊䝗⤖ᯝ ྠ᫬ᐇ⾜䝰䞊䝗 㡰ḟ ฎ⌮䝺䝁䞊䝗ᩘ 1,000,000 ኻᩋ䝺䝁䞊䝗ᩘ 0 ᐇ⾜᫬㛫 41 minutes 䝇䝹䞊䝥䝑䝖 24,300 䝺䝁䞊䝗 / ศ ᖹ⾜ฎ⌮⋡ 0.92 ၥ㢟Ⅼ ୪⾜ฎ⌮⋡䛿ᚲ䛪1௨ୗ䛸䛺䛳䛶䛧䜎䛖 ゎỴ⟇ 䝇䝹䞊䝥䝑䝖ྥୖ䛾Ⅽ䛻୪⾜䝻䞊䝗䜢᳨ウ䛩䜛 #salesforcedevjp
  31. 31. ୪⾜ฎ⌮ vs ༢୍䝆䝵䝤䛾䝇䝹䞊䝥䝑䝖 350000 分 1300000 / ) 数250000 ドー200000 コレ150000 (ト100000 ップ50000 ール0 ス並列処理 #salesforcedevjp 㡰ḟᐇ⾜ • ୪⾜ᛶ䛻ၥ㢟᭷䜚 㡰ḟฎ⌮ 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
  32. 32. ୪⾜䝕䞊䝍䝻䞊䝗
  33. 33. ୪⾜ㄞ䜏㎸䜏: ᝿ᐃ䝥䝷䞁 Thread Thread Thread Thread Thread Thread Thread Thread Thread Thread Thread Thread Thread Thread Thread Thread #salesforcedevjp • 複数ジョブ • 100䝞䝑䝏 • 10,000 䝺䝁䞊䝗㻌/䝞䝑䝏 • 䝖䞊䝍䝹100୓䝺䝁䞊䝗 時間
  34. 34. ୪ิ䝻䞊䝗: 䝆䝵䝤タᐃ #salesforcedevjp
  35. 35. ௨ୗ䛻ὀ┠ § 䝻䝑䜽䛿୪ิᐇ⾜䛻ከ኱䛺ᙳ㡪䜢୚䛘䜛 – ฎ⌮⬟ຊ䜢↓㥏䛻䛧䛶䛧䜎䛖 – 䝇䝹䞊䝥䝑䝖䛾పୗ – 䜶䝷䞊 § ෌ᐇ⾜䛿䛩䜉䛶䜢ゎỴ䛩䜛䜟䛡䛷䛿䛺䛔 #salesforcedevjp
  36. 36. 䝕䝰 ୪ิฎ⌮ v1
  37. 37. ୪ิ䝻䞊䝗㻌v1 䜎䛸䜑 ྠ᫬ᐇ⾜䝰䞊䝗 ୪ิ ฎ⌮䝺䝁䞊䝗ᩘ 404200 ኻᩋ䝺䝁䞊䝗ᩘ 198600 ᐇ⾜᫬㛫 10ศ 䝇䝹䞊䝥䝑䝖 25,000䝺䝁䞊䝗 / ศ ᖹ⾜ฎ⌮⋡ 15.79 ၥ㢟Ⅼ 䝻䝑䜽౛እ䛜Ⓨ⏕䚹㻌䝃䞊䝞䛿ฎ⌮䜢ศᩓ䛥䛫䛶⾜䛳䛯䛜䚸䝇䝹䞊䝥䝑䝖䛿ୖ䛜䜙䛺䛛䛳䛯 ゎỴ⟇ 㡰ḟ䝰䞊䝗䛷ືస䛩䜛䛛䚸䝻䝑䜽䜢⟶⌮䛩䜛 #salesforcedevjp
  38. 38. ୪⾜ฎ⌮v1 vs 㡰ḟฎ⌮ : 䝇䝹䞊䝥䝑䝖 350000 分 1300000 / ) 数250000 ドー200000 コレ150000 (トッ100000 プー50000 ルス0 並列処理 #salesforcedevjp ୪⾜ᐇ⾜䚷v1 • 㧗䛔୪⾜ฎ⌮⋡ • 䝻䝑䜽䛻䜘䜛䝇䝹䞊䝥䝑䝖䛜పୗ 㡰ḟฎ⌮ ୪⾜ฎ⌮ v1 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
  39. 39. ᭱㐺໬!! § ᭱㐺䛺䝕䞊䝍䝻䞊䝗䛾䛯䜑䛾䜻䞊䝽䞊䝗䛂㻌 䛃 § ealize – ᢕᥱ – 䝻䝑䜽䛿୪⾜ᛶ䛸䝇䝹䞊䝥䝑䝖䜢㜼ᐖ § nvestigate – ㄪᰝ – ఱ䛜䝻䝑䜽䛾ཎᅉ䛛 § lan – ィ⏬ – 䝻䝑䜽䜢䝁䞁䝖䝻䞊䝹䛩䜛 #salesforcedevjp
  40. 40. 䝕䝰 ୪⾜䝻䞊䝗㻌v2 䝇䜻䞊䝬ኚ᭦䛻䜘䜛䝻䝑䜽ゎ㝖
  41. 41. ୪ิฎ⌮v2 : ⤖ᯝ ྠ᫬ᐇ⾜䝰䞊䝗 ୪⾜ ฎ⌮䝺䝁䞊䝗ᩘ 100୓䝺䝁䞊䝗 ኻᩋ䝺䝁䞊䝗ᩘ 0 ᐇ⾜᫬㛫 2ศ36⛊ 䝇䝹䞊䝥䝑䝖 400,000䝺䝁䞊䝗 / ศ ᖹ⾜ฎ⌮⋡ 19 ၥ㢟Ⅼ 䛺䛧 ゎỴ⟇ n/a #salesforcedevjp
  42. 42. ୪⾜ฎ⌮v2 vs 㡰ḟฎ⌮ : 䝇䝹䞊䝥䝑䝖 350000 分 300000 1/ ) 250000 数ド200000 ーコ150000 レ(ト100000 ップ50000 ール0 ス並行処理 #salesforcedevjp ୪⾜ฎ⌮ v2 • 㧗䛔୪⾜ᛶ • 㧗䝇䝹䞊䝥䝑䝖 㡰ḟฎ⌮ ୪⾜ฎ⌮ v1 ୪⾜ฎ⌮ v2 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
  43. 43. 䝻䝑䜽䛾⟶⌮ … ὀព䛧䛺䛡䜜䜀䛺䜙䛺䛔䝫䜲䞁䝖 § ୺ᚑ㛵ಀ § ཧ↷㛵ಀ § ✚䜏ୖ䛢㞟ィ㡯┠ § 䝖䝸䜺 § 䝽䞊䜽䝣䝻䞊䝹䞊䝹 § 䜾䝹䞊䝥䝯䞁䝞䞊䝅䝑䝥䝻䝑䜽 #salesforcedevjp
  44. 44. 䝻䝑䜽䛿⟶⌮ฟ᮶䜛 § 䝻䝑䜽䛾ཎᅉ䛸䛺䜛せᅉ䜢᤼㝖䛩䜛 – 䝖䝸䜺䚸䝽䞊䜽䝣䝻䞊䜢୍᫬ⓗ䛻೵Ṇ䛩䜛 – 䝣䜯䜲䝹ෆ䛾䝕䞊䝍䜢䝋䞊䝖䛩䜛 #salesforcedevjp
  45. 45. Parallel load: Sample results ྠ᫬ᐇ⾜䝰䞊䝗 Parallel ฎ⌮䝺䝁䞊䝗ᩘ 1 million ኻᩋ䝺䝁䞊䝗ᩘ 0 ᐇ⾜᫬㛫 4 minutes ᡤせ᫬㛫 1 hour 䝇䝹䞊䝥䝑䝖 250,000 records per minute ᖹ⾜ฎ⌮⋡ 16.5 ၥ㢟Ⅼ Minimal overhead due to locks ゎỴ⟇ Remove all unnecessary locks #salesforcedevjp
  46. 46. 䝇䝻䝑䝖ไᚚ/ ୪⾜䝕䞊䝍䝻䞊䝗
  47. 47. 䝇䝻䝑䝖ไᚚ䛷䛾䝕䞊䝍䝻䞊䝗 § 䝇䝹䞊䝥䝑䝖䛸ᖹ⾜ไ䜢⪃៖䛧䛯䝇䝻䝑䝖䝸䞁䜾 – ୪⾜ฎ⌮ཬ䜃䝻䞊䝕䜱䞁䜾 – 㠀ྠᮇฎ⌮䛾ඃඛ㡰఩௜䛡 § 」㞧䛺䝆䝵䝤䛻䛚䛡䜛㜼ᐖせᅉ䜢⟶⌮䛩䜛 – 䝕䞊䝍䛾䝍䜲䝭䞁䜾 – 」ᩘ䛾䝻䝑䜽 #salesforcedevjp
  48. 48. 䝇䝻䝑䝖ไᚚ䝴䞊䝇䜿䞊䝇1:᝿ᐃ Thread Thread Thread Thread Thread Thread Thread Thread Thread Thread Thread Thread Thread Thread Thread Thread #salesforcedevjp 時間 • 100 ジョブ • 1 䝞䝑䝏/䝆䝵䝤 • 10,000 䝺䝁䞊䝗/䝞䝑䝏 • 100୓䝺䝁䞊䝗
  49. 49. 䝇䝻䝑䝖ไᚚ vs 㡰ḟฎ⌮ : 䝇䝹䞊䝥䝑䝖 350000 300000 250000 200000 150000 100000 50000 0 #salesforcedevjp 䝇䝻䝑䝖ไᚚ v1 • ప䛔୪ิᛶ • ప䛔䝇䝹䞊䝥䝑䝖 㡰ḟ 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 Degree of Parallelism Throughput Records/Min ୪⾜ฎ⌮ v1 ୪⾜ฎ⌮ v2 䝇䝻䝑䝖ไᚚ v1
  50. 50. Controlled Feed Use Case 2: Expected plan Thread Thread Thread Thread Thread Thread Thread Thread Thread Thread Thread Thread Thread Thread Thread Thread #salesforcedevjp Time • 10 ジョブ • 10 䝞䝑䝏/䝆䝵䝤 • 10,000 䝺䝁䞊䝗/䝞䝑䝏 • 100୓䝖䞊䝍䝹䝺䝁䞊䝗
  51. 51. 䝇䝻䝑䝖ไᚚ vs 㡰ḟฎ⌮ : 䝇䝹䞊䝥䝑䝖 350000 分 300000 1/ 250000 ) 数ド200000 ーコ150000 レ(ト100000 ップ50000 ール0 ス並列処理 #salesforcedevjp 䝇䝻䝑䝖ไᚚᐇ⾜ • ୪⾜ᛶ䜢ᢚ䛘䜛 • ᮇᚅ㏻䜚䛾䝇䝹䞊䝥䝑䝖 㡰ḟ Parallel 1 ୪⾜ฎ⌮ v2 䝇䝻䝑䝖ไᚚ 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
  52. 52. 䜎䛸䜑 § ୪ิ䝕䞊䝍䝻䞊䝗䛾䛯䜑䛾䜻䞊䝽䞊䝗䛂㻌 䛃 § ealize – ᢕᥱ – 䝻䝑䜽䛿୪⾜ᛶ䛸䝇䝹䞊䝥䝑䝖䜢㜼ᐖ § nvestigate – ㄪᰝ – ఱ䛜䝻䝑䜽䛾ཎᅉ䛛 § lan – ィ⏬ – 䝻䝑䜽䜢䝁䞁䝖䝻䞊䝹䛩䜛 #salesforcedevjp
  53. 53. Q & A #salesforcedevjp Mitsuhiro Okamoto Developer Program Manager @mitsuhiro
  54. 54. Thank you

×