Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

AP4R on RubyConf2007

3,361 views

Published on

The presentation material at RubyConf2007 at Chrlotte

Published in: Business, Technology
  • Be the first to comment

AP4R on RubyConf2007

  1. 1. RubyConf 07 ( 2 Nov. 2007 ) AP4R Kiwamu Kato & Shun’ichi Shinohara Future Architect, Inc. 1
  2. 2. Introduction 2
  3. 3. kiwamu • Majored in Civil Engineering • Have enhanced and maintained messaging middle-ware in our company 3
  4. 4. kiwamu • Started Ruby last year • Leader of the AP4R team • http://d.hatena.ne.jp/kiwamu/ • The first month of marriage 4
  5. 5. shino • Programmer, I love coding ! • The language of the year is C or Haskell 5
  6. 6. shino • Made a debut to the OSS world last year • http://d.hatena.ne.jp/ita-wasa • Sunday physicist, interested in the string theory 6
  7. 7. Corporate profile • Future Architect, Inc. • http://www.future.co.jp/en/index.html • Aiming for optimization and synergy of business and IT • Headquarters: Tokyo, Japan • Foundation: November 28, 1989 • Number of employees: 594 (as of June 2007) • Revenue: ¥14,831M (2006) 7
  8. 8. Agenda 1. Overview ( the most important part. ) 2. Lightweight 3. Demo 4. Robust 5. Other features 6. Conclusion 8
  9. 9. 1. Overview 9
  10. 10. 1-1. What is AP4R ? 1-2. Typical use cases 1-3. What are advantages of AP4R ? 1-4. Case examples 10
  11. 11. What is AP4R ? 11
  12. 12. What is AP4R ? • A messaging library by Ruby • Asynchronous Processing for Ruby • Loose coupling by messaging 12
  13. 13. Why did we start AP4R ? • Developed a messaging middle-ware in house • Named “RtFA” 13
  14. 14. RtFA • Pure Java • Proprietary protocol and API (≠ JMS) • Time-proven on various systems 14
  15. 15. One of illustrative cases of “RtFA” Mission critical system in transportation business http://japan.internet.com/linuxtoday/20061228/5.html 15
  16. 16. Project summary • Over 100 million package data per day • Parallel processing • Load distribution 16
  17. 17. Project summary • Approx. 100 servers • Used “RtFA” 17
  18. 18. http://www.pragmaticprogrammer.com/titles/fr_j2r/ http://www.oreilly.co.jp/books/9784873113203/ 18
  19. 19. From RtFA to AP4R • Based on experiences of Java, carefully designed to keep good concepts and added “agility” • Focused on ease of use API, configuration 19
  20. 20. Open source • Released as OSS last year • Take advantages of the community • For “Enterprise” 20
  21. 21. Typical use cases of AP4R • Quicker response to users • Load distribution 21
  22. 22. Quicker response to users 22
  23. 23. If quick response is not needed, execute it later 23
  24. 24. Synchronously Client Server Sequential tasks 24
  25. 25. Synchronously Client Server A B C D E Sequential tasks 24
  26. 26. Better solution Client Server Messaging Server 25
  27. 27. Better solution Client Server Messaging Server A B 25
  28. 28. Asynchronously Client Server Messaging Server A B 26
  29. 29. Asynchronously Client Server Messaging Server A B C D E 26
  30. 30. Typical cases • Logging • Sending mail • Creating heavy summary data 27
  31. 31. Advanced case • Intersystem connection 28
  32. 32. Moreover, load distribution 29
  33. 33. Easy to scale out Client Server Messaging Server 30
  34. 34. Easy to scale out Client Server Messaging Servers 31
  35. 35. There are some solutions that also use messaging service in the ( Ruby ) world. 32
  36. 36. What are AP4R’s advantages? 33
  37. 37. Lightweight and Robust 34
  38. 38. Lightweight ? 35
  39. 39. 36
  40. 40. Ruby’s advantages • Dynamic • Succinct • Open class • Meta programming • etc.. 37
  41. 41. AP4R realized • Ease of use • Simple API / configuration • Seamless collaboration with Rails • All-in-one support • Not only development • But also test and operation support 38
  42. 42. One of the advantages is Lightweight 39
  43. 43. Robust ? 40
  44. 44. AP4R guarantees a message delivery 41
  45. 45. If a message delivery is NOT guaranteed,... 42
  46. 46. it may cause message lost or message duplication 43
  47. 47. In other words... 44
  48. 48. •No items despite payment ! •Double withdrawals ! 45
  49. 49. Messages should be delivered in a reliable way, 46
  50. 50. No matter what happens • Database trouble • Network trouble • Server down • Busy and timeout • ... etc 47
  51. 51. The other advantage is Robust 48
  52. 52. AP4R is .. 49
  53. 53. Lightweight and Robust 50
  54. 54. Lightweight Robust Simple API Recovery Test support Flexible SAF system design 51
  55. 55. Case examples 52
  56. 56. •Online game (not opened yet to the public) • Interactivemediums.com 53
  57. 57. Online game 54
  58. 58. What is it ? • From England • Massively multiplayer web-based strategy game • Late alpha now; private beta testing in January 2008 55
  59. 59. They use AP4R for • Mail sending • Report generation • Data analysis routines • A variety of processor-intensive processes including database access 56
  60. 60. Scale • Two dedicated AP4R servers • Four Rails processes per AP4R • A few hundred messages an hour • MySQL to store messages 57
  61. 61. Interactivemediums.com 58
  62. 62. http://interactivemediums.com/ 59
  63. 63. What is it ? • From the U.S. • Provide mobile messaging solutions for clients, as well as internal products like TextMe for Business ( http://textmeforbusiness.com ) • In production since July 2007 60
  64. 64. http://textmeforbusiness.com/ 61
  65. 65. They use AP4R for • Intersystem connection • Inbound queue • Outbound queue • Routing queue • Text processing 62
  66. 66. Alert service on SMS AP4R queues acceptance inbound Text processing delivery delivery outbound delivery processing Third Party routing application 63
  67. 67. Scale • Three AP4R processes • Four mongrel clusters and so forth • Now at least a thousand messages a day ( Several thousand messages a day soon ) • MySQL to store messages 64
  68. 68. What did they choose AP4R ? 65
  69. 69. Questionnaire results • “Easy to distribute a kind of load across a number of servers” • “Easy to scale” • “Development is active” • “Development team had real-world experience with the technology” 66
  70. 70. Coffee break 67
  71. 71. :Firefox => (c) 2007 Mozilla Japan 68
  72. 72. Only in Japan ? Foxkeh :Firefox => (c) 2007 Mozilla Japan 69
  73. 73. Duke :Java => https://duke.dev.java.net/ 70
  74. 74. cute ! / cute ? 71
  75. 75. Mascot matters 72
  76. 76. :AP4R => ... 73
  77. 77. Copyright © 2007 by Kanican Licensed under Creative Commons License. 74
  78. 78. What / Why is that ? > “AP4R”.sub(/4/, ”A”) > “APAR”.downcase > “apar” • Three-banded armadillo Dictionary http://www.alc.co.jp/ Photo http://www.flickr.com/photos/jeffclow/29738818/ 75
  79. 79. Copyright © 2007 by Kanican Licensed under Creative Commons License. 76
  80. 80. Copyright © 2007 by Kanican 77
  81. 81. His name is Maro 78
  82. 82. Coffee break is done :-) 79
  83. 83. 2. Lightweight 80
  84. 84. Lightweight Robust Simple API Recovery Test support Flexible SAF system design 81
  85. 85. 2-1. Simple API 2-2. Flexible system design 82
  86. 86. Simple API 83
  87. 87. API to put messages 84
  88. 88. ap4r.async_to({destination}, {data} [,{options}]) There’s also a block style c.f. RDoc: http://ap4r.rubyforge.org/doc/ 85
  89. 89. Class AsyncShopController < ApplicationController def order # synchronous part o = Order.new(:name => params[:name]) o.save! ap4r.async_to({:action => ‘payment’}, {:order_id => o.id}) redirect_to ... end def payment # asynchronous part Payment.new(:order_id => params[:order_id]).save! ... render :text => ‘true’ end end 86
  90. 90. Process flow 87
  91. 91. User Apache Rails AP4R 88
  92. 92. User Apache Rails AP4R order message put 89
  93. 93. User Apache Rails AP4R dispatch 90
  94. 94. User Apache Rails AP4R payment 91
  95. 95. Flexible system design 92
  96. 96. Typical design 93
  97. 97. User Apache Rails AP4R 94
  98. 98. Load distribution for AP4R servers 95
  99. 99. User Apache Rails AP4R 96
  100. 100. Separate servers for sync/async 97
  101. 101. User Apache Rails AP4R sync async 98
  102. 102. Separate servers including reverse proxy 99
  103. 103. User Apache Rails AP4R URL rewrite 100
  104. 104. User Apache Rails AP4R URL rewrite HTTP 100
  105. 105. User Apache Rails AP4R URL rewrite HTTP 101
  106. 106. Dispatcher Producer Messaging Consumer AP4R put dispatch Generally put get 102
  107. 107. Protocols • Default is HTTP POST • Now supported • SOAP • XML-RPC • dRuby 103
  108. 108. Flow management 104
  109. 109. Two kinds of tasks processing number time Task “busy” many short Task “heavy” few very long 105
  110. 110. Configuration dispatchers: - targets: queue.very_busy.* threads: 10 modify_rules: url: proc {|url| url.host = quot;busy.async.hostquot;} - targets: queue.very_heavy.* threads: 1 modify_rules: url: proc {|url| url.host = quot;heavy.async.hostquot;} 106
  111. 111. busy URL rewrite heavy 107
  112. 112. Lightweight Robust Simple API Recovery Test support Flexible SAF system design 108
  113. 113. 3. Demo 109
  114. 114. Outline • Shopping store application • Order and payment 110
  115. 115. Shopping store app. Client Server A B order payment 111
  116. 116. Better solution AP4R Client Rails Rails × 3 112
  117. 117. Synch’ly order AP4R Client Rails Rails × 3 A order 113
  118. 118. Async’ly payment AP4R Client Rails Rails × 3 A B order payment 114
  119. 119. Monitor message status AP4R Client Rails Rails × 3 100 75 50 25 0 115
  120. 120. Outline • Use mongrel cluster • Stress by ab • Monitor queue retaining status 116
  121. 121. 117
  122. 122. 118
  123. 123. Agenda 1. Overview ( the most important part. ) 2. Lightweight 3. Demo 4. Robust 5. Other features 6. Conclusion 119
  124. 124. 4. Robust 120
  125. 125. Lightweight Robust Simple API Recovery Test support Flexible SAF system design 121
  126. 126. Stored And Forward 122
  127. 127. Ensure Atomicity of application transaction and putting messages 123
  128. 128. Rails app-DB AP4R msg-DB CRUD Store SAF create commit Forward message put insert /commit SAF delete ✓ 124
  129. 129. Enterprise Integration Patterns: Designing, Building, and Deploying Messaging Solutions by Gregor Hohpe,Bobby Woolf Guaranteed Delivery [122] 125
  130. 130. Details 126
  131. 131. Typical flow to create async. messages 127
  132. 132. start end 128
  133. 133. start CRUD end 128
  134. 134. start CRUD commit end 128
  135. 135. start CRUD commit message put / commit end 128
  136. 136. start CRUD commit message put / commit end data 128
  137. 137. start CRUD commit message put / commit end data messages 128
  138. 138. Ensure Atomicity of application transaction and putting messages 129
  139. 139. On updating database • If error occurs, • Any following messages must NOT be queued 130
  140. 140. On updating database • If everything succeeded, • All following messages must be queued 131
  141. 141. ☺ Relief 132
  142. 142. Lightweight and Robust 133
  143. 143. Ap4r::Client#transaction Class ShopController < ApplicationController def order # synchronous side ap4r.transaction do o = Order.new(:name => params[:order_id]) o.save! ap4r.async_to({:action => ‘payment’}, {:order_id => o.id}) end redirect_to ... end end 134
  144. 144. Process flow changes in queueing sequence 135
  145. 145. Behind ap4r.transaction ap4r.transaction do ... # CRUD on database ap4r.async_to(...) end 136
  146. 146. Behind ap4r.transaction ap4r.transaction do ... # CRUD on database Insert into SAF table ap4r.async_to(...) end 136
  147. 147. Behind ap4r.transaction ap4r.transaction do ... # CRUD on database Insert into SAF table ap4r.async_to(...) end Commit database 136
  148. 148. Behind ap4r.transaction ap4r.transaction do ... # CRUD on database Insert into SAF table ap4r.async_to(...) end Commit database Queue messages 136
  149. 149. Behind ap4r.transaction ap4r.transaction do ... # CRUD on database Insert into SAF table ap4r.async_to(...) Store end Commit database Queue messages 136
  150. 150. Behind ap4r.transaction ap4r.transaction do ... # CRUD on database Insert into SAF table ap4r.async_to(...) Store end Commit database Queue messages Forward 136
  151. 151. Sequence in detail Without SAF 137
  152. 152. start CRUD commit message put / commit end data messages 138
  153. 153. start CRUD commit message put / commit end data messages 138
  154. 154. start CRUD commit message put / commit end data messages 139
  155. 155. start CRUD commit message put / commit end data messages 139
  156. 156. start CRUD commit message put / commit ☠ end data messages 139
  157. 157. start CRUD commit message put / commit ☠ end data messages 139
  158. 158. start CRUD commit message put / commit ☠ end data messages 139
  159. 159. start CRUD commit counterchange message put / commit end data messages 140
  160. 160. start CRUD message put / commit counterchange commit end data messages 141
  161. 161. start CRUD message put / commit commit end data messages 142
  162. 162. start CRUD message put / commit commit end data messages 142
  163. 163. start CRUD message put / commit commit end data messages 143
  164. 164. start CRUD message put / commit commit end data messages 143
  165. 165. start CRUD message put / commit commit ☠ end data messages 143
  166. 166. start CRUD message put / commit commit ☠ end data messages 143
  167. 167. start CRUD message put / commit commit ☠ end data messages 143
  168. 168. Better Safe than Sorry • Database trouble • Network trouble • Server down • busy and timeout • ... etc 144
  169. 169. ☺ Embrace every exceptions ☠ 145
  170. 170. Sequence in detail With SAF 146
  171. 171. CRUD start SAF create commit message put / commit SAF update end data messages 147
  172. 172. CRUD start SAF create commit message put / commit SAF update end data messages 148
  173. 173. CRUD start SAF create commit message put / commit SAF update end data messages 148
  174. 174. CRUD start SAF create commit message put / commit SAF update end data messages 149
  175. 175. CRUD start SAF create commit message put / commit SAF update end data messages 149
  176. 176. CRUD start SAF create commit message put / commit SAF update end data messages 149
  177. 177. CRUD start SAF create commit message put / commit SAF update ☺ end data messages 149
  178. 178. CRUD start SAF create commit message put / commit SAF update ☺ end data messages 149
  179. 179. CRUD start SAF create commit message put / commit SAF update ☺ end data messages 149
  180. 180. CRUD start SAF create commit message put / commit SAF update end data messages 150
  181. 181. CRUD start SAF create commit message put / commit SAF update end data messages 150
  182. 182. CRUD start SAF create commit message put / commit SAF update end data messages 151
  183. 183. CRUD start SAF create commit message put / commit SAF update end data messages 151
  184. 184. CRUD start SAF create commit message put / commit SAF update end data messages messages 151
  185. 185. CRUD start SAF create commit message put / commit SAF update ☺ end data messages messages 151
  186. 186. CRUD start SAF create commit message put / commit Recoverable SAF update ☺ end data messages messages 151
  187. 187. CRUD start SAF create commit message put / commit SAF update ☺ end data messages messages 151
  188. 188. CRUD start SAF create commit message put / commit SAF update ☺ end data messages messages 151
  189. 189. CRUD start SAF create commit message put / commit SAF update end data messages 152
  190. 190. CRUD start SAF create commit message put / commit SAF update end data messages 152
  191. 191. CRUD start SAF create commit message put / commit SAF update end data messages 153
  192. 192. CRUD start SAF create commit message put / commit SAF update end data messages messages 153
  193. 193. CRUD start SAF create commit message put / commit SAF update ☺ end data messages messages 153
  194. 194. CRUD start SAF create commit message put / commit Duplicated SAF update ☺ end data messages messages 153
  195. 195. CRUD start SAF create commit message put / commit SAF update ☺ end data messages messages 153
  196. 196. CRUD start SAF create commit message put / commit SAF update ☺ end data messages messages 153
  197. 197. Ensure Atomicity of application transaction and putting message 154
  198. 198. at-least-once between Producer and Channel 155
  199. 199. Lightweight Robust Simple API Recovery Test support Flexible SAF system design 156
  200. 200. 5. Other features 157
  201. 201. • Messages recovery when some troubles happen • Collaboration with Capistrano (2.x) • Test support 158
  202. 202. Test support 159
  203. 203. TDD / BDD 160
  204. 204. ☺ It makes us feel relieved in developing an application 161
  205. 205. On async app. • Test matters also, but .. • Multiple processes communicating via N/W • take time • annoying to setup 162
  206. 206. Two ways of testing • Functional test • no N/W communication • fast but restricted • Async test • using N/W communication • slow but close to actual 163
  207. 207. Functional test • Provided a queue stub • Executed in memory, quickly 164
  208. 208. code sample test code here is against a ‘fictional’ on-line shop application, which has an order action (sync) and payment action (async). 165
  209. 209. def test_order post :order, :order => {:item => quot;introduction to AP4Rquot;} assert_response :redirect assert_redirected_to :action => 'index' messages = @controller.ap4r.queued_messages # ... (1) assert_equal 1, messages.keys.size, quot;should have messages in just ONE queuequot; assert messages.key?(quot;queue.async_shop.paymentquot;), quot;queue name is INCORRECTquot; # ... (2) assert_equal 1, messages[quot;queue.async_shop.paymentquot;].size, quot;should have just ONE message for paymentquot; assert_match /order_id=d+/, messages[quot;queue.async_shop.paymentquot;].first[:body], quot;parameter order_id should be included with a numeric valuequot; # ... (3) end 1. get (stubbed)queued messages 2. assert the queue name 3. assert the message content 166
  210. 210. functional test rake .. CRUD message put Stub assert assert 167
  211. 211. Async test • Get on reality • Test everything, e.g serialization, cooperation. 168
  212. 212. code sample test code here is against a ‘fictional’ application. 169
  213. 213. [RAILS_ROOT]/test/async/ap4r_test_helper.rb ENV[quot;RAILS_ENVquot;] = quot;testquot; require File.expand_path(File.dirname(__FILE__) + quot;/../../config/environmentquot;) require quot;ap4r/service_handlerquot; ap4r_test_helper = Ap4r::ServiceHandler.new require 'test_help' class Test::Unit::TestCase self.use_transactional_fixtures = false self.use_instantiated_fixtures = false # Add more helper methods to be used by all tests here... cattr_accessor :ap4r_helper def ap4r_helper @@ap4r_helper end def with_services(&block) ap4r_helper.with_services(&block) end end Test::Unit::TestCase.ap4r_helper = ap4r_test_helper 170
  214. 214. [RAILS_ROOT]/test/async/async_shop_test.rb require quot;#{File.dirname(__FILE__)}/ap4r_test_helperquot; require 'net/http' class AsyncShopTest < Test::Unit::TestCase def test_http_dispatch ap4r_helper.stop_dispatchers # ... (1) assert_rows_added(Order, 1) { # ... (3) do_order # ... (2) } assert_rows_added(Payment, 1) { # ... (6) ap4r_helper.start_dispatchers # ... (4) ap4r_helper.wait_all_done # ... (5) } end private # Requests to <tt>async_shop/order</tt>. def do_order(item_name = quot;test itemquot;) Net::HTTP.start(quot;localhostquot;, 3000, nil, nil) do |http| http.request_post(quot;/async_shop/orderquot;, quot;order[item]=#{item_name}quot;) do |res| #nop end end end def assert_rows_added(model, rows) rows_before = model.count yield rows_after = model.count assert_equal rows, rows_after - rows_before, quot;table '#{model.table_name}' should count up by #{rows}quot; end end 171
  215. 215. stop dispatcher threads 1. request to Rails with HTTP POST 2. assert one row added to orders table 3. start dispatcher threads 4. wait for async action is completed 5. assert one row added to payments table 6. 172
  216. 216. async test rake .. service control HTTP CRUD message put assert HTTP CRUD assert 173
  217. 217. Summary • Two test methods. • Functional test • Async test • It makes us feel relieved to develop an application 174
  218. 218. 6. Conclusion 175
  219. 219. AP4R is .. 176
  220. 220. Lightweight and Robust 177
  221. 221. Lightweight Robust Simple API Recovery Test support Flexible SAF system design 178
  222. 222. • Messaging with simple API, simple configuration • Seamless collaboration with Rails • Flexible system design 179
  223. 223. • Automatically invocation of async. processing by dispatcher • Guaranteed delivery by SAF • Test support • Recovery and deployment 180
  224. 224. Future Plans 181
  225. 225. Smooth operation ver.0.3.x ☑ Daemonize ☑ URL-rewrite filter ☑ DLQ / SAF recovery □ Protocols: Stomp, HTTP 182
  226. 226. Monitoring ver.0.4.x □ Coordinate with Cacti, ZABBIX, etc. □ Many AP4R processes □ Interactive management tool 183
  227. 227. Step to large scale ver.0.5.x □ Dynamic configuration □ Automatic recovery □ Long run 24/7 184
  228. 228. Others □ Security ☑ Application testing support □ Blocking queues □ Ruby off Rails 185
  229. 229. AP4R@RubyForge • wiki: http://ap4r.rubyforge.org/wiki/wiki.pl?HomePage • source: svn co http://ap4r.rubyforge.org/svn/trunk/ap4r • ML: http://rubyforge.org/mailman/listinfo/ap4r-user Your patches are welcome! Thank you. by Kanican 186
  230. 230. • Future Architect logo belongs to Future Architect, Inc. Japan. All rights reserved. • Rails logo is trademarks of David Heinemeier Hansson. All rights reserved. • The Ruby logo is copyrighted (c) 2006, Yukihiro Matsumoto. It is released under the terms of the Creative Commons Attribution-ShareAlike 2.5 License. 187

×