How to build a scalable SNS via Polling & Push
•
24 likes•4,642 views
Agenda ● Variety of sync mechanisms ● Polling & Push ● RESTful API Design ● API Blueprint
Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Viewers also liked
Viewers also liked (14)
Similar to How to build a scalable SNS via Polling & Push
Similar to How to build a scalable SNS via Polling & Push (20)
More from Mu Chun Wang
More from Mu Chun Wang (20)
Recently uploaded
Recently uploaded (20)
How to build a scalable SNS via Polling & Push
- 1. How to build a scalable SNS via Polling & Push Kewang 三竹資訊
- 2. 2 Who I am ● 王慕羣 ● Java / Node.js / AngularJS ● SQL-like / HBase ● Mentor: NIU CSIE GitHub: kewang Google Play: kewang Facebook: kewangtw Linkedin: kewangtw Slideshare: kewang Mail: cpckewang@gmail.com
- 4. Who Mitake is 三竹資訊 大家都唸 Mitake
- 5. Who Mitake is 三竹資訊 大家都唸 Mitake ，但我們公司都唸 Mitake
- 6. Who Mitake is 三竹資訊 Mitake 不唸作 MiTAC 啊！！！
- 14. Who Mitake is 三竹資訊 ● 簡訊平台 ● 行動下單：不計其數 ● 行動銀行：臺銀、土銀、富邦、台新、聯邦、臺 企銀、遠銀、華南、澳盛、郵局、合庫、渣打 ... 等 18 家 ● 產壽險：
- 15. Who Mitake is 三竹資訊 ● 簡訊平台 ● 行動下單：不計其數 ● 行動銀行：臺銀、土銀、富邦、台新、聯邦、臺 企銀、遠銀、華南、澳盛、郵局、合庫、渣打 ... 等 18 家 ● 產壽險：全球、明台、新光、新安東京、富邦 ... 等
- 16. Who Mitake is 三竹資訊 ● 簡訊平台 ● 行動下單：不計其數 ● 行動銀行：臺銀、土銀、富邦、台新、聯邦、臺企 銀、遠銀、華南、澳盛、郵局、合庫、渣打 ... 等 18 家 ● 產壽險：全球、明台、新光、新安東京、富邦 ... 等 ● 其他：
- 17. Who Mitake is 三竹資訊 ● 簡訊平台 ● 行動下單：不計其數 ● 行動銀行：臺銀、土銀、富邦、台新、聯邦、臺企 銀、遠銀、華南、澳盛、郵局、合庫、渣打 ... 等 18 家 ● 產壽險：全球、明台、新光、新安東京、富邦 ... 等 ● 其他： udn 買東西、手機逛週年慶、財政園地、證交 所、綜所稅申報 ... 等
- 23. 23 Agenda ● Variety of sync mechanisms ● Polling & Push ● RESTful API Design ● API Blueprint
- 24. 24 Variety of sync mechanisms
- 25. 25 Variety of sync mechanisms ● Polling ● Comet ● Long Polling ● WebSocket ● Engine.IO (Socket.IO)
- 31. 31 Polling ● Pros – Easy to implement ● Cons – No efficiency
- 32. 32 Comet a never died HTTP request
- 40. 40 Comet ● Pros – Save more network traffic ● Cons – Blocking IO issue – Always get server response, can't send another request
- 41. 41 Long Polling send a long time request repeatedly when response received
- 49. 49 Long Polling ● Pros – Save a little network traffic – Can send another request ● Cons – I don't know
- 50. 50 WebSocket FDX communications channels over a single TCP connection
- 57. 57 WebSocket ● Pros – Bidirectional communication – Save more network traffic ● Cons – Proxy & Firewall issue
- 58. 58 Engine.IO communication layer for Socket.IO
- 59. 59 Engine.IO - Socket.IO history 0.1~0.6.2 0.6.3~0.6.17 0.7.0~0.9.17 1.0.0~now websocket ✓ ✓ ✓ ✓ server-events ✓ flashsocket ✓ ✓ ✓ htmlfile ✓ ✓ xhr-multipart ✓ ✓ xhr-polling ✓ ✓ ✓ jsonp-polling ✓ polling ✓
- 61. 61 Engine.IO - Overview ● Ensure the most reliable realtime communication
- 62. 62 Engine.IO - Overview ● Ensure the most reliable realtime communication ● Always establishes a long-polling connection first
- 63. 63 Engine.IO - Overview ● Ensure the most reliable realtime communication ● Always establishes a long-polling connection first – then tries to upgrade to better transports that are "tested" on the side
- 64. 64 Engine.IO - Upgrade transport seamlessly
- 65. 65 Engine.IO - Upgrade transport seamlessly ● Switches from polling to another transport in between polling cycles
- 66. 66 Engine.IO - Upgrade transport seamlessly ● Switches from polling to another transport in between polling cycles ● To ensure no messages are lost, the upgrade packet will only be sent once all the buffers of the existing transport are flushed and the transport is considered paused
- 75. 75 Engine.IO ● Pros – Auto-switch different network scenario ● Cons – Use particular protocol at server & client
- 77. 77 Why we don't use WebSocket ● Server – Many corporate proxies block WebSocket traffic ● Client – Many personal firewalls and antivirus softwares block WebSocket traffic
- 80. 80 Push != Notification Push Notification Notification Push Notification Notification
- 81. 81 Push - Overview ● Non-IM – Breaking news notification – Remittance notification – Call off work notification – Email notification – ......etc. ● IM – Message notification
- 82. 82 How it works?
- 83. 83 When unsafe APIs done - Server 1.Store affected devices' ID & API ID to database 2.Send Push to affected devices
- 84. 84 When unsafe APIs done - Client ● Send an unified GET request – GET /datas
- 85. 85 When unsafe APIs done - Client ● Send an unified GET request – GET /datas ● Push off – apply frequency rapidly to achieve realtime, but waste resources
- 86. 86 When unsafe APIs done - Client ● Send an unified GET request – GET /datas ● Push off – apply frequency rapidly to achieve realtime, but waste resources ● Push on – apply frequency long to save resources – when Push comes, send request to get newest data to achieve realtime
- 87. 87 Push - Push off client server
- 88. 88 Push - Push off client server T0 T1 T2
- 89. 89 Push - Push off client server T0 T1 T2 T4 T5 T6
- 90. 90 Push - Push off client server T0 T1 T2 T4 T5 T6 T8 T9 T10
- 91. 91 Push - Push on client server SNS
- 92. 92 Push - Push on client server T0 T1 T2 SNS
- 93. 93 Push - Push on client server T0 T1 T2 T22 T23 T24 SNS
- 94. 94 Push - Push on client server T0 T1 T2 T22 T23 T24 SNS T30
- 95. 95 Push - Push on client server T0 T1 T2 T22 T23 T24 T31 T32 T33 SNS T30
- 96. 96 Fallback ● Don't polling everything ● Every GET APIs provide UI operation
- 97. 97 Fallback
- 99. 99 RESTful API Design ● HTTP method ● Common examples ● Is Graph API RESTful? ● Actual example ● Internal & Open API
- 100. 100 GET - Retrieve a resource ● A request to the server should never change the resource state
- 101. 101 GET - Retrieve a resource ● A request to the server should never change the resource state ● Incidental side effects are OK – Like logging
- 102. 102 GET - Retrieve a resource ● A request to the server should never change the resource state ● Incidental side effects are OK – Like logging ● Common response code – 200 OK
- 103. 103 GET - Retrieve a resource ● A request to the server should never change the resource state ● Incidental side effects are OK – Like logging ● Common response code – 200 OK – 301 Moved Permanently
- 104. 104 GET - Retrieve a resource ● A request to the server should never change the resource state ● Incidental side effects are OK – Like logging ● Common response code – 200 OK – 301 Moved Permanently – 404 Not Found (related to DELETE) – 410 Gone (related to DELETE)
- 105. 105 DELETE - Remove a resource ● A request to the server should destroy the resource & never refer to it again
- 106. 106 DELETE - Remove a resource ● A request to the server should destroy the resource & never refer to it again ● Common response code – 200 OK – 202 Accepted – 204 No Content
- 107. 107 POST
- 109. 109 POST ● POST-to-append ● Overloaded POST
- 110. 110 POST-to-append ● A request to the server should create a new resource
- 111. 111 POST-to-append ● A request to the server should create a new resource ● Common response code – 201 Created
- 112. 112 POST-to-append ● A request to the server should create a new resource ● Common response code – 201 Created (plus "Location" header)
- 113. 113 POST-to-append ● A request to the server should create a new resource ● Common response code – 201 Created (plus "Location" header) – 202 Accepted
- 114. 114 Overloaded POST ● Providing a block of data, such as the result of submitting a form, to a data-handling process
- 115. 115 Overloaded POST ● Providing a block of data, such as the result of submitting a form, to a data-handling process ● "Data-handling process" can be anything
- 116. 116 Overloaded POST ● Providing a block of data, such as the result of submitting a form, to a data-handling process ● "Data-handling process" can be anything – POST /users/sort
- 117. 117 Overloaded POST ● Providing a block of data, such as the result of submitting a form, to a data-handling process ● "Data-handling process" can be anything – POST /users/sort – POST /login
- 118. 118 Overloaded POST ● Providing a block of data, such as the result of submitting a form, to a data-handling process ● "Data-handling process" can be anything – POST /users/sort – POST /login – ...etc.
- 119. 119 PUT - Update a resource ● A request to the server should modify the resource
- 120. 120 PUT - Update a resource ● A request to the server should modify the resource ● Common response code – 200 OK – 204 No Content
- 122. 122 Idempotent & Safe ● Idempotent – A HTTP method can be called many times without different outcomes
- 123. 123 Idempotent & Safe ● Idempotent – A HTTP method can be called many times without different outcomes ● Safe – Do not modify resources
- 124. 124 Idempotent & Safe Idempotent Safe GET yes yes DELETE yes no POST no no PUT yes no
- 125. 125 Common examples
- 126. 126 GET /users
- 127. 127 GET /users ● Retrieve all users' brief
- 128. 128 GET /users ● Retrieve all users' brief { "users": [ { "name": "kewang", "slogan": "Hello World" }, { "name": "Tony Stark", "slogan": "I'm Iron Man" }, { "name": "America captain", "slogan": "U.S.A." } ] }
- 130. 130 GET /users/:user_id ● Retrieve a specific user information
- 131. 131 GET /users/:user_id ● Retrieve a specific user information { "name": "kewang", "slogan": "Hello World", "birthday": "19831108", "sex": 0, "email": [ "cpckewang@gmail.com", "kewang@mitake.com.tw" ], "company": "mitake" }
- 133. 133 GET /users/:user_id/avatars ● Retrieve a specific user's all avatars' URL
- 134. 134 GET /users/:user_id/avatars ● Retrieve a specific user's all avatars' URL { "avatars": [ { "url": "http://www.s3.com/abc.png", "created": 1413385358 }, { "url": "http://www.s3.com/def.png", "created": 1401239876 }, { "url": "http://www.s3.com/ghi.png", "created": 1348303844 } ] }
- 136. 136 GET /users/:user_id/avatar ● Retrieve a specific user's avatar URL
- 137. 137 GET /users/:user_id/avatar ● Retrieve a specific user's newest avatar URL
- 138. 138 GET /users/:user_id/avatar ● Retrieve a specific user's newest avatar URL { "url": "http://www.s3.com/abc.png" }
- 139. 139 Is Graph API RESTful? Facebook powered
- 140. 140 Graph API - Overview
- 141. Graph API - Overview ● Everything is a "node" has a unique ID
- 142. 142 Graph API - Overview ● Everything is a "node" has a unique ID ● Two nodes connect each other with a "edge"
- 143. 143 Graph API - Overview ● Everything is a "node" has a unique ID ● Two nodes connect each other with a "edge" ● A node information contains some "fields"
- 144. 144 Graph API - Are two people friends? ● GET /{user-a-id}/friends/{user-b-id}
- 145. 145 Graph API - Does someone like a Page? ● GET /{user-id}/likes/{page-id}
- 146. 146 Graph API - Publishing new Status Updates ● POST /{user-id}/feed ● POST /{page-id}/feed
- 147. 147 Graph API - Uploading Photos ● POST /{user-id}/photos ● POST /{page-id}/photos ● POST /{album-id}/photos
- 148. 148 So, Graph API is RESTful?
- 149. 149 Graph API isn't RESTful Deprecating the REST API
- 150. 150 An actual example Kick a user from system
- 151. 151 V1 - Kick a user from system
- 152. 152 V1 - Kick a user from system ● Kick by myself
- 153. 153 V1 - Kick a user from system ● Kick by myself – DELETE /me
- 154. 154 V1 - Kick a user from system ● Kick by myself – DELETE /me ● Kick somebody
- 155. 155 V1 - Kick a user from system ● Kick by myself – DELETE /me ● Kick somebody – DELETE /:user_id
- 156. 156 V2 - Restore a user to system
- 157. 157 V2 - Restore a user to system ● PUT /:user_id
- 158. 158 V2 - Restore a user to system ● PUT /:user_id – Restore somebody to system
- 159. 159 V2 - Restore a user to system ● PUT /:user_id – Restore somebody to system – Change somebody information
- 160. 160 V3 - Update a user status
- 161. 161 V3 - Update a user status ● PUT /:user_id/status
- 162. 162 V3 - Update a user status ● PUT /:user_id/status – Kick somebody from system
- 163. 163 V3 - Update a user status ● PUT /:user_id/status – Kick somebody from system – Restore somebody to system
- 164. 164 Internal & Open API
- 165. 165 Internal API design guideline
- 166. 166 Internal API design guideline ● Minimize data size
- 167. 167 Internal API design guideline ● Minimize data size ● Merge operations
- 168. 168 Internal API design guideline ● Minimize data size ● Merge operations ● Field names are short & convoluted
- 169. 169 Internal API design guideline ● Minimize data size ● Merge operations ● Field names are short & convoluted ● Customize request & response for official apps
- 170. 170 Open API design guideline
- 171. 171 Open API design guideline ● Minimize data size
- 172. 172 Open API design guideline ● Minimize data size ● Field names are simple & clear
- 173. 173 Open API design guideline ● Minimize data size ● Field names are simple & clear ● Build request & response generally
- 174. 174 Differences - GET /users Internal API { "us": [{ "id": "2d3f1a", "nm": "Kewang", "el": "kkk@mail.com" }, { "id": "9a4f57", "nm": "Hans", "el": "hhh@mail.com" }] } Open API { "users": [{ "id": "2d3f1a", "name": "Kewang" }, { "id": "9a4f57", "name": "Hans" }] }
- 175. 175 API Blueprint
- 176. 176 API Blueprint ● Introduction ● Aglio ● API-Mock ● Postman
- 177. 177 Introduction ● Web API Language ● Pure Markdown ● Design for Humans ● Understandable by Machines ● Powerful Tooling ● Easy Lifecycle
- 178. 178 Hello World
- 179. 179 Complex example
- 180. 180 Aglio - API Blueprint renderer
- 181. 181 Aglio - Complex example
- 183. 183 Live DEMO
- 184. 184 References
- 185. 185 References
- 186. 186 References ● Browser 與 Server 持續同步的作法介紹 ● Comet (programming) ● Engine.IO: the realtime engine ● Engine.IO Protocol ● API Design Optimized for Mobile Platform ● HTTP GET with request body ● Does Google treat 404 and 410 status code differen tly
- 187. 187
- 188. 188