Advanced CouchDB phpday.it

1. CouchDB relax

2. CouchDB relax Sander van de Graaf @svdgraaf

3. http://joind.in/talk/view/3009

4. RELAX

5. CONTENTS • Introduction • PHP Usage • Replication/Scalability • Backend usage • Couchapps • Other stuff (GeoCouch!)

6. CouchDB relax

8. NOSQL

9. IT’S A MOVEMENT

10. 1998

19. Carlo Strozzi

20. NOSQL == Not Only SQL

21. “[The NoSQL movement] departs from the relational model altogether, it should therefore have been called more appropriately ‘NoREL’, or something to that effect.” - Carlo Strozzi

22. CouchDB relax

31. NUTSHELL

32. SPEED

33. APPEND ONLY

34. NO REPAIR NEEDED

35. COMPACTING

36. HTTP SERVER

37. CAP

38. CouchDB CAP

39. CouchDB EVENTUALLY CONSISTENT

40. FULL REST API

41. HTTP METHODS • GET • SELECT • PUT • UPDATE • POST • INSERT • DELETE • DELETE • COPY • ...

43. { "total_rows": 2, "offset": 0, "rows": [ { "id": "abc089b91fadfcf67a438e73f426f71d", "key": [ "2011-05-07 13:52:15", "abc089b91fadfcf67a438e73f426f71d" ], "value": { "_id": "abc089b91fadfcf67a438e73f426f71d", "_rev": "2-a38e08996586cd774b0b5c3006659de0", "message": "A2 HUISARTSENPOST BARNEVELD NIJKERKERWEG 119 3771LA 119" } }, { "id": "abc089b91fadfcf67a438e73f4286fd5", "key": [ "2011-05-07 14:00:16", "abc089b91fadfcf67a438e73f4286fd5" ], "value": { "_id": "abc089b91fadfcf67a438e73f4286fd5", "_rev": "2-691929b5f3d6cde58f27d62681c7b599", "message": "A1 LEEUWERIKSWEIDE 183 WAGENINGEN 6708LH 183" } } ] }

44. REPLICATION

47. CONTENTS • Introduction • PHP Usage • Replication/Scalability • Backend usage • Couchapps • Other stuff

48. RELAX

49. PHP USAGE

50. PHP LIBRARIES • PHPillow (LGPL) • PHP Object Freezer (BSD) • PHP On Couch (GPL 2 / 3) • PHP CouchDB Extension (PHP license) • SAG for CouchDB (apache) • Doctrine 2 CouchDB ODM

51. PHP ON COUCH <?PHP // setup connection for couchdb $client = new Couchdb_Client('http://ponies.couchone.com:5984','rainbows'); // fetch a document $doc = $client->getDoc('awesome_pony'); // updating document $doc->type = "awesome"; try { $client->storeDoc($doc); } catch (Exception $e) { echo "Document storage failed : " . $e->getMessage(); }

53. REPLICATION

54. DEFINITION “Replication is the process of sharing information so as to ensure consistency between redundant resources, such as software or hardware components, to improve reliability, fault-tolerance, or accessibility.” Source: wikipedia

56. CouchDB relax

57. CouchDB relax CouchDB relax

58. CouchDB relax CouchDB relax CouchDB relax CouchDB relax

59. CouchDB relax CouchDB relax

60. CouchDB relax CouchDB relax CouchDB relax

61. US NL CouchDB relax CouchDB relax BE CouchDB relax

62. USE CASE IDG Publishers

66. IDG US IDG AUSTRALIA CouchDB relax CouchDB relax CouchDB relax CouchDB relax CouchDB relax CouchDB relax CouchDB relax IDG NL IDG DE

67. IDG DE IDG Italy IDG Poland IDG Hungary IDG NL IDG US IDG Romania IDG Sweden IDG Brazil IDG UK IDG Spania IDG Portugal IDG Ghana IDG Japan IDG Thailand IDG Vietnam

68. IMPLICATION Every single bit of IDG content published worldwide, is available anywhere, anytime in msec.

69. P2P WEB

70. “World Domination”

71. HOW DO I DO THAT?

72. CLUSTERING “The fun stuff ”

74. loadbalancer ...n CouchDB relax CouchDB relax

75. CHALLENGES • Large amounts of data • Large views (with big/long map/reduce queries) • LOTS of trafﬁc • Location based partitions • For fun and proﬁt

76. MAP/REDUCE

77. INPUT IP Bytes 212.122.174.13 18271 212.122.174.13 191726 212.122.174.13 198 74.119.8.111 91272 74.119.8.111 8371 212.122.174.13 43

78. MAPPER => REDUCER IP Bytes 18271 191726 212.122.174.13 198 43 91272 74.119.8.111 8371

79. AFTER REDUCE IP Bytes 212.122.174.13 210238 74.119.8.111 99643

80. PARTITION INPUT Partition IP Bytes 0 212.122.174.13 18271 0 212.122.174.13 191726 0 212.122.174.13 198 1 74.119.8.111 91272 1 74.119.8.111 8371 0 212.122.174.13 43

81. MAPPER => REDUCER Partition IP Bytes 18271 191726 0 212.122.174.13 198 43 91272 1 74.119.8.111 8371

82. AFTER REDUCE IP Bytes 212.122.174.13 210238 74.119.8.111 99643

83. CLUSTERING OPTIONS • CouchDB Lounge • Pillow • BigCouch

84. LOUNGE • partitioning/clustering • Nginx module • meebo.com • ‘easy’ • http://tilgovi.github.com/couchdb-lounge/

85. LOUNGE • dumb_proxy => proxy for simple PUT/GET’s • smart_proxy => proxy for map/reduce over shards • replicator => updates all copies, redundantly

86. nginx dumb_proxy ...n CouchDB relax CouchDB relax

87. nginx smart_proxy ...n CouchDB relax CouchDB relax

88. Bonus: other nginx modules work too

89. PILLOW • Erlang based • router/rereducer (map/reduce over multiple systems) • In development (but promising!) • https://github.com/khellan/Pillow

90. BIGCOUCH • Fork • 100% api compatible • Open Source/Commercial • https://cloudant.com/#!/solutions/bigcouch

92. BACKEND USAGE

93. PROXIED CouchDB relax

94. DIRECT CouchDB relax

95. NOSQL && SQL HYBRID • onSave, onCommit hooks available in every major framework • onSave -> make a JSON representation of your object, and PUT it to couchdb (#protip: only ‘public’ data) • sql db is leading, you don’t care about versioning in couchdb • youcan use your data directly from couchdb within your frontend javascript

96. MODEL <?php class Pony extends Application_models { public function toArray() { $data = $this->_getData(); // we don't want any private data pushed unset($data['created_on']); unset($data['created_by']); unset($data['access_level']); unset($data['private_data']); $data['tags'] = $this->getTags(); $data['categories'] = $this->getCategories(); $data['rainbows'] = 'double'; return $data; } }

97. AFTER_SAVE <?php class article_module extends admin_module { public function after_save() { parent::after_save(); $data = $this->toJson(); // store the document in couchdb $res = CouchDB::put($data); // set the local id and revision, so we can // use it the next time $this->_id = $res->_id; $this->_rev = $res->_rev; } }

98. PROXY RewriteEngine On RewriteRule /data/(.*) http://127.0.0.1:5984/db/$1 [P,L]

99. JSONP ?callback=foobar foobar(...)

100. JAVASCRIPT <script type="text/javascript"> $.getJSON("/db/ponies/_design/ponies/_view/best-ponies?include_docs=true", function(res){ for(i in res.rows) { doc = res.rows[i].doc; // do stuff } }); </script>

102. COUCHAPP

103. “Distributed, scalable, web applications you say? omgwtfbbq!?!1!!!one1!1!eleven”

104. _attachments

105. CouchDB relax CouchDB relax CouchDB relax

106. INSTALLATION Couchapp 0.7.5

107. $ couchapp init

108. LAYOUT

109. $ couchapp push http://ponies.couchone.com:5984/rainbows

110. get your own copy: $ couchapp clone http://svdgraaf.couchone.com/couchappdemo/_design/↩ phpday phpday

111. https://github.com/brandon-beacher/couchapp-tmbundle

113. OTHER STUFF

114. REWRITES

115. _REWRITE

116. STEP 1 $ curl "http://ponies.couchone.com/rainbows/_design/ponies/ _view/best-ponies?descending=true&limit=5&key=”foobar”"

118. STEP 1 { .... "rewrites": [{ "from": "/best-5-ponies", "to": "ponies/_view/best-ponies", "method": "GET", "query": { "descending": true, "limit": 5, "key": "foobar" } }] }

119. STEP 1 $ curl "http://ponies.couchone.com/rainbows/_design/ponies/ _view/best-ponies?descending=true&limit=5&key=”foobar”"

120. STEP 1 $ curl "http://ponies.couchone.com/rainbows/_design/ponies/ _rewrite/best-5-ponies"

121. STEP 2 [vhosts] awesomeponies.com = /rainbows/_design/ponies/_rewrite

122. STEP 2 $ curl "http://ponies.couchone.com/rainbows/_design/ponies/ _rewrite/best-5-ponies"

123. STEP 2 $ curl "http://awesomeponies.com/best-5-ponies"

125. _CHANGES

126. RELAX

127. $ curl -X GET "http://ponies.couchone.com/rainbows/_changes"

128. {"results":[ ], "last_seq":0}

129. curl -X PUT http://ponies.couchone.come/rainbows/foobar -d '{"type":"awesome"}'

130. {"results":[ { "seq":1, "id":"foobar", "changes":[{"rev":"1-aaa8e2a031bca334f50b48b6682fb486"}] } ], "last_seq":1}

131. {"results":[ { "seq":1, "id":"foobar", "changes":[{"rev":"1-aaa8e2a031bca334f50b48b6682fb486"}] }, { "seq":2, "id":"foobar2", "changes":[{"rev":"1-e18422e6a82d0f2157d74b5dcf457997"}] } ], "last_seq":2}

132. _CHANGES OPTIONS • ?since • Longpolling • Continuous

133. SINCE $ curl -X GET "http://ponies.couchone.com/rainbows/_changes?since=20"

134. LONGPOLLING curl -X GET "http://ponies.couchone.com/rainbows/_changes?feed=longpoll&since=2"

135. CONTINOUS curl -X GET "http://ponies.couchone.com/rainbows/_changes?feed=continuous&since=2"

137. FILTERS

138. FILTERS/IMPORTANT.JS function(doc, req) { if(doc.priority == 'high') { return true; } return false; }

139. $ curl -X GET "http://ponies.couchone.com/rainbows/_changes?feed=continuous&filter=app/important

140. FILTERS/CITY.JS function(doc, req) { if(doc.city.name == req.query.name) { return true; } return false; }

141. curl -X GET "http://ponies.couchone.com/rainbows/_changes?feed=continuous&filter=app/name&name=verona

142. SHOWS

143. function(doc, req) { return { body: "Hello World" } }

144. curl -X "http://db.example.com/db/_design/foobar/_show/showfunction/docid"

145. function(doc) { return { "code": 302, "body": "See other", "headers": { "Location": doc.target } }; }

146. USE CASE CDN

147. From public hash to private ﬁle /m/m1gz4l4a1clb.jpg a2d85de382c8672925174e731a1d2a89

148. storage (http based) 5 Varnish proprietary middleware 1 2 webserver 3 4 mysql

149. storage (http based) 4 Varnish nginx + mod_cache 1 2 3 middleware

150. function(doc) { return { "code": 302, "body": "See other", "headers": { "Location": doc.target } }; }

151. { "_id": "m1gz4l4a1clb", "_rev": "2-d7855bba9445f446c0b71f044e796c28", "content-type": "image/jpeg", "target": "http://10.16.10.1/a2d85de382c8672925174e731a1d2a89", }

152. LUCENE

153. [external] fti=/path/to/python /path/to/couchdb-lucene/tools/couchdb-external-hook.py [httpd_db_handlers] _fti = {couch_httpd_external, handle_external_req, <<"fti">>}

154. FULLTEXT/BY_MESSAGE/BY_QUERY.JS function(doc) { var ret=new Document(); ret.add(doc.message); ret.add(new Date(doc.datetime)); return ret; }

155. curl -X GET "http://db.example.com/db/_fti/_design/app/by-query?q=foobar"

156. GEOCOUCH https://github.com/vmx/couchdb

158. GEOCOUCH • Supports bbox, polygon, lines, etc. • fork (for now) • outputs via lists, georss possible • directly useable by google maps • can read GIS data • combined with _changes makes interesting use cases possible

159. SPATIAL INDEX in spatial/points.js function(doc) { if (doc.geo && doc.geo.latitude != '' && doc.geo.longitude != '') { emit( { type: "Point", coordinates: [ parseFloat(doc.geo.latitude), parseFloat(doc.geo.longitude) ] }, [ doc._id, doc ] ); } }

162. Worldwide search http://db.example.com/couchappdemo/_design/phpday/_spatial/points?bbox=0,0,180,90 {"update_seq":3,"rows":[ { "id":"Verona", "bbox":[10.991748,45.438367,10.991748,45.438367], "value":[ "Verona", [10.991748,45.438367] ] }] }

163. bbox=west,south,east,north

165. COUCHDB FOR IOS https://github.com/couchbaselabs/iOS-Couchbase

166. http://www.iriscouch.com/

167. http://www.couchbase.com/downloads

168. Q?

169. http://joind.in/talk/view/3009

Advanced CouchDB phpday.it

Advanced CouchDB phpday.it

Recommended

More Related Content

Similar to Advanced CouchDB phpday.it

Similar to Advanced CouchDB phpday.it(20)

Recently uploaded

Recently uploaded(20)

Advanced CouchDB phpday.it