MySQL::Replication
what is it?
It's a replacement for
MySQL Replication
why?
first...
what is replication?
synchronising data
between multiple databases
db1 db2
why replicate?
examples...
redundancy
database client client client client client client client client
database on client client client client client client client client
database on fire client client client client client client client client
database on fire client client client client client client client client
database on fire client client client client client client client client
arhhhh!
If we had replicated
to a hot standby
clients could be moved
 
 
another example
performance
database client
database client client
database client client client
database client client client client
database client client client client client
database client client client client client client
database client client client client client client client
database client client client client client client client client
database client client client client client client client client
database client client client client client client client client
arhhhh!
If we replicated
for scale out
load would be spread
 
 
 
 
 
 
 
 
so...
why replicate?
because it gives you benefits
compared to having
a single database
so...
how does it work?
in particular...
how does MySQL Replication work?
 
master
master slave
master client slave
master client binlog slave
master client binlog slave relay
master client binlog slave relay
the cool thing is
each slave
can be a master
to other slaves
master slave slave slave
master slave slave master/slave slave
master master/slave slave master/slave slave slave
this works fine
but...
slaves can only have a  single  master
master master/slave slave master/slave slave slave
so?
that means...
no multiple masters
slave master master master
slave master master master
why not?
don't ask me
because I don't know
but...
we can  emulate  multiple masters
to do this
we have to
setup a ring topology
what?
each slave
is a master
to another slave
master slave
master slave master/slave
master master/slave master/slave slave
master/slave master/slave master/slave master/slave
 
so...
how does this achieve
multiple master replication?
by having each master
write  all  queries
to it's binlog
...including the queries
it replicated
from  its  master
huh?
queries are passed
around the ring
kind of like
pass the parcel
 
why is this a problem?
when the ring breaks
 
 
 
 
 
arhhhh!
so does everything else
...and recovery is hard
why?
infinite loops!
what do you mean?
each master
is responsible
for termination
of its own queries
around the ring...
in other words
queries don't replicate infinitely
because
a master won't relay
its own queries twice
 
so...
what happens if...
a database goes down?
we recover by
...reconnecting the links
...and restarting replication
?
 
 
(OpenOffice doesn't do the arrow I wanted)
hang on...
who was responsible
for terminating
the failed database's queries?
hrmmm...
if there are queries
from the failed database
still replicating
that haven't yet terminated
 
arhhhh
Infinite queries, infinitely replicating
so...
how can I achieve
multiple master replication
...that is fault tolerant
...and is easy to recover from?
MySQL::Replication
It's a replacement
for MySQL's built-in replication
it's a client/server model
each master runs the server
slave master master master
slave master master server
slave server master server
slave server server server
each slave runs the client
...and the client
can run multiple instances
(one for each server)
(think peer-to-peer)
slave server server server
client(s) server server server
and if something breaks
client(s) server server server
client(s) server server server
client(s) server server server
only the peer connection is effected
client(s) server server server
client(s) server server server
and if the failed database recovers
we restart from where we left off
client(s) server server server
client(s) server server server
otherwise
we just fail it out
and can take our time rebuilding
it was redundant anyway
right?
:)
ohh
...and no infinite loops
why?
since we connect
directly to the server
no intermediary connections
can break
there's more...
relay caching
say what?
instead of connecting
directly to the master
the client
connects to a local relay
if it doesn't have
the queries we want
it will get them for us
example
multiple clients
in data center  A
...each connect
to a server
in data center  B
...each download the same queries
what a waste
...in bandwidth
...and server load
so...
instead
...each connect
to a local relay
now...
only a single client (the relay)
connects to the server
...saves bandwidth
...saves server load
:)
so...
CPAN?
almost ready
client/server working in production
I just need to do the documentation
then I'll put it on CPAN
what about the relay stuff?
It's nearly complete...
I just need to finish
...a couple more test harnesses
...and write the documentation
FAQ
what happens with collisions?
when two databases
update the same record
It's a race condition
solution?
solve it at the application layer
use a broker for global IDs
or shard writes
questions?
[email_address]
Upcoming SlideShare
Loading in...5
×

MySQL::Replication (Melbourne Perl Mongers 2011-07)

4,978

Published on

3 Comments
8 Likes
Statistics
Notes
  • Awesome, I just finished a design doc to do almost the same thing to solve the same issues. I'm going to go look at your work. :)
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • It was for a talk given at Melbourne Perl Mongers. I used ’because’ and ’so’ as breathers between sections of the talk and to wait for any questions about what I just talked about.
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Seriously, dude, you need to work on how you put slides together. Having a slide with the word 'because' on it makes me want to hit you in the face.
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
No Downloads
Views
Total Views
4,978
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
125
Comments
3
Likes
8
Embeds 0
No embeds

No notes for slide

MySQL::Replication (Melbourne Perl Mongers 2011-07)

  1. 1. MySQL::Replication
  2. 2. what is it?
  3. 3. It's a replacement for
  4. 4. MySQL Replication
  5. 5. why?
  6. 6. first...
  7. 7. what is replication?
  8. 8. synchronising data
  9. 9. between multiple databases
  10. 10. db1 db2
  11. 11. why replicate?
  12. 12. examples...
  13. 13. redundancy
  14. 14. database client client client client client client client client
  15. 15. database on client client client client client client client client
  16. 16. database on fire client client client client client client client client
  17. 17. database on fire client client client client client client client client
  18. 18. database on fire client client client client client client client client
  19. 19. arhhhh!
  20. 20. If we had replicated
  21. 21. to a hot standby
  22. 22. clients could be moved
  23. 25. another example
  24. 26. performance
  25. 27. database client
  26. 28. database client client
  27. 29. database client client client
  28. 30. database client client client client
  29. 31. database client client client client client
  30. 32. database client client client client client client
  31. 33. database client client client client client client client
  32. 34. database client client client client client client client client
  33. 35. database client client client client client client client client
  34. 36. database client client client client client client client client
  35. 37. arhhhh!
  36. 38. If we replicated
  37. 39. for scale out
  38. 40. load would be spread
  39. 49. so...
  40. 50. why replicate?
  41. 51. because it gives you benefits
  42. 52. compared to having
  43. 53. a single database
  44. 54. so...
  45. 55. how does it work?
  46. 56. in particular...
  47. 57. how does MySQL Replication work?
  48. 59. master
  49. 60. master slave
  50. 61. master client slave
  51. 62. master client binlog slave
  52. 63. master client binlog slave relay
  53. 64. master client binlog slave relay
  54. 65. the cool thing is
  55. 66. each slave
  56. 67. can be a master
  57. 68. to other slaves
  58. 69. master slave slave slave
  59. 70. master slave slave master/slave slave
  60. 71. master master/slave slave master/slave slave slave
  61. 72. this works fine
  62. 73. but...
  63. 74. slaves can only have a single master
  64. 75. master master/slave slave master/slave slave slave
  65. 76. so?
  66. 77. that means...
  67. 78. no multiple masters
  68. 79. slave master master master
  69. 80. slave master master master
  70. 81. why not?
  71. 82. don't ask me
  72. 83. because I don't know
  73. 84. but...
  74. 85. we can emulate multiple masters
  75. 86. to do this
  76. 87. we have to
  77. 88. setup a ring topology
  78. 89. what?
  79. 90. each slave
  80. 91. is a master
  81. 92. to another slave
  82. 93. master slave
  83. 94. master slave master/slave
  84. 95. master master/slave master/slave slave
  85. 96. master/slave master/slave master/slave master/slave
  86. 98. so...
  87. 99. how does this achieve
  88. 100. multiple master replication?
  89. 101. by having each master
  90. 102. write all queries
  91. 103. to it's binlog
  92. 104. ...including the queries
  93. 105. it replicated
  94. 106. from its master
  95. 107. huh?
  96. 108. queries are passed
  97. 109. around the ring
  98. 110. kind of like
  99. 111. pass the parcel
  100. 113. why is this a problem?
  101. 114. when the ring breaks
  102. 120. arhhhh!
  103. 121. so does everything else
  104. 122. ...and recovery is hard
  105. 123. why?
  106. 124. infinite loops!
  107. 125. what do you mean?
  108. 126. each master
  109. 127. is responsible
  110. 128. for termination
  111. 129. of its own queries
  112. 130. around the ring...
  113. 131. in other words
  114. 132. queries don't replicate infinitely
  115. 133. because
  116. 134. a master won't relay
  117. 135. its own queries twice
  118. 137. so...
  119. 138. what happens if...
  120. 139. a database goes down?
  121. 140. we recover by
  122. 141. ...reconnecting the links
  123. 142. ...and restarting replication
  124. 143. ?
  125. 146. (OpenOffice doesn't do the arrow I wanted)
  126. 147. hang on...
  127. 148. who was responsible
  128. 149. for terminating
  129. 150. the failed database's queries?
  130. 151. hrmmm...
  131. 152. if there are queries
  132. 153. from the failed database
  133. 154. still replicating
  134. 155. that haven't yet terminated
  135. 157. arhhhh
  136. 158. Infinite queries, infinitely replicating
  137. 159. so...
  138. 160. how can I achieve
  139. 161. multiple master replication
  140. 162. ...that is fault tolerant
  141. 163. ...and is easy to recover from?
  142. 164. MySQL::Replication
  143. 165. It's a replacement
  144. 166. for MySQL's built-in replication
  145. 167. it's a client/server model
  146. 168. each master runs the server
  147. 169. slave master master master
  148. 170. slave master master server
  149. 171. slave server master server
  150. 172. slave server server server
  151. 173. each slave runs the client
  152. 174. ...and the client
  153. 175. can run multiple instances
  154. 176. (one for each server)
  155. 177. (think peer-to-peer)
  156. 178. slave server server server
  157. 179. client(s) server server server
  158. 180. and if something breaks
  159. 181. client(s) server server server
  160. 182. client(s) server server server
  161. 183. client(s) server server server
  162. 184. only the peer connection is effected
  163. 185. client(s) server server server
  164. 186. client(s) server server server
  165. 187. and if the failed database recovers
  166. 188. we restart from where we left off
  167. 189. client(s) server server server
  168. 190. client(s) server server server
  169. 191. otherwise
  170. 192. we just fail it out
  171. 193. and can take our time rebuilding
  172. 194. it was redundant anyway
  173. 195. right?
  174. 196. :)
  175. 197. ohh
  176. 198. ...and no infinite loops
  177. 199. why?
  178. 200. since we connect
  179. 201. directly to the server
  180. 202. no intermediary connections
  181. 203. can break
  182. 204. there's more...
  183. 205. relay caching
  184. 206. say what?
  185. 207. instead of connecting
  186. 208. directly to the master
  187. 209. the client
  188. 210. connects to a local relay
  189. 211. if it doesn't have
  190. 212. the queries we want
  191. 213. it will get them for us
  192. 214. example
  193. 215. multiple clients
  194. 216. in data center A
  195. 217. ...each connect
  196. 218. to a server
  197. 219. in data center B
  198. 220. ...each download the same queries
  199. 221. what a waste
  200. 222. ...in bandwidth
  201. 223. ...and server load
  202. 224. so...
  203. 225. instead
  204. 226. ...each connect
  205. 227. to a local relay
  206. 228. now...
  207. 229. only a single client (the relay)
  208. 230. connects to the server
  209. 231. ...saves bandwidth
  210. 232. ...saves server load
  211. 233. :)
  212. 234. so...
  213. 235. CPAN?
  214. 236. almost ready
  215. 237. client/server working in production
  216. 238. I just need to do the documentation
  217. 239. then I'll put it on CPAN
  218. 240. what about the relay stuff?
  219. 241. It's nearly complete...
  220. 242. I just need to finish
  221. 243. ...a couple more test harnesses
  222. 244. ...and write the documentation
  223. 245. FAQ
  224. 246. what happens with collisions?
  225. 247. when two databases
  226. 248. update the same record
  227. 249. It's a race condition
  228. 250. solution?
  229. 251. solve it at the application layer
  230. 252. use a broker for global IDs
  231. 253. or shard writes
  232. 254. questions?
  233. 255. [email_address]
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×