Uploaded byPavlo Baron
From Hand To Mouth (@pavlobaron)
The document discusses moving away from traditional backend architectures towards more real-time and distributed systems. It advocates for storing data in a distributed database and processing it asynchronously in batches to improve user experience. Concrete examples are given of using protocols like ProtoBufs over REST, writing data from mobile clients to partitions, and performing analytics on batches of data later.
More Related Content
Similar to From Hand To Mouth (@pavlobaron)
From Hand To Mouth (@pavlobaron)
- 1.
- 2. Pavlo Baron pavlo.baron@codecentric.de @pavlobaron
- 3.
- 4. Or at leastwhat you call a backend
- 5. Your application server willjust slow down your machines to keep step with them
- 6. Or you willslow down your incoming traffic to give your application server a chance to breathe
- 7. User experience: order food atMcDonalds, get queued and receive the ordered food through mail 3 days later
- 8. User experience withsome psychology: order food at McDonalds, get queued, leave the restaurant with color pictures of your food and wait 3 days for its delivery
- 9. Every single data abstractionlayer only helps ruin the atmosphere through heating
- 10. Hey, man, you carryaround a damn USS Enterprise in your pocket And you can run a damn Babylon 5 in a cluster
- 11. And in theend, it's always store/update/ delete/read/search/ process. Isn't it?
- 12. So why drivea clown car when you can have a Ferrari full of these?
- 13. Why not justlive from hand to mouth?
- 14. You haz this: Youruser. Totally drunk in a football stadium. Just wants to inform the world that Ronaldo has scored a goal. Shaking hand, lots of typos and confusion.
- 15. You want this: Lethim protocol the fact at any cost. Look at it later, together with further facts to make a picture of the sports fact.
- 16. Concrete: Mobile clients writefrom everywhere, buffer, read occasionally, post-processing completely behind the scenes, like statistics etc.
- 17. Even more concrete: Immediate,reliable, massive writes. Analytics and processing in a batch afterwards. No need to be exact to the second and 100% data complete
- 18.
- 19. Very concrete: ProtoBufs, REST Round Robin local REST, local Erlang client Buffer Web Apps, R, tools etc.
- 20. Infrastructure can besolved. More challenging is how to get structured records out of a bunch of facts written by shaking hands. That's different story tho.
- 21. HTF does itwork?
- 22.
- 23. Da ring X bitinteger space 0 <= N <= 2 ^ X or: 2 x Pi 0 <= A <= 2 x Pi x(N) = cos(A) y(N) = sin(A)
- 24. Da cluster 12 partitions(constant) 3 nodes, 4 vnodes each add node 4 nodes, 3 vnodes each Alternatives: 3 nodes, 2 x 5 + 1 x 2 vnodes container based
- 25. Da quorum V: vnodesholding a key W: write quorum R: read quorum DW, PW, PR W > 0.5 * V R+W>V
- 26. Insert key Key =“foo” # = N, W = 2 (sloppy quorum) replicate N ok
- 27. Add node co py leave leave co py y leave p co
- 28. Lookup key (sloppy N quorum) Value = “bar” Key = “foo” # = N, R = 2
- 29. Remove node copy leave
- 30. write RM2 Da Gossip RM1 Clock table Value Update log stable clock Replica clock updates Value Executed operation table
- 31. Da vector clocks Node1 1,0,0 2,2,0 3,2,0 4,3,3 Node 2 1,1,0 1,2,0 1,3,3 4,4,3 Node 3 1,0,1 1,2,2 1,2,3 4,3,4
- 32. Da MapReduce query = Node X "Alice" map reduce hit list map map N= N= N= „Alice" "Alice" "Alice" Node A Node B Node C
- 33. Da hinted handoff N:node, G: group including N node(N) is unavailable replicate to G or store data(N) locally hint handoff for later node(N) is alive handoff data to node(N)
- 34. Key = “foo”,# = N -> Replica handoff hint = true fails Key = “foo” N replicate
- 35. Replica handoff recovers
- 36. And that's byfar not all: You can search through index with data locality. You can tag objects and query through 2i. You can add hooks.
- 37. You don't evenneed to store anything. Just distribute your calculation, search, batch
- 38. Have a lookat tendita.com
- 39. So it's evencomfy to load your clown car with these
- 40.
- 41. Most images originatefrom istockphoto.com except few ones taken from Wikipedia or Flickr (CC) and product pages/ publicly available presentations or generated through public online generators