Download as PDF, PPTX
Uploaded bySang-Kyu Park
Replikativ
The document discusses building distributed reactive systems without network APIs. It explains that the problem with networking is that messages can be delayed, dropped, delivered out of order, or duplicated. This is not a problem that can be solved through programming techniques alone. The document introduces CRDTs (Conflict-Free Replicated Data Types), which use concepts from order theory like semilattices to allow for commutative, associative, and idempotent operations even in an asynchronous distributed system. It also discusses using Erlang and OTP for building fault-tolerant distributed systems based on the let it crash philosophy. Finally, it introduces replikativ, an open-source Clojure framework that aims to provide a scalable and distributed
Replikativ
- 1. replikativ Building services withoutnetwork API (CRDT와 OTP를 중심으로)
- 2. re-frame app-db components HiccupReagent VDOM React DOM f1 f2 f3 eventhandler
- 3. re-frame app-db components HiccupReagent VDOM React DOM f1 f2 f3 eventhandler Server
- 4. re-frame app-db components HiccupReagent VDOM React DOM f1 f2 f3 eventhandler Server ● send/receive ● serialization/deserialization ● packing/unpacking ● error/exception handling ● routing
- 5. re-frame app-db components HiccupReagent VDOM React DOM f1 f2 f3 eventhandler Server ● send/receive ● serialization/deserialization ● packing/unpacking ● error/exception handling ● routing Compelexity!
- 6. re-frame app-db components HiccupReagent VDOM React DOM f1 f2 f3 eventhandler Server ● send/receive ● serialization/deserialization ● packing/unpacking ● error/exception handling ● routing Compelexity!Not Reactive
- 7. wanted app-db components HiccupReagent VDOM React DOM f1 f2 f3 eventhandler Server Reactive Simple
- 8. wanted app-dbServer ● 내가 데이타를app-db에 쓰면 저절로 Server에 전달되었으면… ● Server에서 필요한 데이타가 저절로 app-db로 전달되어 있으면…
- 9. the problem isthe networking ClientServer reactive c/s programming
- 10. ClientServer Server ServerServer Server Server Client Client Client Client the reactivity ofthe distributed system the problem is the networking
- 11. Hard Disk Sector File Memory Data,Time Data FlowConceptialLayer Implementation Layer Physical Layer Reactive Programming Imperative Programming Abstraction Message, Network API ??? One Machine Distributed System Lan Electronic Wiring the problem is the networking
- 12. Leslie Lamport, 2013 Turing AwardWinner ● 논문 : Time, Clocks, and the Ordering of Events in a Distributed System, 1978 ● “Most people view concurrency as a programming problem or a language problem. I regard it as a physics problem” - Interviewed by Erik Meijer ● Yes, the problem is the networking. ● But it is not a programming or a language problem but a physics problem. the problem is the networking the transition of problem domain
- 13. ● handled bymathematics ○ game ○ AI ○ Algorithms ○ CSP ● how does the network problem be handled by math? the problem is the networking the physics problems
- 14. Marc Shapiro ● 논문: Conflict-Free Replicated Data Types, 2011 ● a data structure which can be updated independently and concurrently without coordination between the replicas ● use ○ Riak : NoSQL ○ Roshi : SoundCloud ○ Game : Angry Birds, Leage of Legends ○ TomTom : navigation data sync ○ akka distributed data CRDT
- 15. The messages is ●Delayed ● Dropped ● Delivered out-of-order ● Duplicated 이것은 프로그래밍 기술로 풀 문제가 아니다. CRDT
- 16. The messages is ●Delayed ⇒ Associative ● Dropped ● Delivered out-of-order ⇒ Commutative ● Duplicated ⇒ Idempotent CRDT
- 17. ● Associative (A ●B) ● C = A ● (B ● C) 메세지를 어떻게 묶든 결과는 같다. ● Commutative A ● B = B ● A 메세지가 순서가 바뀌어도 결과는 같다. ● Idempoent A ● A = A 메세지가 중복되어도 결과는 같다. CRDT
- 18. ● 수학의 한분과인 순서론(Order Theory)중 부분순서집합인 Semilattice를 이용하여 CRDT의 존재를 증명. ● 메세지가 전달되는 순서에 관한 문제를 하나의 추상적 물리현상으로 보고 수학적으로 분석. ● 위상학(Topology)와 범주론(Category Theory)과 관련 CRDT
- 20. ● Counter ○ G-Counter ○PN-Counter ● Register ○ LWW-Register ○ MV-Register ● Set ○ G-Set ○ 2P-Set ○ LWW-Set ○ PN-Set ○ OR-Set ● Map ○ ORMap ● Graph ○ DAG ● Array ○ RGA CRDTs
- 21. ● OTP(Open TelecomPlatform) ○ written in Erlang ○ used by Erlang ○ 미들웨어 ● Erlang ○ nine-nineth의 고가용성의 concurrent, functional, dynamically typed, and immutable language ○ Fault-tolerent ● Lightweight Process ○ 얼랭 프로그램을 구성하는 단위. ○ 프로그램은 얼랭 VM에서 관리 ○ OS process도 thread도 아님 ○ 어떤 자원도 서로 공유하지 않음. ○ 다만 메세지만 주고 받음. ○ Actor Model OTP
- 22. ● Erlang’s LetIt Crash Philosophy ● No Defensive Programming ○ Using exceptions and 'error handling' adds complexity ● Supervisor/Worker ● 에러 발생하면 해당 process를 죽이고 재시작. ● Robust System OTP Supervisor Worker
- 23. Hard Disk Sector File Memory Data,Time Data FlowConceptialLayer Implementation Layer Physical Layer Reactive Programming Imperative Programming Abstraction Message, Network API CRDT, OTP One Machine Distributed System Lan Electronic Wiring the problem is the networking
- 24. ● an open,scalable and distributive infrastructure for a data-driven community of applications ● written in clojure. ● CRDT based. ● subprojects ○ kabel ■ pluggable, asynchronous nerwork IO ■ pub/sub architecture ○ konserve ■ pluggable, asynchronous storage IO ■ backend : CouchDB, Riak, LevelDB ○ superv.async ■ expand core.async ■ Erlang/OTP supervisor ■ Let It Crash ● 단 2명이 5년에 걸쳐 만듦 ○ 완성도는 좋으나 아직은 실험적인 듯... replikativ
- 26.
- 27.