Wiring Hacker Synapses
Mustafa K. Isik Scott Lewis
codesurgeonblog.com composent.com
Eclipse Communication Framework
EclipseDay at the Googleplex
June 24th, 2008
Mountain View, CA

Who has seen the
screencast?
http://www.vimeo.com/1195398

Live Demo

Collaborative Coding and
Team Tooling in Eclipse

Collaborative Coding
‣ Real-Time Shared Editing with Cola
‣ Motivation
‣ Conceptual Overview
‣ Challenges
‣ Solutions
‣ The Future

Motivation
‣ Enable Pair Programming
‣ Live Code Review
‣ Help, Tutor, Mentor
‣ Tap into Domain/API Knowledge
‣ Independent of Geographic Restrictions
‣ Resilient to Location Limitations

Make Coding More Social,
Effective & Fun

Conceptual Overview
‣ 2 Participants working on the same
document
‣ text file, java source code, etc.
‣ On different machines
‣ changes sent over the network
‣ Consistent document state
‣ No unintended changes

time
Conflict Free Ideal
Anna Zoë
initial consistent doc initial consistent doc
ins(a)
ins(a)
ins(a)
index a
index a
ins(0)
ins(0) ins(a)
ins(0) ins(a) index 0
index 0

time
Review
Anna Zoë
initial consistent doc initial consistent doc
Some Editing
Consistent Documents
Intention Preserving Changes

Review (cont.)
‣ No conflicts because
‣ Anna generates change on common document
state, sends to Zoë
‣ Zoë updates unmodified local doc with
incoming change
‣ Zoë generates change on updated document,
sends to Anna
‣ Anna updates common doc state with incoming
change
‣ Strictly Sequential Execution ➭ ideal, but unrealistic

Challenge
‣ Out of Order sending/receptionZoë
Anna
time initial consistent doc initial consistent doc
ins(a)
ins(a)
index a
ins(0)
ins(0) ins(a) ins(0)
index 0 index 0 index a
ins(0) ins(a)
index 0 index a

Solution
‣ In-order Message sending/reception
‣ network protocols to the rescue
‣ build on TCP
‣ build on application level protocol relying
on TCP, e.g. XMPP
‣ ECF provides for abstraction from
underlying protocol, XMPPS, Skype, etc.

Challenge
‣ Text Editor Responsiveness
‣ Local changes need to be applied
immediately
‣ Network Latency
‣ Messages crossing “on the wire”
‣ Immediate application of local changes and
Network Latency ➭ High Probability for
Conflicts

time
Cross-on-Wire Conflict
Anna Zoë
initial consistent doc initial consistent doc
ins(a) ins(0)
ins(a) ins(0)
index a index 0
ins(0) ins(a) ins(0) ins(a)
index 0 index 0 index a

Solution
‣ Resolution mechanism for conflicting,
mutually directed changes ➭ operational
transformations
‣ precondition: locally applied operation
and incoming remote operation originate
from same document state
‣ postcondition: transformed remote
operation ready for intention-preserving
application to local document

Operational
Transformations
‣ How to determine origination state/
compatibility?
‣ stamp each locally generated operation
with counters
‣ local operation count
‣ remote operation count

Operational
Transformations (cont.)
‣ compare counters on incoming remote
operation & conflicting, already applied
local operation
‣ cola operational transformation
‣ input: conflicting incoming remote op &
already applied local op
‣ output: updated remote op, ready for
local application, e.g. index update

Operational Transformation for Cross-
time on-Wire Conflict Resolution
Anna Zoë
initial consistent doc initial consistent doc
ins(a) ins(0)
ins(a) ins(0)
index a index 0
coopt( ins(0) ins(a) )➭ ins(0) coopt( ins(a) ins(0) )➭ ins(a+L)
no update necessary! updated index!
ins(0) ins(a) ins(0) ins(a+L)
index 0 index 0 index a + L

Pseudocode:
Resolving Cross-On-Wire
//assert operation compatibility, i.e. same
origination state
Assert.isTrue(
localOp.sentCount == remoteOp.receivedCount );
if ( localOp.isIns && remoteOp.isIns ) {
if( localOp.index < remoteOp.index){
//move remoteOp.index right by length of
localOp
} else if (localOp.index == remoteOp.index) {
//notion of docOwner, consistently clarify
preference
} else if (localOp.index > remoteOp.index) {
//do nothing to remote op, apply without
modification
}
}

Combinatorial
Explosion
‣ Determine atomic operations, e.g. del, ins
‣ to be transformed against each other
‣ The fewer the better
‣ model compound operations from simple
ones (e.g. replacement as deletion and
insertion)

Combinatorial
Explosion (cont.)
‣ |cases| ≡ |atomic operations| ^ 2 ∈ O(n^2)
‣ |cases| * |index_checks = 3|
‣ still ∈ O(n^2) though
‣ Not runtime problem, but
implementation complexity
‣ The fewer atomic operations the better!

Divergence by More
than one Operation
‣ Generation of multiple local changes while
remote operation is traveling
‣ that is: upon arrival of remote operation,
local doc changed by more than locally
applied operation
‣ precondition for operational
transformation not met

Divergence by More than One
time
Anna
Op. (cont.) Zoë
initial consistent doc initial consistent doc
ins(a) ins(z)
ins(a) ins(z)
ins(v)
ins(a) ins(v) coopt( ins(a) ins(z) )➭ ins(a)
no update necessary!
ins(a) ins(z)
coopt( ins(z) ? )➭ ? coopt( ins(v) ins(z) )➭ ins(v)
no update necessary!
ins(a) ins(v) ins(z)

time
Anna
Resolution Zoë
initial consistent doc initial consistent doc
ins(a) ins(z)
ins(a) ins(z)
ins(v)
ins(a) ins(v) coopt( ins(a) ins(z) )➭ ins(a)
no update necessary!
coopt( ins(z) ins(a) ) ➭ ins(x)
ins(a) ins(z)
where x = z + Length of ins(a)
coopt( ins(v) ins(z) )➭ ins(v)
coopt( ins(x) ins(v) ) ➭ ins(y) no update necessary!
where y = x + Length of ins(v) ins(a) ins(v) ins(z)
ins(a) ins(v) ins(y)

Additional Details
‣ Manage local queue of unacknowledged
operations
‣ add local operations as executed
‣ remove local operations as implicitly
acknowledged by remote operations
appropriate counter
‣ “virtual” update of queued up, applied
local ops’ properties ➮ not in this talk
‣ Helps to introduce notion of state-space
which is being traversed ➮ not in this talk

The Future
‣ More than 2 session participants
‣ Project Sharing
‣ API for Cola and its Model for Optimistic
Concurrency Control
‣ Diagrams, anyone?
‣ Deeper Integration
‣ Multiple Cursors
‣ Highlighted Areas
‣ More on this from Scott ... time left?

Resources
Mustafa’s blog http://codesurgeonblog.com
Scott’s ECF blog http://eclipseecf.blogspot.com/
ECF Eclipse Wiki http://wiki.eclipse.org/
Eclipse_Communication_Framework_Project
ECF Project Home http://www.eclipse.org/ecf/
Cola Screencast
http://www.vimeo.com/1195398
6000+ views in 6 days