Cloud native programming inherently involves working with remote endpoints: microservices, serverless, APIs, WebSockets, SaaS apps, and more. Ballerina is a new programming language that is designed to bring code-first agility to the challenge of integrating across endpoints. As we know, one of the fallacies of distributed computing is "the network is reliable". Resilience is the ability to recover to a working condition after being affected by a serious incident. Integration programmers have to be aware of such failures and also make their programs robust to deal with and recover from such failures. In this session, Afkham will look at how Ballerina helps integration developers write resilient programs and how it helps eliminate or reduce failures due to oversight. He will also demonstrate core resiliency techniques, such as timeout, retry, circuit breaker, distributed transactions, and others.

1. Service Resiliency in Microservices
Afkham Azeez, Vice President, WSO2
November 2018

2. “
Random House Kernerman Webster’s College dictionary
Ability to return to the original form or position after being
affected by a particular alteration.
Ability to recover from illness, depression, adversity or the like.
Resilience

3. Why should I bother?

4. Because real systems need to run in production!

5. Availability is critical in production
MTTF
MTTF + MTTR
A =
E[Uptime]
E[Uptime] + E[Downtime]
A =

6. Traditional approach to improving availability
MTTF
MTTF + MTTR
A =

7. Resilience approach to improving availability
MTTF
MTTF + MTTR
A =

8. Failures are a fact of life.
Don’t avoid failures. Embrace them.

9. Resilience
○ The ability of an app to recover from certain types of failure yet remain
functional from the users’ perspective
○ Users don’t notice it
○ Graceful degradation
○ Resilience is how you achieve the outcome
○ Also called recoverability

10. Features of resilience systems
○ Failures are compartmentalized
○ A resilient system would automatically cut off failing components and
reintegrate them once they are no longer failing

11. MSA and resilience
MSA inherently helps with designing resilient systems

12. Techniques
○ Timeout
○ Retry
○ Failover
○ Load balancing
○ Circuit breaker
○ Transactions

13. Don’t hide the network
○ Calls over the network should always return errors in addition to the
response
○ Network calls should be easily distinguishable

14. var backendRes = backendClientEP->forward("/hello", request);
match backendRes {
http:Response res => {
...
}
error responseError => {
log:printError("Error sending response", err = responseError);
}
}

15. Timeout
○ Hide downstream latency and keep the responsiveness to upstream
○ Prevent waiting forever

16. Timeout Sample
endpoint http:Client backendClientEP {
url: "http://localhost:8080",
timeoutMillis: 2000
};
var backendRes = backendClientEP->forward("/hello", request);
match backendRes {
http:Response res => {
...
}
error responseError => {
string resp = cache.get(“hello”);
...
}
}

17. Demo

18. Retry
○ Transient failures are not uncommon
○ In such cases, a simple retry is sufficient
○ Can be handled by
○ Retry immediately
○ Retry with a delay
○ IMPORTANT: Idempotency should be handled at the application layer

19. Retry Sample
endpoint http:Client backendClientEP {
url: "http://localhost:8080",
retryConfig: {
interval: 3000,
count: 3,
backOffFactor: 2
},
timeoutMillis: 2000
};
Retry after 3000, 6000 & 12000 ms

20. Demo

21. Failover
Switch to an alternate endpoint when the primary fails

22. Failover Sample
endpoint http:FailoverClient foBackendEP {
timeoutMillis: 5000,
failoverCodes: [501, 502, 503],
intervalMillis: 5000,
targets: [
{ url: "http://localhost:3000/mock1" },
{ url: "http://localhost:8080/echo" },
{ url: "http://localhost:8080/mock" }
]
};

23. Demo

24. Load Balancing
Load balance across endpoints

25. Load Balancing Sample
endpoint http:LoadBalanceClient lbBackendEP {
targets: [
{ url: "http://localhost:8080/mock1" },
{ url: "http://localhost:8080/mock2" },
{ url: "http://localhost:8080/mock3" }
],
algorithm: http:ROUND_ROBIN,
timeoutMillis: 5000
};

26. Demo

27. Circuit breaker
○ Some transient failures take much longer to recover
○ Repeatedly retrying may hinder recoverability
○ Retry up to a certain degree and cut off

28. Circuit Breaker
OPEN HALF OPEN
CLOSED
Success
Failure
Delay
Failure
threshold
exceeded

29. Circuit Breaker Sample
endpoint http:Client backendClientEP {
url: "http://localhost:8080",
circuitBreaker: {
rollingWindow: {
timeWindowMillis: 16000,
bucketSizeMillis: 2000
},
failureThreshold: 0.2,
resetTimeMillis: 10000,
statusCodes: [400, 404, 500]
},
timeoutMillis: 2000
};

30. Demo

31. Transactions
µ3
µ1 µ6
µ2 µ4
µ5
Initiator
Participants
S1
S2
S3

32. 2PC Coordination
Completion protocol
alt
Initiator Coordinator
Create-Context()
Micro-Transaction-Context
Commit()
Committed | Aborted | Mixed
Abort()
Aborted | Mixed

33. 2PC Coordination
Durable protocol - triggered by commit
success
Coordinator Participant
Prepare()
Prepared | Read-Only | Aborted | Committed
notify(...Commit...)
Committed
notify(...Abort...)
Aborted

34. 2PC Coordination
Durable protocol - triggered by abort
Coordinator Participant
notify(...Abort...)
Aborted

35. Transactions Sample - initiator
transaction with retries = 2 {
// Calling a local participant
localParticipantDoSomething();
// Calling a remote participant
http:Response res = check participant->get(“/ParticipantService/hi”)
} onretry {
// Code here will execute before retrying
} committed {
// Code here will execute after the initiated transaction
// has committed
} aborted {
// Code here will execute after the initiated transaction
// has been aborted
}

36. Transactions Sample - participant
service<http:Service> ParticipantService bind listener {
@transactions:participant {
oncommit = onTxnCommit,
onabort = onTxnAbort
}
hi(endpoint caller, http:Request req) {
...
}
}
...
function onTxnCommit(string transactionId) {
...
}
function onTxnAbort(string transactionId) {
...
}

37. Demo

38. https://github.com/afkham/
ballerinacon2018_resiliency

39. Q & A

40. THANK YOU