The document discusses various challenges and techniques related to service discovery and load balancing in networking, including the use of overlay networks, dynamic host addition, and health checks. It highlights the importance of proper registration and health monitoring for services, along with methodologies for load balancing and asynchronous communication. Additionally, it provides examples of service composition and asynchronous architectures, emphasizing the use of Redis Pub/Sub for data handling.

1. HTTP
Can we do better?

2. Challenges,
Debt, and
Cost of Production

3. Services

4. Services

5. Discovery
● Overlay Networks
● Registry/Reverse Proxying
● Port Scanning
● DNS

6. Discovery:
Overlay Networks
● Slow
● Rather call it host/container
discovery?
● Probably use Docker - Swarm

7. Discovery: Registry
● Etcd, consul, zookeeper
● Oh no, another cluster?

8. ● How do you add dynamic hosts?
● Proxy could Crash?
● How to find Proxies?
● Connection Draining
● Latency+
● Gateway Partitioning
Discovery:
Reverse Proxies

9. Discovery:
DNS Based
● No Port based Information
● SRV is cool
○ No one uses it
● Some Languages don’t obey TTL.
● Health-check remains external
process.

10. Components of Discovery
● Registry
○ Dynamically Configurable
● Registration
○ Clients emit Events
○ Registrar’s sees lack of events
● Health Check
○ Reachability
○ Load
○ Latency
● Load Balancing
○ Client side
○ Server Side

11. ● Server - Side
○ Proxy
○ Haproxy, httpd, consul, nginx etc.
○ ALB, ELB
● Client - Side
○ Netflix OSS
● Algorithm
● Staleness
Load Balancing
○ Round Robin
○ Weighted
Round-Robin
○ Random
○ URL
○ Least Connections
○ Least Traffic
○ Least Latency
○ Recently Used.

12. Other Essentials
● Request Tracing
● Unified Logging
● Error Reporting
● Wildcard
● HTTP Multicast, Broadcast?

13. Alternate?

14. Sample Service

15. HTTP Way

16. PubSub/Redis

17. PubSub Way

18. Gilmour-Libs
/gilmour-libs

19. Service Responder
func bindListeners(g *G.Gilmour) {
opts := G.NewHandlerOpts()
g.ReplyTo("action.login", echoReply, opts)
}
func main() {
engine := G.Get(redis.MakeRedis("127.0.0.1:6379", ""))
bindListeners(engine)
engine.Start()
}

20. Service Caller
func echoRequest(engine *G.Gilmour, msg string) (string, error) {
req := engine.NewRequest("action.login")
resp, err := req.Execute(G.NewMessage().SetData(msg))
if err != nil {
return "", err
}
var output string
err := resp.Next().GetData(&output)
if err != nil {
return "", err
}
return output, nil
}

21. Request Response
CALLER SERVICE
HELLO?
ALOHA!WRONG NUMBER
SENDER_ID
CODE

22. No/Good Service Discovery
● No querying discovery process
● Call service and not servers

23. Load Balancing
Or
Function Point Scaling

24. Load Balancing - Then
NOTIFICATION
SERVER 1
REPLICATED
LOAD
BALANCER
X.Y.Y.Z
?
CALLER
SERVICE
REGISTER
I, AM
HEALTHY?
NOTIFICATION
SERVER 2
NOTIFICATION
SERVER 3

25. Load Balancing : Now
NOTIFICATION
SERVER 2
GILMOUR
MANAGER.NOTIFICATION
NOTIFICATION
SERVER 1
NOTIFICATION
SERVER 3
MANAGER.NOTIFICATION
MANAGER.NOTIFICATION
CALLER
SERVICE
MANAGER.NOTIFICATION
{“NOTIFICATION”: “DATA”}

26. No/Good Load Balancing
● Has Capacity, Will Serve
● Message delivered to one and all
● Fittest node acquires lock first
● No need for Connection draining

27. And, How?

28. Just this
func bindListeners(g *G.Gilmour) {
opts := G.NewHandlerOpts().SetGroup("exclusive")
g.ReplyTo("echo", echoReply, opts)
}

29. Tracing, much?

30. $: redis-cli MONITOR
1509330865.376313 [0 127.0.0.1:59770] "PUBSUB" "NUMSUB" "gilmour.request.echo"
1509330865.376388 [0 127.0.0.1:59768] "SUBSCRIBE" "gilmour.response.1643e3fc-6362-4a2d-853b-35e8447a63f2"
1509330865.376502 [0 127.0.0.1:59770] "PUBLISH" "gilmour.request.echo" "{"data":"Hello:
World","code":200,"sender":"1643e3fc-6362-4a2d-853b-35e8447a63f2"}"
1509330865.376764 [0 127.0.0.1:59718] "SET" "1643e3fc-6362-4a2d-853b-35e8447a63f2exclusive"
"1643e3fc-6362-4a2d-853b-35e8447a63f2exclusive" "NX" "EX" "600"
1509330865.376928 [0 127.0.0.1:59718] "PUBLISH" "gilmour.response.1643e3fc-6362-4a2d-853b-35e8447a63f2"
"{"data":"Pong Hello:
World","code":200,"sender":"gilmour.response.1643e3fc-6362-4a2d-853b-35e8447a63f2"}"
1509330865.377133 [0 127.0.0.1:59768] "UNSUBSCRIBE" "gilmour.response.1643e3fc-6362-4a2d-853b-35e8447a63f2"

31. Network practices, much?

32. How?
# Server Side Timeouts
opts := G.NewHandlerOpts().SetTimeout(500)
g.ReplyTo("echo", echoReply, opts)
# Client Side Timeouts
opts := G.NewRequestOpts().SetTimeout(400)
req := engine.NewRequestWithOpts("action.login", opts)

33. Logging

34. How?
G.Logf("action.login %v", "message")
G.Log("action.login")

35. Errors

36. What do you do with Errors
● Status Code >= 300 is reported.
● Policy can be:
○ Publish
○ Queue
○ Ignore
func (r *Redis) SetErrorPolicy(policy string) error {
if policy != errorPolicyQueue &&
policy != errorPolicyPublish &&
policy != errorPolicyIgnore {
return errors.New("Invalid policy"))
}
r.errorPolicy = policy
return nil
}

37. Error Sample

38. Asynchronous Architectures

39. HTTP won’t, but
● Queues
● Celery
● SideKiq
● Kafka
● SQS
● Pravega
● Nats
… Can

40. Asynchronous
EVENT.POWER_UP
EVENT.*
EVENT.*
GROUP: REVENUE
FP SCALING
PUSH MESSAGE
SERVERADS/MARKETINGS
ERVER
UNLOCK LEVEL
BADGE
SERVICE
EVENT.*
ADS/
MARKETING
GAME GILMOUR
(Redis PubSub)
EVENT
EVENT.BRIDGE_FALL
FALL FROM BRIDGEFALL FROM
BRIDGEFALL FROM BRIDGE
GROUP: BRIDGE_FALL

41. Logging,
Revisited

42. Service Responder
func bindListeners(g *G.Gilmour) {
opts := G.NewHandlerOpts()
g.Slot("example.log", func(req *G.Request) {
var msg string
if err := req.Data(&msg); err == nil {
log.Println(req.Sender(), "->", msg)
}
}, nil)
}

43. Asynchronous Caller
func sendLog(engine *G.Gilmour) {
line := "WordCount"
engine.Signal("example.log", G.NewMessage().SetData(line))
line := "Fetching Data"
// err will be not nil, if Message for example.log was not received by anyone.
_, err := engine.Signal("example.log", G.NewMessage().SetData(line))
}

44. Composition

45. ● compose - service1 | service2 | composition3
● andand - service1 && service2 && composition3
● oror - service1 || composition2 || service3
● batch
○ (service1; service2; service3) > out
○ service1; composition2; service3 > out
● parallel
● lambda functions

46. Composition
pipe := engine.NewPipe(
engine.NewRequest("example.fetch"),
engine.NewRequest("example.words"),
engine.NewRequest("example.stopfilter"),
engine.NewRequest("example.count"),
engine.NewParallel(
engine.NewRequest("example.popular3"),
engine.NewRequest("example.popular4"),
engine.NewRequest("example.popular5"),
),
)
resp, err := pipe.Execute(G.NewMessage().SetData(url))
expected := []string{}
err := resp.Next().GetData(&expected)

47. Composition
batch := e.NewPipe(
e.NewRequest("weather.fetch"),
e.NewRequest("weather.group"),
e.NewParallel(
e.NewPipe(
e.NewParallel(
e.NewRequest("weather.min"),
e.NewRequest("weather.max"),
)),
e.NewPipe(
e.NewParallel(
e.NewRequest("weather.min"),
e.NewRequest("weather.max"),
))))
resp, _ := batch.Execute(G.NewMessage().SetData("pune"))

48. Demo

49. Examples: Popular Word Count
Given
● A S3 URL pointing to a file that has (lot of) random text
Goal
● Eliminate stop words from the file
● Find most popular 3, 4, and 5 letter words
● Find these in parallel

50. Output
● Popular 3 letter words: [way, for]
● Popular 4 letter words: [Text, copy]
● Popular 5 letter words: [Blind, blind]
Examples: Weather Aggregation

51. Examples: Composition

52. Thank you!
Piyush Verma
@meson10
Oogway
Consulting
http://oogway.in