Let's replicate cache across regions. in SERVERLESS!

1. Serverless
Multi Region Caching

2. Kurt Lee
Technical Leader, Vingle Inc
iOS / Frontend / Backend
kurt@vingle.net
https://github.com/breath103

3. © 2018, Amazon Web Services, Inc. or Its Affiliates. All rights reserved.
Vingle, Interest Network

4. © 2018, Amazon Web Services, Inc. or Its Affiliates. All rights reserved.

5. Q. What’s the point of doing multi region without local cache

6. Things That Will Not Be Covered
a. Why Master-Master Multi Region is SO HARD
b. How to migrate Monolith to Microservice
c. How to migrate to Serverless (Lambda)

7. Things That Will Be Covered
a. How to build multi-region cache replication
b. Include monitoring / backup
c. in Serverless

8. CDN
AP-NORTHEAST-2
User
US-EAST-1
CDN Edge (Lambda)
Edge (Lambda)
Memcached
Memcached
Feed
User
Card

9. Scenario
1) Someone request User A’s profile image from VIRGINIA
2) Put user A’s profile in to cache (VIRGINIA) and return
3) Someone request User A’s profile image on SEOUL
4) Put user A’s profile in to cache (SEOUL) and return
5) User’A updates send profile image change request (VIRGINIA)
6) Update Database and purge cache (VIRGINIA)
7) Someone requests User A’s profile image from SEOUL
8) ???? Old Image ?????

10. Scenario
1) Someone request User A’s profile image from VIRGINIA
2) Put user A’s profile in to cache (VIRGINIA) and return
3) Someone request User A’s profile image on SEOUL
4) Put user A’s profile in to cache (SEOUL) and return
5) User’A updates send profile image change request (VIRGINIA)
6) Update Database and purge cache (VIRGINIA)
7) VIRGINA -> SEOUL, “DEL User’A Profile Image Cache”
or, “SET User’A Profile Image Cache with new image”
8) Someone requests User A’s profile image from SEOUL
9) NEW IMAGE!

11. Challenge:
“How can we propagate
cache change (DEL, SET)
across regions?”

12. Available Solutions
Cross Region
Sync
Cache “GET”
Latency
Cost
DynamoDB
(GlobalTable or
DynamDB Stream)
AWS Supported,
Non-Atomic
At least 1000ms
20~30ms
Auto-Scaled,
Pay-As-You-Go
AWS RDS + DMS
AWS Supported,
Non-Atomic
At least 1000ms
5~10ms
On-Promise,
Instance Based,
Expensive
Memcached No 1~5ms
On-Promise,
Instance Based,
Cheap
Redis No 1~5ms
On-Promise,
Instance Based,
Cheap

13. Netflix does it (Obviously)

14. Netflix does it, BUT
Requires,
1) Kafka Cluster
2) Service Discovery
- Address of kafka in local region
- Address of Replication Proxy on
Other Region(S)
3) EVCache cluster
4) Monitoring, Logging.....

15. EVCache
- Basically, <Memcached + Extra Features>
- Extra Features
- Replication Layer (Capture every commands)
- Secondary Index (Using ElasticSearch)
- But AWS doesn't have "Managed EVCache"

16. So how can we do this in
"Severless" way?

17. CDN
AP-NORTHEAST-2
User
US-EAST-1
CDN
Edge (Lambda)
Edge (Lambda)
Memcached
Memcached
Replicator
Replicator
Firehose
S3 Bucket

18. Replicator (Lambda)
• It only knows "Local Region"
memcached url
• Receive SET / DEL command,
execute it
• Application should know all
other regions Replicators
endpoint
MemcachedReplicator Application
MemcachedReplicator Application
MemcachedReplicator Application

19. Replicator (Lambda)
export interface CacheEvent {
source: {
region: "us-east-1" | "ap-northeast-2";
};
metadata?: {
service: string; // Which service?
operation: string; // Which operation?
};
createdAt: number;
action: (
{
type: "SET";
key: string;
value: string;
lifetime: number;
} | {
type: "DEL";
key: string;
}
)
}
const memcached = new Memcached(process.env.URL);
export async function applyEvents(
events: CacheEvent[]
) {
await Promise.all(events.map(async (event) => {
const action = event.action;
switch (action.type) {
case "SET": {
return await memcached.set(
action.key, action.value, action.lifetime
);
}
case "DEL": {
return await memcached.del(action.key);
}
}
}));
}

20. Q. Application should know
all other regions Replicators endpoint.
A. It's Lambda! ARN is pretty same.
You don't need Service Discovery
npm run deploy:prod -- --region=ap-northeast-2
arn:aws:lambda:ap-northeast-2:12345:function:retriever-prod-receiver
npm run deploy:prod -- --region=us-east-1
arn:aws:lambda:us-east-1:12345:function:retriever-prod-receiver
npm run deploy:prod -- --region=us-west-1
arn:aws:lambda:us-west-1:12345:function:retriever-prod-receiver

21. Thus, At Application (Memcached Client),
const clients = [
"us-east-1",
"ap-west-1",
"ap-northeast-2"
].map(region => new AWS.Lambda({ region }));
async function set(key: string, value: string) {
const local = new MemcachedDriver(process.env.CACHE_URL);
await Promise.all([
local.set(key, value),
...clients.filter(c => c.region !== process.env.AWS_REGION).map((client) =>
client.invokeAsync({
FunctionName: "retriever-prod",
InvokeArgs: JSON.stringify({
source: process.env.AWS_REGION,
action: {
type: "SET", key, value
}
})
}).promise()
)
]);
}

22. Logging can be really complicated
1) For logging, it's better to have centralized bucket on single region
2) Otherwise,
us-east-1 → ap-northeast-2 log is at ap-northeast-2
ap-northeast-2 → us-east-1 log is at us-east-1
3) memcached SET / DEL takes about 5~10ms
4) but cross region access (either KinesisFirehose or Cloudwatch)
takes at least 300ms
5) So, Waiting for Logging is Really Really inefficient
6) And even expensive if you use Lambda. Cost = Duration * Memory

23. The only way to log without extra
latency in lambda:
Console.log + Cloudwatch

24. Replicator (Lambda)
const memcached = new Memcached(process.env.URL);
export async function applyEvents(
events: CacheEvent[]
) {
await Promise.all(events.map(async (event) => {
console.log(JSON.stringify(event));
const action = event.action;
switch (action.type) {
case "SET": {
return await memcached.set(
action.key, action.value, action.lifetime
);
}
case "DEL": {
return await memcached.del(action.key);
}
}
}));
}

25. But how can we "Query" or "Search" that?
- Cloudwatch → ElasticSearch,
- Work fine, A lot of guides, not serverless, Expensive
- Cloudwatch → Kinesis Firehose → S3 → Athena,
- SERVERLESS!
- Link

26. 1) Gunzip Cloudwatch log data,
2) Format, remove invalid data,
3) return!

27. Athena
CREATE EXTERNAL TABLE `prod_events`(
`source` struct<region:string>,
`target` struct<region:string>,
`action` struct<
type:string,
key:string,
value:string,
lifetime:int
>
)
ROW FORMAT SERDE
'org.openx.data.jsonserde.JsonSerDe'
STORED AS INPUTFORMAT
'org.apache.hadoop.mapred.TextInputFormat'
OUTPUTFORMAT
'org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat'
LOCATION
's3://retriever-prod-log/events'

28. Athena

29. QnA