This document discusses using a copy-on-write cache accelerator pattern and Varnish cache to optimize a microservice that allows customers to specify a favorite delivery location. Implementing a persistent cache with Varnish Cache Plus and transaction logging would make the solution robust and scalable by handling cache updates asynchronously and repopulating the cache if needed. This caching approach is well-suited for services with an expected low cache hit rate and improves performance by consuming fewer resources than directly calling the microservice.

1. COW's, microservices and Varnish

2. A copy on write cache
accelerator pattern
or
A cache close to you

3. Me
● Anders Håål
● Software engineer consultant
● anders.haal@ingby.com
● @thenodon
● Currently PostNord Logistics
● Developing customer and recipient services

4. Customer facing servics @PostNord
Logistic production systems
Predictive load → ∫(parcels)
“Low” change rate
Enterprise software solutions
Internet facing services
Microservice design
Polyglot strategy
Continues delivery
Push
data

5. Favorite ServicePoint
● 25% of the parcel volume is
delivered to Servicepoints
● Enable a recipient to specify a
favorite Servicepoint
● Change the default Servicepoint,
based on postal code, to recipients
favorite
700 000 – 1 000 000 parcels/day
10-15% yearly increase

6. Solution
● Change the default to favorite before sorting and
processing
● During validation of EDI
● Lookup
– recipient mobile or email has
chosen a favorite
● Replace
– Servicepoint in EDI
if match
UNA:+.? '
UNB+UNOC:3+000X5766:30+00012306:30+160331:0759+1042++SendIT'
UNH+123456+IFTMIN:S:93A:UN+PLE101'
BGM+++9'
DTM+10:201604051530:203'
TSR+57+73 '
RFF+CU:Ordernummer1'
RFF+FLW:ABSOIDD'
TDT+20'
NAD+CZ+0206700700++Avsändarnamn+Testgatan 2+Nacka++13135+SE'
CTA+IC+Sven Svensson'
NAD+OS+++Webshop+Testgatan 2+Nacka++13135+SE'
NAD+CN+++Mottagarnamn+Hedlunds backe 1+Lerum++44338+SE'
CTA+IC'
COM+0700000000:AL'
COM+sven@gmail.com:EM'
GID+1+1:PC'
PCI+24+00373500454584614380'
UNT+17+123456'
UNZ+1+1042'

7. Favorite Servicepoint
Service
FavoriteSP
EDI validation
system
API
Internal
API
Public API
(auth)
PUT GET DELETE
GET

8. Favorite Servicepoint
Service
FavoriteSP
EDI validation
system
API
Internal
API
Public API
(auth)
PUT GET DELETE
GET
Service operation ~2 ms
Network latency ~110 ms
Total time > 110 ms

9. Favorite Servicepoint
Service
FavoriteSP
EDI validation
system
API
Internal
API
Public API
(auth)
PUT GET DELETE
GET
Service operation ~2 ms
Network latency ~110 ms
Total time > 110 ms
Normal EDI validation per parcel 10-20 ms
TAKES TOO LONG

10. Adding Varnish Plus
Service
FavoriteSP
EDI validation
system
API
Internal
API
Public API
(auth)
Varnish
Cache Plus
PUT GET DELETE
GET

11. Adding Varnish Plus
Service
FavoriteSP
EDI validation
system
API
Internal
API
Public API
(auth)
Varnish
Cache Plus
PUT GET DELETE
GET
Expected hit rate is low
What is the right TTL?

12. Adding COW
Service
FavoriteSP
EDI validation
system
API
Internal
API
Public API
(auth)
Varnish
Cache Plus
PUT GET DELETE
GET
GET
PURGE
Async
call
On PUT →
Cache GET
On DELETE →
Cache PURGE

13. Adding COW
Service
FavoriteSP
EDI validation
system
API
Internal
API
Public API
(auth)
Varnish
Cache Plus
PUT GET DELETE
GET
On PUT →
Cache GET
On DELETE →
Cache PURGE
GET
Header X-force-fetch:true
PURGE
Async
call

14. Completed!
Service
FavoriteSP
EDI validation
system
API
Internal
API
Public API
(auth)
Varnish
Cache Plus
PUT GET DELETE
GET
On PUT →
Cache GET
On DELETE →
Cache PURGE
GET
Header X-force-fetch:true
PURGE
Async
call
{
~90 ms
{
~4 ms

16. What about data consistency
Network down?
Varnish down?
Varnish out of space?
Everything can go wrong!

17. “Transactions”
● Persistent cache
● Varnish Cache Plus – massive storage engine
● Transaction log to enable resending and preserve
transaction order
● Acknowledge/commit the call to Varnish
● Operation to re-populate cache

18. Benefits
● Robust
●
Scalable
● Improved speed
● Consumes the “right” resources
● Cache updates are done out of EDI validation service
execution scope
● Effective for use cases with expected “low” hit rate
● Route to a specific cache based on request parameters

19. Nordic Favorite Servicepoint
Service
FavoriteSP
EDI validation
system
Sweden
API
Internal
API
Public API
(auth)
Varnish
Cache Plus
PUT GET DELETE
Async
call
PUT
EDI validation
system
Denmark
Varnish
Cache Plus
If SE
If DK

20. Thanks for listening!
Questions?