1. BUILDING A SCALABLE SYSTEM FOR
TRACKING SHIPPING PACKAGES

2. WHERE IS MY
PACKAGE?
https://www.etsy.com/listing/180703192/johannes-holst-sailboat-at-sea-1959

3. WHERE IS MY
PACKAGE?
https://www.etsy.com/listing/180703192/johannes-holst-sailboat-at-sea-1959
♥

4. PREMSHREE PILLAI

5. SR. ENGINEER, ETSY
PREVIOUSLY, YAHOO!
@premshree

6. AGENDA
https://www.etsy.com/listing/122960327/moleskine-cover-agenda-leather-cover

7. AGENDA
https://www.etsy.com/listing/122960327/moleskine-cover-agenda-leather-cover
Etsy
Shipping
Tracking Architecture
Monitoring
Q & A

8. empathy
https://www.etsy.com/listing/26849451/the-jetty-sad-girl-empathy-melancholy

9. empathy
noun ˈem-pə-thē
: the feeling that you understand and share
another person's experiences and emotions :
the ability to share someone else's feelings
https://www.etsy.com/listing/26849451/the-jetty-sad-girl-empathy-melancholy

10. https://www.etsy.com/listing/110699162/seeker-of-happiness-canvas-tote-bag-you
RE-IMAGINE
COMMERCE
IN WAYS THAT BUILD
A MORE FULFILLING
AND LASTING WORLD

12. https://www.etsy.com/listing/201864292/cleveland-map-street-map-vintage-print

15. Hong
Kong
https://www.etsy.com/listing/162358227/old-map-of-hong-kong-and-environs-16-x

17. Zagreb
https://www.etsy.com/listing/157215176/croatia-watercolor-painting-art-print

19. ♥ POTTERY
https://www.etsy.com/listing/195729720/handmade-dinner-plate-ceramic-plate

21. SCOTTSDALE, AZ

22. ANDOVER, MA

23. LEFKIS, GREECE

24. https://www.etsy.com/listing/191371121/jael-green-red-sisal-basket-from-kenya

25. 1.5 MILLION ACTIVE SELLERS
https://www.etsy.com/listing/191371121/jael-green-red-sisal-basket-from-kenya

26. 1.5 MILLION ACTIVE SELLERS
21.7 MILLION ACTIVE BUYERS
https://www.etsy.com/listing/191371121/jael-green-red-sisal-basket-from-kenya

27. 1.5 MILLION ACTIVE SELLERS
21.7 MILLION ACTIVE BUYERS
https://www.etsy.com/listing/191371121/jael-green-red-sisal-basket-from-kenya
$1.93 BILLION GMS (2014)

33. CARRIERS 250+
https://www.etsy.com/listing/129521877/original-world-map-watercolor-print

34. IN TRANSIT 500,000+
https://www.etsy.com/listing/129521877/original-world-map-watercolor-print

35. OUT FOR DELIVERY 10,000+
https://www.etsy.com/listing/129521877/original-world-map-watercolor-print

36. SHIPPING CARRIERS

37. FedEx
DHL
USPS
Australian Post
...

39. LET’S DIVE IN

40. mysql

51. tracking_events

52. SHIPPING APIS

53. GET http://production.shippingapis.com/ShippingAPI.dll?API=TrackV2&XML=
%3CTrackFieldRequest+USERID%3D%22022XXXXH6696%22%3E%3CTrackID+ID%3D
%22LZ13XXX3064US%22%3E%3C%2FTrackID%3E%3C%2FTrackFieldRequest%3E

54. GET http://production.shippingapis.com/ShippingAPI.dll?API=TrackV2&XML=
%3CTrackFieldRequest+USERID%3D%22022XXXXH6696%22%3E%3CTrackID+ID%3D
%22LZ13XXX3064US%22%3E%3C%2FTrackID%3E%3C%2FTrackFieldRequest%3E

60. TRACKING STATES

64. INFO_RECEIVED
PACKAGE_RECEIVED
IN_TRANSIT
DELAYED
OUT_FOR_DELIVERY
...
DELIVERED

65. STATE MAP

66. INFO_RECEIVED
“acceptance”
“processed at usps origin
facility"
“accepted at usps origin sort
facility”
“picked up”

67. INFO_RECEIVED
“tendered at fedex office”
“picked up"
“in fedex possession”

68. IN_TRANSIT
“dispatched from usps sort
facility”
“sorting complete"
“local holiday - will attempt
delivery on next working day”
“customs clearance processing

69. SELECT * FROM trackings
WHERE last_access_date < ?

72. SELECT * FROM trackings
WHERE last_access_date < ?

73. ? = now() - 15

75. caveat lector

76. WHAT’S THE PROBLEM?

77. SIZE

78. SELECT * FROM trackings
WHERE last_access_date < ?
LIMIT ?

79. STRAGGLERS

81. SHARDING 101

82. SHOP

83. SHOP

84. USER

85. USER

90. SELECT * FROM trackings
WHERE shard = ? AND
last_access_date < ?

92. QUEUE

93. FAUX QUEUE

95. QUEUE_3_DAY
QUEUE_1_DAY
QUEUE_3_HOUR
QUEUE_1_HOUR
...
QUEUE_STOPPED

96. QUEUE MAP

97. INFO_RECEIVED => QUEUE_1_DAY
PACKAGE_RECEIVED => QUEUE_1_DAY
IN_TRANSIT => QUEUE_1_DAY
OUT_FOR_DELIVERY => QUEUE_1_HOUR
...
DELIVERED => QUEUE_STOPPED

98. INFO_RECEIVED => QUEUE_3_HOUR
PACKAGE_RECEIVED => QUEUE_3_HOUR
IN_TRANSIT => QUEUE_3_HOUR
OUT_FOR_DELIVERY => QUEUE_1_HOUR
...
DELIVERED => QUEUE_STOPPED

99. QUEUE AGE

100. QUEUE_3_DAY => 1*60*60*24*3
QUEUE_1_DAY => 1*60*60*24
QUEUE_3_HOUR => 3*60*60
QUEUE_1_HOUR => 1*60*60

101. SELECT * FROM trackings
WHERE shard = ? AND queue
= ? AND last_access_date < ?

102. SELECT * FROM trackings
WHERE shard = ? AND queue
= ? AND last_access_date < ?

103. last_access_date < time() - age

104. ASYNC

105. GEARMAN

106. unique job key
(shard, queue)

112. IN SUMMARY

113. API
STATE MAP
SHARDING
CRON/ASYNC
FAUX-QUEUE

114. MONITORING

115. StatsD

116. StatsD
StatsD::increment("usps.get_tracking_call");

117. github.com/etsy/statsd/

118. Tracking stats
Gearman worker calls
Tracking calls by provider
Tracking queues
CURL errors

119. tracking numbers

120. update tracking batch

121. get tracking (usps)

122. get tracking (canada post)

123. Tracking stats
Gearman worker calls
Tracking calls by provider
Tracking queues
CURL errors

124. get tracking (usps)

125. Tracking stats
Gearman worker calls
Tracking calls by provider
Tracking queues
CURL errors

126. curl errors (usps)

127. curl errors (fedex)

128. IMAGINARY QUESTIONS
FROM YOU

129. IMAGINARY QUESTIONS
FROM YOU

130. IMAGINARY QUESTIONS
FROM YOU

131. ActiveMQ/RabbitMQ?

132. webhooks?

133. THANK YOU
https://www.etsy.com/listing/173815205/namaste-art-print

134. @premshree
https://joind.in/15376
https://www.etsy.com/listing/173815205/namaste-art-print

135. QUESTIONS
https://www.etsy.com/listing/241859251/original-rainbow-abstract-question-mark