Track and Trace
• When problems start, how do you
• tail -f logﬁle: which logﬁle?
• Who dealt with the user request?
• Which webserver?
• Which minion?
• An agent runs on every Perl program
• Connects to a hub via TCP
• Establishes a bidirectional link between us
and each web-head/worker
• Logging, stats and commands ﬂow between
• Your own C&C system