This document discusses a command line tool called hotdog for interacting with DataDog. It summarizes that hotdog allows users to search for hosts on DataDog using tag expressions and instance IDs. It works by parsing expressions, retrieving host tag mappings from the DataDog API, building an index of host-tag relations, evaluating the expression against the index, and outputting results. The presenter then discusses how Treasure Data uses DataDog for monitoring and is hiring.

1. Copyright ©2015 Treasure Data. All Rights Reserved.
hotdog
a TD tool for DD
Yamashita, Yuu (@yyuu)
Treasure Data, Inc.
yuu@treasure-data.com
March 11, 2015

2. Copyright ©2015 Treasure Data. All Rights Reserved.
about:yyuu
• 2014/10 - Software Engineer at Treasure Data Inc.
• Past: Infrastructure Engineer at Geisha Tokyo Entertainment Inc.
• OSS
• pyenv (github: yyuu/pyenv)
• xtc (bitbucket: yyuu/xtc)
• jetty-nosql-memcached (github: yyuu/jetty-nosql-memcached)
• botornado (github: yyuu/botornado)
• sbt-extras (github: paulp/sbt-extras)
2

3. Copyright ©2015 Treasure Data. All Rights Reserved.
about:yyuu
3

4. Copyright ©2015 Treasure Data. All Rights Reserved.
Today’s topic
• hotdog - yet another command line tool for DataDog
4

5. Copyright ©2015 Treasure Data. All Rights Reserved.
What’s DD?
5

6. Copyright ©2015 Treasure Data. All Rights Reserved.
What’s DD?
6
• System metrics
• dd-agent
• Application metrics
• fluent-logger
• td-agent
• fluent-plugin-metricsense

7. Copyright ©2015 Treasure Data. All Rights Reserved.
TD on DD
7

8. Copyright ©2015 Treasure Data. All Rights Reserved.
TD on DD
• We have *MANY* servers on DD
• BIG DATA NEEDS PROCESSORS
• We have many DataDog monitors to detect failures
• Send notification from DD to Slack and/or PagerDuty
8

9. Copyright ©2015 Treasure Data. All Rights Reserved.
TD ❤️ DD
9

10. Copyright ©2015 Treasure Data. All Rights Reserved.
Today’s topic
• hotdog - yet another command line tool for DataDog
10

11. Copyright ©2015 Treasure Data. All Rights Reserved.
TD 😱 PD
11
So, what is i-XXXXXXXX?

12. Copyright ©2015 Treasure Data. All Rights Reserved.
How to get node info of i-XXXXXXXX?
• AWS Management Console
• DataDog Infrastructure List
• Chef / Knife
• …
12

13. Copyright ©2015 Treasure Data. All Rights Reserved.
My solution; hotdog
13

14. Copyright ©2015 Treasure Data. All Rights Reserved.
My solution; hotdog
• Search hosts from DD tags
• Search hosts from instance-id
• Support expressions on DD tags
• AND, OR, NOT
• wildcard characters (‘*’, ‘?’, ‘[‘, ‘]’)
• Supports cooperation with pssh/GNU Parallel
14

15. Copyright ©2015 Treasure Data. All Rights Reserved.
How hotdog works
• Parse given expression by PEG parser
15

16. Copyright ©2015 Treasure Data. All Rights Reserved.
How hotdog works
• Parse given expression by PEG parser
• Get tag->hosts map from “GET /api/v1/tags/hosts”
16

17. Copyright ©2015 Treasure Data. All Rights Reserved.
How hotdog works
• Parse given expression by PEG parser
• Get tag->hosts map from “GET /api/v1/tags/hosts”
• Build host<->tag index on on-memory SQLite
17

18. Copyright ©2015 Treasure Data. All Rights Reserved.
How hotdog works
• Parse given expression by PEG parser
• Get tag->hosts map from “GET /api/v1/tags/hosts”
• Build host<->tag index on on-memory SQLite
• Evaluate the expression
18

19. Copyright ©2015 Treasure Data. All Rights Reserved.
How hotdog works
• Parse given expression by PEG parser
• Get tag->hosts map from “GET /api/v1/tags/hosts”
• Build host<->tag index on on-memory SQLite
• Evaluate the expression
• Print the results in specified format (text, json, yam, etc.)
19
small expression interpreter based on parslet

20. Copyright ©2015 Treasure Data. All Rights Reserved.
How hotdog works
• https://github.com/yyuu/hotdog
20

21. Copyright ©2015 Treasure Data. All Rights Reserved.
WE ARE HIRING!
21
Check: www.treasuredata.com