Basic TF Query Language Power searching
Query Language Outline <ul><li>Formulating Queries Basics </li></ul><ul><li>SEARCH and WHERE </li></ul><ul><li>Regular Exp...
Query language <ul><li>A way of asking the datastore to extract a list nodes, and a subset of their attributes </li></ul><...
Formulating Queries in the UI
Generic Query Interface <ul><li>Enter query, may split over several lines for clarity </li></ul><ul><li>Press ‘Run Query’ ...
Previous Queries <ul><li>Under the search query window a list of previous queries is available </li></ul><ul><ul><li>Click...
Saving Previous Queries <ul><li>Use “//” to provide a title to your query </li></ul><ul><li>Title is shown in Previous Que...
SEARCH and WHERE
SEARCH <ul><li>Looks through all the data and returns a set of nodes based on the specified node kind </li></ul><ul><li>SE...
Adding a WHERE <ul><li>SEARCH Host </li></ul><ul><ul><li>Collect the set of nodes of kind “Host” </li></ul></ul><ul><li>Ad...
Some Simple Conditions <ul><li>Equality </li></ul><ul><ul><li>a = b </li></ul></ul><ul><li>Not equals </li></ul><ul><ul><l...
More Advanced Conditions <ul><li>a HAS SUBSTRING b </li></ul><ul><ul><li>Works if the attribute contains the given string ...
Multiple Conditions and Arithmetic <ul><li>Multiple conditions if you join them with logical operations </li></ul><ul><ul>...
Special Conditions <ul><li>DEFINED </li></ul><ul><ul><li>This condition is true if the node has the attribute </li></ul></...
A Word on Performance a = b a HAS SUBWORD b a HAS SUBSTRING b a MATCHES b Full use of datastore indexes Full use of datast...
Alternatives to MATCHES <ul><li>Case insensitive search </li></ul><ul><ul><li>WHERE name MATCHES ‘(?i)fred’ (0.20s) </li><...
Query Exercises
Exercise 1 <ul><li>SEARCH host  </li></ul><ul><li>SEARCH Host </li></ul><ul><li>SEARCH SoftwareInstance </li></ul><ul><li>...
Exercise 2 <ul><li>SEARCH Host WHERE os = &quot;SunOS 5.10&quot;  </li></ul><ul><li>SEARCH Host WHERE os <> &quot;SunOS 5....
Exercise 3 <ul><li>SEARCH Host WHERE os = &quot;SunOS 5.10&quot; AND vendor = &quot;Sun Microsystems&quot; </li></ul><ul><...
Regular Expression Basics
Character and Anchoring Matches <ul><li>Characters match themselves, except </li></ul><ul><ul><li>. ^ $ * + ? { } [ ]  | (...
General Matches <ul><li>.   A period matches any character </li></ul><ul><li>[ abcdef]  square brackets match any one of t...
Repeated Characters <ul><li>Very often want to repeat/extend matches to more than a single character </li></ul><ul><li>Som...
Exercise 4 <ul><li>SEARCH Host WHERE name matches '01app$' </li></ul><ul><li>SEARCH Host WHERE name matches '02aPP$' </li>...
Exercise 5 <ul><li>SEARCH Host WHERE name matches '(?i)0*App$' AND vendor matches '(?i)^sun' </li></ul><ul><li>SEARCH Host...
Controlling Display with SHOW
SHOW <ul><li>By default results set nodes are displayed by showing the “summary attributes” </li></ul><ul><ul><li>Defined ...
Exercise 6 <ul><li>SEARCH Host WHERE name matches '(?i)0*App$' AND vendor NOT matches '(?i)^sun' SHOW * </li></ul><ul><li>...
Putting it all together
Query Format <ul><li>Basic format of Query: </li></ul><ul><ul><li>SEARCH <kinds> [where clause] [show clause] </li></ul></...
Final Exercises
Exercise 7 <ul><li>For all hosts show following attributes: </li></ul><ul><ul><li>hostname, host type </li></ul></ul><ul><...
Exercise 8 <ul><li>Refine the query from the last labs : </li></ul><ul><ul><li>To display all hosts except for windows </l...
<ul><li>Online Documentation: </li></ul><ul><ul><li>http://www.tideway.com/confluence/display/81/Search+and+Reporting+Serv...
Upcoming SlideShare
Loading in...5
×

Addmi 10.5-basic query-language

505

Published on

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
505
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
22
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • The search is useful to find Hosts where we have not been able to determine the CPU type.
  • Note: “search host” is of course meant to fail.
  • Note: the last query is meant to fail.
  • Note: The second query is meant to fail.
  • Addmi 10.5-basic query-language

    1. 1. Basic TF Query Language Power searching
    2. 2. Query Language Outline <ul><li>Formulating Queries Basics </li></ul><ul><li>SEARCH and WHERE </li></ul><ul><li>Regular Expression Basics </li></ul><ul><li>Controlling Display </li></ul><ul><li>Putting it all together </li></ul>
    3. 3. Query language <ul><li>A way of asking the datastore to extract a list nodes, and a subset of their attributes </li></ul><ul><ul><li>Performs the same function as SQL in a relational database </li></ul></ul><ul><li>It forms the basis of every datastore read </li></ul><ul><ul><li>Used internally by Atrium Discovery </li></ul></ul><ul><ul><li>Used when viewing built-in reports </li></ul></ul><ul><ul><li>Used for user-generated queries </li></ul></ul><ul><ul><li>Used for ad-hoc user queries </li></ul></ul><ul><ul><li>Used within TPL (patterns) </li></ul></ul><ul><li>There are special keywords to describe what to pick: SEARCH, WHERE </li></ul><ul><li>And keywords to decide how to display it: SHOW, ORDER </li></ul>
    4. 4. Formulating Queries in the UI
    5. 5. Generic Query Interface <ul><li>Enter query, may split over several lines for clarity </li></ul><ul><li>Press ‘Run Query’ to see results </li></ul><ul><li>Results can be exported as CSV or XML </li></ul>
    6. 6. Previous Queries <ul><li>Under the search query window a list of previous queries is available </li></ul><ul><ul><li>Clicking these copies the query into the search box </li></ul></ul>
    7. 7. Saving Previous Queries <ul><li>Use “//” to provide a title to your query </li></ul><ul><li>Title is shown in Previous Queries section </li></ul><ul><li>Can drag and drop query to bookmark </li></ul><ul><ul><li>Query will work on any Atrium Discovery instance </li></ul></ul>
    8. 8. SEARCH and WHERE
    9. 9. SEARCH <ul><li>Looks through all the data and returns a set of nodes based on the specified node kind </li></ul><ul><li>SEARCH kind </li></ul><ul><ul><li>Find all nodes of given kind </li></ul></ul><ul><ul><li>e.g. SEARCH SoftwareInstance </li></ul></ul><ul><li>These nodes are displayed as a list </li></ul>
    10. 10. Adding a WHERE <ul><li>SEARCH Host </li></ul><ul><ul><li>Collect the set of nodes of kind “Host” </li></ul></ul><ul><li>Add limiting conditions </li></ul><ul><li>WHERE os_type = 'Solaris' </li></ul><ul><ul><li>Restrict the set of nodes to those that contain a specific string in the “os_type” attribute </li></ul></ul><ul><li>WHERE <attribute> [conditional statement] </li></ul>
    11. 11. Some Simple Conditions <ul><li>Equality </li></ul><ul><ul><li>a = b </li></ul></ul><ul><li>Not equals </li></ul><ul><ul><li>a <> b </li></ul></ul><ul><li>Similarly </li></ul><ul><ul><li>a > b </li></ul></ul><ul><ul><li>a < b </li></ul></ul><ul><ul><li>a >= b </li></ul></ul><ul><ul><li>a <= b </li></ul></ul>
    12. 12. More Advanced Conditions <ul><li>a HAS SUBSTRING b </li></ul><ul><ul><li>Works if the attribute contains the given string </li></ul></ul><ul><li>a HAS SUBWORD b </li></ul><ul><ul><li>Works if the attribute contains the given word </li></ul></ul><ul><li>a MATCHES b </li></ul><ul><ul><li>Lets you use pattern matching (Regular Expressions) </li></ul></ul><ul><li>a IN [b, c, d, e] </li></ul><ul><ul><li>Lets you search for a number of possible values </li></ul></ul>
    13. 13. Multiple Conditions and Arithmetic <ul><li>Multiple conditions if you join them with logical operations </li></ul><ul><ul><li>AND </li></ul></ul><ul><ul><li>OR </li></ul></ul><ul><ul><li>NOT </li></ul></ul><ul><li>Brackets are useful to aid readability, and ensure precedence </li></ul><ul><ul><li>( ) </li></ul></ul><ul><li>Integer arithmetic only </li></ul><ul><ul><li>+ - * / </li></ul></ul>
    14. 14. Special Conditions <ul><li>DEFINED </li></ul><ul><ul><li>This condition is true if the node has the attribute </li></ul></ul><ul><ul><li>The value is not important </li></ul></ul><ul><ul><li>Useful given the dynamic nature of the datastore </li></ul></ul><ul><li>SEARCH Host WHERE processor_type NOT DEFINED </li></ul>
    15. 15. A Word on Performance a = b a HAS SUBWORD b a HAS SUBSTRING b a MATCHES b Full use of datastore indexes Full use of datastore indexes Partial use of datastore indexes No use of datastore indexes SPEED
    16. 16. Alternatives to MATCHES <ul><li>Case insensitive search </li></ul><ul><ul><li>WHERE name MATCHES ‘(?i)fred’ (0.20s) </li></ul></ul><ul><ul><li>WHERE name HAS SUBWORD ‘fred’ (0.03s) </li></ul></ul><ul><li>Multiple word match </li></ul><ul><ul><li>WHERE os MATCHES ‘(?i)Microsoft.+Datacenter’ (0.14s) </li></ul></ul><ul><ul><li>WHERE os HAS SUBWORD 'Microsoft Datacenter‘ (0.09s) </li></ul></ul><ul><li>A list of possible values </li></ul><ul><ul><li>WHERE name MATCHES ‘(?i)TSL-D’ (0.24s) </li></ul></ul><ul><ul><li>WHERE name IN ['tsl-dkirby','tsl-dtweed','tsl-duncan'] (0.08s) </li></ul></ul>
    17. 17. Query Exercises
    18. 18. Exercise 1 <ul><li>SEARCH host </li></ul><ul><li>SEARCH Host </li></ul><ul><li>SEARCH SoftwareInstance </li></ul><ul><li>SEARCH DiscoveredFile </li></ul><ul><li>Use “//” to title a report </li></ul><ul><ul><li>Drag and drop to create a bookmark </li></ul></ul>
    19. 19. Exercise 2 <ul><li>SEARCH Host WHERE os = &quot;SunOS 5.10&quot; </li></ul><ul><li>SEARCH Host WHERE os <> &quot;SunOS 5.10&quot; </li></ul><ul><li>SEARCH Host WHERE name has substring &quot; Win &quot; </li></ul><ul><li>SEARCH Host WHERE name has subword &quot;Win&quot; </li></ul>
    20. 20. Exercise 3 <ul><li>SEARCH Host WHERE os = &quot;SunOS 5.10&quot; AND vendor = &quot;Sun Microsystems&quot; </li></ul><ul><li>SEARCH Host WHERE os = &quot;SunOS 5.10&quot; AND vendor NOT matches &quot;Sun Microsystems&quot; </li></ul>
    21. 21. Regular Expression Basics
    22. 22. Character and Anchoring Matches <ul><li>Characters match themselves, except </li></ul><ul><ul><li>. ^ $ * + ? { } [ ] | () </li></ul></ul><ul><ul><li>To match one of these you need to put a backslash before it </li></ul></ul><ul><li>Be careful with Foundation pathnames </li></ul><ul><ul><li>Use . to match a full stop </li></ul></ul><ul><ul><li>Use to match a (Windows) path separator </li></ul></ul><ul><li>Anchoring matches </li></ul><ul><ul><li>^expr – match only at start of string </li></ul></ul><ul><ul><li>expr$ – match only at end of string </li></ul></ul>
    23. 23. General Matches <ul><li>. A period matches any character </li></ul><ul><li>[ abcdef] square brackets match any one of the things inside </li></ul><ul><li>[^123] match anything NOT contained within the brackets </li></ul><ul><li>a|b matches one thing or the other </li></ul><ul><li>Brackets group things together: (Fred)|(Fiona) matches either name </li></ul>
    24. 24. Repeated Characters <ul><li>Very often want to repeat/extend matches to more than a single character </li></ul><ul><li>Sometimes called wildcard matching </li></ul><ul><li>? matches 0 or 1 occurrences </li></ul><ul><li>+ matches 1 or more occurrences </li></ul><ul><li>* matches 0 or more occurrences </li></ul><ul><li>Control case matching </li></ul><ul><ul><li>(?i) means ignore case of matches </li></ul></ul>
    25. 25. Exercise 4 <ul><li>SEARCH Host WHERE name matches '01app$' </li></ul><ul><li>SEARCH Host WHERE name matches '02aPP$' </li></ul><ul><li>SEARCH Host WHERE name matches '(?i)02aPP$' </li></ul>
    26. 26. Exercise 5 <ul><li>SEARCH Host WHERE name matches '(?i)0*App$' AND vendor matches '(?i)^sun' </li></ul><ul><li>SEARCH Host WHERE name matches '(?i)0*App$' AND vendor NOT matches '(?i)^sun' </li></ul>
    27. 27. Controlling Display with SHOW
    28. 28. SHOW <ul><li>By default results set nodes are displayed by showing the “summary attributes” </li></ul><ul><ul><li>Defined in the taxonomy </li></ul></ul><ul><li>SHOW * </li></ul><ul><ul><li>All attributes defined in the taxonomy </li></ul></ul><ul><li>SHOW att1, att2 </li></ul><ul><ul><li>Only named attributes of the nodes </li></ul></ul><ul><li>SHOW summary, attr3 </li></ul><ul><ul><li>Summary attributes defined in the taxonomy, plus attr3 </li></ul></ul><ul><li>SHOW att1 AS ‘col1’ , att2 AS ‘col2’ </li></ul><ul><ul><li>Rename the column headings </li></ul></ul>
    29. 29. Exercise 6 <ul><li>SEARCH Host WHERE name matches '(?i)0*App$' AND vendor NOT matches '(?i)^sun' SHOW * </li></ul><ul><li>SEARCH Host WHERE name matches '(?i)0*App$' AND vendor NOT matches '(?i)^sun' SHOW hostname, os_type </li></ul><ul><li>SEARCH Host WHERE name matches '(?i)0*App$' AND vendor matches '(?i)^sun' </li></ul><ul><li>SEARCH Host WHERE name matches '(?i)0*App$' AND vendor matches '(?i)^sun' SHOW summary, ram AS &quot;Memory&quot; </li></ul>
    30. 30. Putting it all together
    31. 31. Query Format <ul><li>Basic format of Query: </li></ul><ul><ul><li>SEARCH <kinds> [where clause] [show clause] </li></ul></ul><ul><ul><li><kinds> – Used to specify the nodes (objects) to search within </li></ul></ul><ul><ul><li>[where clause] – Optional – filters the current set of nodes </li></ul></ul><ul><ul><ul><li>A WHERE clause evaluates an attribute </li></ul></ul></ul><ul><ul><ul><li>WHERE hostname = “itapp” </li></ul></ul></ul><ul><ul><li>[show clause] – Optional – defines information to return </li></ul></ul><ul><ul><ul><li>SHOW summary, attribute1, attrbute2 </li></ul></ul></ul><ul><ul><ul><li>SHOW hostname, os </li></ul></ul></ul>
    32. 32. Final Exercises
    33. 33. Exercise 7 <ul><li>For all hosts show following attributes: </li></ul><ul><ul><li>hostname, host type </li></ul></ul><ul><ul><li>all os attributes </li></ul></ul><ul><li>Let’s refine the query: </li></ul><ul><ul><li>Limit the query to only windows hosts </li></ul></ul><ul><ul><li>Show attributes a bit more interesting to windows hosts </li></ul></ul><ul><ul><li>Add the virtual attribute </li></ul></ul><ul><li>Refine the query again: </li></ul><ul><ul><li>Limit by windows hosts that have any service pack details </li></ul></ul><ul><li>Refine the query once again: </li></ul><ul><ul><li>Limit by windows hosts that DO NOT have service pack details </li></ul></ul>
    34. 34. Exercise 8 <ul><li>Refine the query from the last labs : </li></ul><ul><ul><li>To display all hosts except for windows </li></ul></ul><ul><ul><li>Remove service pack limitation </li></ul></ul><ul><ul><li>Remove the windows specific attributes </li></ul></ul><ul><ul><li>Add attributes that are more interesting to non windows hosts </li></ul></ul><ul><li>Refine again by: </li></ul><ul><ul><li>Limit by hosts that have power supply status details </li></ul></ul><ul><ul><li>Show package count totals </li></ul></ul>
    35. 35. <ul><li>Online Documentation: </li></ul><ul><ul><li>http://www.tideway.com/confluence/display/81/Search+and+Reporting+Service </li></ul></ul><ul><ul><li>http://docs.python.org/dev/howto/regex.html </li></ul></ul><ul><ul><li>http://www.tideway.com/configipedia/Writing_Efficient_Regex </li></ul></ul>Further Resources
    1. A particular slide catching your eye?

      Clipping is a handy way to collect important slides you want to go back to later.

    ×