This document summarizes a talk about using parsers to extract structured data from unstructured text. It introduces Treetop, a Ruby library for building parsers based on Parsing Expression Grammars (PEGs). It provides examples of using Treetop grammars to parse configuration files and discusses techniques like using Ruby code in grammars and handling nested structures. The document notes that Treetop parsers are fast and linear time but can be memory intensive for large inputs. It recommends Treetop and links to further reading on PEGs and the Treetop library.

1. TREETOP: Making Parsers Fun
Monday October 7th @ Capital Factory

2. Say what? Why do I need a Parser?
Argentina.blft # real world example Big Lever Gears Format
propertyUrlPrefix = "propiedad";
landingPageUrlPrefix = "alquileres-vacaciones";
propertyReviewsPrefix = "reviews";
propertyReviewsWritePrefix = "reviews/write";
propertyReviewsConfirmPrefix = "reviews/confirm";
propertyReviewsResponsePrefix = "reviews/response";

4. This could work…

5. But…
Locale =
{
es_AR;
}
BrandId = 68;
BrandConfig
{
OmnitureAccount
{
Dev = "homeawayardev";
}
linguaEnabled = false;
}
Nesting
Enum
Number
String
Boolean

6. Ok… maybe not
# this just extracts the values, we haven’t
even
# begun to set the correct type or handle the
# nesting
/([a-zA-Z0-9])+=([a-zA-Z]+[^;]*|'"'[^"]*'"'|[0-
9]+|(true|false))/

8. Ok, so we need a Parser

10. .tt (Treetop) Grammar Files

11. Using your grammars inRuby
$tt foo.treetop bar.treetop
$tt foo.treetop -o foogrammar.rb

12. Using Parsers in Ruby

13. Let’s try this again

14. What about nesting?

16. But It gets better!

17. Use Ruby in your .tt files!

18. Alternate Method

19. Bringing it all together

20. Bringing it all together Part II

22. Gotchas
• First rule must match the document

23. On Parsing Expression Grammars
Parsing expression grammars (PEGs) are an alternative to
context free grammars for formally specifying syntax, and
packrat parsers are parsers for PEGs that operate in guaranteed
linear time through the use of memoization.
• Linear time, fast!
• Memory hog, storage proportional to the total input size
• Not suitable for natural language processing

24. Further Reading
Parsing Expression Grammars
• http://en.wikipedia.org/wiki/Parsing_expression_grammar
• http://bford.info/packrat/
Treetop
• http://treetop.rubyforge.org/
• http://github.com/nathansobo/treetop/tree/master
• https://groups.google.com/forum/#!forum/treetop-dev

25. Thank You!
Patrick Ritchie
@pritchie