The document discusses using Ruby and the Nokogiri library for parsing HTML and XML, primarily for documentarians. It provides examples of scraping data from web pages, including how to extract tables and process content. Additionally, it lists resources for further learning about Ruby scripting.

1. CHOPPING UP HTML
Ruby and Nokogiri for Documentarians
Wendy Shaffer @wshaffer74
Vmware, Inc

2. I’M SURROUNDED BY HTML AND XML

3. WHAT IS RUBY?
• Scripting language
• Many of you have probably heard of “Ruby on Rails”, used to build
websites
• https://www.ruby-lang.org/en/
• You can probably do most of what I’m describing in any language – pick
one you like

4. WHAT IS NOKOGIRI?
• Nokogiri is a library (“gem”) for Ruby that parses HTML and XML
• http://www.nokogiri.org/
• This means it “understands” how these languages are put together and
can identify information based on the HTML or XML structure
• <p class=“important”>This is some very important text.</p>

5. EXAMPLE 1: SCRAPING WIKI TABLES
Spreadsheet

6. EXAMPLE 1: CODE SNIPPETS
#Find the tables in the document
tables = page.xpath("//table")
#Process table rows
def process_row(row)
elements = row.children
z = Array.new
elements.each{|element|
#substitute all spaces with a
plain space to get rid of non-breaking
spaces
z <<
element.text.gsub(/[[:space:]]/, '
').strip
}
return z
end

7. EXAMPLE 2: SCRAPING A WEB REPORT

8. EXAMPLE 2: CODE SNIPPET
when "Clarity"
score_list[score] = score_page.css('table#tableSummary
tr.scorecardRow td')[2].text

9. OTHER NEAT STUFF YOU CAN DO
• Automate downloads of reports from web apps
• Figure out the number of links, images, etc. in a document
• Modify HTML output after publishing

10. MORE RESOURCES
• The Bastards Book of Ruby by Dan Nguyen:
http://ruby.bastardsbook.com/
• Why’s Poignant Guide to Ruby: https://poignant.guide/
• Everyday Scripting with Ruby: For Teams, Testers, and You by Brian
Marick