This document discusses using UNIX pipes for web scraping. It describes scraping content from various Japanese adult websites, extracting metadata like title, images, description, and date. It lists over 30 website domains and YAML configuration files used to scrape specific pages and elements from each site.

4. Plagger
the UNIX pipe programming for web 2.0

7. author: Tatsuhiko Miyagawa
handle: http://www.asahi.com/
extract: <!-- End of Headline --><div class="day">.*?
</div>(.*?)<!-- google_ad_section_end
extract_capture: body

9. 1046034.yaml dmm_monthly.yaml himemix.yaml r-shodo.yaml
10musume.yaml duga.yaml ittele.yaml s-angels.yaml
1pondo.yaml eroerooyaji.yaml movie-girls.yaml sexyrose.yaml
99bb.yaml eroseven.yaml newplay-st.com.yaml smooth-movie.yaml
ademusume.yaml exotica-xxx.yaml omany.yaml sukiyakiav.yaml
caribbeancom.yaml g-area.yaml onacle.yaml tokyo-247.yaml
cutie82.yaml h4610.yaml peachbomb.yaml tora-tora-tora.yaml

13. custom_feed_handle: http://www.caribbeancom.com/
index2.htmcustom_feed_follow_link: http://www.caribbeancom.com/
moviepages/.*?.html
handle: http://www.caribbeancom.com/moviepages/.*?extract: <META
name="keywords" content="(.*?)">.*?<h2>(.*?)</h2>.*?<p
class="between2"></p>.*?<img
src="(.*?)".*?<textarea class=text_basic name=textarea wrap=VIRTUAL
cols=55 rows=8>(.*?)</textarea>
.*?<td class="value">(d{4}/d{2}/d{2})</td>
extract_capture: tags title img body date
extract_date_format: %Y/%m/%d