SlideShare a Scribd company logo
Submit Search
Upload
Introduce of the parallel distributed Crawler with scraping Dynamic HTML
Report
Share
Kei Shiratsuchi
Senior Web Developer at Oh My Glasses, Inc
Follow
•
1 like
•
1,798 views
1
of
27
Introduce of the parallel distributed Crawler with scraping Dynamic HTML
•
1 like
•
1,798 views
Report
Share
Download Now
Download to read offline
Technology
動的HTMLスクレイピング対応並列分散クローラのご紹介 札幌Ruby会議02
Read more
Kei Shiratsuchi
Senior Web Developer at Oh My Glasses, Inc
Follow
Recommended
Web+GISという視点から見たGISの方向性 by
Web+GISという視点から見たGISの方向性
Hidenori Fujimura
1.8K views
•
49 slides
AngularJS - $http & $resource Services by
AngularJS - $http & $resource Services
Eyal Vardi
25.3K views
•
25 slides
Expressを使ってみた by
Expressを使ってみた
Atsuhiro Takiguchi
886 views
•
33 slides
AngularJS Routing by
AngularJS Routing
Eyal Vardi
10.1K views
•
13 slides
Dart and AngularDart by
Dart and AngularDart
Loc Nguyen
1K views
•
31 slides
hachioji.pm #40 : asynchronous in JS by
hachioji.pm #40 : asynchronous in JS
Kotaro Kawashima
536 views
•
7 slides
More Related Content
What's hot
AngularJS Services by
AngularJS Services
Eyal Vardi
9.8K views
•
28 slides
Code igniter parameter passing techniques by
Code igniter parameter passing techniques
Rakhitha Ratnayake
17.6K views
•
1 slide
Let's Build A Gutenberg Block | WordCamp Europe 2018 by
Let's Build A Gutenberg Block | WordCamp Europe 2018
Lara Schenck
2.5K views
•
51 slides
Div id by
Div id
Nelson Dionizio
95 views
•
1 slide
Backbone by
Backbone
Glenn De Backer
809 views
•
55 slides
Understanding backbonejs by
Understanding backbonejs
Nick Lee
2.6K views
•
39 slides
What's hot
(20)
AngularJS Services by Eyal Vardi
AngularJS Services
Eyal Vardi
•
9.8K views
Code igniter parameter passing techniques by Rakhitha Ratnayake
Code igniter parameter passing techniques
Rakhitha Ratnayake
•
17.6K views
Let's Build A Gutenberg Block | WordCamp Europe 2018 by Lara Schenck
Let's Build A Gutenberg Block | WordCamp Europe 2018
Lara Schenck
•
2.5K views
Div id by Nelson Dionizio
Div id
Nelson Dionizio
•
95 views
Backbone by Glenn De Backer
Backbone
Glenn De Backer
•
809 views
Understanding backbonejs by Nick Lee
Understanding backbonejs
Nick Lee
•
2.6K views
Hack tutorial by Wakana Yoshizawa
Hack tutorial
Wakana Yoshizawa
•
7.8K views
Flask and Angular: An approach to build robust platforms by Ayush Sharma
Flask and Angular: An approach to build robust platforms
Ayush Sharma
•
197 views
JQuery introduction by Pradeep Saraswathi
JQuery introduction
Pradeep Saraswathi
•
638 views
[Srijan Wednesday Webinars] Routing in Drupal 8: Decoupling hook_menu by Srijan Technologies
[Srijan Wednesday Webinars] Routing in Drupal 8: Decoupling hook_menu
Srijan Technologies
•
1.1K views
Clojure Workshop: Web development by Sytac
Clojure Workshop: Web development
Sytac
•
328 views
The hitchhiker's guide to the Webpack - Sara Vieira - Codemotion Amsterdam 2017 by Codemotion
The hitchhiker's guide to the Webpack - Sara Vieira - Codemotion Amsterdam 2017
Codemotion
•
765 views
Introduction to AngularJS For WordPress Developers by Caldera Labs
Introduction to AngularJS For WordPress Developers
Caldera Labs
•
5.6K views
Angular Promises and Advanced Routing by Alexe Bogdan
Angular Promises and Advanced Routing
Alexe Bogdan
•
3.2K views
Creating effective ruby gems by Ben Zhang
Creating effective ruby gems
Ben Zhang
•
980 views
Backbone js in drupal core by Marcin Wosinek
Backbone js in drupal core
Marcin Wosinek
•
1K views
Javascript - Beyond-jQuery by Tanner Moushey ❖ Mission Lab - WordPress Agency
Javascript - Beyond-jQuery
Tanner Moushey ❖ Mission Lab - WordPress Agency
•
171 views
Ruby on Rails Intro by zhang tao
Ruby on Rails Intro
zhang tao
•
294 views
Introducing AngularJS by Loc Nguyen
Introducing AngularJS
Loc Nguyen
•
661 views
Express Presentation by aaronheckmann
Express Presentation
aaronheckmann
•
1K views
Recently uploaded
NET Conf 2023 Recap by
NET Conf 2023 Recap
Lee Richardson
10 views
•
71 slides
Ransomware is Knocking your Door_Final.pdf by
Ransomware is Knocking your Door_Final.pdf
Security Bootcamp
59 views
•
46 slides
Unit 1_Lecture 2_Physical Design of IoT.pdf by
Unit 1_Lecture 2_Physical Design of IoT.pdf
StephenTec
12 views
•
36 slides
"Running students' code in isolation. The hard way", Yurii Holiuk by
"Running students' code in isolation. The hard way", Yurii Holiuk
Fwdays
17 views
•
34 slides
Special_edition_innovator_2023.pdf by
Special_edition_innovator_2023.pdf
WillDavies22
18 views
•
6 slides
Zero to Automated in Under a Year by
Zero to Automated in Under a Year
Network Automation Forum
15 views
•
23 slides
Recently uploaded
(20)
NET Conf 2023 Recap by Lee Richardson
NET Conf 2023 Recap
Lee Richardson
•
10 views
Ransomware is Knocking your Door_Final.pdf by Security Bootcamp
Ransomware is Knocking your Door_Final.pdf
Security Bootcamp
•
59 views
Unit 1_Lecture 2_Physical Design of IoT.pdf by StephenTec
Unit 1_Lecture 2_Physical Design of IoT.pdf
StephenTec
•
12 views
"Running students' code in isolation. The hard way", Yurii Holiuk by Fwdays
"Running students' code in isolation. The hard way", Yurii Holiuk
Fwdays
•
17 views
Special_edition_innovator_2023.pdf by WillDavies22
Special_edition_innovator_2023.pdf
WillDavies22
•
18 views
Zero to Automated in Under a Year by Network Automation Forum
Zero to Automated in Under a Year
Network Automation Forum
•
15 views
Future of Indian ConsumerTech by Kapil Khandelwal (KK)
Future of Indian ConsumerTech
Kapil Khandelwal (KK)
•
22 views
Future of AR - Facebook Presentation by ssuserb54b561
Future of AR - Facebook Presentation
ssuserb54b561
•
15 views
Evolving the Network Automation Journey from Python to Platforms by Network Automation Forum
Evolving the Network Automation Journey from Python to Platforms
Network Automation Forum
•
13 views
PharoJS - Zürich Smalltalk Group Meetup November 2023 by Noury Bouraqadi
PharoJS - Zürich Smalltalk Group Meetup November 2023
Noury Bouraqadi
•
132 views
Voice Logger - Telephony Integration Solution at Aegis by Nirmal Sharma
Voice Logger - Telephony Integration Solution at Aegis
Nirmal Sharma
•
39 views
Piloting & Scaling Successfully With Microsoft Viva by Richard Harbridge
Piloting & Scaling Successfully With Microsoft Viva
Richard Harbridge
•
12 views
Business Analyst Series 2023 - Week 3 Session 5 by DianaGray10
Business Analyst Series 2023 - Week 3 Session 5
DianaGray10
•
300 views
Democratising digital commerce in India-Report by Kapil Khandelwal (KK)
Democratising digital commerce in India-Report
Kapil Khandelwal (KK)
•
18 views
Data Integrity for Banking and Financial Services by Precisely
Data Integrity for Banking and Financial Services
Precisely
•
25 views
The Research Portal of Catalonia: Growing more (information) & more (services) by CSUC - Consorci de Serveis Universitaris de Catalunya
The Research Portal of Catalonia: Growing more (information) & more (services)
CSUC - Consorci de Serveis Universitaris de Catalunya
•
80 views
6g - REPORT.pdf by Liveplex
6g - REPORT.pdf
Liveplex
•
10 views
Microsoft Power Platform.pptx by Uni Systems S.M.S.A.
Microsoft Power Platform.pptx
Uni Systems S.M.S.A.
•
53 views
Case Study Copenhagen Energy and Business Central.pdf by Aitana
Case Study Copenhagen Energy and Business Central.pdf
Aitana
•
16 views
MVP and prioritization.pdf by rahuldharwal141
MVP and prioritization.pdf
rahuldharwal141
•
31 views
Introduce of the parallel distributed Crawler with scraping Dynamic HTML
19.
require 'rubygems' require 'sinatra' post
'/' do url = params[:url] data = params[:data] store(url, data) next_url = process(url) next_url end
20.
//
==UserScript== // @name greasi_scraper // @namespace http://libelabo.jp/ // @include http://images.google.co.jp/* // @require http://ajax.googleapis.com/ajax/libs/jquery/1.3.1/jquery.min.js // ==/UserScript== function postData(data) { var postData = $.param({url: location.href, data: JSON.stringify(data)}); GM_xmlhttpRequest({ method: "POST", url: "http://libelabo.jp/greasi/", headers: {'Content-type':'application/x-www-form-urlencoded'}, data: postData, onload: function(xhr){ location.href = xhr.responseText } }); }