Growth Hackers Toronto Web Scraping & Lead Generation
•Download as PPTX, PDF•
2 likes•361 views
Slides from the talk on November 17, 2015.
Recommended
More Related Content
Viewers also liked
Viewers also liked (16)
Similar to Growth Hackers Toronto Web Scraping & Lead Generation
Similar to Growth Hackers Toronto Web Scraping & Lead Generation (20)
Recently uploaded
Recently uploaded (20)
Growth Hackers Toronto Web Scraping & Lead Generation
- 1. Growth Hacking Toronto @GrowthCollectiv @Dean_h_wang @FundThrough @TopHat @EventMobi @GuruLink Connect with us: N: EventMobi-Guest P: emguests #GrowthHackingTO Web Scraping and Lead Generation By: Dean Wang
- 2. Web Scraping and Lead Generation November 17, 2015
- 3. About Me Dean Wang Data Intelligence Manager at Influitive @dean_h_wang linkedin.com/in/deanwang1
- 4. • Data Nightmares • What is web scraping? • Learning web scraping • Demo set up • Legal stuff and prevention • More to explore • Demo results Outline
- 5. Data Nightmares Is your database a strength or weakness?
- 6. Dude, where are my prospects?
- 10. Data Refresh/Appending Incomplete entries • Contact information • Segmentation Outdated entries • Person has left the company • Company has grown or shrunk • Company was acquired
- 11. Innovative Marketing Initiatives Personalize communications with targeted prospects Target prospects with strong signals More possibilities for ongoing campaigns
- 12. Traditional Solutions Manual data entry • Time and resource intensive List buying • Quality problems
- 13. New Solutions Web Scraping (the subject of this talk!) Accessing a website’s API Not always available
- 15. The World Wide Web • Full of information, directories • Some webpages are semi-structured • Web scraping can take advantage of this structure
- 16. “Web Scraping” as an Intern What an intern or scraper would do: • Find a list of useful pages • Click through each page • Note down the useful information
- 18. The Structure of Webpages
- 19. The Structure of Webpages HTML tags “mark up” pages <tag> <div> <span> <a href=“www.influitive.com”>
- 20. The Structure of Webpages With a few exceptions, each opening tag has a closing tag <div></div> <span></span> <h3></h3> <a href=“www.influitive.com”></a>
- 21. The Structure of Webpages Tags can contain text or even other tags within them <div>This text is in the div element</div> <nested> <tag></tag> </nested> <a href=“www.influitive.com”>Influitive</a>
- 22. The Structure of Webpages
- 23. CSS Selectors CSS Selectors enable you to communicate exactly what tags you want to select An advanced technique that gives you more control over scraping e.g. div, div a, h3.class
- 24. Structure and Web Scraping Web scraping takes advantages of HTML tags The tags must be somewhat regular on every page in order for the web scraping to work
- 27. Legal Stuff Is it legal? • A site’s terms of use often forbid it • Information may be otherwise publicly available • Case of QVC v Resultly • A gray area in general
- 28. Preventing Web Scraping • robots.txt • Restrictions on number of page views allowed • Blacklisting IP addresses • CAPTCHAs • Slight variations in website design
- 29. More to Explore Links and Tools
- 31. Different Web Scraping Tools Chrome Extensions Data Miner/Scraper Webscraper ScraperWiki Mozenda Knowledge of Python required Scrapy Selenium + Beautiful Soup
- 33. Questions?
- 34. Thank You
Editor's Notes
- Inbound marketing – some prospects may never respond Outbound marketing – still need to identify prospects More strategic
- Strategic vs tactical
- Marketing automation Initiatives outside of the inbox (not email)
- Traditional vs. cool and new, separate slide with last two slides
- Too dry
- Too dry
- It’s useful to understand, but it’s not completely necessary for web scraping Another language that accomplishes the same thing is XPath
- “The holy grail of scraping” You can only scrape effectively when…