Scraping

235 views

Published on

Published in: Technology, Business
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
235
On SlideShare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
2
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • Know the suburb you live inKnow the place you’re moving toFind the place to move to
  • Where do I get the data?
  • Scraping

    1. 1. FREE data available. ** Just scrape it
    2. 2. Public vs. Private data
    3. 3. “Paid” sourcesSurveysResearch and experimentsOfficial statisticsInternal data
    4. 4. Get whatever you need,whenever you need.
    5. 5. What is scraping?
    6. 6. HTML/CSS
    7. 7. Dynamic sites?
    8. 8. AJAX, REST, SOAP, RSS
    9. 9. And APIs too?
    10. 10. Documents?
    11. 11. How?
    12. 12. In whatever way you preferPythonPerlC#Java
    13. 13. So hard?
    14. 14. Tools“Scraper” chrome extensionwebharvy.com - desktop toolmozenda.com - SaaS solutiongrepsr.com - another SaaS solution
    15. 15. Maybe a little bit more technical.
    16. 16. SeleniumTwillRobot= Browser automation
    17. 17. Where’s the catch?
    18. 18. Be responsibleName your user agentCheck what you can/cannot use on the website.Never copy and paste content
    19. 19. But be persistentInduce delaysEmulate browserDistribute trafficProxies“Tor” network
    20. 20. Other issues? Legal!
    21. 21. BizWorld
    22. 22. Project BizWorld is a free tool .... that uses multiple sources to create an integrated picture of abusiness, group of businesses or an industry.Use it to research your target business market, potential partners orcompetition. Or even use it to monitor aspects of your own business.
    23. 23. Market research and reviewCustomer researchCompetitor researchCompany image in the Media
    24. 24. What We Pull in and TrackLinkedInTwitterBusinessWebsiteBizWorldFacebookBusinesskeywords industrysubsidiaries& outletsGoogle/webSocial mediaactivityThemes
    25. 25. How you can pull the dataFlexible filterPivot with drill-downDetailed listingCreate shortlist
    26. 26. OpportunityanalysisBizWorldPulldata viaAPIresultsYourdata$publish$$
    27. 27. ozplace.com.au(shadow)
    28. 28. ozplace=Research & FindThe place to live and buy in
    29. 29. Price/RentProfileTransportEnvironment
    30. 30. Everything is scrape-able.en.wikipedia.org/wiki/Open_data

    ×