THE SCRAPING
PROBLEM
IN TICKETING
Martin Zetterlund,
Founder of Sentor Managed Security Services
SCRAPING DEFINED
“To automatically interact with a
website through the use of a computer
program.“
We also include manual ...
HOW COMMON IS IT
• Our clients averaged 23 % of
scraped traffic.
• Those hardest hit experienced in
excess of 50 % of scra...
SCRAPING: AN INDUSTRY
“The companies stealing your data or scraping your tickets might have more
resources than you expect...
THE LEGAL SITUATION
• Unclear and fairly complicated
situation. Ticketmaster seem to
have had some success in US and
have ...
OUR CLIENT
• Major ticket vendor based in London
with a world-wide reputation
• Experienced issues with waiting
room engag...
THE PROBLEM
Second hand sales
• Half empty venues
due to tout having
problem selling
tickets
• Bad PR
Site overload
• Will...
LESSONS LEARNED
Our client attempted to solve the problem by using manual log analysis to find
IP addresses that hit their...
HOW TO STOP SCRAPING
• Rate limiting
Against non-distributed scrapers
• Captcha
Effects user experience but can
be effecti...
KEY SUCCESS FACTORS
Resources
Needs both dev and
analysis to work together
over time
Technology
Platform needs to be
able ...
SCRAPESENTRY IN BRIEF
BENEFITS OF SCRAPESENTRY
• Correlates scraper IP information over a
global network of clients
• Real-time analysis of user...
THANK YOU!
martin.zetterlund@sentormss.com
For more information visit: www.ScrapeSentry.com
CONTACT ME:
Upcoming SlideShare
Loading in …5
×

The scraping problem in ticketing - Martin Zetterlund from Sentor

565 views

Published on

This is the presentation that Martin Zetterlund gave at Ticketing Technology Forum in London in March 2014.

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
565
On SlideShare
0
From Embeds
0
Number of Embeds
230
Actions
Shares
0
Downloads
6
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

The scraping problem in ticketing - Martin Zetterlund from Sentor

  1. 1. THE SCRAPING PROBLEM IN TICKETING Martin Zetterlund, Founder of Sentor Managed Security Services
  2. 2. SCRAPING DEFINED “To automatically interact with a website through the use of a computer program.“ We also include manual labor in low cost countries that are sometimes utilized to do the work of a computer program
  3. 3. HOW COMMON IS IT • Our clients averaged 23 % of scraped traffic. • Those hardest hit experienced in excess of 50 % of scraping traffic to their sites. Data from The Scraping Threat Report 2014
  4. 4. SCRAPING: AN INDUSTRY “The companies stealing your data or scraping your tickets might have more resources than you expect and they change their behavior in order to bypass safety methods” Examples of professional scraping companies that sell their services to anyone are screenscraper.com and mozenda.com
  5. 5. THE LEGAL SITUATION • Unclear and fairly complicated situation. Ticketmaster seem to have had some success in US and have an interesting case against higs tickets right now. • O‟leary won the right to call resellers „worthless ticket touts‟ in ads.
  6. 6. OUR CLIENT • Major ticket vendor based in London with a world-wide reputation • Experienced issues with waiting room engaging due to scrapers • Negative publicity as well as site overload
  7. 7. THE PROBLEM Second hand sales • Half empty venues due to tout having problem selling tickets • Bad PR Site overload • Will cause more expensive infrastructure • Bad user experience Functionality issues • Seats/tickets unavailable to users due to bots occupying them by use of website functionality
  8. 8. LESSONS LEARNED Our client attempted to solve the problem by using manual log analysis to find IP addresses that hit their site or seemed automated in their behavior. Time consuming Reactive No automated blocking Old data logs
  9. 9. HOW TO STOP SCRAPING • Rate limiting Against non-distributed scrapers • Captcha Effects user experience but can be effective against scrapers • Blacklisting IP addresses, User-agents http://www.scrapesentry.com for more details
  10. 10. KEY SUCCESS FACTORS Resources Needs both dev and analysis to work together over time Technology Platform needs to be able to adapt and evolve over time Persistence Problem will not go away. Staff for long time.
  11. 11. SCRAPESENTRY IN BRIEF
  12. 12. BENEFITS OF SCRAPESENTRY • Correlates scraper IP information over a global network of clients • Real-time analysis of user behaviour on website in order to identify scrapers • Continuously developed to stay ahead of scrapers • Supported 24/7 by security analysts and security operators from the Sentor Security Operations Centre.
  13. 13. THANK YOU! martin.zetterlund@sentormss.com For more information visit: www.ScrapeSentry.com CONTACT ME:

×