Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Case Study on Property Portal Data Security


Published on

Your listing data is valuable.  Scraping it NOT good for distribution of your listings to your competitors and fraudsters.  Controlling your listing data is good business - protects your value, saves on costs and maximizes revenue. This session explores the specific of how one property portal found strong ROI with bot detection protecting their listings.

Published in: Internet
  • Be the first to comment

  • Be the first to like this

Case Study on Property Portal Data Security

  1. 1. Securing Property Portals Lamudi Case Study
  2. 2. Platforms Portals Brokerages MLS Customers Premium Brands Distil in Real Estate and Premium Brands
  3. 3. The New Threat Landscape of APBs Advanced Persistent Bots (APBs)... Advanced Mimick human behavior Load JavaScript Load external resources Support cookies Browser automation (Selenium, PhantomJS) Persistent Dynamic IP rotation Distribute attacks across IP addresses Hide behind anonymous and peer-to-peer proxies 2015 Distil Bad Bot Report
  4. 4. Homegrown Solutions Are Ineffective Creates a poor user experience Bots appear human in logs Defeated by distributed IP attacks Defeated by advanced bots Labor intensive Defeated by low and slow crawlers Defeated by CAPTCHA farms Distributed attacks hard to pinpoint Defeated by peer-to-peer / proxies Reduces conversions by up to 27% Reactive in nature Reactive in nature
  5. 5. Web App Security Requires Complementary Solutions l DDoS Mitigation Firewall WAF Distil Bot Protection Core Competency Volumetric attacks on infrastructure Network layer attacks Application coding exploits Automated abuse, misuse, and attacks (scraping, fraud, account takeover, etc.) Techniques Scrubbing centers, Large pipes Access Control Lists (ACLs), Rules-Based App layer understanding, ACLs, Rules-Based Real-time Analysis, Fingerprinting, Honeypotting, Machine learning, Behavioral modeling
  6. 6. Survey Respondents 100 real estate executives representing over 600,000 realtors 14 real estate portal operators running 400,000 real estate websites 2015 Real Estate Web Scraping Survey
  7. 7. ○ 50% - 75% of bot traffic is from Consumer ISPs ○ 7 of top 10 sources of bad bots are Consumer ISPs ○ Most Consumer ISPs had 1,500+ IPs with bots Highlights of Bot Sources on Real Estate Websites The Facts on Scraping Real Estate Data Top 7 Consumer ISPs with Bot Traffic 1 Comcast 2 Time Warner Cable 3 Verizon FIOS 4 Charter 5 Cox 6 CenturyLink 7 AT&T Uverse Highlights of Bot Sophistication ○ 18-45% Automated browsers - mimicking humans ○ 14-25% Already in bot database - fingerprinted, known bots ○ 16-42% Slow crawlers - recycling IPs and user agents
  8. 8. About Lamudi 30+ Countries 900,000+ Listings 660+ Employees Property portal focused exclusively on emerging markets
  9. 9. Lamudi Bad Bot Challenges Bad Bot Challenges Bad guys scraped listing data to duplicate listings, impact SEO, and compete w/Lamudi Bots are spamming listing agent/owner contact forms & reducing agent retention & satisfaction 15,000 bad bot requests per minute (15x human traffic) caused slowdowns WAF-based IP blocking system used enginering time and was ineffective
  10. 10. Lamudi Selection Criteria Bot Detection and Mitigation Solution Requirements Support a complex deployment across several AWS instances with Akamai Block web scrapers and spammers without impacting human visitors Accurately identify good bots vs. bad bots Increase website availability and speed Detect automated browsing tools Simple setup for 30+ domains Little or no maintenance, “self-optimizing” solution
  11. 11. Lamudi Results with Distil Results with ROI No more scraping data → unique listings = better SEO No more form spam to agents → higher value leads = $$ Less time addressing agent complaints → Rev. Retention = $$ Increased website performance → Faster site = better SEO Save 100 engineering hours/mo. → More resources! Save $$ “Distil is the best anti-bot and anti-scraper protection solution available, hands down.” Oliver Fiege, CTO, Lamudi
  12. 12. How the Distil Bot Detection Solution Works As web traffic passes through Distil, the system 1. Fingerprints each incoming connection and compares it to our Known Violators Database 1. If it’s a new fingerprint, validates the browser to determine if it’s a Bot or Not 1. “No Silver Bullet” - Distil randomizes a battery of challenges to find bots and remain spoof- proof from the bot coders 1. Based on your settings, Distil automatically tags, challenges, or blocks the bot
  13. 13. Sticky Bot Tracking With No Impact On Real Users Device Fingerprinting Fingerprints stick to the bot even if it attempts to reconnect from random IP addresses or hide behind an anonymous proxy or peer-to-peer network Tracks distributed attacks that would normally fly under the radar Without Distil With Distil Without Impacting Users Sharing the Same IP Avoids blocking residential users or organizations that might share the same NAT as the bot or botnet
  14. 14. Browser Validation Detects all known browser automation tools, such as Selenium and Phantom JS Protects against browser spoofing by validating each incoming request as self reported Advanced Bot Detection Increases Accuracy Behavioral Modeling and Machine Learning Machine-learning algorithms pinpoint behavioral anomalies specific to your site’s unique traffic patterns Self optimizing algorithms improve bot detection and mitigation without manual configuration
  15. 15. Awards and Analyst Recognition “Analyzing behavior provides the best chance of detecting and blocking bot- driven attacks.” 5 Stars across the board.“ Verdict: For monitoring the impact of bots on a network this is the tool one needs.” The only anti-bot solution to be included in Gartner’s Online Fraud Detection Market Guide Ovum puts Distil Networks On The Radar. “Clear innovation compared to similar services.”
  16. 16. QUESTIONS….COMMENTS? C H A R L I E @ D I S T I L N E T W O R K S . C O M 1.703.962.1614 OR CALL CHARLIE ON