Clickfraud bot signatures2_wordy

494 views

Published on

Published in: Technology, Design
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
494
On SlideShare
0
From Embeds
0
Number of Embeds
4
Actions
Shares
0
Downloads
0
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Clickfraud bot signatures2_wordy

  1. 1. ClickFraud:An Overviewof theProblemBrendanKittsData MiningAleksander KolczLegalAndy CooksonSteve SantorelliScott SteinJon ZiegerChris WeinsteinFinancial OpsLisa LarsenAnthony LopezTim SloaneViruses / MalwareJeff WilliamsZiv MadorBlack OpsRon MillsTier 2 SupportBridget BidlackJoseph MorrisonJason DorseyKimberly LancasterLarry GoldenMichael GrochauMSN SearchZijian ZhengPR / PlanningJenifer HandlerProgram ManagementBen HerdBrian BurdickBrendan KittsEmail SPAMGeoff HultenDevelopmentKazHank HoekJulien BeasleyKen PierceBinu JohnRajeev PrasadTier 3 SupportMichael GrochauMatt RiceDisplay Ads PlatformAlam AliPrasanthNirSystems IntegrationMartin MarkovBusijnessDevelopmentCorey Rosemond
  2. 2. "I think something has to be doneabout this really, really quickly,because I think, potentially, itthreatens our business model,"George Reyes, Google ChiefFinancial Officer“Click fraud is the biggestthreat to the Interneteconomy” George Reyes,Google Chief FinancialOfficerIntroductionPrevalenceDetection MethodsExamplesAuction TheoryConclusions“Anyone who says this is not areal challenge is kidding you”,John Slade, Senior DirectorProduct Management, Yahoo!Search AdvertisinhClick Fraud
  3. 3. Click Fraud: ReactionsPerplexed:"Its hard to tell how big theproblem is, but people arelooking at it closer and closeras the cost of searchadvertising goes up," JohnSquire, VP BusinessDevelopment, CoremetricsPhilosophical:"Click fraud is like a bigelephant standing in the middleof the living room. Everyonesees it and knows its there, butno one is quite sure what to doabout it." Lisa Wehr, President,Oneupweb,Hysterical:“Click fraud is ‘rampant’ and‘staggering’…. it could wipeout ROI in searchmarketing...”, StephenMesser, CEO, LinkShare
  4. 4. “Click Fraud: The Google Killer”(WebProNews)“Click Fraud: Is It Happening to YOU?”(French)“Click Fraud Looms as Search Engine Threat”(Associated Press)“The Google Bomb”z(Silicon Valley Daily News)“Fraud ABig Threat” (MSNBC)
  5. 5. Clickfraud Threats
  6. 6. Clicker Viruses and Trojan HorsesFirst observedin 2000 !
  7. 7. 115 computer botnet, Google syndication clicker
  8. 8. 100,000 computer botnet, Clickbot.ABots instructed to click no more than 20 times per day. clickon ads at a number of adult-oriented sites, which weredelivered by a common Web address: www.asdbiz.biz,according to Panda officials. This would bring up adult sites,such as girlsascats.com and virgin-clitors.com. Both of thesesites are registered to a possibly-fictional entity dubbedBeatOn in Kirov, Russia. Attempts to reach the Web-siteowners were not successful.
  9. 9. Source: http://www.benedelman.org/spyware/images/yahoo-apr06/4/Spyware – spyware link creationThe Spyware - Click-FraudConnection -- and Yahoos RoleRevisited - Ben EdelmanThis traffic is notable bothbecause it resulted fromspyware installed on my test PCwithout my consent, andbeause it resulted frominsertion of advertising linksinto third parties web sites(without their consent). Inparticular, as shown below, thistraffic was predicated onQklinkserver inserting link intothe New York Times web site,without its consent and withoutany on-screen labeling.On a test PC with Qklinkserver, Iobserved numerous extraneoushyperlinks inserted into thirdparties sites. See e.g. the NewYork Times site below. Note thestray hyperlink labeled "primeminister" -- words that are notactually a hyperlink on the"real" New York Times site, asviewed on uninfected test PCs.The Spyware - Click-FraudConnection -- and Yahoos RoleRevisited - Ben EdelmanThis traffic is notable bothbecause it resulted fromspyware installed on my test PCwithout my consent, andbeause it resulted frominsertion of advertising linksinto third parties web sites(without their consent). Inparticular, as shown below, thistraffic was predicated onQklinkserver inserting link intothe New York Times web site,without its consent and withoutany on-screen labeling.On a test PC with Qklinkserver, Iobserved numerous extraneoushyperlinks inserted into thirdparties sites. See e.g. the NewYork Times site below. Note thestray hyperlink labeled "primeminister" -- words that are notactually a hyperlink on the"real" New York Times site, asviewed on uninfected test PCs.http://www.benedelman.org/spyware/images/yahoo-apr06/4/video.wmvQklinkserver.com, Searchdistribution.net, Intermixs Sirsearch
  10. 10. Spyware window openerThis page gives screenshots showingon-screen displays after I requestedSmartBargains. 180solutions openeda popup substantially covering myinitial SmartBargains window. Thepopups traffic flowed from180solutions to Nbcsearch, then toDitto.com, on to Yahoo Overture, andfinally to a Yahoo advertiser -- allwithout me clicking on any sponsoredlink.Interestingly and unusually, theharmed Yahoo advertiser here isSmartBargains itself -- the same siteI had initially requested. The neteffect of this click fraud is to show theuser the site the user had requested-- but to show that site also in asecond ("double") window. Sinceusers end up at the requested site,users may not notice that anything iswrong. But from an advertisersperspective, something is verywrong: This process asksSmartBargains to pay YahooOverture PPC fees forSmartBargains own organic traffic --a bad deal, since Yahoo Overture isproviding SmartBargains with no newleads and no genuine value.All testing occurred on March 2,2006.This page gives screenshots showingon-screen displays after I requestedSmartBargains. 180solutions openeda popup substantially covering myinitial SmartBargains window. Thepopups traffic flowed from180solutions to Nbcsearch, then toDitto.com, on to Yahoo Overture, andfinally to a Yahoo advertiser -- allwithout me clicking on any sponsoredlink.Interestingly and unusually, theharmed Yahoo advertiser here isSmartBargains itself -- the same siteI had initially requested. The neteffect of this click fraud is to show theuser the site the user had requested-- but to show that site also in asecond ("double") window. Sinceusers end up at the requested site,users may not notice that anything iswrong. But from an advertisersperspective, something is verywrong: This process asksSmartBargains to pay YahooOverture PPC fees forSmartBargains own organic traffic --a bad deal, since Yahoo Overture isproviding SmartBargains with no newleads and no genuine value.All testing occurred on March 2,2006.Source: http://www.benedelman.org/spyware/images/yahoo-apr06/4/180 solutions
  11. 11. Ads not doing it for your webpage?Try porn instead!Spyware that replaces Contextual Banneradshttp://www.techshout.com/internet/2005/27/a-trojan-horse-program-that-targets-google-ads-has-been-detected-by-an-indian-web-publisher/
  12. 12. Search Chaff generators
  13. 13. IP Rotators: Anonymizer.com
  14. 14. Open Proxy Listshttp://www.samair.ru/proxy/
  15. 15. Commercial Clicker programs
  16. 16. Clicking Agent
  17. 17. Smart HitBot
  18. 18. RobinHood
  19. 19. iFaker
  20. 20. Agloco: Pay you for clicking!
  21. 21. Human Clicking Operations• Data World, New Delhi (1)– Rajiv Kumar, CEO– Sells the names of web sites that pay people to clickon internet ads for 350 rupees ($6.74).– Kumar claims to have recruited 300 clickers duringthe last year or so. “We’ve been doing this for a yearand a half and haven’t heard of any problems,” Kumarsays.
  22. 22. Human Clicking Operations• Shipranet, New Delhi (1)– Jagriti Bora, CEO– Bora has claimed to have recruited about 1,000clickers.– She says its perfectly all right to click on ads. “There’snothing wrong with looking through a shop windoweven if you don’t buy.”, she says.
  23. 23. Legal cases
  24. 24. $90million(Moneybag)Google "failed to take any significant measures to track or prevent clickfraud," and "fails to adequately warn its existing and potential customersabout the existence of click fraud."Google "failed to take any significant measures to track or prevent clickfraud," and "fails to adequately warn its existing and potential customersabout the existence of click fraud."Google: Lanes Gifts
  25. 25. Yahoo case: Checkmate Strategic Group$4.95million(Moneybag)Yahoo breached its contract with class members,… by charging and/orovercharging Class Members for clicks that were click fraud, clickthrough fraud, fraudulent clicks, click spam, invalid clicks, unwantedclicks, unqualified clicks, improper clicks, non-converting clicks,inadequately converting clicks, clicks that were not reasonably expectedby ClassMembers.Yahoo breached its contract with class members,… by charging and/orovercharging Class Members for clicks that were click fraud, clickthrough fraud, fraudulent clicks, click spam, invalid clicks, unwantedclicks, unqualified clicks, improper clicks, non-converting clicks,inadequately converting clicks, clicks that were not reasonably expectedby ClassMembers.
  26. 26. Yahoo case: Crafts by VeronicaOngoing!$$$$In spite of Defendants’ promise and duty not to place ads in perniciousspyware programs, Defendants have done just that…. By placing ClassMembers’ ads into illegal platforms such as spyware programs, Defendantswrongfully collect high search engine advertising fees for ads that are actuallyshown in contexts that are worth far less, if anything…. [allowing searchengines to] pocket the difference… Defendants also caused Class Members’ads to appear within “typosquatting” web sites… [which are] illegal under theAnti-Cybersquatting Consumer Protection Act. Charges: (i) Civil Conspiracy,(ii) unjust enrichment, (iii) breach of contract (iv) violation of NJ Consumerfraud act. Plaintiff demands a trial by jury on all issues so triable.In spite of Defendants’ promise and duty not to place ads in perniciousspyware programs, Defendants have done just that…. By placing ClassMembers’ ads into illegal platforms such as spyware programs, Defendantswrongfully collect high search engine advertising fees for ads that are actuallyshown in contexts that are worth far less, if anything…. [allowing searchengines to] pocket the difference… Defendants also caused Class Members’ads to appear within “typosquatting” web sites… [which are] illegal under theAnti-Cybersquatting Consumer Protection Act. Charges: (i) Civil Conspiracy,(ii) unjust enrichment, (iii) breach of contract (iv) violation of NJ Consumerfraud act. Plaintiff demands a trial by jury on all issues so triable.
  27. 27. Samuel Lassoff vrs Google– Samuel Lassoff sues Google, Class action on behalfof residents of NY and NJ.Ongoing!$$$$Charges: (i) breach of contract, (ii) negligence, (iii)unjust enrichment and (iv) unfair business practices.Charges: (i) breach of contract, (ii) negligence, (iii)unjust enrichment and (iv) unfair business practices.
  28. 28. Microsoft vs Eric Lam and family• 1.5 million in damages• Sued for 750KThe Web Giant confronts Microsoft, a slumping stock – and a surge of swindlers clicking on ads, Anthony Effinger and Jonathan Thaw, Bloomberg Markets, May
  29. 29. How do you detect click fraud?
  30. 30. Production performance – Quality separation and Revenue retainedProduction performance – Quality separation and Revenue retained01002003004005006007008009001000CPA billed CPA filtered0102030405060708090100CPA billedCPA billed1. Quality billed is extremely stable, even when the environment is subjected to massive shifts intraffic quality.2. Red line shows Quality of filtered, which is the Quality of the traffic that we pulled out. It fluctuateswildly. Blue shows Quality billed which hovers within 20% over an extended period of time.
  31. 31. TrafficScorerSmart-pricingadCenter Delivery EngineMB LogScoresAdsImpression /ad callCall for adsPublisherWeb SiteARTEMIS Real-time Scoring EngineMinerva FiltrationSystemadCenterReportsDump andLoadOLTPDatabaseBI DatabaseRedirectionServerMR LogsFR LogsMC LogsNew siteClick on AdAdvertiserquality andtargetingsettingsAd rotationmodulePublisherPipelineAdvertiserPipelinePubCenterReportsPublisherDatabaseKPI ExcelKPI CubeKPIPipelineBot TelemetryInstrumentationServerBlacklist CaptureSystemAutomatedCrawlerSystemBot TelemetryInstrumentationServerF.MSN.Com LogGlaux DataBridgeOLS StatsMinerva trainingpipelineCPA Glaux OLSFileFeature StatsCFR row byrowSnapshotCall forpayloadFWBCasesFraudOpsAdUnitBTISPayloadBotpayloadTelemetryThird partyand internaldata sourcesThird partyand internalcrawlersTop Bad FeedMicrosoft filtrationtechnologyarchitecture
  32. 32. Bot SignaturesBot Signatures0.8 1 1.2 1.4 1.6 1.8 2 2.2 2.4 2.600.050.10.150.20.250.30.350.4filtration rate (1==average for population)probabilityfraudnon-fraudBot signatures: Method foridentifying bots in a standardformatAllows investigators tocalculate true positive, falsepositive rates. For examplegraphs like the one at left
  33. 33. Making money by Measuring “fraud”
  34. 34. Google Attacks the Third Parties“third party firms significantly over-estimate the […] amount of “click fraud”.“In one case where 800 paid clicks weremarked as “fraudulent”, the rate ofconversion for these clicks was 5.1%,which compared favorably with 5.8%overall conversion rate”
  35. 35. Click Quality EnginesClick Quality Engines
  36. 36. Customer control• Valid / invalid is the wrong way to think about thisproblem– legally, operationally, statistically.– Even customers recognize that invalid clicks are always “shadesof grey”• Retroactive credits– Nobody likes them• CPA– Also fraud risk• 3rdparties– Google is currently fighting 3rdparties instead of working withthem– 3rdparty revenue models include rev recovery and law suits!
  37. 37. Customer control– Advertiser controls the filtration system– Even poor quality clicks may be profitable – junkbonds– No more Invalid/valid– Reduce retroactive credits to near zero– Uses the massive distributed system of advertisers toshut down revenue to fraudsters– Quality bucket is a targeting variable just like age,gender, time-of-day, publisher-site.– 3rdparties plug in their own fraud engines
  38. 38. Click Fraud: An Overview of theProblem• Kitts, B., Zhang, Jingying, Roux, A., Mills, R. (2013), Click Fraud Detectionwith Bot Signatures, Proceedings of the 2013 IEEE Conference onIntelligence and Security Informatics (ISI IEEE 2013), June, Seattle, WA.

×