Skip to main content
UploadLanguage (EN)Support
BusinessMobileSocial MediaMarketingTechnologyArt & PhotosCareerDesignEducationPresentations & Public SpeakingGovernment & NonprofitHealthcareInternetLawLeadership & ManagementAutomotiveEngineeringSoftwareRecruiting & HRRetailSalesServicesScienceSmall Business & EntrepreneurshipFoodEnvironmentEconomy & FinanceData & AnalyticsInvestor RelationsSportsSpiritualNews & PoliticsTravelSelf ImprovementReal EstateEntertainment & HumorHealth & MedicineDevices & HardwareLifestyle
BOAZ Bigdata, profile picture
Uploaded byBOAZ Bigdata
57 views

제 24회 보아즈(BOAZ) 빅데이터 컨퍼런스 - [HEART] : H ARM E STIMATION A ND R ANKING OF T RAFFIC FOR CACHE-HIT-RATIO RECOVERY IN E-COMMERCE LOGS

데이터 분석 프로젝트를 진행한 HEART 팀에서는 아래와 같은 프로젝트를 진행했습니다. Harm Estimation And Ranking of Traffic for Cache-Hit-Ratio Recovery in E-Commerce Logs 25기 김지수 숙명여자대학교 데이터사이언스학과 25기 서영우 고려대학교 통계학과

Data & Analytics◦
Download to read offline
HARM ESTIMATION AND RANKING OF TRAFFIC FOR CACHE-HIT-RATIO RECOVERY IN E-COMMERCE LOGS NOT ALL BOTS POLLUTE THE CACHE.... JISOO KIM YOUNGWOO SEO 25 JULY, 2026 PRESENTED BY DATE HEART:
TEAM. CACHEMERE JISOO KIM SOOKYMUNG WOMEN’S UNIVERSITY DEPARTMENT OF DATA SCIENCE YOUNGWOO SEO KOREA UNIVERSITY DEPARTMENT OF STATISTICS Email : sallysooo@sookmyung.ac.kr Github : sallysooo Email : maple5795@korea.ac.kr Github : maple5795
TEAM. CACHEMERE JISOO KIM SOOKYMUNG WOMEN’S UNIVERSITY DEPARTMENT OF DATA SCIENCE YOUNGWOO SEO KOREA UNIVERSITY DEPARTMENT OF STATISTICS Email : sallysooo@sookmyung.ac.kr Github : sallysooo Email : maple5795@korea.ac.kr Github : maple5795
Crawler A Crawler B p-8812 Both are bots. UA filtering treats them identically. 1. INTRODUCTION. 07:09:42 GET /p-9474.html 200 12481 07:09:49 GET /p-3312.html 200 9204 07:10:03 GET /p-9474.html 304 0 07:10:21 GET /p-3312.html 304 0 07:09:42 GET /wyszukiwanie-query15688.html 200 8113 07:09:43 GET /kartapdf-9293.html 200 6402 07:09:43 GET /wyszukiwanie-query13698.html 200 7856 07:09:44 GET /kartapdf-9107.html 200 6120 p-9474 p-3312 c-317 q15688 k-9293 q13698 k-9107 CACHE (4 slots) CACHE (4 slots) 인기페이지2개를재방문→ distinct URI 2개 cold URI 4개를1초안에훑고재방문0 다음사람: GET /p-8812.html → HIT! O 다음사람: GET /p-8812.html → MISS X “WARMS” the cache “POLLUTES” the cache
Human Human Cache Cache 응답 Origin 응답 1. INTRODUCTION. ≈ 50% HIT! of web traffic is automated. [1] Zhang et.al., “Rethinking web cache design for the AI era,” in Proc. ACM Symp. on Cloud Computing (SoCC), 2025 [1] MISS 사람의요청 GET /p-9474.html : 12,481 bytes O X 앱+DB가새로생성 빠름· origin이보낸byte 0 B 느림· origin이보낸byte 12,481 B origin = 상품페이지를실제로만들어내는본서버(앱+ DB) origin egress = origin이내보낸바이트= 클라우드요금(GB당과금)
195.181.168.164 - - [24/Jan/2019:10:47:31 +0330] "GET /static/bundle-bundle_jqGrid_head.min.css HTTP/1.1" 200 2260 "https://www.zanbil.ir/orderAdministration/console/186308" "Mozilla/5.0 (Windows NT10.0; Win64; x64; rv:64.0) Gecko/20100101 Firefox/64.0" "-" 89.196.178.224 - - [24/Jan/2019:10:47:31 +0330] "GET /image/61522/productModel/200x200 HTTP/1.1" 200 7013 "https://www.zanbil.ir/m/filter/b2%2Cp65" "Mozilla/5.0 (Android 5.1.1; Mobile; rv:59.0) Gecko/59.0 Firefox/59.0" "-" 89.196.178.224 - - [24/Jan/2019:10:47:31 +0330] "GET /image/62066/productModel/200x200 HTTP/1.1" 200 6210 "https://www.zanbil.ir/m/filter/b2%2Cp65" "Mozilla/5.0 (Android 5.1.1; Mobile; rv:59.0) Gecko/59.0 Firefox/59.0" "-" 89.196.178.224 - - [24/Jan/2019:10:47:31 +0330] "GET /image/62456/productModel/200x200 HTTP/1.1" 200 7252 "https://www.zanbil.ir/m/filter/b2%2Cp65" "Mozilla/5.0 (Android 5.1.1; Mobile; rv:59.0) Gecko/59.0 Firefox/59.0" "-" 89.196.178.224 - - [24/Jan/2019:10:47:31 +0330] "GET /image/61734/productModel/200x200 HTTP/1.1" 200 9601 "https://www.zanbil.ir/m/filter/b2%2Cp65" "Mozilla/5.0 (Android 5.1.1; Mobile; rv:59.0) Gecko/59.0 Firefox/59.0" "-" 89.196.178.224 - - [24/Jan/2019:10:47:31 +0330] "GET /image/61734/productModel/200x200 HTTP/1.1" 200 9601 "https://www.zanbil.ir/m/filter/b2%2Cp65" "Mozilla/5.0 (Android 5.1.1; Mobile; rv:59.0) Gecko/59.0 Firefox/59.0" "-" 5.202.218.194 - - [24/Jan/2019:10:47:32 +0330] "GET /image/8795/specialSale?role=e1 HTTP/1.1" 200 11 "https://www.zanbil.ir/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/71.0.3578.98 Safari/537.36" "-" 5.202.218.194 - - [24/Jan/2019:10:47:32 +0330] "GET /image/8795/specialSale?role=e1 HTTP/1.1" 200 11 "https://www.zanbil.ir/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/71.0.3578.98 Safari/537.36" "-"... 1. PROBLEM A : NO ANSWER LABEL. AUC=0.99xx... ACC=0.98xx...? AUC=0.99xx... ACC=0.98xx...? No ground truth in raw log data... Circular Dillemma! Circular Dillemma!
같은browser UA이지만하나는human, 하나는crawler v88 07:09:49 GET /p-9474.html 200 12481 "Mozilla/5.0 … Chrome/78.0.3904" v88 07:09:57 GET /c-317_322.html 200 4102 "Mozilla/5.0 … Chrome/78.0.3904" v2571 07:09:42 GET /p-8812.html 200 11903 "Mozilla/5.0 … Chrome/78.0.3904" v2571 07:09:43 GET /p-8813.html 200 11855 "Mozilla/5.0 … Chrome/78.0.3904" 1. PROBLEM B : ACTION OVERLAP. UA로는완벽한구분불가능.
“CONSEQUENCE-GROUNDED” “CONSEQUENCE-GROUNDED” “CONSEQUENCE-GROUNDED” “CONSEQUENCE-GROUNDED” 1. SOLUTION FOR A & B ? “이게Bot인가?” UA label 필요→ Circular ← identity “이게캐시에어떤영향을줬나?” ← Consequence!! Cache Simulator가측정→ label이필요없음
Previous Researches What it does Limitations → Our Insights ① AI 시대캐시재설계 Zhang, SoCC’25 · Cloudflare 자동화가locality를깬다→ 캐시구조·정책을재설계 (workload-aware tiers) 캐시는바꾸지만"어떤트래픽을뺄까"는안물음 ② 로그기반봇탐지 Tan&Kumar’02 · Doran&Gokhale’11 · Xu’18 · Iliou’19 UA·행동으로봇의정체(identity)를분류 라벨= UA 휴리스틱→ circular browser UA 위장봇을오분류하는문제점(Iliou) ③ 캐시오염공격+ 탐지 INCS (Hemmatpour et al.) 봇이in-network 캐시(P4/DPU)를오염시킴 → data plane에서tabular ML로탐지 실데이터가아닌합성clean 환경+ rarity로갈림 → 탐지기와라벨생성기구분불가(circular) ★HEART (ours) real 로그에서identity가아닌 결과(cache-harm)로“라벨없이” ranking! "실로그+ 결과기반+ 라벨-프리"를모두결합 2. RELATED WORKS.
2. DATASET. EClog main corpus replayed requests days – 6개월치실제트래픽로그 unique URIs – hash key sessions – harm을매기고랭킹하는대상 12.7M 183 577,775 635,082 Zanbil auxiliary corpus requests · 5 days 10.4M 우리가모델링하는캐시= 를동시에담을수있음 고유URI의1% = 5,778개객체 No Bot Labels in these logs! EClog가원리상못하는것만담당: ① Agnostic : 방법론이한데이터셋에만국한되지않음을증명 ② 실IP 보유: UA와무관한독립앵커(FCrDNS) 데이터센터IP에서 브라우저UA로위장한자동화를포착 EClog에서발견한75개browser UA scraper가여기서도등장함을Zanbil 내부의 실제IP를근거로발견(824개)
p-9474 q15688 q13698 k-9293 q17852 p-9474 3. CACHE HARM? – ALGORITHM 1 07:09:42 /p-9474.html human88 ✓ 07:09:42 /wyszukiwanie-query15688.html scraperA ✗ 07:09:43 /kartapdf-9293.html scraperA ✗ 07:09:50 /p-9474.html human88 ✓ ← cold insert ← /p-9474.html 이밀려남 ← MISS! X → harm[scraperA] += 1 harm[세션] = 그세션이쫓아낸물건을나중에사람이miss한횟수 CACHE (4 slots) 사람이miss 했다→ 그물건을마지막으로쫓아낸세션에게1점부여 ① ② ③
3. CACHE HARM? Fig. 4. Attributed harm vs. leave-one-session-out victim-hit gain. Spearman ρ=0.80; zero-harm controls have median gain 0; dashed line is identity. x축: Algoirithm1으로만든attributed harm – 한번replay 해서쫓아낸세션에게1점부여하는로직 y축: 진짜정답– 세션하나를빼고전체를다시재생해서측정한 counterfactual n = 539 : 세션에대해실제로측정해본sampling 샘플개수 ρ = 0.80 : Spearman 값이0.80이므로x와y의값이같은순서를 낸다는것을알수있음→ ranking 로직으로사용가능함을확인!
전부막을필요없이, 소수만찾으면된다. 48.8% of all cache harm 상위1% 세션에서나옴· Gini 0.91 3. CACHE HARM? What? : 세션을harm 큰순으로세우고누적harm을그린곡선 점선: harm이고르게퍼졌다면이대각선 파란선: 실제– 왼쪽끝에서이미수직으로치솟는모습 Fig. 6. Concentration of per-session attributed cache-harm on real traffic (LRU, 1% cache).
3. CACHE HARM? 전체attributed cache harm = 402,523건 11.4% of ALL cache harm – 단75개세션에서 전체세션의0.012% (75 / 635,082) · browser UA를쓰지만scraper처럼행동 UA가놓친상위층harm(24.0%)의절반가까이를이75개가차지 UA identity로만거르면가장치명적인세션을놓친다. bot 24.8% 그외 12,6% 75 sessions 11.4% 상위1% 밖의나머지99% 세션 51.2% UA가잡음 UA가못잡음 24.0% └──상위1% 세션(6,351개) = 48.8% ──┘
4. METHODOLOGY.
Input 4. METHODOLOGY. - CACHE HARM RANKER bot label은Train · Evaluation · Corpus selection 어느곳에도쓰이지않음! HistGBM Fig. 5. Permutation importance for held-out Spearman correlation. static_ratio dominates; req_count ranks only 5th — the harm ranker is not a volume heuristic in disguise. Target Model Prediction Train set Exclude 세션당feature 20개 log(1 + harm) – heavy-tail이라압축(70%가0) 5-fold OOF (data leakage 방지) human 포함전체635,082 세션→ UA tag로corpus를고르지않음 intent_ratio – UA tagging 규칙과겹치는유일한feature라서제외
4. METHODOLOGY. - EVALUATION PROTOCOL Bypass (ours) Bypass (ours) Block (차단) flagged된요청→ 403 및service denial Cost of false positive = 사람들이사이트를못씀...!! 즉, 시스템이human에대해BOT이라고오탐하면아예 쇼핑몰접근이막히는문제발생 flagged된요청→ origin에서정상serving ✓ 공유cache에insert/evict만안할뿐 Cost of false positive = SLOW response (느린응답) 따라서오탐이발생해도human은안전!! 1. Same Volume 모든방법을똑같은요청량(전체의20%)을 제거한지점에서비교 캐시에서요청을많이빼면경쟁자가줄어CHR이저절로오르기때문 2. Victim Definition 아래Rule을통과한“진짜human”만CHR count
5. EXPERIMENT RESULTS. – real traffic · LRU · 1% cache Fig. 7. Real-traffic operating curves: (a) victim-CHR recovery vs. bypassed volume; (b) recovery vs. human collateral. harm ranking (ours) tracks the attributed-harm reference and matches UA blocking at a fraction of the collateral. 47x +1.12 pp fewer sessions, same CHR UA block – status quo 166,622 sessions · collat 0% · egress −3.7% +1.04 pp ours – harm ranking 3,551 sessions · collat 0.18% · egress −5.2%
5. EXPERIMENT RESULTS. 0.5% Cache 캐시가극히작으면bot들이전부경쟁자이므로 UA blocking이유리할수밖에없음 +1.06 +1.06 +1.58 +1.58 ours UA-block 1% Cache (main) similar +1.04 +1.04 +1.12 +1.12 ours UA-block 5% Cache ★ Bot이대신해주던cache warming까지지워 버리므로UA blocking이CHR을떨어뜨림 +0.59 +0.59 -0.27 -0.27 ours UA-block 캐시가커질수록bot을지우는건손해– Not All Bots Pollute the Cache!
5. EXPERIMENT RESULTS. Q. 그냥volume/rarity로자르면되잖아? CHR Recovery는항상human collateral과함께읽어야한다. Volume Cold-URI Rarity -0.06 -0.03 +4.27 요청많은순으로제거→ CHR 회복없음 distinct URI 많은순→ 회복없음 겉보기엔최고! 그러나human session의28.4%를삭제함(무려85,364명) rarity가이긴게아니라, cache가원래못챙기던사람들을명단에서지워버린것. 지운사람의baseline hit율0.08 vs 남은사람0.62 – 생존편향
5. EXPERIMENT RESULTS. - 탐지정확도≠ 캐시보호 Fig. 8. CPI-bot ROC-AUC before and after training-set decontamination (mean±std, five seeds). Every architecture improves. Decontam 후탐지AUC : 0.77 → 0.92 (all architectures) Ours : harm ranking은+1.02pp를3,551 session · collat 0.18%로달성! CHR: 사람CHR 회복은+0.98 → +1.00 (사실상불변) → bot 자체는더잘찾아도캐시회복율은똑같음(불변) 심지어detector는153k~154k 만큼의세션을지우며사람도3%+ 삭제함
Questions for Limitations... Answers Q1. Target & Evaluation 모두같은Simulator? 4 policies x 3 용량x byte-weighted에서전부결론재확인완료 Q2. 실시간이아님– offline · periodic 과거5개월만학습→ 다음달+1.00pp (UA 차단+0.84보다높음) 명단리스트가아닌“model”을재배포 Q3. victim 정의도결국휴리스틱? 3종정의(plausible · strict · loose)로민감도분석완료 Q4. 실제cache의추가특징(TTL · 객체크기· 멀티티어) 모델링? byte-weighted replay에서도+1.02pp로회복확인 6. LIMITATIONS? 4가지교체정책(LRU / LFU / SIEVE / admission) 3가지캐시용량(0.5% / 1% / 5%) → 설정을바꿔도harm 랭킹이계속통하므로특정시뮬레이터의우연이아님을증명 → harm ranking (ours)이크기인식환경에서도살아남음. 단byte reference는 +1.42로약간의손실은있으므로실운영자는self 비용모델로harm을다시매기는게좋음 TTL, multi-tier → FUTURE WORK!
7. FINAL TAKEAWAYS. E-commerce Log에서cache harm은소수에극단적으로집중되며, UA labeling은해당부분을상당수놓친다. Gini 0.91 · 상위1%가harm의48.8% · browser-UA 75세션이전체의11% 1 1 1 1 UA identity가아닌결과(cache-harm)로랭킹하면, 봇라벨없이UA 차단과같은CHR을47× 적은개입으로회복 그리고캐시가커지면UA 차단은오히려손해— Not All Bots Pollute the Cache!! 2 2 2 2 탐지정확도≠ 캐시보호 탐지기정화로ROC-AUC +0.15인데CHR 회복은0 — 두문제는별개, 완화는신원이아니라측정된결과에근거해야함 3 3 3 3 답없는질문(봇이냐)에서, 답있는질문(캐시를망쳤나)으로의전환.
THANK YOU! THANK YOU! THANK YOU! THANK YOU! 분석25기김지수 분석25기서영우 TEAM CACHEMERE.