SlideShare a Scribd company logo
1 of 22
Download to read offline
!!
T4
Mobile!Testing!
10/16/2014!9:45:00!AM!
!
Top Ten Attacks to Break
Mobile Apps
!
Presented by:
Jon Hagar
Grand Software Testing
!
!
!
Brought(to(you(by:(
(
(
(
340!Corporate!Way,!Suite!300,!Orange!Park,!FL!32073!
888G268G8770!H!904G278G0524!H!sqeinfo@sqe.com!H!www.sqe.com
Jon Hagar
Grand Software Testing
Jon Hagar is an independent consultant working in software product integrity,
testing, verification, and validation. Jon publishes regularly on testing, including
the book Software Test Attacks to Break Mobile and Embedded Devices
(breakingembeddedsoftware.com). For more than thirty years, he has worked in
software engineering, particularly testing, supporting projects which include
control systems (avionics and auto), spacecraft, mobile-smart devices, and
attack testing of smart phones. Jon is an editor for ISO, IEEE, and OMG
standards.
	
  
	
  
	
  
	
  
	
  
	
  
	
  
	
  
	
  
	
  
	
  
	
  
	
  
	
  
	
  
	
  
	
  
	
  
	
  
	
  
Jon$Hagar$Copy$right$2014$ So4ware$Test$A:acks$to$Break$Mobile$and$Embedded$Devices$ 1$
TOP$10$SOFTWARE$TEST$ATTACKS$
TO$BREAK$MOBILE$SOFTWARE$
STARWEST$2014$
Jon$Hagar$
embedded@ecentral.com$
jon.d.hagar@gmail.com$
Grand$So4ware$TesJng$
Web:$
h:p://breakingembeddedso4ware.wordpress.com/$
Jon$Hagar$Copy$right$2014$ So4ware$Test$A:acks$to$Break$Mobile$and$Embedded$Devices$ 2$2$
AGENDA$
•  DefiniJons$for$this$session$
•  RiskQbased$tesJng$concepts$for$mobile$
•  Exploratory$tesJng$concepts$for$$mobile$
•  My$top$10$Mobile$So4ware$(app)$a:acks$
•  Wrap$up$
Jon$Hagar$Copy$right$2014$ So4ware$Test$A:acks$to$Break$Mobile$and$Embedded$Devices$ 3$3$
$
MOBILE,$SMART,$AND$HANDHELD$
•  As$the$names$implies,$these$are$devices—small,$held$in$the$hand,$o4en$
connected$to$communicaJon$networks,$including$
•  Cell$and$smart$phones$–$apps$$
•  Tablets$
•  Medical$devices$
•  Typically$have:$
•  Many$of$the$problems$of$classic$“embedded”$systems$
•  The$power$of$PCs/IT$
•  More$user$interface$(UI)$than$classic$embedded$systems$
•  (RelaJvely)$Fast$updates$
•  Mobile$devices$are$“evolving”$with$more$power,$resources,$apps,$etc.$$
•  Mobile$is$the$“hot”$area$of$computers$and$so4ware$currently$
•  TesJng$rules$and$concepts$are$“evolving”$
STARWEST$
Jon$Hagar$Copy$right$2014$ So4ware$Test$A:acks$to$Break$Mobile$and$Embedded$Devices$ 4$4$
TYPES$OF$MOBILE$APPS$
•  NaJve$ApplicaJons$
•  Local$to$device$
•  Hybrid$ApplicaJons$
•  Local$to$device$but$
interacts$w/internet$
•  Web$ApplicaJons$
•  Not$local$to$device.$
All$interacJons$on$
internet$
STARWEST$
Jon$Hagar$Copy$right$2014$ So4ware$Test$A:acks$to$Break$Mobile$and$Embedded$Devices$ 5$5$
MOBILE$TESTING$DEFINITIONS$
•  Mobile$ApplicaJon$TesJng$is$tesJng$the$applicaJon$in$a$support$
environment$or$on$a$mobile$device$
•  System$Level$Mobile$Device$TesJng$is$tesJng$the$hardware$and$operaJng$
system$
•  Does$the$OperaJng$System$install?$$$
•  Does$the$device$power$on?$Do$the$LED$lights$work$as$expected?$$$
•  Does$the$ba:ery$charge$when$the$AC$adapter$is$plugged$into$the$device?$
•  Mobile$Phone$TesJng$should$have$some$different$approaches$to$tesJng$
•  Mobile$System$TesJng$incorporates$tesJng$more$than$one$applicaJon$and$
can$combine$hardware,$so4ware,$firmware,$along$with$other$applicaJons$
•  Mobile$TesJng$–$can/should$be$all$of$the$above$
$
Be$clear$when$using$this$terminology.$If$you$are$only$tesJng$apps$on$mobile$phones,$
then$state$“mobile$apps$tesJng.”$Use$mobile$tesJng$when$you$are$tesJng$mobile$
websites,$mobile$hybrid$apps,$mobile$hardware,$etc.$$
STARWEST$$
Jon$Hagar$Copy$right$2014$ So4ware$Test$A:acks$to$Break$Mobile$and$Embedded$Devices$ 6$6$
DEFINING$SKILL$SET$FOR$
THE$MOBILE$TESTER$
•  Some$exposure$or$knowledge$about$products$from$the$domain$in$which$you$
are$tesJng:$$aerospace,$medical,$automobile$manufacturing,$airplanes,$
factory$systems,$roboJcs,$regulated$environments,$etc.$
•  Some$knowledge$of:$hard$sciences:$$math,$physics,$electronics,$engineering,$
etc.$for$logical$thought$processes$
•  So4ware$sciences:$$psychology,$philosophy,$sociology,$human$factors$(human$
machine$interface)$for$creaJve$&$conceptual$thought$processes$
•  Tester$skill$
•  Planning,$design$techniques,$pa:erns$of$errors,$intuiJon,$criJcal$thinking,$“so4$skills,”$$
communicaJon,$observaJon,$and$mental$models$[ISTQB$and$AST$have$“lists”]$
Chapter$1$–$So4ware$Test$A:acks$to$Break$Mobile$&$Embedded$Devices$
STARWEST$
Jon$Hagar$Copy$right$2014$ So4ware$Test$A:acks$to$Break$Mobile$and$Embedded$Devices$ 7$7$
WHAT$IS$AN$ATTACK?$
•  A$pa:ern$(of$tesJng)$based$on$a$common$mode$of$failure$seen$
over$and$over$
•  Maybe$seen$as$a$negaJve,$when$it$is$really$a$posi%ve(
•  Goes$a4er$the$“bugs”$that$may$be$in$the$so4ware$
•  May$include$or$use$classic$test$techniques$and$test$concepts$
•  Lee$Copeland’s$book$on$test$design$
•  Many$other$good$books$
•  A$Pa:ern$(more$than$a$process),$which$must$be$modified$for$the$
context$at$hand,$to$do$the$tesJng$$
•  Testers$learn$these$in$a$domain$a4er$years$and$form$a$mental$
model$(most$good$testers$a:ack)$
STARWEST$
Jon$Hagar$Copy$right$2014$ So4ware$Test$A:acks$to$Break$Mobile$and$Embedded$Devices$ 8$8$
WHY$ATTACK?$
•  A:acking$your$so4ware$is$in$part,$$the$process$of$a:empJng$to$
demonstrate$a$system$(hardware,$firmware,$so4ware$and$operaJons)$$
does$not$meet$requirements,$funcJonal$and$nonQfuncJonal$objecJves$
•  Embedded/handheld$so4ware$tesJng$must$include$"the$
system"$(hardware,$so4ware,$operaJons,$users)$
•  A:acking$common$modes$of$failure,$especially$where$the$applicaJon$
is$engaged$and$visible$by$the$user.$
Attack your enemy with approaches to include:
Tools Levels
Attacks Techniques
STARWEST$
Jon$Hagar$Copy$right$2014$ So4ware$Test$A:acks$to$Break$Mobile$and$Embedded$Devices$ 9$9$
KINDS$OF$ATTACKS$
•  Whi:aker$offers$a$good$starJng$point$for$so4ware$
a:acks$in$general$that$can$be$applied$to$mobile:$
•  User$Interface$A:acks$
•  Data$and$ComputaJon$
•  File$System$Interface$
•  So4ware/OS$Interface$
•  Whi:aker’s$“How$to$Break$So4ware”$lists$23$a:acks$
•  Plus$he$has$other$books$on$a:acks,$security,$web,$exploratory,$and$tours$in$tesJng$
•  “So4ware$Test$A:acks$to$Break$Mobile$and$
Embedded$Devices”$lists$32$a:acks$and$8$sub$a:acks$
STARWEST$
Jon$Hagar$Copy$right$2014$ So4ware$Test$A:acks$to$Break$Mobile$and$Embedded$Devices$ 10$10$
MOBILE$RISK$AREAS$TO$CONSIDER$
•  There$are$many$risk$to$$consider,$but$you$cannot$test$everything$
•  Risk(s)$based$tesJng$$helps$$bound$the$test$scope$problem$
•  TesJng$is$about$providing$informaJon$and$understanding$
•  ExploraJon$gets$you$started$with$whatever$you$have$(or$don’t$
have)$
STARWEST$
Jon$Hagar$Copy$right$2014$ So4ware$Test$A:acks$to$Break$Mobile$and$Embedded$Devices$ 11$11$
SAMPLE$MOBILE$PRODUCT$RISKS$
TESTERS$SHOULD$CONSIDER$
•  Environment$and$input$factors$
•  Environment$–$heat,$noise,$sun,$water,$etc.$
•  Hardware$–$calibraJon,$uniqueness,$manufacturing,$etc.$
•  Electronics$–$noise,$power,$ba:eries,$etc.$
•  CommunicaJons$
•  Interfaces$types$
•  Hardware$
•  Human$$
•  Network$$
•  So4ware$
•  Output$—$noise$influences,$D2A,$representaJon,$etc.$
•  Complexity—use$/$size$of$the$system$$
STARWEST$
Jon$Hagar$Copy$right$2014$ So4ware$Test$A:acks$to$Break$Mobile$and$Embedded$Devices$ 12$12$
RISK=BASED$
TESTING$$
(ISO$29119)$
•  Address,$miJgate,$a:ack$and$reJre$product$risks$
•  PrioriJze$risks$Q$tests:$
•  PotenJal$problems$Q$$Consequences$and$effects$
•  Occurrences$–$likelihood$or$chance$of$happening$
•  Impacts$–$what$happens$
•  Take$consistent$acJon$from$the$beginning$(proposal)$
to$the$end$(reJrement)$of$the$product$or$lifecycle$
•  Risks$&$prioriJzing$should$dictate$the$test$a:acks$
STARWEST$
Jon$Hagar$Copy$right$2014$ So4ware$Test$A:acks$to$Break$Mobile$and$Embedded$Devices$ 13$13$
EXPLORATORY$TESTING$=$DEFINITION$
•  QuoJng$James$Bach:$$“The$plainest$definiJon$of$exploratory$
tesJng$is$test$design$and$test$execuJon$at$the$same$Jme.$This$is$
the$opposite$of$scripted$tesJng$(predefined$test$procedures,$
whether$manual$or$automated).$Exploratory$tests,$unlike$
scripted$tests,$are$not$defined$in$advance$and$carried$out$
precisely$according$to$plan.”$
h:p://www.saJsfice.com/arJcles/what_is_et.shtml$
STARWEST$
Jon$Hagar$Copy$right$2014$ So4ware$Test$A:acks$to$Break$Mobile$and$Embedded$Devices$ 14$14$
EXPLORATORY$TESTING$IN$MOBILE$
•  Rapid$feedback$
•  Learning$
•  Upfront$rapid$
learning$
•  A:acking$
•  Address$Risk$(s)$
•  Independent$assessment$
•  Target$a$defect$
•  Prototyping$
•  Need$info$
•  Test$beyond$the$
requirements$
STARWEST$
Jon$Hagar$Copy$right$2014$ So4ware$Test$A:acks$to$Break$Mobile$and$Embedded$Devices$ 15$15$
NUMBER$10:$FUNCTIONAL$ATTACK$(33)$
•  Have$an$outline$$or$charter$(top$level$plan$and/or$risk$list)$
•  Create$a$flip$chart,$notecard,$state$model,$mind$map$or$some$representaJon$of$
each$test$$task$$
•  No$“heavyweight$documentaJon$of$the$“test$case”$
•  See$Exploratory$Charter$(test$objecJve)$
•  Have$a$Target$concept$$or$charter$$(Risk,$A:ack,$Bug,$Learning,$…)$
•  VerificaJon$checking$of$requirements$(necessary$but$not$sufficient)$$
•  Have$a$schedule/Jme$box$(short$$test$cycles$=$Planning$to$report)$
•  Do$the$test$
•  Design$test$
•  Execute$test$$
•  Learn$about$the$product:$change$the$risk$list,$modify/add$tests,$and$so$on$
•  Repeat$as$needed$
STARWEST$
Jon$Hagar$Copy$right$2014$ So4ware$Test$A:acks$to$Break$Mobile$and$Embedded$Devices$ 16$16$
EXAMPLE$MIND$MAP$FROM$A$TRAVEL$APP$
Jon$Hagar$Copy$right$2014$ So4ware$Test$A:acks$to$Break$Mobile$and$Embedded$Devices$ 17$17$
•  Download$either$Twi:er$or$Facebook$onto$a$device$$
•  Start$either$downloaded$app$
•  From$another$device,$send$an$email$to$the$device’s$email$account$
•  Immediately$send$a$tweet$or$post$a$status$$
•  ConJnue$to$do$engage$Twi:er$or$Facebook$app$for$at$least$1$minute$
•  Record$email$noJficaJon$and$Jme$when$sent$and$received$
•  What$other$observaJons$occurred?$
9:$NOTIFICATION$TEST$ATTACK$(18)$
Jon$Hagar$Copy$right$2014$ So4ware$Test$A:acks$to$Break$Mobile$and$Embedded$Devices$ 18$18$
ATTACK$TEST$CASE$EXAMPLE:$$
INTERRUPTS$ON$MOBILE$PHONES$
•  Go$to$your$App$store$and$choose$an$applicaJon$to$download$
•  While$the$downloading$is$occurring,$call$the$mobile$phone$
•  Record$observaJons$with$the$download$
•  You$may$need$to$rely$on$observing$a$log$file$while$implemenJng$these$
tests$
•  If$it$fails,$what$kind$of$error$recovery$occurs?$Can$you$repeat$any$
errors?$
Jon$Hagar$Copy$right$2014$ So4ware$Test$A:acks$to$Break$Mobile$and$Embedded$Devices$ 19$19$
8:$ATTACK$SCENARIOS$(12)$
•  Tests$consider$usage,$operaJons,$interface$interacJons$and$
integraJons$$
•  Interface$points$include:$$hardware,$firmware,$so4ware,$data$
exchange,$network$communicaJon$and$combinaJons$
•  How$each$interface$point$integrates$with$another$interface$point$
•  Tests$include$how$the$applicaJon$is$used$endQtoQend$$
•  Tests$to$combine$how$the$enJre$system$interacts$as$well$as$how$
porJons$interact$with$one$another$and$depending$on$complexity$
•  Note:$ConfiguraJon$tests$with$regards$to$how$so4ware$behaves$based$
on$various$configuraJons$of$devices,$operaJng$systems$
Jon$Hagar$Copy$right$2014$ So4ware$Test$A:acks$to$Break$Mobile$and$Embedded$Devices$ 20$
IN$SCENARIO$TESTING$:$TIMING$SUBQ$ATTACK$
When$Time$interacts$with$the$so4ware,$events,$inputs,$
and$outputs,$here's$a$checklist$of$things$to$look$for$and$
consider$(where$bugs$lurk)$in$sequences/stories$
$
•  Order$problems$
•  Too$Long$$
•  Too$Fast$
•  Not$at$right$Jme$mark$or$point$$
•  Late$$
•  Late$or$early$
•  Early$$
•  Deadlocked$caused$by$a$race$
condiJon$(hard$to$find)$$
•  Extra$input$or$output$events$$
•  Missing$events$$
•  Wrong$input/output$within$events$$
Jon$Hagar$Copy$right$2014$ So4ware$Test$A:acks$to$Break$Mobile$and$Embedded$Devices$ 21$21$
•  When$to$apply$this$a:ack?$$…when$your$app/device$has$games$
•  What$faults$make$this$a:ack$successful?$$$…games$are$complex$
•  Who$conducts$this$a:ack?$$…see$chart$on$Roles$
•  Where$is$this$a:ack$conducted?$$…throughout$lifecycle$and$in$environments$
•  How$to$determine$if$the$a:ack$exposes$failures?$
•  Unhappy$“users”$
•  Bugs$found$
•  See$checklist$
7:$ATTACK$TESTING$MOBILE$GAMES$(26)$
Mobile Device Game Testing
(2 years ago gaming was 60% or more of Mobile App downloads)
Jon$Hagar$Copy$right$2014$ So4ware$Test$A:acks$to$Break$Mobile$and$Embedded$Devices$ 22$22$
•  The$developer(s)—see$A:acks$1,$2,$and$3.$
•  The$app$game$architect$or$director$
•  OnQteam$game$tester(s)$$
•  InQcompany$“dog$food”$testers$
•  Independent$test$players$$
•  Mass$beta$trials$
•  Not$a$tester—Finally,$consider$who$should$not$be$playing$
Note%on%roles:%During(the(tes%ng(effort(and(as(it(progresses,(don’t(
forget(that(there(are(many(different(user(roles$
ROLES$TO$PLAY$IN$THE$GAME$
(ANY$MANY$OTHER$APPS)$
Jon$Hagar$Copy$right$2014$ So4ware$Test$A:acks$to$Break$Mobile$and$Embedded$Devices$ 23$23$
•  Refine$checklist$to$context$scope$
•  Define$a$role$$
•  Watch$what$is$happening$with$this$role$
•  Define$a$usage$(scenario$or$set$of$funcJons$to$Play$the$game)$
•  Guided$exploraJons$or$ad$hoc$
•  Stress,$unusual$$cases,$explore$opJons$
•  Capture$understanding,$risk,$observaJons,$etc.$
•  Checklist$(watch$for$confusion)$
•  Run$Exploratory$A:ack$
•  Learn$
•  ReQplanQdesign$
•  Watch$for$Bias$
•  Switch$testers$
•  Repeat$
$
$
$
GAME$ATTACK$PATTERN$
Jon$Hagar$Copy$right$2014$ So4ware$Test$A:acks$to$Break$Mobile$and$Embedded$Devices$ 24$24$
6:$BREAKING$SOFTWARE$WITH$HARDWARE$
AND$SYSTEM$OPERATION$(9)$
•  Classic$subQa:ack$example$to$consider$is$checking$ba:ery$
power$impacts$
•  Not$Common$to$IT/PC$tesJng$
•  Large$impacts$to$users$(if$ba:ery$is$drained)$
•  Relates$to$hardware$and$basic$operaJon$acJviJes$$
•  Requires$systems$thinking$
•  May$require$use$of$specialized$test$environments$and$
support$test$tools$
Jon$Hagar$Copy$right$2014$ So4ware$Test$A:acks$to$Break$Mobile$and$Embedded$Devices$ 25$25$
DOCUMENTING$YOUR$TEST$CONDITIONS$
FOR$THE$CHARGING$OF$BATTERY$TEST$
Credit$to:$Jean$Ann$Harrison$$2013$
Jon$Hagar$Copy$right$2014$ So4ware$Test$A:acks$to$Break$Mobile$and$Embedded$Devices$ 26$26$
5:$ATTACKING$WITH$SIMULATION$(AND$EMULATION)$(17)$
•  TesJng$with$real$hardware$is$advised,$but$
•  Has$limitaJons$
•  Can$require$a$lot$of$equipment$
•  You$need$the$hardware$$
•  FragmentaJon$
•  Many$mobile$people$test$using$simulators$and/
or$emulators$
Jon$Hagar$Copy$right$2014$ So4ware$Test$A:acks$to$Break$Mobile$and$Embedded$Devices$ 27$27$
PROS$AND$CON$(SAMPLING)$OF$SIMULATION$AND$
EMULATION$
•  Pro$
•  Can$start$early$
•  Can$support$virtual$tesJng$
•  Can$support$automaJon$
•  Con$
•  Will$miss$some$kinds$of$bugs$
•  May$not$transfer$to$the$actual$hardware$
•  May$require$special$skills$and$efforts$to$set$up$
•  Modeling$(if$used)$can$be$very$tricky$
Jon$Hagar$Copy$right$2014$ So4ware$Test$A:acks$to$Break$Mobile$and$Embedded$Devices$ 28$28$
4:$DEVELOPER$LEVEL$ATTACK:$WHITE$BOX$(1$&$2)$
•  Between$20QandQ30$percent$of$errors$can$be$found$by$developerQled$structural$
tesJng$
•  When$combined$with$#3,$a$testers$job$becomes$much$more$interesJng$
•  Industry$has$known$this$tesJng$from$the$beginning$and$yet$it$is$underused$
•  Priority$is$high$
•  “Official”$Tester$should$know$it,$advocate$for$it$and$even$“do$it”$someJmes$
•  Two$basic$A:acks$
•  Data$
•  Logic$
Jon$Hagar$Copy$right$2014$ So4ware$Test$A:acks$to$Break$Mobile$and$Embedded$Devices$ 29$29$
3:$STATIC$CODE$ANALYSIS$(SCA)$ATTACK$(3)$
•  This$acJvity$does$NOT$execute$the$code$
•  Can$be$done$“early”$in$the$lifecycle$
•  A$be:er$term$is$just$“analysis,”$but…………$
•  For$the$code,$we$use$a$tool$to$“analyze”$for$certain$types$of$errors$
•  Tools$are$commercial$although$some$open$source$tools$exist$
•  SCA$finds$the$“hard$to$find”$errors$
•  Many$test$teams$take$this$effort$over$since$programmers$“don’t$have$the$Jme”$
•  Issues:$
•  False$posiJves$
•  When$to$do$
•  When$to$repeat$
•  Efforts$can$(should)$$include$analyzing$models,$requirements,$and$other$arJfacts$
Jon$Hagar$Copy$right$2014$ So4ware$Test$A:acks$to$Break$Mobile$and$Embedded$Devices$ 30$30$
1$&$2:$ATTACK$SECURITY$$
•  Apply$when$the$device$is$mobile$and$has$
•  Account$numbers$
•  UserQids$and$passwords$
•  LocaJon$tags$
•  Restricted$data$$
•  $Current$$authenJcaJon$approaches$in$use$on$mobile$devices$
•  ServerQbased$
•  Registry$(user/password)$
•  LocaJon$or$deviceQbased$
•  ProfileQbased$
PRIVAC
Y
Jon$Hagar$Copy$right$2014$ So4ware$Test$A:acks$to$Break$Mobile$and$Embedded$Devices$ 31$31$
THE$CURRENT$SECURITY$SITUATION$
•  Mobile$systems$are$highly$integrated$hardware–so4ware–system$
soluJons$which:$
•  Must$be$highly$trustworthy$since$they$handle$sensiJve$data$$
•  O4en$perform$criJcal$tasks$
•  Security$holes$and$problems$abound$
•  Coverity$Scan$2010$Open$Source$Integrity$Report$Q$Android$
•  staJc$analysis$test$a:ack$found$0.47$defects$per$1,000$SLOC$$
•  359$defects$in$total,$88$of$which$were$considered$“high$risk”$in$
the$security$domain$
•  OS#hole#Andriod#with#Angry#Birds#$(researchers$Jon$Oberheide$and$Zach$
Lanier)$
•  Robots$and$Drones$rumored$to$be$a:acked$
•  Cars$$and$medical$devices$being$hacked$
Jon$Hagar$Copy$right$2014$ So4ware$Test$A:acks$to$Break$Mobile$and$Embedded$Devices$ 32$32$
EMBEDDED/MOBILE$SECURITY$CONCERNS$
•  Fraud$–$IdenJty$
•  Worms,$virus,$etc.$
•  Fault$injecJon$
•  Processing$on$the$run$
•  Hacks$impact$
•  Power$
•  Memory$
•  CPU$usage$
•  Eavesdropping$–$yes$everyone$can$hear$you$
•  Hijacking$
•  ClickQjacking$
•  Voice/Screen$
•  Physical$Hacks$
•  File$snooping$
•  Lost$phone$
Jon$Hagar$Copy$right$2014$ So4ware$Test$A:acks$to$Break$Mobile$and$Embedded$Devices$ 33$33$
SECURITY$ATTACKS$$$
(ONLY$A$STARTING$POINT)$
•  A:ack$28$PenetraJon$A:ack$Test $$
•  A:ack$28.1$PenetraJon$Sub–A:acks:$AuthenJcaJon$—$Password$A:ack $$
•  A:ack$28.2$Sub–A:ack$Fuzz$Test$$
•  A:ack$29:$InformaJon$The4—Stealing$Device$Data $$
•  A:ack$29.1$Sub$A:ack$–IdenJty$Social$Engineering $$
•  A:ack$30:$Spoofing$A:acks$$
•  A:ack$30.1$LocaJon$and/or$User$Profile$Spoof$Sub–A:ack$
•  A:ack$30.2$GPS$Spoof$Sub–A:ack $$
•  A:ack$31:$A:acking$Viruses$on$the$Run$in$Factories$or$PLCs$
Jon$Hagar$Copy$right$2014$ So4ware$Test$A:acks$to$Break$Mobile$and$Embedded$Devices$ 34$34$
WARNINGS$WHEN$CONDUCTING$
SECURITY$ATTACKS$
!  Security$a:acks$must$be$done$with$the$knowledge$and$approval$of$owners$of$
the$system$and$so4ware$
!  Severe$legal$implicaJons$exist$in$this$area$
!  Many$of$these$a:acks$must$be$done$in$a$lab$(sandbox)$
!  In$these$a:acks,$I$tell$you$conceptually$how$to$“drive$a$car$very$fast$(150$miles$
an$hour)$but$there$are$places$to$do$this$with$a$car$legally$(a$race$track)$and$
places$where$you$will$get$a$Jcket$(most$public$streets)”$
!  Be$forewarned$Q$Do$not$a:ack$you$favorite$app$on$your$phone$or$connected$
server$without$the$right$permissions$due$to$the$legal$implicaJons$
Jon$Hagar$Copy$right$2014$ So4ware$Test$A:acks$to$Break$Mobile$and$Embedded$Devices$ 35$35$
WRAP$UP$
•  I$gave$my$top$10,$but$your$a:acks$can$and$will$be$different$
•  Understanding$your$local$context$and$error$pa:erns$is$important$$
(one$size$does$NOT$fit$all)$
•  A:acks$are$pa:erns…you$sJll$must$THINK$and$tailor$
$
Jon$Hagar$Copy$right$2014$ So4ware$Test$A:acks$to$Break$Mobile$and$Embedded$Devices$ 36$36$
MOBILE$ATTACK$CLASSIFICATION$
•  Developer$A:acks$(unit/code$tesJng)$$
•  Control$System$A:acks$$
•  HardwareQSo4ware$A:acks $$
•  Mobile$and$Embedded$So4ware$Domain$A:acks$$
•  Time$A:acks$(Performance) $$
•  Human$User$Interface$A:acks$$$
•  Smart$and/or$Mobile$Phone$FuncJonal$App$A:acks $$
•  Mobile/Embedded$Security$A:acks $$
•  Generic$A:acks$$
•  FuncJonal,$mind$mapping,$and$combinatorial$tests$
Jon$Hagar$Copy$right$2014$ So4ware$Test$A:acks$to$Break$Mobile$and$Embedded$Devices$ 37$
MORE$ATTACKS$$
(FROM$SOFTWARE$TEST$ATTACKS$TO$BREAK$MOBILE$AND$EMBEDDED$
DEVICES)$
•  A:ack$1:$StaJc$Code$Analysis $$
•  A:ack$2:$Finding$White–Box$Data$ComputaJon$Bugs$$
•  A:ack$3:$White–Box$Structural$Logic$Flow$Coverage$
•  A:ack$4:$Finding$Hardware–System$Unhandled$Uses$in$So4ware$
•  A:ack$5:$$HwQSw$and$SwQHw$signal$Interface$Bugs$
•  A:ack$6:$Long$DuraJon$Control$A:ack$Runs $$
•  A:ack$7:$$Breaking$So4ware$Logic$and/or$Control$Laws$
•  A:ack$8:$Forcing$the$Unusual$Bug$Cases $$
•  A:ack$9$Breaking$So4ware$with$Hardware$and$System$OperaJons$
•  9.1$Sub–A:ack:$Breaking$Ba:ery$Power $$
•  A:ack$10:$Finding$Bugs$in$Hardware–So4ware$CommunicaJons $$
•  A:ack$11:$Breaking$So4ware$Error$Recovery $$
•  A:ack$12:$Interface$and$IntegraJon$TesJng $$
•  12.1$Sub–A:ack:$ConfiguraJon$IntegraJon$EvaluaJon $$
•  A:ack$13:$Finding$Problems$in$So4ware–System$Fault$Tolerance$
•  A:ack$14:$Breaking$Digital$So4ware$CommunicaJons $$
•  A:ack$15:$Finding$Bugs$in$the$Data $$
•  A:ack$16:$Bugs$in$System–So4ware$ComputaJon $$
•  A:ack$17:$$Using$SimulaJon$and$SJmulaJon$to$Drive$So4ware$A:acks$
•  A:ack$18:$Bugs$in$Timing$Interrupts$and$Priority$Inversion$
•  A:ack$19:$Finding$Time$Related$Bugs $$
•  A:ack$20:$Time$Related$Scenarios,$Stories$and$Tours $$
•  A:ack$21:$Performance$TesJng$IntroducJon $$
•  A:ack$22:$Finding$SupporJng$(User)$DocumentaJon$Problems$
•  Sub–A:ack$22.1:$$Confirming$Install–ability $$
•  A:ack$23:$Finding$Missing$or$Wrong$Alarms $$
•  A:ack$24:$Finding$Bugs$in$Help$Files $$
•  A:ack$25:$Finding$Bugs$in$Apps $$
•  A:ack$26:$TesJng$Mobile$and$Embedded$Games $$
•  A:ack$27:$A:acking$App–Cloud$Dependencies $$
•  A:ack$28$PenetraJon$A:ack$Test $$
•  A:ack$28.1$PenetraJon$Sub–A:acks:$AuthenJcaJon$—$Password$A:ack $$
•  A:ack$28.2$Sub–A:ack$Fuzz$Test$$
•  A:ack$29:$InformaJon$The4—Stealing$Device$Data $$
•  A:ack$29.1$Sub$A:ack$–IdenJty$Social$Engineering $$
•  A:ack$30:$Spoofing$A:acks $$
•  A:ack$30.1$LocaJon$and/or$User$Profile$Spoof$Sub–A:ack$
•  A:ack$30.2$GPS$Spoof$Sub–A:ack $$
•  A:ack$31:$A:acking$Viruses$on$the$Run$in$Factories$or$PLCs$
•  A:ack$32:$Using$Combinatorial$Tests $$
•  A:ack$33:$A:acking$FuncJonal$Bugs $$
Jon$Hagar$Copy$right$2014$ So4ware$Test$A:acks$to$Break$Mobile$and$Embedded$Devices$ 38$38$
SUMMARY:$THANK$YOU$(IDEAS$USED$FROM)$
•  James$Whi:aker$(a:acks)$
•  Elisabeth$Hendrickson$(simulaJons)$
•  Lee$Copeland$(techniques)$
•  Brian$Merrick$(tesJng)$
•  James$Bach$(exploratory$&$tours)$
•  Cem$Kaner$$(test$thinking)$
•  Many$teachers$
•  GeneraJons$past$and$future$
•  Books,$references,$etc.$
Jon$Hagar$Copy$right$2014$ So4ware$Test$A:acks$to$Break$Mobile$and$Embedded$Devices$ 39$39$
BOOK$LIST$(MY$FAVORITES)$
•  “So;ware(Test(A?acks(to(Break(Mobile(and(Embedded(Devices”((
–(Jon(Hagar,(to(be(published(in(2013(
•  “How$to$Break$So4ware”$James$Whi:aker,$2003$
•  And$his$other$“How$To$Break…”$books$
•  “TesJng$Embedded$So4ware”$Broeckman$and$Notenboom,$2003$
•  “A$PracJJoner’s$Guide$to$So4ware$Test$Design”$Copeland,$2004$
•  “A$PracJJoner’s$Handbook$for$RealQTime$Analysis”$Klein$et.$al.,$1993$
•  “Computer$Related$Risks”,$Neumann,$1995$
•  “Safeware:$System$Safety$and$Computers”,$Leveson,$1995$
•  Honorable$menJons:$
•  “Embedded$System$and$So4ware$ValidaJon”$Roychoudhury,$2009$
•  “Systems$TesJng$with$an$A}tude”$Petschenik$$2005$
•  “So4ware$System$TesJng$and$Quality$Assurance”$Beizer,$1987$
•  “TesJng$Computer$So4ware”$Kaner$et.$al.,$1988$
•  “SystemaJc$So4ware$TesJng”$Craig$&$Jaskiel,$2001$
•  “Managing$the$TesJng$Process”$Black,$2002$
Jon$Hagar$Copy$right$2014$ So4ware$Test$A:acks$to$Break$Mobile$and$Embedded$Devices$ 40$40$
MORE RESOURCES
•  www.sJckyminds.com$–$CollecJon$of$test$info$
•  www.embedded.com$–$info$on$a:acks$
•  www.sqaforums.com$Q$Mobile$Devices,$Mobile$Apps$Q$Embedded$Systems$
TesJng$forum$
$
•  AssociaJon$of$So4ware$TesJng$
–  BBST$Classes$h:p://www.tesJngeducaJon.org/BBST/$
•  Your$favorite$search$engine$
•  Our$web$sites$and$blogs$(listed$on$front$page)$

More Related Content

Viewers also liked

Pairwise Testing Explained
Pairwise Testing ExplainedPairwise Testing Explained
Pairwise Testing ExplainedTechWell
 
Introducing Keyword-Driven Test Automation
Introducing Keyword-Driven Test AutomationIntroducing Keyword-Driven Test Automation
Introducing Keyword-Driven Test AutomationTechWell
 
Test Estimation in Practice
Test Estimation in PracticeTest Estimation in Practice
Test Estimation in PracticeTechWell
 
Introduction to Selenium and WebDriver
Introduction to Selenium and WebDriverIntroduction to Selenium and WebDriver
Introduction to Selenium and WebDriverTechWell
 
Simplify Project and Portfolio Planning with “Real Options”
Simplify Project and Portfolio Planning with “Real Options”Simplify Project and Portfolio Planning with “Real Options”
Simplify Project and Portfolio Planning with “Real Options”TechWell
 
Using DevOps to Improve Software Quality in the Cloud
Using DevOps to Improve Software Quality in the CloudUsing DevOps to Improve Software Quality in the Cloud
Using DevOps to Improve Software Quality in the CloudTechWell
 
Test Automation Patterns: Issues and Solutions
Test Automation Patterns: Issues and SolutionsTest Automation Patterns: Issues and Solutions
Test Automation Patterns: Issues and SolutionsTechWell
 
Software Managers: Their Place in Agile
Software Managers: Their Place in AgileSoftware Managers: Their Place in Agile
Software Managers: Their Place in AgileTechWell
 
End-to-End Testing with the Heuristic Software Test Model
End-to-End Testing with the Heuristic Software Test ModelEnd-to-End Testing with the Heuristic Software Test Model
End-to-End Testing with the Heuristic Software Test ModelTechWell
 

Viewers also liked (9)

Pairwise Testing Explained
Pairwise Testing ExplainedPairwise Testing Explained
Pairwise Testing Explained
 
Introducing Keyword-Driven Test Automation
Introducing Keyword-Driven Test AutomationIntroducing Keyword-Driven Test Automation
Introducing Keyword-Driven Test Automation
 
Test Estimation in Practice
Test Estimation in PracticeTest Estimation in Practice
Test Estimation in Practice
 
Introduction to Selenium and WebDriver
Introduction to Selenium and WebDriverIntroduction to Selenium and WebDriver
Introduction to Selenium and WebDriver
 
Simplify Project and Portfolio Planning with “Real Options”
Simplify Project and Portfolio Planning with “Real Options”Simplify Project and Portfolio Planning with “Real Options”
Simplify Project and Portfolio Planning with “Real Options”
 
Using DevOps to Improve Software Quality in the Cloud
Using DevOps to Improve Software Quality in the CloudUsing DevOps to Improve Software Quality in the Cloud
Using DevOps to Improve Software Quality in the Cloud
 
Test Automation Patterns: Issues and Solutions
Test Automation Patterns: Issues and SolutionsTest Automation Patterns: Issues and Solutions
Test Automation Patterns: Issues and Solutions
 
Software Managers: Their Place in Agile
Software Managers: Their Place in AgileSoftware Managers: Their Place in Agile
Software Managers: Their Place in Agile
 
End-to-End Testing with the Heuristic Software Test Model
End-to-End Testing with the Heuristic Software Test ModelEnd-to-End Testing with the Heuristic Software Test Model
End-to-End Testing with the Heuristic Software Test Model
 

Similar to Top Ten Attacks to Break Mobile Apps

Soft Skills You Need Are Not Always Taught in Class
Soft Skills You Need Are Not Always Taught in ClassSoft Skills You Need Are Not Always Taught in Class
Soft Skills You Need Are Not Always Taught in ClassTechWell
 
IoT Software Testing Challenges: The IoT World Is Really Different
IoT Software Testing Challenges: The IoT World Is Really DifferentIoT Software Testing Challenges: The IoT World Is Really Different
IoT Software Testing Challenges: The IoT World Is Really DifferentTechWell
 
Top IOT Testing Challenges Webinar with Jon Hagar
Top IOT Testing Challenges Webinar with Jon HagarTop IOT Testing Challenges Webinar with Jon Hagar
Top IOT Testing Challenges Webinar with Jon HagarXBOSoft
 
JoeHelfrichResume0701
JoeHelfrichResume0701JoeHelfrichResume0701
JoeHelfrichResume0701Joe Helfrich
 
IoT Software Testing Challenges: The IoT World Is Really Different
IoT Software Testing Challenges: The IoT World Is Really DifferentIoT Software Testing Challenges: The IoT World Is Really Different
IoT Software Testing Challenges: The IoT World Is Really DifferentTechWell
 
Implement Combinatorial Test Patterns for Better Mobile and IoT Testing
Implement Combinatorial Test Patterns for Better Mobile and IoT TestingImplement Combinatorial Test Patterns for Better Mobile and IoT Testing
Implement Combinatorial Test Patterns for Better Mobile and IoT TestingJosiah Renaudin
 
XBOSoft Mobile Security Webinar with Jon D. Hagar
XBOSoft Mobile Security Webinar with Jon D. HagarXBOSoft Mobile Security Webinar with Jon D. Hagar
XBOSoft Mobile Security Webinar with Jon D. HagarXBOSoft
 
Java certification course in Coimbatore|Java Learning Center Coimbatore|Java ...
Java certification course in Coimbatore|Java Learning Center Coimbatore|Java ...Java certification course in Coimbatore|Java Learning Center Coimbatore|Java ...
Java certification course in Coimbatore|Java Learning Center Coimbatore|Java ...sathyancegon
 
How to Create a Risk Based Testing Strategy With Simulators, Emulators, and R...
How to Create a Risk Based Testing Strategy With Simulators, Emulators, and R...How to Create a Risk Based Testing Strategy With Simulators, Emulators, and R...
How to Create a Risk Based Testing Strategy With Simulators, Emulators, and R...Perfecto by Perforce
 
Patterns of Automation: Simplify Your Test Code
Patterns of Automation: Simplify Your Test CodePatterns of Automation: Simplify Your Test Code
Patterns of Automation: Simplify Your Test CodeTechWell
 
Android automation tools
Android automation toolsAndroid automation tools
Android automation toolsSSGMCE SHEGAON
 
Mobile testing trends webinar PPT
Mobile testing trends webinar PPTMobile testing trends webinar PPT
Mobile testing trends webinar PPTXBOSoft
 
The Internet of Things in Action: Testing Anki’s OVERDRIVE Racing Game
The Internet of Things in Action: Testing Anki’s OVERDRIVE Racing GameThe Internet of Things in Action: Testing Anki’s OVERDRIVE Racing Game
The Internet of Things in Action: Testing Anki’s OVERDRIVE Racing GameJosiah Renaudin
 
A Look into the Future: App Testing and Quality in 2025
A Look into the Future: App Testing and Quality in 2025A Look into the Future: App Testing and Quality in 2025
A Look into the Future: App Testing and Quality in 2025TechWell
 
Appium: the Superhero of Mobile testing
Appium: the Superhero of Mobile testingAppium: the Superhero of Mobile testing
Appium: the Superhero of Mobile testingInfotek Solutions
 
Real-Time Contextual and Social Relevance in Mobile
Real-Time Contextual and Social Relevance in MobileReal-Time Contextual and Social Relevance in Mobile
Real-Time Contextual and Social Relevance in MobileTechWell
 
Matt Lee - Resume 2015
Matt Lee - Resume 2015Matt Lee - Resume 2015
Matt Lee - Resume 2015Matt Lee
 

Similar to Top Ten Attacks to Break Mobile Apps (20)

Soft Skills You Need Are Not Always Taught in Class
Soft Skills You Need Are Not Always Taught in ClassSoft Skills You Need Are Not Always Taught in Class
Soft Skills You Need Are Not Always Taught in Class
 
IoT Software Testing Challenges: The IoT World Is Really Different
IoT Software Testing Challenges: The IoT World Is Really DifferentIoT Software Testing Challenges: The IoT World Is Really Different
IoT Software Testing Challenges: The IoT World Is Really Different
 
Top IOT Testing Challenges Webinar with Jon Hagar
Top IOT Testing Challenges Webinar with Jon HagarTop IOT Testing Challenges Webinar with Jon Hagar
Top IOT Testing Challenges Webinar with Jon Hagar
 
JoeHelfrichResume0701
JoeHelfrichResume0701JoeHelfrichResume0701
JoeHelfrichResume0701
 
IoT Software Testing Challenges: The IoT World Is Really Different
IoT Software Testing Challenges: The IoT World Is Really DifferentIoT Software Testing Challenges: The IoT World Is Really Different
IoT Software Testing Challenges: The IoT World Is Really Different
 
Implement Combinatorial Test Patterns for Better Mobile and IoT Testing
Implement Combinatorial Test Patterns for Better Mobile and IoT TestingImplement Combinatorial Test Patterns for Better Mobile and IoT Testing
Implement Combinatorial Test Patterns for Better Mobile and IoT Testing
 
XBOSoft Mobile Security Webinar with Jon D. Hagar
XBOSoft Mobile Security Webinar with Jon D. HagarXBOSoft Mobile Security Webinar with Jon D. Hagar
XBOSoft Mobile Security Webinar with Jon D. Hagar
 
Java certification course in Coimbatore|Java Learning Center Coimbatore|Java ...
Java certification course in Coimbatore|Java Learning Center Coimbatore|Java ...Java certification course in Coimbatore|Java Learning Center Coimbatore|Java ...
Java certification course in Coimbatore|Java Learning Center Coimbatore|Java ...
 
Java
JavaJava
Java
 
How to Create a Risk Based Testing Strategy With Simulators, Emulators, and R...
How to Create a Risk Based Testing Strategy With Simulators, Emulators, and R...How to Create a Risk Based Testing Strategy With Simulators, Emulators, and R...
How to Create a Risk Based Testing Strategy With Simulators, Emulators, and R...
 
Patterns of Automation: Simplify Your Test Code
Patterns of Automation: Simplify Your Test CodePatterns of Automation: Simplify Your Test Code
Patterns of Automation: Simplify Your Test Code
 
Android automation tools
Android automation toolsAndroid automation tools
Android automation tools
 
Mobile testing trends webinar PPT
Mobile testing trends webinar PPTMobile testing trends webinar PPT
Mobile testing trends webinar PPT
 
The Internet of Things in Action: Testing Anki’s OVERDRIVE Racing Game
The Internet of Things in Action: Testing Anki’s OVERDRIVE Racing GameThe Internet of Things in Action: Testing Anki’s OVERDRIVE Racing Game
The Internet of Things in Action: Testing Anki’s OVERDRIVE Racing Game
 
A Look into the Future: App Testing and Quality in 2025
A Look into the Future: App Testing and Quality in 2025A Look into the Future: App Testing and Quality in 2025
A Look into the Future: App Testing and Quality in 2025
 
Appium: the Superhero of Mobile testing
Appium: the Superhero of Mobile testingAppium: the Superhero of Mobile testing
Appium: the Superhero of Mobile testing
 
BSides LA/PDX
BSides LA/PDXBSides LA/PDX
BSides LA/PDX
 
Real-Time Contextual and Social Relevance in Mobile
Real-Time Contextual and Social Relevance in MobileReal-Time Contextual and Social Relevance in Mobile
Real-Time Contextual and Social Relevance in Mobile
 
RR2 CV
RR2 CVRR2 CV
RR2 CV
 
Matt Lee - Resume 2015
Matt Lee - Resume 2015Matt Lee - Resume 2015
Matt Lee - Resume 2015
 

More from TechWell

Failing and Recovering
Failing and RecoveringFailing and Recovering
Failing and RecoveringTechWell
 
Instill a DevOps Testing Culture in Your Team and Organization
Instill a DevOps Testing Culture in Your Team and Organization Instill a DevOps Testing Culture in Your Team and Organization
Instill a DevOps Testing Culture in Your Team and Organization TechWell
 
Test Design for Fully Automated Build Architecture
Test Design for Fully Automated Build ArchitectureTest Design for Fully Automated Build Architecture
Test Design for Fully Automated Build ArchitectureTechWell
 
System-Level Test Automation: Ensuring a Good Start
System-Level Test Automation: Ensuring a Good StartSystem-Level Test Automation: Ensuring a Good Start
System-Level Test Automation: Ensuring a Good StartTechWell
 
Build Your Mobile App Quality and Test Strategy
Build Your Mobile App Quality and Test StrategyBuild Your Mobile App Quality and Test Strategy
Build Your Mobile App Quality and Test StrategyTechWell
 
Testing Transformation: The Art and Science for Success
Testing Transformation: The Art and Science for SuccessTesting Transformation: The Art and Science for Success
Testing Transformation: The Art and Science for SuccessTechWell
 
Implement BDD with Cucumber and SpecFlow
Implement BDD with Cucumber and SpecFlowImplement BDD with Cucumber and SpecFlow
Implement BDD with Cucumber and SpecFlowTechWell
 
Develop WebDriver Automated Tests—and Keep Your Sanity
Develop WebDriver Automated Tests—and Keep Your SanityDevelop WebDriver Automated Tests—and Keep Your Sanity
Develop WebDriver Automated Tests—and Keep Your SanityTechWell
 
Eliminate Cloud Waste with a Holistic DevOps Strategy
Eliminate Cloud Waste with a Holistic DevOps StrategyEliminate Cloud Waste with a Holistic DevOps Strategy
Eliminate Cloud Waste with a Holistic DevOps StrategyTechWell
 
Transform Test Organizations for the New World of DevOps
Transform Test Organizations for the New World of DevOpsTransform Test Organizations for the New World of DevOps
Transform Test Organizations for the New World of DevOpsTechWell
 
The Fourth Constraint in Project Delivery—Leadership
The Fourth Constraint in Project Delivery—LeadershipThe Fourth Constraint in Project Delivery—Leadership
The Fourth Constraint in Project Delivery—LeadershipTechWell
 
Resolve the Contradiction of Specialists within Agile Teams
Resolve the Contradiction of Specialists within Agile TeamsResolve the Contradiction of Specialists within Agile Teams
Resolve the Contradiction of Specialists within Agile TeamsTechWell
 
Pin the Tail on the Metric: A Field-Tested Agile Game
Pin the Tail on the Metric: A Field-Tested Agile GamePin the Tail on the Metric: A Field-Tested Agile Game
Pin the Tail on the Metric: A Field-Tested Agile GameTechWell
 
Agile Performance Holarchy (APH)—A Model for Scaling Agile Teams
Agile Performance Holarchy (APH)—A Model for Scaling Agile TeamsAgile Performance Holarchy (APH)—A Model for Scaling Agile Teams
Agile Performance Holarchy (APH)—A Model for Scaling Agile TeamsTechWell
 
A Business-First Approach to DevOps Implementation
A Business-First Approach to DevOps ImplementationA Business-First Approach to DevOps Implementation
A Business-First Approach to DevOps ImplementationTechWell
 
Databases in a Continuous Integration/Delivery Process
Databases in a Continuous Integration/Delivery ProcessDatabases in a Continuous Integration/Delivery Process
Databases in a Continuous Integration/Delivery ProcessTechWell
 
Mobile Testing: What—and What Not—to Automate
Mobile Testing: What—and What Not—to AutomateMobile Testing: What—and What Not—to Automate
Mobile Testing: What—and What Not—to AutomateTechWell
 
Cultural Intelligence: A Key Skill for Success
Cultural Intelligence: A Key Skill for SuccessCultural Intelligence: A Key Skill for Success
Cultural Intelligence: A Key Skill for SuccessTechWell
 
Turn the Lights On: A Power Utility Company's Agile Transformation
Turn the Lights On: A Power Utility Company's Agile TransformationTurn the Lights On: A Power Utility Company's Agile Transformation
Turn the Lights On: A Power Utility Company's Agile TransformationTechWell
 

More from TechWell (20)

Failing and Recovering
Failing and RecoveringFailing and Recovering
Failing and Recovering
 
Instill a DevOps Testing Culture in Your Team and Organization
Instill a DevOps Testing Culture in Your Team and Organization Instill a DevOps Testing Culture in Your Team and Organization
Instill a DevOps Testing Culture in Your Team and Organization
 
Test Design for Fully Automated Build Architecture
Test Design for Fully Automated Build ArchitectureTest Design for Fully Automated Build Architecture
Test Design for Fully Automated Build Architecture
 
System-Level Test Automation: Ensuring a Good Start
System-Level Test Automation: Ensuring a Good StartSystem-Level Test Automation: Ensuring a Good Start
System-Level Test Automation: Ensuring a Good Start
 
Build Your Mobile App Quality and Test Strategy
Build Your Mobile App Quality and Test StrategyBuild Your Mobile App Quality and Test Strategy
Build Your Mobile App Quality and Test Strategy
 
Testing Transformation: The Art and Science for Success
Testing Transformation: The Art and Science for SuccessTesting Transformation: The Art and Science for Success
Testing Transformation: The Art and Science for Success
 
Implement BDD with Cucumber and SpecFlow
Implement BDD with Cucumber and SpecFlowImplement BDD with Cucumber and SpecFlow
Implement BDD with Cucumber and SpecFlow
 
Develop WebDriver Automated Tests—and Keep Your Sanity
Develop WebDriver Automated Tests—and Keep Your SanityDevelop WebDriver Automated Tests—and Keep Your Sanity
Develop WebDriver Automated Tests—and Keep Your Sanity
 
Ma 15
Ma 15Ma 15
Ma 15
 
Eliminate Cloud Waste with a Holistic DevOps Strategy
Eliminate Cloud Waste with a Holistic DevOps StrategyEliminate Cloud Waste with a Holistic DevOps Strategy
Eliminate Cloud Waste with a Holistic DevOps Strategy
 
Transform Test Organizations for the New World of DevOps
Transform Test Organizations for the New World of DevOpsTransform Test Organizations for the New World of DevOps
Transform Test Organizations for the New World of DevOps
 
The Fourth Constraint in Project Delivery—Leadership
The Fourth Constraint in Project Delivery—LeadershipThe Fourth Constraint in Project Delivery—Leadership
The Fourth Constraint in Project Delivery—Leadership
 
Resolve the Contradiction of Specialists within Agile Teams
Resolve the Contradiction of Specialists within Agile TeamsResolve the Contradiction of Specialists within Agile Teams
Resolve the Contradiction of Specialists within Agile Teams
 
Pin the Tail on the Metric: A Field-Tested Agile Game
Pin the Tail on the Metric: A Field-Tested Agile GamePin the Tail on the Metric: A Field-Tested Agile Game
Pin the Tail on the Metric: A Field-Tested Agile Game
 
Agile Performance Holarchy (APH)—A Model for Scaling Agile Teams
Agile Performance Holarchy (APH)—A Model for Scaling Agile TeamsAgile Performance Holarchy (APH)—A Model for Scaling Agile Teams
Agile Performance Holarchy (APH)—A Model for Scaling Agile Teams
 
A Business-First Approach to DevOps Implementation
A Business-First Approach to DevOps ImplementationA Business-First Approach to DevOps Implementation
A Business-First Approach to DevOps Implementation
 
Databases in a Continuous Integration/Delivery Process
Databases in a Continuous Integration/Delivery ProcessDatabases in a Continuous Integration/Delivery Process
Databases in a Continuous Integration/Delivery Process
 
Mobile Testing: What—and What Not—to Automate
Mobile Testing: What—and What Not—to AutomateMobile Testing: What—and What Not—to Automate
Mobile Testing: What—and What Not—to Automate
 
Cultural Intelligence: A Key Skill for Success
Cultural Intelligence: A Key Skill for SuccessCultural Intelligence: A Key Skill for Success
Cultural Intelligence: A Key Skill for Success
 
Turn the Lights On: A Power Utility Company's Agile Transformation
Turn the Lights On: A Power Utility Company's Agile TransformationTurn the Lights On: A Power Utility Company's Agile Transformation
Turn the Lights On: A Power Utility Company's Agile Transformation
 

Recently uploaded

Presentation on how to chat with PDF using ChatGPT code interpreter
Presentation on how to chat with PDF using ChatGPT code interpreterPresentation on how to chat with PDF using ChatGPT code interpreter
Presentation on how to chat with PDF using ChatGPT code interpreternaman860154
 
[2024]Digital Global Overview Report 2024 Meltwater.pdf
[2024]Digital Global Overview Report 2024 Meltwater.pdf[2024]Digital Global Overview Report 2024 Meltwater.pdf
[2024]Digital Global Overview Report 2024 Meltwater.pdfhans926745
 
Artificial Intelligence: Facts and Myths
Artificial Intelligence: Facts and MythsArtificial Intelligence: Facts and Myths
Artificial Intelligence: Facts and MythsJoaquim Jorge
 
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time AutomationFrom Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time AutomationSafe Software
 
IAC 2024 - IA Fast Track to Search Focused AI Solutions
IAC 2024 - IA Fast Track to Search Focused AI SolutionsIAC 2024 - IA Fast Track to Search Focused AI Solutions
IAC 2024 - IA Fast Track to Search Focused AI SolutionsEnterprise Knowledge
 
Scaling API-first – The story of a global engineering organization
Scaling API-first – The story of a global engineering organizationScaling API-first – The story of a global engineering organization
Scaling API-first – The story of a global engineering organizationRadu Cotescu
 
🐬 The future of MySQL is Postgres 🐘
🐬  The future of MySQL is Postgres   🐘🐬  The future of MySQL is Postgres   🐘
🐬 The future of MySQL is Postgres 🐘RTylerCroy
 
A Call to Action for Generative AI in 2024
A Call to Action for Generative AI in 2024A Call to Action for Generative AI in 2024
A Call to Action for Generative AI in 2024Results
 
Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024The Digital Insurer
 
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...Miguel Araújo
 
Powerful Google developer tools for immediate impact! (2023-24 C)
Powerful Google developer tools for immediate impact! (2023-24 C)Powerful Google developer tools for immediate impact! (2023-24 C)
Powerful Google developer tools for immediate impact! (2023-24 C)wesley chun
 
2024: Domino Containers - The Next Step. News from the Domino Container commu...
2024: Domino Containers - The Next Step. News from the Domino Container commu...2024: Domino Containers - The Next Step. News from the Domino Container commu...
2024: Domino Containers - The Next Step. News from the Domino Container commu...Martijn de Jong
 
Histor y of HAM Radio presentation slide
Histor y of HAM Radio presentation slideHistor y of HAM Radio presentation slide
Histor y of HAM Radio presentation slidevu2urc
 
The 7 Things I Know About Cyber Security After 25 Years | April 2024
The 7 Things I Know About Cyber Security After 25 Years | April 2024The 7 Things I Know About Cyber Security After 25 Years | April 2024
The 7 Things I Know About Cyber Security After 25 Years | April 2024Rafal Los
 
A Year of the Servo Reboot: Where Are We Now?
A Year of the Servo Reboot: Where Are We Now?A Year of the Servo Reboot: Where Are We Now?
A Year of the Servo Reboot: Where Are We Now?Igalia
 
Finology Group – Insurtech Innovation Award 2024
Finology Group – Insurtech Innovation Award 2024Finology Group – Insurtech Innovation Award 2024
Finology Group – Insurtech Innovation Award 2024The Digital Insurer
 
Understanding Discord NSFW Servers A Guide for Responsible Users.pdf
Understanding Discord NSFW Servers A Guide for Responsible Users.pdfUnderstanding Discord NSFW Servers A Guide for Responsible Users.pdf
Understanding Discord NSFW Servers A Guide for Responsible Users.pdfUK Journal
 
08448380779 Call Girls In Friends Colony Women Seeking Men
08448380779 Call Girls In Friends Colony Women Seeking Men08448380779 Call Girls In Friends Colony Women Seeking Men
08448380779 Call Girls In Friends Colony Women Seeking MenDelhi Call girls
 
GenCyber Cyber Security Day Presentation
GenCyber Cyber Security Day PresentationGenCyber Cyber Security Day Presentation
GenCyber Cyber Security Day PresentationMichael W. Hawkins
 
Factors to Consider When Choosing Accounts Payable Services Providers.pptx
Factors to Consider When Choosing Accounts Payable Services Providers.pptxFactors to Consider When Choosing Accounts Payable Services Providers.pptx
Factors to Consider When Choosing Accounts Payable Services Providers.pptxKatpro Technologies
 

Recently uploaded (20)

Presentation on how to chat with PDF using ChatGPT code interpreter
Presentation on how to chat with PDF using ChatGPT code interpreterPresentation on how to chat with PDF using ChatGPT code interpreter
Presentation on how to chat with PDF using ChatGPT code interpreter
 
[2024]Digital Global Overview Report 2024 Meltwater.pdf
[2024]Digital Global Overview Report 2024 Meltwater.pdf[2024]Digital Global Overview Report 2024 Meltwater.pdf
[2024]Digital Global Overview Report 2024 Meltwater.pdf
 
Artificial Intelligence: Facts and Myths
Artificial Intelligence: Facts and MythsArtificial Intelligence: Facts and Myths
Artificial Intelligence: Facts and Myths
 
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time AutomationFrom Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
 
IAC 2024 - IA Fast Track to Search Focused AI Solutions
IAC 2024 - IA Fast Track to Search Focused AI SolutionsIAC 2024 - IA Fast Track to Search Focused AI Solutions
IAC 2024 - IA Fast Track to Search Focused AI Solutions
 
Scaling API-first – The story of a global engineering organization
Scaling API-first – The story of a global engineering organizationScaling API-first – The story of a global engineering organization
Scaling API-first – The story of a global engineering organization
 
🐬 The future of MySQL is Postgres 🐘
🐬  The future of MySQL is Postgres   🐘🐬  The future of MySQL is Postgres   🐘
🐬 The future of MySQL is Postgres 🐘
 
A Call to Action for Generative AI in 2024
A Call to Action for Generative AI in 2024A Call to Action for Generative AI in 2024
A Call to Action for Generative AI in 2024
 
Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024
 
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
 
Powerful Google developer tools for immediate impact! (2023-24 C)
Powerful Google developer tools for immediate impact! (2023-24 C)Powerful Google developer tools for immediate impact! (2023-24 C)
Powerful Google developer tools for immediate impact! (2023-24 C)
 
2024: Domino Containers - The Next Step. News from the Domino Container commu...
2024: Domino Containers - The Next Step. News from the Domino Container commu...2024: Domino Containers - The Next Step. News from the Domino Container commu...
2024: Domino Containers - The Next Step. News from the Domino Container commu...
 
Histor y of HAM Radio presentation slide
Histor y of HAM Radio presentation slideHistor y of HAM Radio presentation slide
Histor y of HAM Radio presentation slide
 
The 7 Things I Know About Cyber Security After 25 Years | April 2024
The 7 Things I Know About Cyber Security After 25 Years | April 2024The 7 Things I Know About Cyber Security After 25 Years | April 2024
The 7 Things I Know About Cyber Security After 25 Years | April 2024
 
A Year of the Servo Reboot: Where Are We Now?
A Year of the Servo Reboot: Where Are We Now?A Year of the Servo Reboot: Where Are We Now?
A Year of the Servo Reboot: Where Are We Now?
 
Finology Group – Insurtech Innovation Award 2024
Finology Group – Insurtech Innovation Award 2024Finology Group – Insurtech Innovation Award 2024
Finology Group – Insurtech Innovation Award 2024
 
Understanding Discord NSFW Servers A Guide for Responsible Users.pdf
Understanding Discord NSFW Servers A Guide for Responsible Users.pdfUnderstanding Discord NSFW Servers A Guide for Responsible Users.pdf
Understanding Discord NSFW Servers A Guide for Responsible Users.pdf
 
08448380779 Call Girls In Friends Colony Women Seeking Men
08448380779 Call Girls In Friends Colony Women Seeking Men08448380779 Call Girls In Friends Colony Women Seeking Men
08448380779 Call Girls In Friends Colony Women Seeking Men
 
GenCyber Cyber Security Day Presentation
GenCyber Cyber Security Day PresentationGenCyber Cyber Security Day Presentation
GenCyber Cyber Security Day Presentation
 
Factors to Consider When Choosing Accounts Payable Services Providers.pptx
Factors to Consider When Choosing Accounts Payable Services Providers.pptxFactors to Consider When Choosing Accounts Payable Services Providers.pptx
Factors to Consider When Choosing Accounts Payable Services Providers.pptx
 

Top Ten Attacks to Break Mobile Apps

  • 1. !! T4 Mobile!Testing! 10/16/2014!9:45:00!AM! ! Top Ten Attacks to Break Mobile Apps ! Presented by: Jon Hagar Grand Software Testing ! ! ! Brought(to(you(by:( ( ( ( 340!Corporate!Way,!Suite!300,!Orange!Park,!FL!32073! 888G268G8770!H!904G278G0524!H!sqeinfo@sqe.com!H!www.sqe.com
  • 2. Jon Hagar Grand Software Testing Jon Hagar is an independent consultant working in software product integrity, testing, verification, and validation. Jon publishes regularly on testing, including the book Software Test Attacks to Break Mobile and Embedded Devices (breakingembeddedsoftware.com). For more than thirty years, he has worked in software engineering, particularly testing, supporting projects which include control systems (avionics and auto), spacecraft, mobile-smart devices, and attack testing of smart phones. Jon is an editor for ISO, IEEE, and OMG standards.                                        
  • 3. Jon$Hagar$Copy$right$2014$ So4ware$Test$A:acks$to$Break$Mobile$and$Embedded$Devices$ 1$ TOP$10$SOFTWARE$TEST$ATTACKS$ TO$BREAK$MOBILE$SOFTWARE$ STARWEST$2014$ Jon$Hagar$ embedded@ecentral.com$ jon.d.hagar@gmail.com$ Grand$So4ware$TesJng$ Web:$ h:p://breakingembeddedso4ware.wordpress.com/$ Jon$Hagar$Copy$right$2014$ So4ware$Test$A:acks$to$Break$Mobile$and$Embedded$Devices$ 2$2$ AGENDA$ •  DefiniJons$for$this$session$ •  RiskQbased$tesJng$concepts$for$mobile$ •  Exploratory$tesJng$concepts$for$$mobile$ •  My$top$10$Mobile$So4ware$(app)$a:acks$ •  Wrap$up$
  • 4. Jon$Hagar$Copy$right$2014$ So4ware$Test$A:acks$to$Break$Mobile$and$Embedded$Devices$ 3$3$ $ MOBILE,$SMART,$AND$HANDHELD$ •  As$the$names$implies,$these$are$devices—small,$held$in$the$hand,$o4en$ connected$to$communicaJon$networks,$including$ •  Cell$and$smart$phones$–$apps$$ •  Tablets$ •  Medical$devices$ •  Typically$have:$ •  Many$of$the$problems$of$classic$“embedded”$systems$ •  The$power$of$PCs/IT$ •  More$user$interface$(UI)$than$classic$embedded$systems$ •  (RelaJvely)$Fast$updates$ •  Mobile$devices$are$“evolving”$with$more$power,$resources,$apps,$etc.$$ •  Mobile$is$the$“hot”$area$of$computers$and$so4ware$currently$ •  TesJng$rules$and$concepts$are$“evolving”$ STARWEST$ Jon$Hagar$Copy$right$2014$ So4ware$Test$A:acks$to$Break$Mobile$and$Embedded$Devices$ 4$4$ TYPES$OF$MOBILE$APPS$ •  NaJve$ApplicaJons$ •  Local$to$device$ •  Hybrid$ApplicaJons$ •  Local$to$device$but$ interacts$w/internet$ •  Web$ApplicaJons$ •  Not$local$to$device.$ All$interacJons$on$ internet$ STARWEST$
  • 5. Jon$Hagar$Copy$right$2014$ So4ware$Test$A:acks$to$Break$Mobile$and$Embedded$Devices$ 5$5$ MOBILE$TESTING$DEFINITIONS$ •  Mobile$ApplicaJon$TesJng$is$tesJng$the$applicaJon$in$a$support$ environment$or$on$a$mobile$device$ •  System$Level$Mobile$Device$TesJng$is$tesJng$the$hardware$and$operaJng$ system$ •  Does$the$OperaJng$System$install?$$$ •  Does$the$device$power$on?$Do$the$LED$lights$work$as$expected?$$$ •  Does$the$ba:ery$charge$when$the$AC$adapter$is$plugged$into$the$device?$ •  Mobile$Phone$TesJng$should$have$some$different$approaches$to$tesJng$ •  Mobile$System$TesJng$incorporates$tesJng$more$than$one$applicaJon$and$ can$combine$hardware,$so4ware,$firmware,$along$with$other$applicaJons$ •  Mobile$TesJng$–$can/should$be$all$of$the$above$ $ Be$clear$when$using$this$terminology.$If$you$are$only$tesJng$apps$on$mobile$phones,$ then$state$“mobile$apps$tesJng.”$Use$mobile$tesJng$when$you$are$tesJng$mobile$ websites,$mobile$hybrid$apps,$mobile$hardware,$etc.$$ STARWEST$$ Jon$Hagar$Copy$right$2014$ So4ware$Test$A:acks$to$Break$Mobile$and$Embedded$Devices$ 6$6$ DEFINING$SKILL$SET$FOR$ THE$MOBILE$TESTER$ •  Some$exposure$or$knowledge$about$products$from$the$domain$in$which$you$ are$tesJng:$$aerospace,$medical,$automobile$manufacturing,$airplanes,$ factory$systems,$roboJcs,$regulated$environments,$etc.$ •  Some$knowledge$of:$hard$sciences:$$math,$physics,$electronics,$engineering,$ etc.$for$logical$thought$processes$ •  So4ware$sciences:$$psychology,$philosophy,$sociology,$human$factors$(human$ machine$interface)$for$creaJve$&$conceptual$thought$processes$ •  Tester$skill$ •  Planning,$design$techniques,$pa:erns$of$errors,$intuiJon,$criJcal$thinking,$“so4$skills,”$$ communicaJon,$observaJon,$and$mental$models$[ISTQB$and$AST$have$“lists”]$ Chapter$1$–$So4ware$Test$A:acks$to$Break$Mobile$&$Embedded$Devices$ STARWEST$
  • 6. Jon$Hagar$Copy$right$2014$ So4ware$Test$A:acks$to$Break$Mobile$and$Embedded$Devices$ 7$7$ WHAT$IS$AN$ATTACK?$ •  A$pa:ern$(of$tesJng)$based$on$a$common$mode$of$failure$seen$ over$and$over$ •  Maybe$seen$as$a$negaJve,$when$it$is$really$a$posi%ve( •  Goes$a4er$the$“bugs”$that$may$be$in$the$so4ware$ •  May$include$or$use$classic$test$techniques$and$test$concepts$ •  Lee$Copeland’s$book$on$test$design$ •  Many$other$good$books$ •  A$Pa:ern$(more$than$a$process),$which$must$be$modified$for$the$ context$at$hand,$to$do$the$tesJng$$ •  Testers$learn$these$in$a$domain$a4er$years$and$form$a$mental$ model$(most$good$testers$a:ack)$ STARWEST$ Jon$Hagar$Copy$right$2014$ So4ware$Test$A:acks$to$Break$Mobile$and$Embedded$Devices$ 8$8$ WHY$ATTACK?$ •  A:acking$your$so4ware$is$in$part,$$the$process$of$a:empJng$to$ demonstrate$a$system$(hardware,$firmware,$so4ware$and$operaJons)$$ does$not$meet$requirements,$funcJonal$and$nonQfuncJonal$objecJves$ •  Embedded/handheld$so4ware$tesJng$must$include$"the$ system"$(hardware,$so4ware,$operaJons,$users)$ •  A:acking$common$modes$of$failure,$especially$where$the$applicaJon$ is$engaged$and$visible$by$the$user.$ Attack your enemy with approaches to include: Tools Levels Attacks Techniques STARWEST$
  • 7. Jon$Hagar$Copy$right$2014$ So4ware$Test$A:acks$to$Break$Mobile$and$Embedded$Devices$ 9$9$ KINDS$OF$ATTACKS$ •  Whi:aker$offers$a$good$starJng$point$for$so4ware$ a:acks$in$general$that$can$be$applied$to$mobile:$ •  User$Interface$A:acks$ •  Data$and$ComputaJon$ •  File$System$Interface$ •  So4ware/OS$Interface$ •  Whi:aker’s$“How$to$Break$So4ware”$lists$23$a:acks$ •  Plus$he$has$other$books$on$a:acks,$security,$web,$exploratory,$and$tours$in$tesJng$ •  “So4ware$Test$A:acks$to$Break$Mobile$and$ Embedded$Devices”$lists$32$a:acks$and$8$sub$a:acks$ STARWEST$ Jon$Hagar$Copy$right$2014$ So4ware$Test$A:acks$to$Break$Mobile$and$Embedded$Devices$ 10$10$ MOBILE$RISK$AREAS$TO$CONSIDER$ •  There$are$many$risk$to$$consider,$but$you$cannot$test$everything$ •  Risk(s)$based$tesJng$$helps$$bound$the$test$scope$problem$ •  TesJng$is$about$providing$informaJon$and$understanding$ •  ExploraJon$gets$you$started$with$whatever$you$have$(or$don’t$ have)$ STARWEST$
  • 8. Jon$Hagar$Copy$right$2014$ So4ware$Test$A:acks$to$Break$Mobile$and$Embedded$Devices$ 11$11$ SAMPLE$MOBILE$PRODUCT$RISKS$ TESTERS$SHOULD$CONSIDER$ •  Environment$and$input$factors$ •  Environment$–$heat,$noise,$sun,$water,$etc.$ •  Hardware$–$calibraJon,$uniqueness,$manufacturing,$etc.$ •  Electronics$–$noise,$power,$ba:eries,$etc.$ •  CommunicaJons$ •  Interfaces$types$ •  Hardware$ •  Human$$ •  Network$$ •  So4ware$ •  Output$—$noise$influences,$D2A,$representaJon,$etc.$ •  Complexity—use$/$size$of$the$system$$ STARWEST$ Jon$Hagar$Copy$right$2014$ So4ware$Test$A:acks$to$Break$Mobile$and$Embedded$Devices$ 12$12$ RISK=BASED$ TESTING$$ (ISO$29119)$ •  Address,$miJgate,$a:ack$and$reJre$product$risks$ •  PrioriJze$risks$Q$tests:$ •  PotenJal$problems$Q$$Consequences$and$effects$ •  Occurrences$–$likelihood$or$chance$of$happening$ •  Impacts$–$what$happens$ •  Take$consistent$acJon$from$the$beginning$(proposal)$ to$the$end$(reJrement)$of$the$product$or$lifecycle$ •  Risks$&$prioriJzing$should$dictate$the$test$a:acks$ STARWEST$
  • 9. Jon$Hagar$Copy$right$2014$ So4ware$Test$A:acks$to$Break$Mobile$and$Embedded$Devices$ 13$13$ EXPLORATORY$TESTING$=$DEFINITION$ •  QuoJng$James$Bach:$$“The$plainest$definiJon$of$exploratory$ tesJng$is$test$design$and$test$execuJon$at$the$same$Jme.$This$is$ the$opposite$of$scripted$tesJng$(predefined$test$procedures,$ whether$manual$or$automated).$Exploratory$tests,$unlike$ scripted$tests,$are$not$defined$in$advance$and$carried$out$ precisely$according$to$plan.”$ h:p://www.saJsfice.com/arJcles/what_is_et.shtml$ STARWEST$ Jon$Hagar$Copy$right$2014$ So4ware$Test$A:acks$to$Break$Mobile$and$Embedded$Devices$ 14$14$ EXPLORATORY$TESTING$IN$MOBILE$ •  Rapid$feedback$ •  Learning$ •  Upfront$rapid$ learning$ •  A:acking$ •  Address$Risk$(s)$ •  Independent$assessment$ •  Target$a$defect$ •  Prototyping$ •  Need$info$ •  Test$beyond$the$ requirements$ STARWEST$
  • 10. Jon$Hagar$Copy$right$2014$ So4ware$Test$A:acks$to$Break$Mobile$and$Embedded$Devices$ 15$15$ NUMBER$10:$FUNCTIONAL$ATTACK$(33)$ •  Have$an$outline$$or$charter$(top$level$plan$and/or$risk$list)$ •  Create$a$flip$chart,$notecard,$state$model,$mind$map$or$some$representaJon$of$ each$test$$task$$ •  No$“heavyweight$documentaJon$of$the$“test$case”$ •  See$Exploratory$Charter$(test$objecJve)$ •  Have$a$Target$concept$$or$charter$$(Risk,$A:ack,$Bug,$Learning,$…)$ •  VerificaJon$checking$of$requirements$(necessary$but$not$sufficient)$$ •  Have$a$schedule/Jme$box$(short$$test$cycles$=$Planning$to$report)$ •  Do$the$test$ •  Design$test$ •  Execute$test$$ •  Learn$about$the$product:$change$the$risk$list,$modify/add$tests,$and$so$on$ •  Repeat$as$needed$ STARWEST$ Jon$Hagar$Copy$right$2014$ So4ware$Test$A:acks$to$Break$Mobile$and$Embedded$Devices$ 16$16$ EXAMPLE$MIND$MAP$FROM$A$TRAVEL$APP$
  • 11. Jon$Hagar$Copy$right$2014$ So4ware$Test$A:acks$to$Break$Mobile$and$Embedded$Devices$ 17$17$ •  Download$either$Twi:er$or$Facebook$onto$a$device$$ •  Start$either$downloaded$app$ •  From$another$device,$send$an$email$to$the$device’s$email$account$ •  Immediately$send$a$tweet$or$post$a$status$$ •  ConJnue$to$do$engage$Twi:er$or$Facebook$app$for$at$least$1$minute$ •  Record$email$noJficaJon$and$Jme$when$sent$and$received$ •  What$other$observaJons$occurred?$ 9:$NOTIFICATION$TEST$ATTACK$(18)$ Jon$Hagar$Copy$right$2014$ So4ware$Test$A:acks$to$Break$Mobile$and$Embedded$Devices$ 18$18$ ATTACK$TEST$CASE$EXAMPLE:$$ INTERRUPTS$ON$MOBILE$PHONES$ •  Go$to$your$App$store$and$choose$an$applicaJon$to$download$ •  While$the$downloading$is$occurring,$call$the$mobile$phone$ •  Record$observaJons$with$the$download$ •  You$may$need$to$rely$on$observing$a$log$file$while$implemenJng$these$ tests$ •  If$it$fails,$what$kind$of$error$recovery$occurs?$Can$you$repeat$any$ errors?$
  • 12. Jon$Hagar$Copy$right$2014$ So4ware$Test$A:acks$to$Break$Mobile$and$Embedded$Devices$ 19$19$ 8:$ATTACK$SCENARIOS$(12)$ •  Tests$consider$usage,$operaJons,$interface$interacJons$and$ integraJons$$ •  Interface$points$include:$$hardware,$firmware,$so4ware,$data$ exchange,$network$communicaJon$and$combinaJons$ •  How$each$interface$point$integrates$with$another$interface$point$ •  Tests$include$how$the$applicaJon$is$used$endQtoQend$$ •  Tests$to$combine$how$the$enJre$system$interacts$as$well$as$how$ porJons$interact$with$one$another$and$depending$on$complexity$ •  Note:$ConfiguraJon$tests$with$regards$to$how$so4ware$behaves$based$ on$various$configuraJons$of$devices,$operaJng$systems$ Jon$Hagar$Copy$right$2014$ So4ware$Test$A:acks$to$Break$Mobile$and$Embedded$Devices$ 20$ IN$SCENARIO$TESTING$:$TIMING$SUBQ$ATTACK$ When$Time$interacts$with$the$so4ware,$events,$inputs,$ and$outputs,$here's$a$checklist$of$things$to$look$for$and$ consider$(where$bugs$lurk)$in$sequences/stories$ $ •  Order$problems$ •  Too$Long$$ •  Too$Fast$ •  Not$at$right$Jme$mark$or$point$$ •  Late$$ •  Late$or$early$ •  Early$$ •  Deadlocked$caused$by$a$race$ condiJon$(hard$to$find)$$ •  Extra$input$or$output$events$$ •  Missing$events$$ •  Wrong$input/output$within$events$$
  • 13. Jon$Hagar$Copy$right$2014$ So4ware$Test$A:acks$to$Break$Mobile$and$Embedded$Devices$ 21$21$ •  When$to$apply$this$a:ack?$$…when$your$app/device$has$games$ •  What$faults$make$this$a:ack$successful?$$$…games$are$complex$ •  Who$conducts$this$a:ack?$$…see$chart$on$Roles$ •  Where$is$this$a:ack$conducted?$$…throughout$lifecycle$and$in$environments$ •  How$to$determine$if$the$a:ack$exposes$failures?$ •  Unhappy$“users”$ •  Bugs$found$ •  See$checklist$ 7:$ATTACK$TESTING$MOBILE$GAMES$(26)$ Mobile Device Game Testing (2 years ago gaming was 60% or more of Mobile App downloads) Jon$Hagar$Copy$right$2014$ So4ware$Test$A:acks$to$Break$Mobile$and$Embedded$Devices$ 22$22$ •  The$developer(s)—see$A:acks$1,$2,$and$3.$ •  The$app$game$architect$or$director$ •  OnQteam$game$tester(s)$$ •  InQcompany$“dog$food”$testers$ •  Independent$test$players$$ •  Mass$beta$trials$ •  Not$a$tester—Finally,$consider$who$should$not$be$playing$ Note%on%roles:%During(the(tes%ng(effort(and(as(it(progresses,(don’t( forget(that(there(are(many(different(user(roles$ ROLES$TO$PLAY$IN$THE$GAME$ (ANY$MANY$OTHER$APPS)$
  • 14. Jon$Hagar$Copy$right$2014$ So4ware$Test$A:acks$to$Break$Mobile$and$Embedded$Devices$ 23$23$ •  Refine$checklist$to$context$scope$ •  Define$a$role$$ •  Watch$what$is$happening$with$this$role$ •  Define$a$usage$(scenario$or$set$of$funcJons$to$Play$the$game)$ •  Guided$exploraJons$or$ad$hoc$ •  Stress,$unusual$$cases,$explore$opJons$ •  Capture$understanding,$risk,$observaJons,$etc.$ •  Checklist$(watch$for$confusion)$ •  Run$Exploratory$A:ack$ •  Learn$ •  ReQplanQdesign$ •  Watch$for$Bias$ •  Switch$testers$ •  Repeat$ $ $ $ GAME$ATTACK$PATTERN$ Jon$Hagar$Copy$right$2014$ So4ware$Test$A:acks$to$Break$Mobile$and$Embedded$Devices$ 24$24$ 6:$BREAKING$SOFTWARE$WITH$HARDWARE$ AND$SYSTEM$OPERATION$(9)$ •  Classic$subQa:ack$example$to$consider$is$checking$ba:ery$ power$impacts$ •  Not$Common$to$IT/PC$tesJng$ •  Large$impacts$to$users$(if$ba:ery$is$drained)$ •  Relates$to$hardware$and$basic$operaJon$acJviJes$$ •  Requires$systems$thinking$ •  May$require$use$of$specialized$test$environments$and$ support$test$tools$
  • 15. Jon$Hagar$Copy$right$2014$ So4ware$Test$A:acks$to$Break$Mobile$and$Embedded$Devices$ 25$25$ DOCUMENTING$YOUR$TEST$CONDITIONS$ FOR$THE$CHARGING$OF$BATTERY$TEST$ Credit$to:$Jean$Ann$Harrison$$2013$ Jon$Hagar$Copy$right$2014$ So4ware$Test$A:acks$to$Break$Mobile$and$Embedded$Devices$ 26$26$ 5:$ATTACKING$WITH$SIMULATION$(AND$EMULATION)$(17)$ •  TesJng$with$real$hardware$is$advised,$but$ •  Has$limitaJons$ •  Can$require$a$lot$of$equipment$ •  You$need$the$hardware$$ •  FragmentaJon$ •  Many$mobile$people$test$using$simulators$and/ or$emulators$
  • 16. Jon$Hagar$Copy$right$2014$ So4ware$Test$A:acks$to$Break$Mobile$and$Embedded$Devices$ 27$27$ PROS$AND$CON$(SAMPLING)$OF$SIMULATION$AND$ EMULATION$ •  Pro$ •  Can$start$early$ •  Can$support$virtual$tesJng$ •  Can$support$automaJon$ •  Con$ •  Will$miss$some$kinds$of$bugs$ •  May$not$transfer$to$the$actual$hardware$ •  May$require$special$skills$and$efforts$to$set$up$ •  Modeling$(if$used)$can$be$very$tricky$ Jon$Hagar$Copy$right$2014$ So4ware$Test$A:acks$to$Break$Mobile$and$Embedded$Devices$ 28$28$ 4:$DEVELOPER$LEVEL$ATTACK:$WHITE$BOX$(1$&$2)$ •  Between$20QandQ30$percent$of$errors$can$be$found$by$developerQled$structural$ tesJng$ •  When$combined$with$#3,$a$testers$job$becomes$much$more$interesJng$ •  Industry$has$known$this$tesJng$from$the$beginning$and$yet$it$is$underused$ •  Priority$is$high$ •  “Official”$Tester$should$know$it,$advocate$for$it$and$even$“do$it”$someJmes$ •  Two$basic$A:acks$ •  Data$ •  Logic$
  • 17. Jon$Hagar$Copy$right$2014$ So4ware$Test$A:acks$to$Break$Mobile$and$Embedded$Devices$ 29$29$ 3:$STATIC$CODE$ANALYSIS$(SCA)$ATTACK$(3)$ •  This$acJvity$does$NOT$execute$the$code$ •  Can$be$done$“early”$in$the$lifecycle$ •  A$be:er$term$is$just$“analysis,”$but…………$ •  For$the$code,$we$use$a$tool$to$“analyze”$for$certain$types$of$errors$ •  Tools$are$commercial$although$some$open$source$tools$exist$ •  SCA$finds$the$“hard$to$find”$errors$ •  Many$test$teams$take$this$effort$over$since$programmers$“don’t$have$the$Jme”$ •  Issues:$ •  False$posiJves$ •  When$to$do$ •  When$to$repeat$ •  Efforts$can$(should)$$include$analyzing$models,$requirements,$and$other$arJfacts$ Jon$Hagar$Copy$right$2014$ So4ware$Test$A:acks$to$Break$Mobile$and$Embedded$Devices$ 30$30$ 1$&$2:$ATTACK$SECURITY$$ •  Apply$when$the$device$is$mobile$and$has$ •  Account$numbers$ •  UserQids$and$passwords$ •  LocaJon$tags$ •  Restricted$data$$ •  $Current$$authenJcaJon$approaches$in$use$on$mobile$devices$ •  ServerQbased$ •  Registry$(user/password)$ •  LocaJon$or$deviceQbased$ •  ProfileQbased$ PRIVAC Y
  • 18. Jon$Hagar$Copy$right$2014$ So4ware$Test$A:acks$to$Break$Mobile$and$Embedded$Devices$ 31$31$ THE$CURRENT$SECURITY$SITUATION$ •  Mobile$systems$are$highly$integrated$hardware–so4ware–system$ soluJons$which:$ •  Must$be$highly$trustworthy$since$they$handle$sensiJve$data$$ •  O4en$perform$criJcal$tasks$ •  Security$holes$and$problems$abound$ •  Coverity$Scan$2010$Open$Source$Integrity$Report$Q$Android$ •  staJc$analysis$test$a:ack$found$0.47$defects$per$1,000$SLOC$$ •  359$defects$in$total,$88$of$which$were$considered$“high$risk”$in$ the$security$domain$ •  OS#hole#Andriod#with#Angry#Birds#$(researchers$Jon$Oberheide$and$Zach$ Lanier)$ •  Robots$and$Drones$rumored$to$be$a:acked$ •  Cars$$and$medical$devices$being$hacked$ Jon$Hagar$Copy$right$2014$ So4ware$Test$A:acks$to$Break$Mobile$and$Embedded$Devices$ 32$32$ EMBEDDED/MOBILE$SECURITY$CONCERNS$ •  Fraud$–$IdenJty$ •  Worms,$virus,$etc.$ •  Fault$injecJon$ •  Processing$on$the$run$ •  Hacks$impact$ •  Power$ •  Memory$ •  CPU$usage$ •  Eavesdropping$–$yes$everyone$can$hear$you$ •  Hijacking$ •  ClickQjacking$ •  Voice/Screen$ •  Physical$Hacks$ •  File$snooping$ •  Lost$phone$
  • 19. Jon$Hagar$Copy$right$2014$ So4ware$Test$A:acks$to$Break$Mobile$and$Embedded$Devices$ 33$33$ SECURITY$ATTACKS$$$ (ONLY$A$STARTING$POINT)$ •  A:ack$28$PenetraJon$A:ack$Test $$ •  A:ack$28.1$PenetraJon$Sub–A:acks:$AuthenJcaJon$—$Password$A:ack $$ •  A:ack$28.2$Sub–A:ack$Fuzz$Test$$ •  A:ack$29:$InformaJon$The4—Stealing$Device$Data $$ •  A:ack$29.1$Sub$A:ack$–IdenJty$Social$Engineering $$ •  A:ack$30:$Spoofing$A:acks$$ •  A:ack$30.1$LocaJon$and/or$User$Profile$Spoof$Sub–A:ack$ •  A:ack$30.2$GPS$Spoof$Sub–A:ack $$ •  A:ack$31:$A:acking$Viruses$on$the$Run$in$Factories$or$PLCs$ Jon$Hagar$Copy$right$2014$ So4ware$Test$A:acks$to$Break$Mobile$and$Embedded$Devices$ 34$34$ WARNINGS$WHEN$CONDUCTING$ SECURITY$ATTACKS$ !  Security$a:acks$must$be$done$with$the$knowledge$and$approval$of$owners$of$ the$system$and$so4ware$ !  Severe$legal$implicaJons$exist$in$this$area$ !  Many$of$these$a:acks$must$be$done$in$a$lab$(sandbox)$ !  In$these$a:acks,$I$tell$you$conceptually$how$to$“drive$a$car$very$fast$(150$miles$ an$hour)$but$there$are$places$to$do$this$with$a$car$legally$(a$race$track)$and$ places$where$you$will$get$a$Jcket$(most$public$streets)”$ !  Be$forewarned$Q$Do$not$a:ack$you$favorite$app$on$your$phone$or$connected$ server$without$the$right$permissions$due$to$the$legal$implicaJons$
  • 20. Jon$Hagar$Copy$right$2014$ So4ware$Test$A:acks$to$Break$Mobile$and$Embedded$Devices$ 35$35$ WRAP$UP$ •  I$gave$my$top$10,$but$your$a:acks$can$and$will$be$different$ •  Understanding$your$local$context$and$error$pa:erns$is$important$$ (one$size$does$NOT$fit$all)$ •  A:acks$are$pa:erns…you$sJll$must$THINK$and$tailor$ $ Jon$Hagar$Copy$right$2014$ So4ware$Test$A:acks$to$Break$Mobile$and$Embedded$Devices$ 36$36$ MOBILE$ATTACK$CLASSIFICATION$ •  Developer$A:acks$(unit/code$tesJng)$$ •  Control$System$A:acks$$ •  HardwareQSo4ware$A:acks $$ •  Mobile$and$Embedded$So4ware$Domain$A:acks$$ •  Time$A:acks$(Performance) $$ •  Human$User$Interface$A:acks$$$ •  Smart$and/or$Mobile$Phone$FuncJonal$App$A:acks $$ •  Mobile/Embedded$Security$A:acks $$ •  Generic$A:acks$$ •  FuncJonal,$mind$mapping,$and$combinatorial$tests$
  • 21. Jon$Hagar$Copy$right$2014$ So4ware$Test$A:acks$to$Break$Mobile$and$Embedded$Devices$ 37$ MORE$ATTACKS$$ (FROM$SOFTWARE$TEST$ATTACKS$TO$BREAK$MOBILE$AND$EMBEDDED$ DEVICES)$ •  A:ack$1:$StaJc$Code$Analysis $$ •  A:ack$2:$Finding$White–Box$Data$ComputaJon$Bugs$$ •  A:ack$3:$White–Box$Structural$Logic$Flow$Coverage$ •  A:ack$4:$Finding$Hardware–System$Unhandled$Uses$in$So4ware$ •  A:ack$5:$$HwQSw$and$SwQHw$signal$Interface$Bugs$ •  A:ack$6:$Long$DuraJon$Control$A:ack$Runs $$ •  A:ack$7:$$Breaking$So4ware$Logic$and/or$Control$Laws$ •  A:ack$8:$Forcing$the$Unusual$Bug$Cases $$ •  A:ack$9$Breaking$So4ware$with$Hardware$and$System$OperaJons$ •  9.1$Sub–A:ack:$Breaking$Ba:ery$Power $$ •  A:ack$10:$Finding$Bugs$in$Hardware–So4ware$CommunicaJons $$ •  A:ack$11:$Breaking$So4ware$Error$Recovery $$ •  A:ack$12:$Interface$and$IntegraJon$TesJng $$ •  12.1$Sub–A:ack:$ConfiguraJon$IntegraJon$EvaluaJon $$ •  A:ack$13:$Finding$Problems$in$So4ware–System$Fault$Tolerance$ •  A:ack$14:$Breaking$Digital$So4ware$CommunicaJons $$ •  A:ack$15:$Finding$Bugs$in$the$Data $$ •  A:ack$16:$Bugs$in$System–So4ware$ComputaJon $$ •  A:ack$17:$$Using$SimulaJon$and$SJmulaJon$to$Drive$So4ware$A:acks$ •  A:ack$18:$Bugs$in$Timing$Interrupts$and$Priority$Inversion$ •  A:ack$19:$Finding$Time$Related$Bugs $$ •  A:ack$20:$Time$Related$Scenarios,$Stories$and$Tours $$ •  A:ack$21:$Performance$TesJng$IntroducJon $$ •  A:ack$22:$Finding$SupporJng$(User)$DocumentaJon$Problems$ •  Sub–A:ack$22.1:$$Confirming$Install–ability $$ •  A:ack$23:$Finding$Missing$or$Wrong$Alarms $$ •  A:ack$24:$Finding$Bugs$in$Help$Files $$ •  A:ack$25:$Finding$Bugs$in$Apps $$ •  A:ack$26:$TesJng$Mobile$and$Embedded$Games $$ •  A:ack$27:$A:acking$App–Cloud$Dependencies $$ •  A:ack$28$PenetraJon$A:ack$Test $$ •  A:ack$28.1$PenetraJon$Sub–A:acks:$AuthenJcaJon$—$Password$A:ack $$ •  A:ack$28.2$Sub–A:ack$Fuzz$Test$$ •  A:ack$29:$InformaJon$The4—Stealing$Device$Data $$ •  A:ack$29.1$Sub$A:ack$–IdenJty$Social$Engineering $$ •  A:ack$30:$Spoofing$A:acks $$ •  A:ack$30.1$LocaJon$and/or$User$Profile$Spoof$Sub–A:ack$ •  A:ack$30.2$GPS$Spoof$Sub–A:ack $$ •  A:ack$31:$A:acking$Viruses$on$the$Run$in$Factories$or$PLCs$ •  A:ack$32:$Using$Combinatorial$Tests $$ •  A:ack$33:$A:acking$FuncJonal$Bugs $$ Jon$Hagar$Copy$right$2014$ So4ware$Test$A:acks$to$Break$Mobile$and$Embedded$Devices$ 38$38$ SUMMARY:$THANK$YOU$(IDEAS$USED$FROM)$ •  James$Whi:aker$(a:acks)$ •  Elisabeth$Hendrickson$(simulaJons)$ •  Lee$Copeland$(techniques)$ •  Brian$Merrick$(tesJng)$ •  James$Bach$(exploratory$&$tours)$ •  Cem$Kaner$$(test$thinking)$ •  Many$teachers$ •  GeneraJons$past$and$future$ •  Books,$references,$etc.$
  • 22. Jon$Hagar$Copy$right$2014$ So4ware$Test$A:acks$to$Break$Mobile$and$Embedded$Devices$ 39$39$ BOOK$LIST$(MY$FAVORITES)$ •  “So;ware(Test(A?acks(to(Break(Mobile(and(Embedded(Devices”(( –(Jon(Hagar,(to(be(published(in(2013( •  “How$to$Break$So4ware”$James$Whi:aker,$2003$ •  And$his$other$“How$To$Break…”$books$ •  “TesJng$Embedded$So4ware”$Broeckman$and$Notenboom,$2003$ •  “A$PracJJoner’s$Guide$to$So4ware$Test$Design”$Copeland,$2004$ •  “A$PracJJoner’s$Handbook$for$RealQTime$Analysis”$Klein$et.$al.,$1993$ •  “Computer$Related$Risks”,$Neumann,$1995$ •  “Safeware:$System$Safety$and$Computers”,$Leveson,$1995$ •  Honorable$menJons:$ •  “Embedded$System$and$So4ware$ValidaJon”$Roychoudhury,$2009$ •  “Systems$TesJng$with$an$A}tude”$Petschenik$$2005$ •  “So4ware$System$TesJng$and$Quality$Assurance”$Beizer,$1987$ •  “TesJng$Computer$So4ware”$Kaner$et.$al.,$1988$ •  “SystemaJc$So4ware$TesJng”$Craig$&$Jaskiel,$2001$ •  “Managing$the$TesJng$Process”$Black,$2002$ Jon$Hagar$Copy$right$2014$ So4ware$Test$A:acks$to$Break$Mobile$and$Embedded$Devices$ 40$40$ MORE RESOURCES •  www.sJckyminds.com$–$CollecJon$of$test$info$ •  www.embedded.com$–$info$on$a:acks$ •  www.sqaforums.com$Q$Mobile$Devices,$Mobile$Apps$Q$Embedded$Systems$ TesJng$forum$ $ •  AssociaJon$of$So4ware$TesJng$ –  BBST$Classes$h:p://www.tesJngeducaJon.org/BBST/$ •  Your$favorite$search$engine$ •  Our$web$sites$and$blogs$(listed$on$front$page)$