SlideShare a Scribd company logo
1 of 22
Download to read offline
!!
T4
Mobile!Testing!
10/16/2014!9:45:00!AM!
!
Top Ten Attacks to Break
Mobile Apps
!
Presented by:
Jon Hagar
Grand Software Testing
!
!
!
Brought(to(you(by:(
(
(
(
340!Corporate!Way,!Suite!300,!Orange!Park,!FL!32073!
888G268G8770!H!904G278G0524!H!sqeinfo@sqe.com!H!www.sqe.com
Jon Hagar
Grand Software Testing
Jon Hagar is an independent consultant working in software product integrity,
testing, verification, and validation. Jon publishes regularly on testing, including
the book Software Test Attacks to Break Mobile and Embedded Devices
(breakingembeddedsoftware.com). For more than thirty years, he has worked in
software engineering, particularly testing, supporting projects which include
control systems (avionics and auto), spacecraft, mobile-smart devices, and
attack testing of smart phones. Jon is an editor for ISO, IEEE, and OMG
standards.
	
  
	
  
	
  
	
  
	
  
	
  
	
  
	
  
	
  
	
  
	
  
	
  
	
  
	
  
	
  
	
  
	
  
	
  
	
  
	
  
Jon$Hagar$Copy$right$2014$ So4ware$Test$A:acks$to$Break$Mobile$and$Embedded$Devices$ 1$
TOP$10$SOFTWARE$TEST$ATTACKS$
TO$BREAK$MOBILE$SOFTWARE$
STARWEST$2014$
Jon$Hagar$
embedded@ecentral.com$
jon.d.hagar@gmail.com$
Grand$So4ware$TesJng$
Web:$
h:p://breakingembeddedso4ware.wordpress.com/$
Jon$Hagar$Copy$right$2014$ So4ware$Test$A:acks$to$Break$Mobile$and$Embedded$Devices$ 2$2$
AGENDA$
•  DefiniJons$for$this$session$
•  RiskQbased$tesJng$concepts$for$mobile$
•  Exploratory$tesJng$concepts$for$$mobile$
•  My$top$10$Mobile$So4ware$(app)$a:acks$
•  Wrap$up$
Jon$Hagar$Copy$right$2014$ So4ware$Test$A:acks$to$Break$Mobile$and$Embedded$Devices$ 3$3$
$
MOBILE,$SMART,$AND$HANDHELD$
•  As$the$names$implies,$these$are$devices—small,$held$in$the$hand,$o4en$
connected$to$communicaJon$networks,$including$
•  Cell$and$smart$phones$–$apps$$
•  Tablets$
•  Medical$devices$
•  Typically$have:$
•  Many$of$the$problems$of$classic$“embedded”$systems$
•  The$power$of$PCs/IT$
•  More$user$interface$(UI)$than$classic$embedded$systems$
•  (RelaJvely)$Fast$updates$
•  Mobile$devices$are$“evolving”$with$more$power,$resources,$apps,$etc.$$
•  Mobile$is$the$“hot”$area$of$computers$and$so4ware$currently$
•  TesJng$rules$and$concepts$are$“evolving”$
STARWEST$
Jon$Hagar$Copy$right$2014$ So4ware$Test$A:acks$to$Break$Mobile$and$Embedded$Devices$ 4$4$
TYPES$OF$MOBILE$APPS$
•  NaJve$ApplicaJons$
•  Local$to$device$
•  Hybrid$ApplicaJons$
•  Local$to$device$but$
interacts$w/internet$
•  Web$ApplicaJons$
•  Not$local$to$device.$
All$interacJons$on$
internet$
STARWEST$
Jon$Hagar$Copy$right$2014$ So4ware$Test$A:acks$to$Break$Mobile$and$Embedded$Devices$ 5$5$
MOBILE$TESTING$DEFINITIONS$
•  Mobile$ApplicaJon$TesJng$is$tesJng$the$applicaJon$in$a$support$
environment$or$on$a$mobile$device$
•  System$Level$Mobile$Device$TesJng$is$tesJng$the$hardware$and$operaJng$
system$
•  Does$the$OperaJng$System$install?$$$
•  Does$the$device$power$on?$Do$the$LED$lights$work$as$expected?$$$
•  Does$the$ba:ery$charge$when$the$AC$adapter$is$plugged$into$the$device?$
•  Mobile$Phone$TesJng$should$have$some$different$approaches$to$tesJng$
•  Mobile$System$TesJng$incorporates$tesJng$more$than$one$applicaJon$and$
can$combine$hardware,$so4ware,$firmware,$along$with$other$applicaJons$
•  Mobile$TesJng$–$can/should$be$all$of$the$above$
$
Be$clear$when$using$this$terminology.$If$you$are$only$tesJng$apps$on$mobile$phones,$
then$state$“mobile$apps$tesJng.”$Use$mobile$tesJng$when$you$are$tesJng$mobile$
websites,$mobile$hybrid$apps,$mobile$hardware,$etc.$$
STARWEST$$
Jon$Hagar$Copy$right$2014$ So4ware$Test$A:acks$to$Break$Mobile$and$Embedded$Devices$ 6$6$
DEFINING$SKILL$SET$FOR$
THE$MOBILE$TESTER$
•  Some$exposure$or$knowledge$about$products$from$the$domain$in$which$you$
are$tesJng:$$aerospace,$medical,$automobile$manufacturing,$airplanes,$
factory$systems,$roboJcs,$regulated$environments,$etc.$
•  Some$knowledge$of:$hard$sciences:$$math,$physics,$electronics,$engineering,$
etc.$for$logical$thought$processes$
•  So4ware$sciences:$$psychology,$philosophy,$sociology,$human$factors$(human$
machine$interface)$for$creaJve$&$conceptual$thought$processes$
•  Tester$skill$
•  Planning,$design$techniques,$pa:erns$of$errors,$intuiJon,$criJcal$thinking,$“so4$skills,”$$
communicaJon,$observaJon,$and$mental$models$[ISTQB$and$AST$have$“lists”]$
Chapter$1$–$So4ware$Test$A:acks$to$Break$Mobile$&$Embedded$Devices$
STARWEST$
Jon$Hagar$Copy$right$2014$ So4ware$Test$A:acks$to$Break$Mobile$and$Embedded$Devices$ 7$7$
WHAT$IS$AN$ATTACK?$
•  A$pa:ern$(of$tesJng)$based$on$a$common$mode$of$failure$seen$
over$and$over$
•  Maybe$seen$as$a$negaJve,$when$it$is$really$a$posi%ve(
•  Goes$a4er$the$“bugs”$that$may$be$in$the$so4ware$
•  May$include$or$use$classic$test$techniques$and$test$concepts$
•  Lee$Copeland’s$book$on$test$design$
•  Many$other$good$books$
•  A$Pa:ern$(more$than$a$process),$which$must$be$modified$for$the$
context$at$hand,$to$do$the$tesJng$$
•  Testers$learn$these$in$a$domain$a4er$years$and$form$a$mental$
model$(most$good$testers$a:ack)$
STARWEST$
Jon$Hagar$Copy$right$2014$ So4ware$Test$A:acks$to$Break$Mobile$and$Embedded$Devices$ 8$8$
WHY$ATTACK?$
•  A:acking$your$so4ware$is$in$part,$$the$process$of$a:empJng$to$
demonstrate$a$system$(hardware,$firmware,$so4ware$and$operaJons)$$
does$not$meet$requirements,$funcJonal$and$nonQfuncJonal$objecJves$
•  Embedded/handheld$so4ware$tesJng$must$include$"the$
system"$(hardware,$so4ware,$operaJons,$users)$
•  A:acking$common$modes$of$failure,$especially$where$the$applicaJon$
is$engaged$and$visible$by$the$user.$
Attack your enemy with approaches to include:
Tools Levels
Attacks Techniques
STARWEST$
Jon$Hagar$Copy$right$2014$ So4ware$Test$A:acks$to$Break$Mobile$and$Embedded$Devices$ 9$9$
KINDS$OF$ATTACKS$
•  Whi:aker$offers$a$good$starJng$point$for$so4ware$
a:acks$in$general$that$can$be$applied$to$mobile:$
•  User$Interface$A:acks$
•  Data$and$ComputaJon$
•  File$System$Interface$
•  So4ware/OS$Interface$
•  Whi:aker’s$“How$to$Break$So4ware”$lists$23$a:acks$
•  Plus$he$has$other$books$on$a:acks,$security,$web,$exploratory,$and$tours$in$tesJng$
•  “So4ware$Test$A:acks$to$Break$Mobile$and$
Embedded$Devices”$lists$32$a:acks$and$8$sub$a:acks$
STARWEST$
Jon$Hagar$Copy$right$2014$ So4ware$Test$A:acks$to$Break$Mobile$and$Embedded$Devices$ 10$10$
MOBILE$RISK$AREAS$TO$CONSIDER$
•  There$are$many$risk$to$$consider,$but$you$cannot$test$everything$
•  Risk(s)$based$tesJng$$helps$$bound$the$test$scope$problem$
•  TesJng$is$about$providing$informaJon$and$understanding$
•  ExploraJon$gets$you$started$with$whatever$you$have$(or$don’t$
have)$
STARWEST$
Jon$Hagar$Copy$right$2014$ So4ware$Test$A:acks$to$Break$Mobile$and$Embedded$Devices$ 11$11$
SAMPLE$MOBILE$PRODUCT$RISKS$
TESTERS$SHOULD$CONSIDER$
•  Environment$and$input$factors$
•  Environment$–$heat,$noise,$sun,$water,$etc.$
•  Hardware$–$calibraJon,$uniqueness,$manufacturing,$etc.$
•  Electronics$–$noise,$power,$ba:eries,$etc.$
•  CommunicaJons$
•  Interfaces$types$
•  Hardware$
•  Human$$
•  Network$$
•  So4ware$
•  Output$—$noise$influences,$D2A,$representaJon,$etc.$
•  Complexity—use$/$size$of$the$system$$
STARWEST$
Jon$Hagar$Copy$right$2014$ So4ware$Test$A:acks$to$Break$Mobile$and$Embedded$Devices$ 12$12$
RISK=BASED$
TESTING$$
(ISO$29119)$
•  Address,$miJgate,$a:ack$and$reJre$product$risks$
•  PrioriJze$risks$Q$tests:$
•  PotenJal$problems$Q$$Consequences$and$effects$
•  Occurrences$–$likelihood$or$chance$of$happening$
•  Impacts$–$what$happens$
•  Take$consistent$acJon$from$the$beginning$(proposal)$
to$the$end$(reJrement)$of$the$product$or$lifecycle$
•  Risks$&$prioriJzing$should$dictate$the$test$a:acks$
STARWEST$
Jon$Hagar$Copy$right$2014$ So4ware$Test$A:acks$to$Break$Mobile$and$Embedded$Devices$ 13$13$
EXPLORATORY$TESTING$=$DEFINITION$
•  QuoJng$James$Bach:$$“The$plainest$definiJon$of$exploratory$
tesJng$is$test$design$and$test$execuJon$at$the$same$Jme.$This$is$
the$opposite$of$scripted$tesJng$(predefined$test$procedures,$
whether$manual$or$automated).$Exploratory$tests,$unlike$
scripted$tests,$are$not$defined$in$advance$and$carried$out$
precisely$according$to$plan.”$
h:p://www.saJsfice.com/arJcles/what_is_et.shtml$
STARWEST$
Jon$Hagar$Copy$right$2014$ So4ware$Test$A:acks$to$Break$Mobile$and$Embedded$Devices$ 14$14$
EXPLORATORY$TESTING$IN$MOBILE$
•  Rapid$feedback$
•  Learning$
•  Upfront$rapid$
learning$
•  A:acking$
•  Address$Risk$(s)$
•  Independent$assessment$
•  Target$a$defect$
•  Prototyping$
•  Need$info$
•  Test$beyond$the$
requirements$
STARWEST$
Jon$Hagar$Copy$right$2014$ So4ware$Test$A:acks$to$Break$Mobile$and$Embedded$Devices$ 15$15$
NUMBER$10:$FUNCTIONAL$ATTACK$(33)$
•  Have$an$outline$$or$charter$(top$level$plan$and/or$risk$list)$
•  Create$a$flip$chart,$notecard,$state$model,$mind$map$or$some$representaJon$of$
each$test$$task$$
•  No$“heavyweight$documentaJon$of$the$“test$case”$
•  See$Exploratory$Charter$(test$objecJve)$
•  Have$a$Target$concept$$or$charter$$(Risk,$A:ack,$Bug,$Learning,$…)$
•  VerificaJon$checking$of$requirements$(necessary$but$not$sufficient)$$
•  Have$a$schedule/Jme$box$(short$$test$cycles$=$Planning$to$report)$
•  Do$the$test$
•  Design$test$
•  Execute$test$$
•  Learn$about$the$product:$change$the$risk$list,$modify/add$tests,$and$so$on$
•  Repeat$as$needed$
STARWEST$
Jon$Hagar$Copy$right$2014$ So4ware$Test$A:acks$to$Break$Mobile$and$Embedded$Devices$ 16$16$
EXAMPLE$MIND$MAP$FROM$A$TRAVEL$APP$
Jon$Hagar$Copy$right$2014$ So4ware$Test$A:acks$to$Break$Mobile$and$Embedded$Devices$ 17$17$
•  Download$either$Twi:er$or$Facebook$onto$a$device$$
•  Start$either$downloaded$app$
•  From$another$device,$send$an$email$to$the$device’s$email$account$
•  Immediately$send$a$tweet$or$post$a$status$$
•  ConJnue$to$do$engage$Twi:er$or$Facebook$app$for$at$least$1$minute$
•  Record$email$noJficaJon$and$Jme$when$sent$and$received$
•  What$other$observaJons$occurred?$
9:$NOTIFICATION$TEST$ATTACK$(18)$
Jon$Hagar$Copy$right$2014$ So4ware$Test$A:acks$to$Break$Mobile$and$Embedded$Devices$ 18$18$
ATTACK$TEST$CASE$EXAMPLE:$$
INTERRUPTS$ON$MOBILE$PHONES$
•  Go$to$your$App$store$and$choose$an$applicaJon$to$download$
•  While$the$downloading$is$occurring,$call$the$mobile$phone$
•  Record$observaJons$with$the$download$
•  You$may$need$to$rely$on$observing$a$log$file$while$implemenJng$these$
tests$
•  If$it$fails,$what$kind$of$error$recovery$occurs?$Can$you$repeat$any$
errors?$
Jon$Hagar$Copy$right$2014$ So4ware$Test$A:acks$to$Break$Mobile$and$Embedded$Devices$ 19$19$
8:$ATTACK$SCENARIOS$(12)$
•  Tests$consider$usage,$operaJons,$interface$interacJons$and$
integraJons$$
•  Interface$points$include:$$hardware,$firmware,$so4ware,$data$
exchange,$network$communicaJon$and$combinaJons$
•  How$each$interface$point$integrates$with$another$interface$point$
•  Tests$include$how$the$applicaJon$is$used$endQtoQend$$
•  Tests$to$combine$how$the$enJre$system$interacts$as$well$as$how$
porJons$interact$with$one$another$and$depending$on$complexity$
•  Note:$ConfiguraJon$tests$with$regards$to$how$so4ware$behaves$based$
on$various$configuraJons$of$devices,$operaJng$systems$
Jon$Hagar$Copy$right$2014$ So4ware$Test$A:acks$to$Break$Mobile$and$Embedded$Devices$ 20$
IN$SCENARIO$TESTING$:$TIMING$SUBQ$ATTACK$
When$Time$interacts$with$the$so4ware,$events,$inputs,$
and$outputs,$here's$a$checklist$of$things$to$look$for$and$
consider$(where$bugs$lurk)$in$sequences/stories$
$
•  Order$problems$
•  Too$Long$$
•  Too$Fast$
•  Not$at$right$Jme$mark$or$point$$
•  Late$$
•  Late$or$early$
•  Early$$
•  Deadlocked$caused$by$a$race$
condiJon$(hard$to$find)$$
•  Extra$input$or$output$events$$
•  Missing$events$$
•  Wrong$input/output$within$events$$
Jon$Hagar$Copy$right$2014$ So4ware$Test$A:acks$to$Break$Mobile$and$Embedded$Devices$ 21$21$
•  When$to$apply$this$a:ack?$$…when$your$app/device$has$games$
•  What$faults$make$this$a:ack$successful?$$$…games$are$complex$
•  Who$conducts$this$a:ack?$$…see$chart$on$Roles$
•  Where$is$this$a:ack$conducted?$$…throughout$lifecycle$and$in$environments$
•  How$to$determine$if$the$a:ack$exposes$failures?$
•  Unhappy$“users”$
•  Bugs$found$
•  See$checklist$
7:$ATTACK$TESTING$MOBILE$GAMES$(26)$
Mobile Device Game Testing
(2 years ago gaming was 60% or more of Mobile App downloads)
Jon$Hagar$Copy$right$2014$ So4ware$Test$A:acks$to$Break$Mobile$and$Embedded$Devices$ 22$22$
•  The$developer(s)—see$A:acks$1,$2,$and$3.$
•  The$app$game$architect$or$director$
•  OnQteam$game$tester(s)$$
•  InQcompany$“dog$food”$testers$
•  Independent$test$players$$
•  Mass$beta$trials$
•  Not$a$tester—Finally,$consider$who$should$not$be$playing$
Note%on%roles:%During(the(tes%ng(effort(and(as(it(progresses,(don’t(
forget(that(there(are(many(different(user(roles$
ROLES$TO$PLAY$IN$THE$GAME$
(ANY$MANY$OTHER$APPS)$
Jon$Hagar$Copy$right$2014$ So4ware$Test$A:acks$to$Break$Mobile$and$Embedded$Devices$ 23$23$
•  Refine$checklist$to$context$scope$
•  Define$a$role$$
•  Watch$what$is$happening$with$this$role$
•  Define$a$usage$(scenario$or$set$of$funcJons$to$Play$the$game)$
•  Guided$exploraJons$or$ad$hoc$
•  Stress,$unusual$$cases,$explore$opJons$
•  Capture$understanding,$risk,$observaJons,$etc.$
•  Checklist$(watch$for$confusion)$
•  Run$Exploratory$A:ack$
•  Learn$
•  ReQplanQdesign$
•  Watch$for$Bias$
•  Switch$testers$
•  Repeat$
$
$
$
GAME$ATTACK$PATTERN$
Jon$Hagar$Copy$right$2014$ So4ware$Test$A:acks$to$Break$Mobile$and$Embedded$Devices$ 24$24$
6:$BREAKING$SOFTWARE$WITH$HARDWARE$
AND$SYSTEM$OPERATION$(9)$
•  Classic$subQa:ack$example$to$consider$is$checking$ba:ery$
power$impacts$
•  Not$Common$to$IT/PC$tesJng$
•  Large$impacts$to$users$(if$ba:ery$is$drained)$
•  Relates$to$hardware$and$basic$operaJon$acJviJes$$
•  Requires$systems$thinking$
•  May$require$use$of$specialized$test$environments$and$
support$test$tools$
Jon$Hagar$Copy$right$2014$ So4ware$Test$A:acks$to$Break$Mobile$and$Embedded$Devices$ 25$25$
DOCUMENTING$YOUR$TEST$CONDITIONS$
FOR$THE$CHARGING$OF$BATTERY$TEST$
Credit$to:$Jean$Ann$Harrison$$2013$
Jon$Hagar$Copy$right$2014$ So4ware$Test$A:acks$to$Break$Mobile$and$Embedded$Devices$ 26$26$
5:$ATTACKING$WITH$SIMULATION$(AND$EMULATION)$(17)$
•  TesJng$with$real$hardware$is$advised,$but$
•  Has$limitaJons$
•  Can$require$a$lot$of$equipment$
•  You$need$the$hardware$$
•  FragmentaJon$
•  Many$mobile$people$test$using$simulators$and/
or$emulators$
Jon$Hagar$Copy$right$2014$ So4ware$Test$A:acks$to$Break$Mobile$and$Embedded$Devices$ 27$27$
PROS$AND$CON$(SAMPLING)$OF$SIMULATION$AND$
EMULATION$
•  Pro$
•  Can$start$early$
•  Can$support$virtual$tesJng$
•  Can$support$automaJon$
•  Con$
•  Will$miss$some$kinds$of$bugs$
•  May$not$transfer$to$the$actual$hardware$
•  May$require$special$skills$and$efforts$to$set$up$
•  Modeling$(if$used)$can$be$very$tricky$
Jon$Hagar$Copy$right$2014$ So4ware$Test$A:acks$to$Break$Mobile$and$Embedded$Devices$ 28$28$
4:$DEVELOPER$LEVEL$ATTACK:$WHITE$BOX$(1$&$2)$
•  Between$20QandQ30$percent$of$errors$can$be$found$by$developerQled$structural$
tesJng$
•  When$combined$with$#3,$a$testers$job$becomes$much$more$interesJng$
•  Industry$has$known$this$tesJng$from$the$beginning$and$yet$it$is$underused$
•  Priority$is$high$
•  “Official”$Tester$should$know$it,$advocate$for$it$and$even$“do$it”$someJmes$
•  Two$basic$A:acks$
•  Data$
•  Logic$
Jon$Hagar$Copy$right$2014$ So4ware$Test$A:acks$to$Break$Mobile$and$Embedded$Devices$ 29$29$
3:$STATIC$CODE$ANALYSIS$(SCA)$ATTACK$(3)$
•  This$acJvity$does$NOT$execute$the$code$
•  Can$be$done$“early”$in$the$lifecycle$
•  A$be:er$term$is$just$“analysis,”$but…………$
•  For$the$code,$we$use$a$tool$to$“analyze”$for$certain$types$of$errors$
•  Tools$are$commercial$although$some$open$source$tools$exist$
•  SCA$finds$the$“hard$to$find”$errors$
•  Many$test$teams$take$this$effort$over$since$programmers$“don’t$have$the$Jme”$
•  Issues:$
•  False$posiJves$
•  When$to$do$
•  When$to$repeat$
•  Efforts$can$(should)$$include$analyzing$models,$requirements,$and$other$arJfacts$
Jon$Hagar$Copy$right$2014$ So4ware$Test$A:acks$to$Break$Mobile$and$Embedded$Devices$ 30$30$
1$&$2:$ATTACK$SECURITY$$
•  Apply$when$the$device$is$mobile$and$has$
•  Account$numbers$
•  UserQids$and$passwords$
•  LocaJon$tags$
•  Restricted$data$$
•  $Current$$authenJcaJon$approaches$in$use$on$mobile$devices$
•  ServerQbased$
•  Registry$(user/password)$
•  LocaJon$or$deviceQbased$
•  ProfileQbased$
PRIVAC
Y
Jon$Hagar$Copy$right$2014$ So4ware$Test$A:acks$to$Break$Mobile$and$Embedded$Devices$ 31$31$
THE$CURRENT$SECURITY$SITUATION$
•  Mobile$systems$are$highly$integrated$hardware–so4ware–system$
soluJons$which:$
•  Must$be$highly$trustworthy$since$they$handle$sensiJve$data$$
•  O4en$perform$criJcal$tasks$
•  Security$holes$and$problems$abound$
•  Coverity$Scan$2010$Open$Source$Integrity$Report$Q$Android$
•  staJc$analysis$test$a:ack$found$0.47$defects$per$1,000$SLOC$$
•  359$defects$in$total,$88$of$which$were$considered$“high$risk”$in$
the$security$domain$
•  OS#hole#Andriod#with#Angry#Birds#$(researchers$Jon$Oberheide$and$Zach$
Lanier)$
•  Robots$and$Drones$rumored$to$be$a:acked$
•  Cars$$and$medical$devices$being$hacked$
Jon$Hagar$Copy$right$2014$ So4ware$Test$A:acks$to$Break$Mobile$and$Embedded$Devices$ 32$32$
EMBEDDED/MOBILE$SECURITY$CONCERNS$
•  Fraud$–$IdenJty$
•  Worms,$virus,$etc.$
•  Fault$injecJon$
•  Processing$on$the$run$
•  Hacks$impact$
•  Power$
•  Memory$
•  CPU$usage$
•  Eavesdropping$–$yes$everyone$can$hear$you$
•  Hijacking$
•  ClickQjacking$
•  Voice/Screen$
•  Physical$Hacks$
•  File$snooping$
•  Lost$phone$
Jon$Hagar$Copy$right$2014$ So4ware$Test$A:acks$to$Break$Mobile$and$Embedded$Devices$ 33$33$
SECURITY$ATTACKS$$$
(ONLY$A$STARTING$POINT)$
•  A:ack$28$PenetraJon$A:ack$Test $$
•  A:ack$28.1$PenetraJon$Sub–A:acks:$AuthenJcaJon$—$Password$A:ack $$
•  A:ack$28.2$Sub–A:ack$Fuzz$Test$$
•  A:ack$29:$InformaJon$The4—Stealing$Device$Data $$
•  A:ack$29.1$Sub$A:ack$–IdenJty$Social$Engineering $$
•  A:ack$30:$Spoofing$A:acks$$
•  A:ack$30.1$LocaJon$and/or$User$Profile$Spoof$Sub–A:ack$
•  A:ack$30.2$GPS$Spoof$Sub–A:ack $$
•  A:ack$31:$A:acking$Viruses$on$the$Run$in$Factories$or$PLCs$
Jon$Hagar$Copy$right$2014$ So4ware$Test$A:acks$to$Break$Mobile$and$Embedded$Devices$ 34$34$
WARNINGS$WHEN$CONDUCTING$
SECURITY$ATTACKS$
!  Security$a:acks$must$be$done$with$the$knowledge$and$approval$of$owners$of$
the$system$and$so4ware$
!  Severe$legal$implicaJons$exist$in$this$area$
!  Many$of$these$a:acks$must$be$done$in$a$lab$(sandbox)$
!  In$these$a:acks,$I$tell$you$conceptually$how$to$“drive$a$car$very$fast$(150$miles$
an$hour)$but$there$are$places$to$do$this$with$a$car$legally$(a$race$track)$and$
places$where$you$will$get$a$Jcket$(most$public$streets)”$
!  Be$forewarned$Q$Do$not$a:ack$you$favorite$app$on$your$phone$or$connected$
server$without$the$right$permissions$due$to$the$legal$implicaJons$
Jon$Hagar$Copy$right$2014$ So4ware$Test$A:acks$to$Break$Mobile$and$Embedded$Devices$ 35$35$
WRAP$UP$
•  I$gave$my$top$10,$but$your$a:acks$can$and$will$be$different$
•  Understanding$your$local$context$and$error$pa:erns$is$important$$
(one$size$does$NOT$fit$all)$
•  A:acks$are$pa:erns…you$sJll$must$THINK$and$tailor$
$
Jon$Hagar$Copy$right$2014$ So4ware$Test$A:acks$to$Break$Mobile$and$Embedded$Devices$ 36$36$
MOBILE$ATTACK$CLASSIFICATION$
•  Developer$A:acks$(unit/code$tesJng)$$
•  Control$System$A:acks$$
•  HardwareQSo4ware$A:acks $$
•  Mobile$and$Embedded$So4ware$Domain$A:acks$$
•  Time$A:acks$(Performance) $$
•  Human$User$Interface$A:acks$$$
•  Smart$and/or$Mobile$Phone$FuncJonal$App$A:acks $$
•  Mobile/Embedded$Security$A:acks $$
•  Generic$A:acks$$
•  FuncJonal,$mind$mapping,$and$combinatorial$tests$
Jon$Hagar$Copy$right$2014$ So4ware$Test$A:acks$to$Break$Mobile$and$Embedded$Devices$ 37$
MORE$ATTACKS$$
(FROM$SOFTWARE$TEST$ATTACKS$TO$BREAK$MOBILE$AND$EMBEDDED$
DEVICES)$
•  A:ack$1:$StaJc$Code$Analysis $$
•  A:ack$2:$Finding$White–Box$Data$ComputaJon$Bugs$$
•  A:ack$3:$White–Box$Structural$Logic$Flow$Coverage$
•  A:ack$4:$Finding$Hardware–System$Unhandled$Uses$in$So4ware$
•  A:ack$5:$$HwQSw$and$SwQHw$signal$Interface$Bugs$
•  A:ack$6:$Long$DuraJon$Control$A:ack$Runs $$
•  A:ack$7:$$Breaking$So4ware$Logic$and/or$Control$Laws$
•  A:ack$8:$Forcing$the$Unusual$Bug$Cases $$
•  A:ack$9$Breaking$So4ware$with$Hardware$and$System$OperaJons$
•  9.1$Sub–A:ack:$Breaking$Ba:ery$Power $$
•  A:ack$10:$Finding$Bugs$in$Hardware–So4ware$CommunicaJons $$
•  A:ack$11:$Breaking$So4ware$Error$Recovery $$
•  A:ack$12:$Interface$and$IntegraJon$TesJng $$
•  12.1$Sub–A:ack:$ConfiguraJon$IntegraJon$EvaluaJon $$
•  A:ack$13:$Finding$Problems$in$So4ware–System$Fault$Tolerance$
•  A:ack$14:$Breaking$Digital$So4ware$CommunicaJons $$
•  A:ack$15:$Finding$Bugs$in$the$Data $$
•  A:ack$16:$Bugs$in$System–So4ware$ComputaJon $$
•  A:ack$17:$$Using$SimulaJon$and$SJmulaJon$to$Drive$So4ware$A:acks$
•  A:ack$18:$Bugs$in$Timing$Interrupts$and$Priority$Inversion$
•  A:ack$19:$Finding$Time$Related$Bugs $$
•  A:ack$20:$Time$Related$Scenarios,$Stories$and$Tours $$
•  A:ack$21:$Performance$TesJng$IntroducJon $$
•  A:ack$22:$Finding$SupporJng$(User)$DocumentaJon$Problems$
•  Sub–A:ack$22.1:$$Confirming$Install–ability $$
•  A:ack$23:$Finding$Missing$or$Wrong$Alarms $$
•  A:ack$24:$Finding$Bugs$in$Help$Files $$
•  A:ack$25:$Finding$Bugs$in$Apps $$
•  A:ack$26:$TesJng$Mobile$and$Embedded$Games $$
•  A:ack$27:$A:acking$App–Cloud$Dependencies $$
•  A:ack$28$PenetraJon$A:ack$Test $$
•  A:ack$28.1$PenetraJon$Sub–A:acks:$AuthenJcaJon$—$Password$A:ack $$
•  A:ack$28.2$Sub–A:ack$Fuzz$Test$$
•  A:ack$29:$InformaJon$The4—Stealing$Device$Data $$
•  A:ack$29.1$Sub$A:ack$–IdenJty$Social$Engineering $$
•  A:ack$30:$Spoofing$A:acks $$
•  A:ack$30.1$LocaJon$and/or$User$Profile$Spoof$Sub–A:ack$
•  A:ack$30.2$GPS$Spoof$Sub–A:ack $$
•  A:ack$31:$A:acking$Viruses$on$the$Run$in$Factories$or$PLCs$
•  A:ack$32:$Using$Combinatorial$Tests $$
•  A:ack$33:$A:acking$FuncJonal$Bugs $$
Jon$Hagar$Copy$right$2014$ So4ware$Test$A:acks$to$Break$Mobile$and$Embedded$Devices$ 38$38$
SUMMARY:$THANK$YOU$(IDEAS$USED$FROM)$
•  James$Whi:aker$(a:acks)$
•  Elisabeth$Hendrickson$(simulaJons)$
•  Lee$Copeland$(techniques)$
•  Brian$Merrick$(tesJng)$
•  James$Bach$(exploratory$&$tours)$
•  Cem$Kaner$$(test$thinking)$
•  Many$teachers$
•  GeneraJons$past$and$future$
•  Books,$references,$etc.$
Jon$Hagar$Copy$right$2014$ So4ware$Test$A:acks$to$Break$Mobile$and$Embedded$Devices$ 39$39$
BOOK$LIST$(MY$FAVORITES)$
•  “So;ware(Test(A?acks(to(Break(Mobile(and(Embedded(Devices”((
–(Jon(Hagar,(to(be(published(in(2013(
•  “How$to$Break$So4ware”$James$Whi:aker,$2003$
•  And$his$other$“How$To$Break…”$books$
•  “TesJng$Embedded$So4ware”$Broeckman$and$Notenboom,$2003$
•  “A$PracJJoner’s$Guide$to$So4ware$Test$Design”$Copeland,$2004$
•  “A$PracJJoner’s$Handbook$for$RealQTime$Analysis”$Klein$et.$al.,$1993$
•  “Computer$Related$Risks”,$Neumann,$1995$
•  “Safeware:$System$Safety$and$Computers”,$Leveson,$1995$
•  Honorable$menJons:$
•  “Embedded$System$and$So4ware$ValidaJon”$Roychoudhury,$2009$
•  “Systems$TesJng$with$an$A}tude”$Petschenik$$2005$
•  “So4ware$System$TesJng$and$Quality$Assurance”$Beizer,$1987$
•  “TesJng$Computer$So4ware”$Kaner$et.$al.,$1988$
•  “SystemaJc$So4ware$TesJng”$Craig$&$Jaskiel,$2001$
•  “Managing$the$TesJng$Process”$Black,$2002$
Jon$Hagar$Copy$right$2014$ So4ware$Test$A:acks$to$Break$Mobile$and$Embedded$Devices$ 40$40$
MORE RESOURCES
•  www.sJckyminds.com$–$CollecJon$of$test$info$
•  www.embedded.com$–$info$on$a:acks$
•  www.sqaforums.com$Q$Mobile$Devices,$Mobile$Apps$Q$Embedded$Systems$
TesJng$forum$
$
•  AssociaJon$of$So4ware$TesJng$
–  BBST$Classes$h:p://www.tesJngeducaJon.org/BBST/$
•  Your$favorite$search$engine$
•  Our$web$sites$and$blogs$(listed$on$front$page)$

More Related Content

Viewers also liked

Pairwise Testing Explained
Pairwise Testing ExplainedPairwise Testing Explained
Pairwise Testing ExplainedTechWell
 
Introducing Keyword-Driven Test Automation
Introducing Keyword-Driven Test AutomationIntroducing Keyword-Driven Test Automation
Introducing Keyword-Driven Test AutomationTechWell
 
Test Estimation in Practice
Test Estimation in PracticeTest Estimation in Practice
Test Estimation in PracticeTechWell
 
Introduction to Selenium and WebDriver
Introduction to Selenium and WebDriverIntroduction to Selenium and WebDriver
Introduction to Selenium and WebDriverTechWell
 
Simplify Project and Portfolio Planning with “Real Options”
Simplify Project and Portfolio Planning with “Real Options”Simplify Project and Portfolio Planning with “Real Options”
Simplify Project and Portfolio Planning with “Real Options”TechWell
 
Using DevOps to Improve Software Quality in the Cloud
Using DevOps to Improve Software Quality in the CloudUsing DevOps to Improve Software Quality in the Cloud
Using DevOps to Improve Software Quality in the CloudTechWell
 
Test Automation Patterns: Issues and Solutions
Test Automation Patterns: Issues and SolutionsTest Automation Patterns: Issues and Solutions
Test Automation Patterns: Issues and SolutionsTechWell
 
Software Managers: Their Place in Agile
Software Managers: Their Place in AgileSoftware Managers: Their Place in Agile
Software Managers: Their Place in AgileTechWell
 
End-to-End Testing with the Heuristic Software Test Model
End-to-End Testing with the Heuristic Software Test ModelEnd-to-End Testing with the Heuristic Software Test Model
End-to-End Testing with the Heuristic Software Test ModelTechWell
 

Viewers also liked (9)

Pairwise Testing Explained
Pairwise Testing ExplainedPairwise Testing Explained
Pairwise Testing Explained
 
Introducing Keyword-Driven Test Automation
Introducing Keyword-Driven Test AutomationIntroducing Keyword-Driven Test Automation
Introducing Keyword-Driven Test Automation
 
Test Estimation in Practice
Test Estimation in PracticeTest Estimation in Practice
Test Estimation in Practice
 
Introduction to Selenium and WebDriver
Introduction to Selenium and WebDriverIntroduction to Selenium and WebDriver
Introduction to Selenium and WebDriver
 
Simplify Project and Portfolio Planning with “Real Options”
Simplify Project and Portfolio Planning with “Real Options”Simplify Project and Portfolio Planning with “Real Options”
Simplify Project and Portfolio Planning with “Real Options”
 
Using DevOps to Improve Software Quality in the Cloud
Using DevOps to Improve Software Quality in the CloudUsing DevOps to Improve Software Quality in the Cloud
Using DevOps to Improve Software Quality in the Cloud
 
Test Automation Patterns: Issues and Solutions
Test Automation Patterns: Issues and SolutionsTest Automation Patterns: Issues and Solutions
Test Automation Patterns: Issues and Solutions
 
Software Managers: Their Place in Agile
Software Managers: Their Place in AgileSoftware Managers: Their Place in Agile
Software Managers: Their Place in Agile
 
End-to-End Testing with the Heuristic Software Test Model
End-to-End Testing with the Heuristic Software Test ModelEnd-to-End Testing with the Heuristic Software Test Model
End-to-End Testing with the Heuristic Software Test Model
 

Similar to Top Ten Attacks to Break Mobile Apps

Soft Skills You Need Are Not Always Taught in Class
Soft Skills You Need Are Not Always Taught in ClassSoft Skills You Need Are Not Always Taught in Class
Soft Skills You Need Are Not Always Taught in ClassTechWell
 
IoT Software Testing Challenges: The IoT World Is Really Different
IoT Software Testing Challenges: The IoT World Is Really DifferentIoT Software Testing Challenges: The IoT World Is Really Different
IoT Software Testing Challenges: The IoT World Is Really DifferentTechWell
 
Top IOT Testing Challenges Webinar with Jon Hagar
Top IOT Testing Challenges Webinar with Jon HagarTop IOT Testing Challenges Webinar with Jon Hagar
Top IOT Testing Challenges Webinar with Jon HagarXBOSoft
 
JoeHelfrichResume0701
JoeHelfrichResume0701JoeHelfrichResume0701
JoeHelfrichResume0701Joe Helfrich
 
IoT Software Testing Challenges: The IoT World Is Really Different
IoT Software Testing Challenges: The IoT World Is Really DifferentIoT Software Testing Challenges: The IoT World Is Really Different
IoT Software Testing Challenges: The IoT World Is Really DifferentTechWell
 
Implement Combinatorial Test Patterns for Better Mobile and IoT Testing
Implement Combinatorial Test Patterns for Better Mobile and IoT TestingImplement Combinatorial Test Patterns for Better Mobile and IoT Testing
Implement Combinatorial Test Patterns for Better Mobile and IoT TestingJosiah Renaudin
 
XBOSoft Mobile Security Webinar with Jon D. Hagar
XBOSoft Mobile Security Webinar with Jon D. HagarXBOSoft Mobile Security Webinar with Jon D. Hagar
XBOSoft Mobile Security Webinar with Jon D. HagarXBOSoft
 
Java certification course in Coimbatore|Java Learning Center Coimbatore|Java ...
Java certification course in Coimbatore|Java Learning Center Coimbatore|Java ...Java certification course in Coimbatore|Java Learning Center Coimbatore|Java ...
Java certification course in Coimbatore|Java Learning Center Coimbatore|Java ...sathyancegon
 
How to Create a Risk Based Testing Strategy With Simulators, Emulators, and R...
How to Create a Risk Based Testing Strategy With Simulators, Emulators, and R...How to Create a Risk Based Testing Strategy With Simulators, Emulators, and R...
How to Create a Risk Based Testing Strategy With Simulators, Emulators, and R...Perfecto by Perforce
 
Patterns of Automation: Simplify Your Test Code
Patterns of Automation: Simplify Your Test CodePatterns of Automation: Simplify Your Test Code
Patterns of Automation: Simplify Your Test CodeTechWell
 
Android automation tools
Android automation toolsAndroid automation tools
Android automation toolsSSGMCE SHEGAON
 
Mobile testing trends webinar PPT
Mobile testing trends webinar PPTMobile testing trends webinar PPT
Mobile testing trends webinar PPTXBOSoft
 
The Internet of Things in Action: Testing Anki’s OVERDRIVE Racing Game
The Internet of Things in Action: Testing Anki’s OVERDRIVE Racing GameThe Internet of Things in Action: Testing Anki’s OVERDRIVE Racing Game
The Internet of Things in Action: Testing Anki’s OVERDRIVE Racing GameJosiah Renaudin
 
A Look into the Future: App Testing and Quality in 2025
A Look into the Future: App Testing and Quality in 2025A Look into the Future: App Testing and Quality in 2025
A Look into the Future: App Testing and Quality in 2025TechWell
 
Appium: the Superhero of Mobile testing
Appium: the Superhero of Mobile testingAppium: the Superhero of Mobile testing
Appium: the Superhero of Mobile testingInfotek Solutions
 
Real-Time Contextual and Social Relevance in Mobile
Real-Time Contextual and Social Relevance in MobileReal-Time Contextual and Social Relevance in Mobile
Real-Time Contextual and Social Relevance in MobileTechWell
 
Matt Lee - Resume 2015
Matt Lee - Resume 2015Matt Lee - Resume 2015
Matt Lee - Resume 2015Matt Lee
 

Similar to Top Ten Attacks to Break Mobile Apps (20)

Soft Skills You Need Are Not Always Taught in Class
Soft Skills You Need Are Not Always Taught in ClassSoft Skills You Need Are Not Always Taught in Class
Soft Skills You Need Are Not Always Taught in Class
 
IoT Software Testing Challenges: The IoT World Is Really Different
IoT Software Testing Challenges: The IoT World Is Really DifferentIoT Software Testing Challenges: The IoT World Is Really Different
IoT Software Testing Challenges: The IoT World Is Really Different
 
Top IOT Testing Challenges Webinar with Jon Hagar
Top IOT Testing Challenges Webinar with Jon HagarTop IOT Testing Challenges Webinar with Jon Hagar
Top IOT Testing Challenges Webinar with Jon Hagar
 
JoeHelfrichResume0701
JoeHelfrichResume0701JoeHelfrichResume0701
JoeHelfrichResume0701
 
IoT Software Testing Challenges: The IoT World Is Really Different
IoT Software Testing Challenges: The IoT World Is Really DifferentIoT Software Testing Challenges: The IoT World Is Really Different
IoT Software Testing Challenges: The IoT World Is Really Different
 
Implement Combinatorial Test Patterns for Better Mobile and IoT Testing
Implement Combinatorial Test Patterns for Better Mobile and IoT TestingImplement Combinatorial Test Patterns for Better Mobile and IoT Testing
Implement Combinatorial Test Patterns for Better Mobile and IoT Testing
 
XBOSoft Mobile Security Webinar with Jon D. Hagar
XBOSoft Mobile Security Webinar with Jon D. HagarXBOSoft Mobile Security Webinar with Jon D. Hagar
XBOSoft Mobile Security Webinar with Jon D. Hagar
 
Java certification course in Coimbatore|Java Learning Center Coimbatore|Java ...
Java certification course in Coimbatore|Java Learning Center Coimbatore|Java ...Java certification course in Coimbatore|Java Learning Center Coimbatore|Java ...
Java certification course in Coimbatore|Java Learning Center Coimbatore|Java ...
 
Java
JavaJava
Java
 
How to Create a Risk Based Testing Strategy With Simulators, Emulators, and R...
How to Create a Risk Based Testing Strategy With Simulators, Emulators, and R...How to Create a Risk Based Testing Strategy With Simulators, Emulators, and R...
How to Create a Risk Based Testing Strategy With Simulators, Emulators, and R...
 
Patterns of Automation: Simplify Your Test Code
Patterns of Automation: Simplify Your Test CodePatterns of Automation: Simplify Your Test Code
Patterns of Automation: Simplify Your Test Code
 
Android automation tools
Android automation toolsAndroid automation tools
Android automation tools
 
Mobile testing trends webinar PPT
Mobile testing trends webinar PPTMobile testing trends webinar PPT
Mobile testing trends webinar PPT
 
The Internet of Things in Action: Testing Anki’s OVERDRIVE Racing Game
The Internet of Things in Action: Testing Anki’s OVERDRIVE Racing GameThe Internet of Things in Action: Testing Anki’s OVERDRIVE Racing Game
The Internet of Things in Action: Testing Anki’s OVERDRIVE Racing Game
 
A Look into the Future: App Testing and Quality in 2025
A Look into the Future: App Testing and Quality in 2025A Look into the Future: App Testing and Quality in 2025
A Look into the Future: App Testing and Quality in 2025
 
Appium: the Superhero of Mobile testing
Appium: the Superhero of Mobile testingAppium: the Superhero of Mobile testing
Appium: the Superhero of Mobile testing
 
BSides LA/PDX
BSides LA/PDXBSides LA/PDX
BSides LA/PDX
 
Real-Time Contextual and Social Relevance in Mobile
Real-Time Contextual and Social Relevance in MobileReal-Time Contextual and Social Relevance in Mobile
Real-Time Contextual and Social Relevance in Mobile
 
RR2 CV
RR2 CVRR2 CV
RR2 CV
 
Matt Lee - Resume 2015
Matt Lee - Resume 2015Matt Lee - Resume 2015
Matt Lee - Resume 2015
 

More from TechWell

Failing and Recovering
Failing and RecoveringFailing and Recovering
Failing and RecoveringTechWell
 
Instill a DevOps Testing Culture in Your Team and Organization
Instill a DevOps Testing Culture in Your Team and Organization Instill a DevOps Testing Culture in Your Team and Organization
Instill a DevOps Testing Culture in Your Team and Organization TechWell
 
Test Design for Fully Automated Build Architecture
Test Design for Fully Automated Build ArchitectureTest Design for Fully Automated Build Architecture
Test Design for Fully Automated Build ArchitectureTechWell
 
System-Level Test Automation: Ensuring a Good Start
System-Level Test Automation: Ensuring a Good StartSystem-Level Test Automation: Ensuring a Good Start
System-Level Test Automation: Ensuring a Good StartTechWell
 
Build Your Mobile App Quality and Test Strategy
Build Your Mobile App Quality and Test StrategyBuild Your Mobile App Quality and Test Strategy
Build Your Mobile App Quality and Test StrategyTechWell
 
Testing Transformation: The Art and Science for Success
Testing Transformation: The Art and Science for SuccessTesting Transformation: The Art and Science for Success
Testing Transformation: The Art and Science for SuccessTechWell
 
Implement BDD with Cucumber and SpecFlow
Implement BDD with Cucumber and SpecFlowImplement BDD with Cucumber and SpecFlow
Implement BDD with Cucumber and SpecFlowTechWell
 
Develop WebDriver Automated Tests—and Keep Your Sanity
Develop WebDriver Automated Tests—and Keep Your SanityDevelop WebDriver Automated Tests—and Keep Your Sanity
Develop WebDriver Automated Tests—and Keep Your SanityTechWell
 
Eliminate Cloud Waste with a Holistic DevOps Strategy
Eliminate Cloud Waste with a Holistic DevOps StrategyEliminate Cloud Waste with a Holistic DevOps Strategy
Eliminate Cloud Waste with a Holistic DevOps StrategyTechWell
 
Transform Test Organizations for the New World of DevOps
Transform Test Organizations for the New World of DevOpsTransform Test Organizations for the New World of DevOps
Transform Test Organizations for the New World of DevOpsTechWell
 
The Fourth Constraint in Project Delivery—Leadership
The Fourth Constraint in Project Delivery—LeadershipThe Fourth Constraint in Project Delivery—Leadership
The Fourth Constraint in Project Delivery—LeadershipTechWell
 
Resolve the Contradiction of Specialists within Agile Teams
Resolve the Contradiction of Specialists within Agile TeamsResolve the Contradiction of Specialists within Agile Teams
Resolve the Contradiction of Specialists within Agile TeamsTechWell
 
Pin the Tail on the Metric: A Field-Tested Agile Game
Pin the Tail on the Metric: A Field-Tested Agile GamePin the Tail on the Metric: A Field-Tested Agile Game
Pin the Tail on the Metric: A Field-Tested Agile GameTechWell
 
Agile Performance Holarchy (APH)—A Model for Scaling Agile Teams
Agile Performance Holarchy (APH)—A Model for Scaling Agile TeamsAgile Performance Holarchy (APH)—A Model for Scaling Agile Teams
Agile Performance Holarchy (APH)—A Model for Scaling Agile TeamsTechWell
 
A Business-First Approach to DevOps Implementation
A Business-First Approach to DevOps ImplementationA Business-First Approach to DevOps Implementation
A Business-First Approach to DevOps ImplementationTechWell
 
Databases in a Continuous Integration/Delivery Process
Databases in a Continuous Integration/Delivery ProcessDatabases in a Continuous Integration/Delivery Process
Databases in a Continuous Integration/Delivery ProcessTechWell
 
Mobile Testing: What—and What Not—to Automate
Mobile Testing: What—and What Not—to AutomateMobile Testing: What—and What Not—to Automate
Mobile Testing: What—and What Not—to AutomateTechWell
 
Cultural Intelligence: A Key Skill for Success
Cultural Intelligence: A Key Skill for SuccessCultural Intelligence: A Key Skill for Success
Cultural Intelligence: A Key Skill for SuccessTechWell
 
Turn the Lights On: A Power Utility Company's Agile Transformation
Turn the Lights On: A Power Utility Company's Agile TransformationTurn the Lights On: A Power Utility Company's Agile Transformation
Turn the Lights On: A Power Utility Company's Agile TransformationTechWell
 

More from TechWell (20)

Failing and Recovering
Failing and RecoveringFailing and Recovering
Failing and Recovering
 
Instill a DevOps Testing Culture in Your Team and Organization
Instill a DevOps Testing Culture in Your Team and Organization Instill a DevOps Testing Culture in Your Team and Organization
Instill a DevOps Testing Culture in Your Team and Organization
 
Test Design for Fully Automated Build Architecture
Test Design for Fully Automated Build ArchitectureTest Design for Fully Automated Build Architecture
Test Design for Fully Automated Build Architecture
 
System-Level Test Automation: Ensuring a Good Start
System-Level Test Automation: Ensuring a Good StartSystem-Level Test Automation: Ensuring a Good Start
System-Level Test Automation: Ensuring a Good Start
 
Build Your Mobile App Quality and Test Strategy
Build Your Mobile App Quality and Test StrategyBuild Your Mobile App Quality and Test Strategy
Build Your Mobile App Quality and Test Strategy
 
Testing Transformation: The Art and Science for Success
Testing Transformation: The Art and Science for SuccessTesting Transformation: The Art and Science for Success
Testing Transformation: The Art and Science for Success
 
Implement BDD with Cucumber and SpecFlow
Implement BDD with Cucumber and SpecFlowImplement BDD with Cucumber and SpecFlow
Implement BDD with Cucumber and SpecFlow
 
Develop WebDriver Automated Tests—and Keep Your Sanity
Develop WebDriver Automated Tests—and Keep Your SanityDevelop WebDriver Automated Tests—and Keep Your Sanity
Develop WebDriver Automated Tests—and Keep Your Sanity
 
Ma 15
Ma 15Ma 15
Ma 15
 
Eliminate Cloud Waste with a Holistic DevOps Strategy
Eliminate Cloud Waste with a Holistic DevOps StrategyEliminate Cloud Waste with a Holistic DevOps Strategy
Eliminate Cloud Waste with a Holistic DevOps Strategy
 
Transform Test Organizations for the New World of DevOps
Transform Test Organizations for the New World of DevOpsTransform Test Organizations for the New World of DevOps
Transform Test Organizations for the New World of DevOps
 
The Fourth Constraint in Project Delivery—Leadership
The Fourth Constraint in Project Delivery—LeadershipThe Fourth Constraint in Project Delivery—Leadership
The Fourth Constraint in Project Delivery—Leadership
 
Resolve the Contradiction of Specialists within Agile Teams
Resolve the Contradiction of Specialists within Agile TeamsResolve the Contradiction of Specialists within Agile Teams
Resolve the Contradiction of Specialists within Agile Teams
 
Pin the Tail on the Metric: A Field-Tested Agile Game
Pin the Tail on the Metric: A Field-Tested Agile GamePin the Tail on the Metric: A Field-Tested Agile Game
Pin the Tail on the Metric: A Field-Tested Agile Game
 
Agile Performance Holarchy (APH)—A Model for Scaling Agile Teams
Agile Performance Holarchy (APH)—A Model for Scaling Agile TeamsAgile Performance Holarchy (APH)—A Model for Scaling Agile Teams
Agile Performance Holarchy (APH)—A Model for Scaling Agile Teams
 
A Business-First Approach to DevOps Implementation
A Business-First Approach to DevOps ImplementationA Business-First Approach to DevOps Implementation
A Business-First Approach to DevOps Implementation
 
Databases in a Continuous Integration/Delivery Process
Databases in a Continuous Integration/Delivery ProcessDatabases in a Continuous Integration/Delivery Process
Databases in a Continuous Integration/Delivery Process
 
Mobile Testing: What—and What Not—to Automate
Mobile Testing: What—and What Not—to AutomateMobile Testing: What—and What Not—to Automate
Mobile Testing: What—and What Not—to Automate
 
Cultural Intelligence: A Key Skill for Success
Cultural Intelligence: A Key Skill for SuccessCultural Intelligence: A Key Skill for Success
Cultural Intelligence: A Key Skill for Success
 
Turn the Lights On: A Power Utility Company's Agile Transformation
Turn the Lights On: A Power Utility Company's Agile TransformationTurn the Lights On: A Power Utility Company's Agile Transformation
Turn the Lights On: A Power Utility Company's Agile Transformation
 

Recently uploaded

MINDCTI Revenue Release Quarter One 2024
MINDCTI Revenue Release Quarter One 2024MINDCTI Revenue Release Quarter One 2024
MINDCTI Revenue Release Quarter One 2024MIND CTI
 
FWD Group - Insurer Innovation Award 2024
FWD Group - Insurer Innovation Award 2024FWD Group - Insurer Innovation Award 2024
FWD Group - Insurer Innovation Award 2024The Digital Insurer
 
"I see eyes in my soup": How Delivery Hero implemented the safety system for ...
"I see eyes in my soup": How Delivery Hero implemented the safety system for ..."I see eyes in my soup": How Delivery Hero implemented the safety system for ...
"I see eyes in my soup": How Delivery Hero implemented the safety system for ...Zilliz
 
Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024The Digital Insurer
 
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...DianaGray10
 
Artificial Intelligence Chap.5 : Uncertainty
Artificial Intelligence Chap.5 : UncertaintyArtificial Intelligence Chap.5 : Uncertainty
Artificial Intelligence Chap.5 : UncertaintyKhushali Kathiriya
 
Apidays New York 2024 - The value of a flexible API Management solution for O...
Apidays New York 2024 - The value of a flexible API Management solution for O...Apidays New York 2024 - The value of a flexible API Management solution for O...
Apidays New York 2024 - The value of a flexible API Management solution for O...apidays
 
MS Copilot expands with MS Graph connectors
MS Copilot expands with MS Graph connectorsMS Copilot expands with MS Graph connectors
MS Copilot expands with MS Graph connectorsNanddeep Nachan
 
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemkeProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemkeProduct Anonymous
 
Cyberprint. Dark Pink Apt Group [EN].pdf
Cyberprint. Dark Pink Apt Group [EN].pdfCyberprint. Dark Pink Apt Group [EN].pdf
Cyberprint. Dark Pink Apt Group [EN].pdfOverkill Security
 
[BuildWithAI] Introduction to Gemini.pdf
[BuildWithAI] Introduction to Gemini.pdf[BuildWithAI] Introduction to Gemini.pdf
[BuildWithAI] Introduction to Gemini.pdfSandro Moreira
 
Why Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire businessWhy Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire businesspanagenda
 
DBX First Quarter 2024 Investor Presentation
DBX First Quarter 2024 Investor PresentationDBX First Quarter 2024 Investor Presentation
DBX First Quarter 2024 Investor PresentationDropbox
 
Apidays New York 2024 - Passkeys: Developing APIs to enable passwordless auth...
Apidays New York 2024 - Passkeys: Developing APIs to enable passwordless auth...Apidays New York 2024 - Passkeys: Developing APIs to enable passwordless auth...
Apidays New York 2024 - Passkeys: Developing APIs to enable passwordless auth...apidays
 
Ransomware_Q4_2023. The report. [EN].pdf
Ransomware_Q4_2023. The report. [EN].pdfRansomware_Q4_2023. The report. [EN].pdf
Ransomware_Q4_2023. The report. [EN].pdfOverkill Security
 
How to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected WorkerHow to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected WorkerThousandEyes
 
Web Form Automation for Bonterra Impact Management (fka Social Solutions Apri...
Web Form Automation for Bonterra Impact Management (fka Social Solutions Apri...Web Form Automation for Bonterra Impact Management (fka Social Solutions Apri...
Web Form Automation for Bonterra Impact Management (fka Social Solutions Apri...Jeffrey Haguewood
 
Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
Strategize a Smooth Tenant-to-tenant Migration and Copilot TakeoffStrategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoffsammart93
 
AXA XL - Insurer Innovation Award Americas 2024
AXA XL - Insurer Innovation Award Americas 2024AXA XL - Insurer Innovation Award Americas 2024
AXA XL - Insurer Innovation Award Americas 2024The Digital Insurer
 
Strategies for Landing an Oracle DBA Job as a Fresher
Strategies for Landing an Oracle DBA Job as a FresherStrategies for Landing an Oracle DBA Job as a Fresher
Strategies for Landing an Oracle DBA Job as a FresherRemote DBA Services
 

Recently uploaded (20)

MINDCTI Revenue Release Quarter One 2024
MINDCTI Revenue Release Quarter One 2024MINDCTI Revenue Release Quarter One 2024
MINDCTI Revenue Release Quarter One 2024
 
FWD Group - Insurer Innovation Award 2024
FWD Group - Insurer Innovation Award 2024FWD Group - Insurer Innovation Award 2024
FWD Group - Insurer Innovation Award 2024
 
"I see eyes in my soup": How Delivery Hero implemented the safety system for ...
"I see eyes in my soup": How Delivery Hero implemented the safety system for ..."I see eyes in my soup": How Delivery Hero implemented the safety system for ...
"I see eyes in my soup": How Delivery Hero implemented the safety system for ...
 
Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024
 
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
 
Artificial Intelligence Chap.5 : Uncertainty
Artificial Intelligence Chap.5 : UncertaintyArtificial Intelligence Chap.5 : Uncertainty
Artificial Intelligence Chap.5 : Uncertainty
 
Apidays New York 2024 - The value of a flexible API Management solution for O...
Apidays New York 2024 - The value of a flexible API Management solution for O...Apidays New York 2024 - The value of a flexible API Management solution for O...
Apidays New York 2024 - The value of a flexible API Management solution for O...
 
MS Copilot expands with MS Graph connectors
MS Copilot expands with MS Graph connectorsMS Copilot expands with MS Graph connectors
MS Copilot expands with MS Graph connectors
 
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemkeProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
 
Cyberprint. Dark Pink Apt Group [EN].pdf
Cyberprint. Dark Pink Apt Group [EN].pdfCyberprint. Dark Pink Apt Group [EN].pdf
Cyberprint. Dark Pink Apt Group [EN].pdf
 
[BuildWithAI] Introduction to Gemini.pdf
[BuildWithAI] Introduction to Gemini.pdf[BuildWithAI] Introduction to Gemini.pdf
[BuildWithAI] Introduction to Gemini.pdf
 
Why Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire businessWhy Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire business
 
DBX First Quarter 2024 Investor Presentation
DBX First Quarter 2024 Investor PresentationDBX First Quarter 2024 Investor Presentation
DBX First Quarter 2024 Investor Presentation
 
Apidays New York 2024 - Passkeys: Developing APIs to enable passwordless auth...
Apidays New York 2024 - Passkeys: Developing APIs to enable passwordless auth...Apidays New York 2024 - Passkeys: Developing APIs to enable passwordless auth...
Apidays New York 2024 - Passkeys: Developing APIs to enable passwordless auth...
 
Ransomware_Q4_2023. The report. [EN].pdf
Ransomware_Q4_2023. The report. [EN].pdfRansomware_Q4_2023. The report. [EN].pdf
Ransomware_Q4_2023. The report. [EN].pdf
 
How to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected WorkerHow to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected Worker
 
Web Form Automation for Bonterra Impact Management (fka Social Solutions Apri...
Web Form Automation for Bonterra Impact Management (fka Social Solutions Apri...Web Form Automation for Bonterra Impact Management (fka Social Solutions Apri...
Web Form Automation for Bonterra Impact Management (fka Social Solutions Apri...
 
Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
Strategize a Smooth Tenant-to-tenant Migration and Copilot TakeoffStrategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
 
AXA XL - Insurer Innovation Award Americas 2024
AXA XL - Insurer Innovation Award Americas 2024AXA XL - Insurer Innovation Award Americas 2024
AXA XL - Insurer Innovation Award Americas 2024
 
Strategies for Landing an Oracle DBA Job as a Fresher
Strategies for Landing an Oracle DBA Job as a FresherStrategies for Landing an Oracle DBA Job as a Fresher
Strategies for Landing an Oracle DBA Job as a Fresher
 

Top Ten Attacks to Break Mobile Apps

  • 1. !! T4 Mobile!Testing! 10/16/2014!9:45:00!AM! ! Top Ten Attacks to Break Mobile Apps ! Presented by: Jon Hagar Grand Software Testing ! ! ! Brought(to(you(by:( ( ( ( 340!Corporate!Way,!Suite!300,!Orange!Park,!FL!32073! 888G268G8770!H!904G278G0524!H!sqeinfo@sqe.com!H!www.sqe.com
  • 2. Jon Hagar Grand Software Testing Jon Hagar is an independent consultant working in software product integrity, testing, verification, and validation. Jon publishes regularly on testing, including the book Software Test Attacks to Break Mobile and Embedded Devices (breakingembeddedsoftware.com). For more than thirty years, he has worked in software engineering, particularly testing, supporting projects which include control systems (avionics and auto), spacecraft, mobile-smart devices, and attack testing of smart phones. Jon is an editor for ISO, IEEE, and OMG standards.                                        
  • 3. Jon$Hagar$Copy$right$2014$ So4ware$Test$A:acks$to$Break$Mobile$and$Embedded$Devices$ 1$ TOP$10$SOFTWARE$TEST$ATTACKS$ TO$BREAK$MOBILE$SOFTWARE$ STARWEST$2014$ Jon$Hagar$ embedded@ecentral.com$ jon.d.hagar@gmail.com$ Grand$So4ware$TesJng$ Web:$ h:p://breakingembeddedso4ware.wordpress.com/$ Jon$Hagar$Copy$right$2014$ So4ware$Test$A:acks$to$Break$Mobile$and$Embedded$Devices$ 2$2$ AGENDA$ •  DefiniJons$for$this$session$ •  RiskQbased$tesJng$concepts$for$mobile$ •  Exploratory$tesJng$concepts$for$$mobile$ •  My$top$10$Mobile$So4ware$(app)$a:acks$ •  Wrap$up$
  • 4. Jon$Hagar$Copy$right$2014$ So4ware$Test$A:acks$to$Break$Mobile$and$Embedded$Devices$ 3$3$ $ MOBILE,$SMART,$AND$HANDHELD$ •  As$the$names$implies,$these$are$devices—small,$held$in$the$hand,$o4en$ connected$to$communicaJon$networks,$including$ •  Cell$and$smart$phones$–$apps$$ •  Tablets$ •  Medical$devices$ •  Typically$have:$ •  Many$of$the$problems$of$classic$“embedded”$systems$ •  The$power$of$PCs/IT$ •  More$user$interface$(UI)$than$classic$embedded$systems$ •  (RelaJvely)$Fast$updates$ •  Mobile$devices$are$“evolving”$with$more$power,$resources,$apps,$etc.$$ •  Mobile$is$the$“hot”$area$of$computers$and$so4ware$currently$ •  TesJng$rules$and$concepts$are$“evolving”$ STARWEST$ Jon$Hagar$Copy$right$2014$ So4ware$Test$A:acks$to$Break$Mobile$and$Embedded$Devices$ 4$4$ TYPES$OF$MOBILE$APPS$ •  NaJve$ApplicaJons$ •  Local$to$device$ •  Hybrid$ApplicaJons$ •  Local$to$device$but$ interacts$w/internet$ •  Web$ApplicaJons$ •  Not$local$to$device.$ All$interacJons$on$ internet$ STARWEST$
  • 5. Jon$Hagar$Copy$right$2014$ So4ware$Test$A:acks$to$Break$Mobile$and$Embedded$Devices$ 5$5$ MOBILE$TESTING$DEFINITIONS$ •  Mobile$ApplicaJon$TesJng$is$tesJng$the$applicaJon$in$a$support$ environment$or$on$a$mobile$device$ •  System$Level$Mobile$Device$TesJng$is$tesJng$the$hardware$and$operaJng$ system$ •  Does$the$OperaJng$System$install?$$$ •  Does$the$device$power$on?$Do$the$LED$lights$work$as$expected?$$$ •  Does$the$ba:ery$charge$when$the$AC$adapter$is$plugged$into$the$device?$ •  Mobile$Phone$TesJng$should$have$some$different$approaches$to$tesJng$ •  Mobile$System$TesJng$incorporates$tesJng$more$than$one$applicaJon$and$ can$combine$hardware,$so4ware,$firmware,$along$with$other$applicaJons$ •  Mobile$TesJng$–$can/should$be$all$of$the$above$ $ Be$clear$when$using$this$terminology.$If$you$are$only$tesJng$apps$on$mobile$phones,$ then$state$“mobile$apps$tesJng.”$Use$mobile$tesJng$when$you$are$tesJng$mobile$ websites,$mobile$hybrid$apps,$mobile$hardware,$etc.$$ STARWEST$$ Jon$Hagar$Copy$right$2014$ So4ware$Test$A:acks$to$Break$Mobile$and$Embedded$Devices$ 6$6$ DEFINING$SKILL$SET$FOR$ THE$MOBILE$TESTER$ •  Some$exposure$or$knowledge$about$products$from$the$domain$in$which$you$ are$tesJng:$$aerospace,$medical,$automobile$manufacturing,$airplanes,$ factory$systems,$roboJcs,$regulated$environments,$etc.$ •  Some$knowledge$of:$hard$sciences:$$math,$physics,$electronics,$engineering,$ etc.$for$logical$thought$processes$ •  So4ware$sciences:$$psychology,$philosophy,$sociology,$human$factors$(human$ machine$interface)$for$creaJve$&$conceptual$thought$processes$ •  Tester$skill$ •  Planning,$design$techniques,$pa:erns$of$errors,$intuiJon,$criJcal$thinking,$“so4$skills,”$$ communicaJon,$observaJon,$and$mental$models$[ISTQB$and$AST$have$“lists”]$ Chapter$1$–$So4ware$Test$A:acks$to$Break$Mobile$&$Embedded$Devices$ STARWEST$
  • 6. Jon$Hagar$Copy$right$2014$ So4ware$Test$A:acks$to$Break$Mobile$and$Embedded$Devices$ 7$7$ WHAT$IS$AN$ATTACK?$ •  A$pa:ern$(of$tesJng)$based$on$a$common$mode$of$failure$seen$ over$and$over$ •  Maybe$seen$as$a$negaJve,$when$it$is$really$a$posi%ve( •  Goes$a4er$the$“bugs”$that$may$be$in$the$so4ware$ •  May$include$or$use$classic$test$techniques$and$test$concepts$ •  Lee$Copeland’s$book$on$test$design$ •  Many$other$good$books$ •  A$Pa:ern$(more$than$a$process),$which$must$be$modified$for$the$ context$at$hand,$to$do$the$tesJng$$ •  Testers$learn$these$in$a$domain$a4er$years$and$form$a$mental$ model$(most$good$testers$a:ack)$ STARWEST$ Jon$Hagar$Copy$right$2014$ So4ware$Test$A:acks$to$Break$Mobile$and$Embedded$Devices$ 8$8$ WHY$ATTACK?$ •  A:acking$your$so4ware$is$in$part,$$the$process$of$a:empJng$to$ demonstrate$a$system$(hardware,$firmware,$so4ware$and$operaJons)$$ does$not$meet$requirements,$funcJonal$and$nonQfuncJonal$objecJves$ •  Embedded/handheld$so4ware$tesJng$must$include$"the$ system"$(hardware,$so4ware,$operaJons,$users)$ •  A:acking$common$modes$of$failure,$especially$where$the$applicaJon$ is$engaged$and$visible$by$the$user.$ Attack your enemy with approaches to include: Tools Levels Attacks Techniques STARWEST$
  • 7. Jon$Hagar$Copy$right$2014$ So4ware$Test$A:acks$to$Break$Mobile$and$Embedded$Devices$ 9$9$ KINDS$OF$ATTACKS$ •  Whi:aker$offers$a$good$starJng$point$for$so4ware$ a:acks$in$general$that$can$be$applied$to$mobile:$ •  User$Interface$A:acks$ •  Data$and$ComputaJon$ •  File$System$Interface$ •  So4ware/OS$Interface$ •  Whi:aker’s$“How$to$Break$So4ware”$lists$23$a:acks$ •  Plus$he$has$other$books$on$a:acks,$security,$web,$exploratory,$and$tours$in$tesJng$ •  “So4ware$Test$A:acks$to$Break$Mobile$and$ Embedded$Devices”$lists$32$a:acks$and$8$sub$a:acks$ STARWEST$ Jon$Hagar$Copy$right$2014$ So4ware$Test$A:acks$to$Break$Mobile$and$Embedded$Devices$ 10$10$ MOBILE$RISK$AREAS$TO$CONSIDER$ •  There$are$many$risk$to$$consider,$but$you$cannot$test$everything$ •  Risk(s)$based$tesJng$$helps$$bound$the$test$scope$problem$ •  TesJng$is$about$providing$informaJon$and$understanding$ •  ExploraJon$gets$you$started$with$whatever$you$have$(or$don’t$ have)$ STARWEST$
  • 8. Jon$Hagar$Copy$right$2014$ So4ware$Test$A:acks$to$Break$Mobile$and$Embedded$Devices$ 11$11$ SAMPLE$MOBILE$PRODUCT$RISKS$ TESTERS$SHOULD$CONSIDER$ •  Environment$and$input$factors$ •  Environment$–$heat,$noise,$sun,$water,$etc.$ •  Hardware$–$calibraJon,$uniqueness,$manufacturing,$etc.$ •  Electronics$–$noise,$power,$ba:eries,$etc.$ •  CommunicaJons$ •  Interfaces$types$ •  Hardware$ •  Human$$ •  Network$$ •  So4ware$ •  Output$—$noise$influences,$D2A,$representaJon,$etc.$ •  Complexity—use$/$size$of$the$system$$ STARWEST$ Jon$Hagar$Copy$right$2014$ So4ware$Test$A:acks$to$Break$Mobile$and$Embedded$Devices$ 12$12$ RISK=BASED$ TESTING$$ (ISO$29119)$ •  Address,$miJgate,$a:ack$and$reJre$product$risks$ •  PrioriJze$risks$Q$tests:$ •  PotenJal$problems$Q$$Consequences$and$effects$ •  Occurrences$–$likelihood$or$chance$of$happening$ •  Impacts$–$what$happens$ •  Take$consistent$acJon$from$the$beginning$(proposal)$ to$the$end$(reJrement)$of$the$product$or$lifecycle$ •  Risks$&$prioriJzing$should$dictate$the$test$a:acks$ STARWEST$
  • 9. Jon$Hagar$Copy$right$2014$ So4ware$Test$A:acks$to$Break$Mobile$and$Embedded$Devices$ 13$13$ EXPLORATORY$TESTING$=$DEFINITION$ •  QuoJng$James$Bach:$$“The$plainest$definiJon$of$exploratory$ tesJng$is$test$design$and$test$execuJon$at$the$same$Jme.$This$is$ the$opposite$of$scripted$tesJng$(predefined$test$procedures,$ whether$manual$or$automated).$Exploratory$tests,$unlike$ scripted$tests,$are$not$defined$in$advance$and$carried$out$ precisely$according$to$plan.”$ h:p://www.saJsfice.com/arJcles/what_is_et.shtml$ STARWEST$ Jon$Hagar$Copy$right$2014$ So4ware$Test$A:acks$to$Break$Mobile$and$Embedded$Devices$ 14$14$ EXPLORATORY$TESTING$IN$MOBILE$ •  Rapid$feedback$ •  Learning$ •  Upfront$rapid$ learning$ •  A:acking$ •  Address$Risk$(s)$ •  Independent$assessment$ •  Target$a$defect$ •  Prototyping$ •  Need$info$ •  Test$beyond$the$ requirements$ STARWEST$
  • 10. Jon$Hagar$Copy$right$2014$ So4ware$Test$A:acks$to$Break$Mobile$and$Embedded$Devices$ 15$15$ NUMBER$10:$FUNCTIONAL$ATTACK$(33)$ •  Have$an$outline$$or$charter$(top$level$plan$and/or$risk$list)$ •  Create$a$flip$chart,$notecard,$state$model,$mind$map$or$some$representaJon$of$ each$test$$task$$ •  No$“heavyweight$documentaJon$of$the$“test$case”$ •  See$Exploratory$Charter$(test$objecJve)$ •  Have$a$Target$concept$$or$charter$$(Risk,$A:ack,$Bug,$Learning,$…)$ •  VerificaJon$checking$of$requirements$(necessary$but$not$sufficient)$$ •  Have$a$schedule/Jme$box$(short$$test$cycles$=$Planning$to$report)$ •  Do$the$test$ •  Design$test$ •  Execute$test$$ •  Learn$about$the$product:$change$the$risk$list,$modify/add$tests,$and$so$on$ •  Repeat$as$needed$ STARWEST$ Jon$Hagar$Copy$right$2014$ So4ware$Test$A:acks$to$Break$Mobile$and$Embedded$Devices$ 16$16$ EXAMPLE$MIND$MAP$FROM$A$TRAVEL$APP$
  • 11. Jon$Hagar$Copy$right$2014$ So4ware$Test$A:acks$to$Break$Mobile$and$Embedded$Devices$ 17$17$ •  Download$either$Twi:er$or$Facebook$onto$a$device$$ •  Start$either$downloaded$app$ •  From$another$device,$send$an$email$to$the$device’s$email$account$ •  Immediately$send$a$tweet$or$post$a$status$$ •  ConJnue$to$do$engage$Twi:er$or$Facebook$app$for$at$least$1$minute$ •  Record$email$noJficaJon$and$Jme$when$sent$and$received$ •  What$other$observaJons$occurred?$ 9:$NOTIFICATION$TEST$ATTACK$(18)$ Jon$Hagar$Copy$right$2014$ So4ware$Test$A:acks$to$Break$Mobile$and$Embedded$Devices$ 18$18$ ATTACK$TEST$CASE$EXAMPLE:$$ INTERRUPTS$ON$MOBILE$PHONES$ •  Go$to$your$App$store$and$choose$an$applicaJon$to$download$ •  While$the$downloading$is$occurring,$call$the$mobile$phone$ •  Record$observaJons$with$the$download$ •  You$may$need$to$rely$on$observing$a$log$file$while$implemenJng$these$ tests$ •  If$it$fails,$what$kind$of$error$recovery$occurs?$Can$you$repeat$any$ errors?$
  • 12. Jon$Hagar$Copy$right$2014$ So4ware$Test$A:acks$to$Break$Mobile$and$Embedded$Devices$ 19$19$ 8:$ATTACK$SCENARIOS$(12)$ •  Tests$consider$usage,$operaJons,$interface$interacJons$and$ integraJons$$ •  Interface$points$include:$$hardware,$firmware,$so4ware,$data$ exchange,$network$communicaJon$and$combinaJons$ •  How$each$interface$point$integrates$with$another$interface$point$ •  Tests$include$how$the$applicaJon$is$used$endQtoQend$$ •  Tests$to$combine$how$the$enJre$system$interacts$as$well$as$how$ porJons$interact$with$one$another$and$depending$on$complexity$ •  Note:$ConfiguraJon$tests$with$regards$to$how$so4ware$behaves$based$ on$various$configuraJons$of$devices,$operaJng$systems$ Jon$Hagar$Copy$right$2014$ So4ware$Test$A:acks$to$Break$Mobile$and$Embedded$Devices$ 20$ IN$SCENARIO$TESTING$:$TIMING$SUBQ$ATTACK$ When$Time$interacts$with$the$so4ware,$events,$inputs,$ and$outputs,$here's$a$checklist$of$things$to$look$for$and$ consider$(where$bugs$lurk)$in$sequences/stories$ $ •  Order$problems$ •  Too$Long$$ •  Too$Fast$ •  Not$at$right$Jme$mark$or$point$$ •  Late$$ •  Late$or$early$ •  Early$$ •  Deadlocked$caused$by$a$race$ condiJon$(hard$to$find)$$ •  Extra$input$or$output$events$$ •  Missing$events$$ •  Wrong$input/output$within$events$$
  • 13. Jon$Hagar$Copy$right$2014$ So4ware$Test$A:acks$to$Break$Mobile$and$Embedded$Devices$ 21$21$ •  When$to$apply$this$a:ack?$$…when$your$app/device$has$games$ •  What$faults$make$this$a:ack$successful?$$$…games$are$complex$ •  Who$conducts$this$a:ack?$$…see$chart$on$Roles$ •  Where$is$this$a:ack$conducted?$$…throughout$lifecycle$and$in$environments$ •  How$to$determine$if$the$a:ack$exposes$failures?$ •  Unhappy$“users”$ •  Bugs$found$ •  See$checklist$ 7:$ATTACK$TESTING$MOBILE$GAMES$(26)$ Mobile Device Game Testing (2 years ago gaming was 60% or more of Mobile App downloads) Jon$Hagar$Copy$right$2014$ So4ware$Test$A:acks$to$Break$Mobile$and$Embedded$Devices$ 22$22$ •  The$developer(s)—see$A:acks$1,$2,$and$3.$ •  The$app$game$architect$or$director$ •  OnQteam$game$tester(s)$$ •  InQcompany$“dog$food”$testers$ •  Independent$test$players$$ •  Mass$beta$trials$ •  Not$a$tester—Finally,$consider$who$should$not$be$playing$ Note%on%roles:%During(the(tes%ng(effort(and(as(it(progresses,(don’t( forget(that(there(are(many(different(user(roles$ ROLES$TO$PLAY$IN$THE$GAME$ (ANY$MANY$OTHER$APPS)$
  • 14. Jon$Hagar$Copy$right$2014$ So4ware$Test$A:acks$to$Break$Mobile$and$Embedded$Devices$ 23$23$ •  Refine$checklist$to$context$scope$ •  Define$a$role$$ •  Watch$what$is$happening$with$this$role$ •  Define$a$usage$(scenario$or$set$of$funcJons$to$Play$the$game)$ •  Guided$exploraJons$or$ad$hoc$ •  Stress,$unusual$$cases,$explore$opJons$ •  Capture$understanding,$risk,$observaJons,$etc.$ •  Checklist$(watch$for$confusion)$ •  Run$Exploratory$A:ack$ •  Learn$ •  ReQplanQdesign$ •  Watch$for$Bias$ •  Switch$testers$ •  Repeat$ $ $ $ GAME$ATTACK$PATTERN$ Jon$Hagar$Copy$right$2014$ So4ware$Test$A:acks$to$Break$Mobile$and$Embedded$Devices$ 24$24$ 6:$BREAKING$SOFTWARE$WITH$HARDWARE$ AND$SYSTEM$OPERATION$(9)$ •  Classic$subQa:ack$example$to$consider$is$checking$ba:ery$ power$impacts$ •  Not$Common$to$IT/PC$tesJng$ •  Large$impacts$to$users$(if$ba:ery$is$drained)$ •  Relates$to$hardware$and$basic$operaJon$acJviJes$$ •  Requires$systems$thinking$ •  May$require$use$of$specialized$test$environments$and$ support$test$tools$
  • 15. Jon$Hagar$Copy$right$2014$ So4ware$Test$A:acks$to$Break$Mobile$and$Embedded$Devices$ 25$25$ DOCUMENTING$YOUR$TEST$CONDITIONS$ FOR$THE$CHARGING$OF$BATTERY$TEST$ Credit$to:$Jean$Ann$Harrison$$2013$ Jon$Hagar$Copy$right$2014$ So4ware$Test$A:acks$to$Break$Mobile$and$Embedded$Devices$ 26$26$ 5:$ATTACKING$WITH$SIMULATION$(AND$EMULATION)$(17)$ •  TesJng$with$real$hardware$is$advised,$but$ •  Has$limitaJons$ •  Can$require$a$lot$of$equipment$ •  You$need$the$hardware$$ •  FragmentaJon$ •  Many$mobile$people$test$using$simulators$and/ or$emulators$
  • 16. Jon$Hagar$Copy$right$2014$ So4ware$Test$A:acks$to$Break$Mobile$and$Embedded$Devices$ 27$27$ PROS$AND$CON$(SAMPLING)$OF$SIMULATION$AND$ EMULATION$ •  Pro$ •  Can$start$early$ •  Can$support$virtual$tesJng$ •  Can$support$automaJon$ •  Con$ •  Will$miss$some$kinds$of$bugs$ •  May$not$transfer$to$the$actual$hardware$ •  May$require$special$skills$and$efforts$to$set$up$ •  Modeling$(if$used)$can$be$very$tricky$ Jon$Hagar$Copy$right$2014$ So4ware$Test$A:acks$to$Break$Mobile$and$Embedded$Devices$ 28$28$ 4:$DEVELOPER$LEVEL$ATTACK:$WHITE$BOX$(1$&$2)$ •  Between$20QandQ30$percent$of$errors$can$be$found$by$developerQled$structural$ tesJng$ •  When$combined$with$#3,$a$testers$job$becomes$much$more$interesJng$ •  Industry$has$known$this$tesJng$from$the$beginning$and$yet$it$is$underused$ •  Priority$is$high$ •  “Official”$Tester$should$know$it,$advocate$for$it$and$even$“do$it”$someJmes$ •  Two$basic$A:acks$ •  Data$ •  Logic$
  • 17. Jon$Hagar$Copy$right$2014$ So4ware$Test$A:acks$to$Break$Mobile$and$Embedded$Devices$ 29$29$ 3:$STATIC$CODE$ANALYSIS$(SCA)$ATTACK$(3)$ •  This$acJvity$does$NOT$execute$the$code$ •  Can$be$done$“early”$in$the$lifecycle$ •  A$be:er$term$is$just$“analysis,”$but…………$ •  For$the$code,$we$use$a$tool$to$“analyze”$for$certain$types$of$errors$ •  Tools$are$commercial$although$some$open$source$tools$exist$ •  SCA$finds$the$“hard$to$find”$errors$ •  Many$test$teams$take$this$effort$over$since$programmers$“don’t$have$the$Jme”$ •  Issues:$ •  False$posiJves$ •  When$to$do$ •  When$to$repeat$ •  Efforts$can$(should)$$include$analyzing$models,$requirements,$and$other$arJfacts$ Jon$Hagar$Copy$right$2014$ So4ware$Test$A:acks$to$Break$Mobile$and$Embedded$Devices$ 30$30$ 1$&$2:$ATTACK$SECURITY$$ •  Apply$when$the$device$is$mobile$and$has$ •  Account$numbers$ •  UserQids$and$passwords$ •  LocaJon$tags$ •  Restricted$data$$ •  $Current$$authenJcaJon$approaches$in$use$on$mobile$devices$ •  ServerQbased$ •  Registry$(user/password)$ •  LocaJon$or$deviceQbased$ •  ProfileQbased$ PRIVAC Y
  • 18. Jon$Hagar$Copy$right$2014$ So4ware$Test$A:acks$to$Break$Mobile$and$Embedded$Devices$ 31$31$ THE$CURRENT$SECURITY$SITUATION$ •  Mobile$systems$are$highly$integrated$hardware–so4ware–system$ soluJons$which:$ •  Must$be$highly$trustworthy$since$they$handle$sensiJve$data$$ •  O4en$perform$criJcal$tasks$ •  Security$holes$and$problems$abound$ •  Coverity$Scan$2010$Open$Source$Integrity$Report$Q$Android$ •  staJc$analysis$test$a:ack$found$0.47$defects$per$1,000$SLOC$$ •  359$defects$in$total,$88$of$which$were$considered$“high$risk”$in$ the$security$domain$ •  OS#hole#Andriod#with#Angry#Birds#$(researchers$Jon$Oberheide$and$Zach$ Lanier)$ •  Robots$and$Drones$rumored$to$be$a:acked$ •  Cars$$and$medical$devices$being$hacked$ Jon$Hagar$Copy$right$2014$ So4ware$Test$A:acks$to$Break$Mobile$and$Embedded$Devices$ 32$32$ EMBEDDED/MOBILE$SECURITY$CONCERNS$ •  Fraud$–$IdenJty$ •  Worms,$virus,$etc.$ •  Fault$injecJon$ •  Processing$on$the$run$ •  Hacks$impact$ •  Power$ •  Memory$ •  CPU$usage$ •  Eavesdropping$–$yes$everyone$can$hear$you$ •  Hijacking$ •  ClickQjacking$ •  Voice/Screen$ •  Physical$Hacks$ •  File$snooping$ •  Lost$phone$
  • 19. Jon$Hagar$Copy$right$2014$ So4ware$Test$A:acks$to$Break$Mobile$and$Embedded$Devices$ 33$33$ SECURITY$ATTACKS$$$ (ONLY$A$STARTING$POINT)$ •  A:ack$28$PenetraJon$A:ack$Test $$ •  A:ack$28.1$PenetraJon$Sub–A:acks:$AuthenJcaJon$—$Password$A:ack $$ •  A:ack$28.2$Sub–A:ack$Fuzz$Test$$ •  A:ack$29:$InformaJon$The4—Stealing$Device$Data $$ •  A:ack$29.1$Sub$A:ack$–IdenJty$Social$Engineering $$ •  A:ack$30:$Spoofing$A:acks$$ •  A:ack$30.1$LocaJon$and/or$User$Profile$Spoof$Sub–A:ack$ •  A:ack$30.2$GPS$Spoof$Sub–A:ack $$ •  A:ack$31:$A:acking$Viruses$on$the$Run$in$Factories$or$PLCs$ Jon$Hagar$Copy$right$2014$ So4ware$Test$A:acks$to$Break$Mobile$and$Embedded$Devices$ 34$34$ WARNINGS$WHEN$CONDUCTING$ SECURITY$ATTACKS$ !  Security$a:acks$must$be$done$with$the$knowledge$and$approval$of$owners$of$ the$system$and$so4ware$ !  Severe$legal$implicaJons$exist$in$this$area$ !  Many$of$these$a:acks$must$be$done$in$a$lab$(sandbox)$ !  In$these$a:acks,$I$tell$you$conceptually$how$to$“drive$a$car$very$fast$(150$miles$ an$hour)$but$there$are$places$to$do$this$with$a$car$legally$(a$race$track)$and$ places$where$you$will$get$a$Jcket$(most$public$streets)”$ !  Be$forewarned$Q$Do$not$a:ack$you$favorite$app$on$your$phone$or$connected$ server$without$the$right$permissions$due$to$the$legal$implicaJons$
  • 20. Jon$Hagar$Copy$right$2014$ So4ware$Test$A:acks$to$Break$Mobile$and$Embedded$Devices$ 35$35$ WRAP$UP$ •  I$gave$my$top$10,$but$your$a:acks$can$and$will$be$different$ •  Understanding$your$local$context$and$error$pa:erns$is$important$$ (one$size$does$NOT$fit$all)$ •  A:acks$are$pa:erns…you$sJll$must$THINK$and$tailor$ $ Jon$Hagar$Copy$right$2014$ So4ware$Test$A:acks$to$Break$Mobile$and$Embedded$Devices$ 36$36$ MOBILE$ATTACK$CLASSIFICATION$ •  Developer$A:acks$(unit/code$tesJng)$$ •  Control$System$A:acks$$ •  HardwareQSo4ware$A:acks $$ •  Mobile$and$Embedded$So4ware$Domain$A:acks$$ •  Time$A:acks$(Performance) $$ •  Human$User$Interface$A:acks$$$ •  Smart$and/or$Mobile$Phone$FuncJonal$App$A:acks $$ •  Mobile/Embedded$Security$A:acks $$ •  Generic$A:acks$$ •  FuncJonal,$mind$mapping,$and$combinatorial$tests$
  • 21. Jon$Hagar$Copy$right$2014$ So4ware$Test$A:acks$to$Break$Mobile$and$Embedded$Devices$ 37$ MORE$ATTACKS$$ (FROM$SOFTWARE$TEST$ATTACKS$TO$BREAK$MOBILE$AND$EMBEDDED$ DEVICES)$ •  A:ack$1:$StaJc$Code$Analysis $$ •  A:ack$2:$Finding$White–Box$Data$ComputaJon$Bugs$$ •  A:ack$3:$White–Box$Structural$Logic$Flow$Coverage$ •  A:ack$4:$Finding$Hardware–System$Unhandled$Uses$in$So4ware$ •  A:ack$5:$$HwQSw$and$SwQHw$signal$Interface$Bugs$ •  A:ack$6:$Long$DuraJon$Control$A:ack$Runs $$ •  A:ack$7:$$Breaking$So4ware$Logic$and/or$Control$Laws$ •  A:ack$8:$Forcing$the$Unusual$Bug$Cases $$ •  A:ack$9$Breaking$So4ware$with$Hardware$and$System$OperaJons$ •  9.1$Sub–A:ack:$Breaking$Ba:ery$Power $$ •  A:ack$10:$Finding$Bugs$in$Hardware–So4ware$CommunicaJons $$ •  A:ack$11:$Breaking$So4ware$Error$Recovery $$ •  A:ack$12:$Interface$and$IntegraJon$TesJng $$ •  12.1$Sub–A:ack:$ConfiguraJon$IntegraJon$EvaluaJon $$ •  A:ack$13:$Finding$Problems$in$So4ware–System$Fault$Tolerance$ •  A:ack$14:$Breaking$Digital$So4ware$CommunicaJons $$ •  A:ack$15:$Finding$Bugs$in$the$Data $$ •  A:ack$16:$Bugs$in$System–So4ware$ComputaJon $$ •  A:ack$17:$$Using$SimulaJon$and$SJmulaJon$to$Drive$So4ware$A:acks$ •  A:ack$18:$Bugs$in$Timing$Interrupts$and$Priority$Inversion$ •  A:ack$19:$Finding$Time$Related$Bugs $$ •  A:ack$20:$Time$Related$Scenarios,$Stories$and$Tours $$ •  A:ack$21:$Performance$TesJng$IntroducJon $$ •  A:ack$22:$Finding$SupporJng$(User)$DocumentaJon$Problems$ •  Sub–A:ack$22.1:$$Confirming$Install–ability $$ •  A:ack$23:$Finding$Missing$or$Wrong$Alarms $$ •  A:ack$24:$Finding$Bugs$in$Help$Files $$ •  A:ack$25:$Finding$Bugs$in$Apps $$ •  A:ack$26:$TesJng$Mobile$and$Embedded$Games $$ •  A:ack$27:$A:acking$App–Cloud$Dependencies $$ •  A:ack$28$PenetraJon$A:ack$Test $$ •  A:ack$28.1$PenetraJon$Sub–A:acks:$AuthenJcaJon$—$Password$A:ack $$ •  A:ack$28.2$Sub–A:ack$Fuzz$Test$$ •  A:ack$29:$InformaJon$The4—Stealing$Device$Data $$ •  A:ack$29.1$Sub$A:ack$–IdenJty$Social$Engineering $$ •  A:ack$30:$Spoofing$A:acks $$ •  A:ack$30.1$LocaJon$and/or$User$Profile$Spoof$Sub–A:ack$ •  A:ack$30.2$GPS$Spoof$Sub–A:ack $$ •  A:ack$31:$A:acking$Viruses$on$the$Run$in$Factories$or$PLCs$ •  A:ack$32:$Using$Combinatorial$Tests $$ •  A:ack$33:$A:acking$FuncJonal$Bugs $$ Jon$Hagar$Copy$right$2014$ So4ware$Test$A:acks$to$Break$Mobile$and$Embedded$Devices$ 38$38$ SUMMARY:$THANK$YOU$(IDEAS$USED$FROM)$ •  James$Whi:aker$(a:acks)$ •  Elisabeth$Hendrickson$(simulaJons)$ •  Lee$Copeland$(techniques)$ •  Brian$Merrick$(tesJng)$ •  James$Bach$(exploratory$&$tours)$ •  Cem$Kaner$$(test$thinking)$ •  Many$teachers$ •  GeneraJons$past$and$future$ •  Books,$references,$etc.$
  • 22. Jon$Hagar$Copy$right$2014$ So4ware$Test$A:acks$to$Break$Mobile$and$Embedded$Devices$ 39$39$ BOOK$LIST$(MY$FAVORITES)$ •  “So;ware(Test(A?acks(to(Break(Mobile(and(Embedded(Devices”(( –(Jon(Hagar,(to(be(published(in(2013( •  “How$to$Break$So4ware”$James$Whi:aker,$2003$ •  And$his$other$“How$To$Break…”$books$ •  “TesJng$Embedded$So4ware”$Broeckman$and$Notenboom,$2003$ •  “A$PracJJoner’s$Guide$to$So4ware$Test$Design”$Copeland,$2004$ •  “A$PracJJoner’s$Handbook$for$RealQTime$Analysis”$Klein$et.$al.,$1993$ •  “Computer$Related$Risks”,$Neumann,$1995$ •  “Safeware:$System$Safety$and$Computers”,$Leveson,$1995$ •  Honorable$menJons:$ •  “Embedded$System$and$So4ware$ValidaJon”$Roychoudhury,$2009$ •  “Systems$TesJng$with$an$A}tude”$Petschenik$$2005$ •  “So4ware$System$TesJng$and$Quality$Assurance”$Beizer,$1987$ •  “TesJng$Computer$So4ware”$Kaner$et.$al.,$1988$ •  “SystemaJc$So4ware$TesJng”$Craig$&$Jaskiel,$2001$ •  “Managing$the$TesJng$Process”$Black,$2002$ Jon$Hagar$Copy$right$2014$ So4ware$Test$A:acks$to$Break$Mobile$and$Embedded$Devices$ 40$40$ MORE RESOURCES •  www.sJckyminds.com$–$CollecJon$of$test$info$ •  www.embedded.com$–$info$on$a:acks$ •  www.sqaforums.com$Q$Mobile$Devices,$Mobile$Apps$Q$Embedded$Systems$ TesJng$forum$ $ •  AssociaJon$of$So4ware$TesJng$ –  BBST$Classes$h:p://www.tesJngeducaJon.org/BBST/$ •  Your$favorite$search$engine$ •  Our$web$sites$and$blogs$(listed$on$front$page)$