Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

How Do Modelers Read UML Diagrams? Preliminary Results from an Eye-Tracking Study

206 views

Published on

ICSE 2018, Gothenburg (Sweden): Poster presentation by Harald Störrle (@stoerrle, Principal IT Consultant at QAware)

(Please download file for better readability / rotated view)

Abstract:
BACKGROUND: Conceptual diagrams are widely used in practice. Previous research suggested layout quality, diagram size, and expertise level are relevant impact factors, while diagram type is not. Surprisingly little is known about how diagrams are read.
OBJECTIVE: Eventually, we want to understand the cognitive processes of diagram and model understanding. In this paper, we study the behavior of modelers while reading UML diagrams in terms of reading strategies and how they affect diagram understanding.
METHOD: We conduct an eye-tracking study with 28 participants, reusing diagrams and items from previous experiments. We record several objective and subjective performance indicators as well as eye movement and pupil dilation.
RESULTS: We discover behavioral regularities and aggregate them into reading strategies which vary with expertise level and diagram type, but not with layout quality.
CONCLUSIONS: Modelers exhibit specific strategies of diagram understanding. Experts employ different strategies than novices, which explains performance differences irrespective of layout quality.

Published in: Software
  • Be the first to comment

  • Be the first to like this

How Do Modelers Read UML Diagrams? Preliminary Results from an Eye-Tracking Study

  1. 1. HowDoModelersReadUMLDiagrams? PreliminaryResultsfromanEye-TrackingStudy :Wefinditobviousthatgoodlayoutassists inunderstandingadiagram.Interes�ngly, previousreasearchfaildtoconfirmthis intui�ondecisively.Butinaseriesofex- periments,wecouldclearlydemonstrate andquan�fythiseffect,showingthat exper�ese,personalability,anddiagram sizearesignificantimpactfactors,but diagramtypeisnot. Background ?Thefirstobviousques�oniswhetheror notourpreviousresultscanbevalidated underchangedcondi�onsa(newexperi- menters,newpopula�on,newmethod). Thesecondques�onis,whetherwecan observeanybehaviorpa�ernsamongour par�cipants,andwhetherthereare recognizablereadingstrategies.Assuming thereare,willdifferenttreatmentsleadto differentstrategies,andwilldifferent popula�onsexhibitdifferentstrategies, too? Inordertostudytheseques�ons,weused eyetrackingtoexplorethebehaviorof modelersreadingdiagramswithgoodand badlayout. ResearchQues�on !Inaseriesofexperiments,wecoulddem- onstrateandquan�fythiseffect,though, showingthatexper�ese,personalability, anddiagramsizearesignificantimpact factors,butdiagramtypeisnot. 29par�cipantsweretestedwithasubset ofthes�mulifrompreviousexperiments. Theexperimentswereconductedbygrad- uatestudents. Observa�ons €Therarecleardifferencesindiagram readingbehaviorsbetweengood/bad layoutsandexperts/novices.Bylookingat thediagramexplora�onstartpoints,scan paths,andbycorrela�ngpupialrydila�on pa�ernswithdiagramelementtypes,it becomesclearthatlayoutflowplaysan importantroleindiagramunderstanding. PreliminaryInsights ...Thedatagivingrisetothepreliminary resultsreportedhereneedamorecom- hprehensiveanalysis. Also,wewouldliketocomplementthem byotherkindsofphysiologicalmeasure- ments(hearrate,skinconduc�vity)to corroborateearlierfindings. Finally,themethodswehaveemployedso farallowonlyverylimitedanalysisofcog- ni�veprocesses.Wearelookingintomore advancedbrainimagingtechnologiesto gathermoreinsight. FutureWork Thedefini�onof“good”and“bad”layoutquality,respec�vely, followstheexis�ngliteratureofaesthe�ccriteria.Observethat therearetwodifferentaspects:thesimplemistakestoavoidon thegraphlevel,andthe“architectural”pa�ernstofollow. LayoutProblems(i.e.crossings,bends)trigger asmuchcogni�veloadasproperelements. =>DiagramSizeisthecurrencyof diagramcomplexity. ExpertsandNovicesexhibitdifferentbehavior whenitcomestodiagramreading. Gooddiagramsarereadfromthetople�corner. Baddiagramsarereadfromthecenterout. =>Flow-followinglayoutsupportsreading. HaraldStörrle1,3) ,NickBaltsen1,2) ,HenrikChristoffersen1,2) ,AnjaM.Maier2) 1) DepartmentofAppliedMathematicsandComputerScience,TechnicalUniversityofDenmark 2) DepartmentofManagementEngineering,TechnicalUniversityofDenmark 3) QAwareGmbH,Munich,Germany good ModelSamples bad Ac�vity Diagram Class Diagram UseCase Diagram 28 Störrle,Harald:”OntheImpactofLayoutQualityto UnderstandingUMLDiagrams”pp.135-142.In:Myerset al.(Eds.):Proc.IEEESymp.VisualLanguagesand Human-CentricCompu�ng(VL/HCC),IEEECS2011 Störrle,H.:”OntheImpactofLayoutQualityto UnderstandingUMLDiagrams:DiagramTypeand Expertise”.In:Costagliolaetal.(Eds.):Proc.IEEESymp. Vis.Lang.&Human-CentricCompu�ng(VL/HCC),IEEECS 2012 Störrle,Harald:”OntheImpactofLayoutQualityto UnderstandingUMLDiagrams:SizeMatters”.In:Dingelet al.(Eds.):Proc.17thIntl.Conf.ModelDrivenEngineering LanguagesandSystems(MoDELS),Springer2014 Maier,AnjaM.,Baltsen,N.,Christoffersen,H.,Störrle,H.: “TowardsDiagramUnderstanding:APilot-Study MeasuringCognitiveWorkloadThroughEye-Tracking”.In: Proc.Intl.Conf.HumanBehaviorinDesign,DesignSociety 2014 BadLayout unnecessarylinebends unnecessarylinecrossings obscuringelements violatenaturalflow GoodLayout consistentcolouring uniformtext(direc�on,font) group/overlaybysimilarity symmetricarrangement -ReadingDirec�on -GraphFollowing -Unstructured ReadingStrategiesPar�cipantsInstrumenta�onSetup (c)2018,HaraldStörrle,stoerrle@acm.org ? 0,0000 0,5000 1,0000 1,5000 2,0000 2,5000 3,0000 3,5000 0 778 1556 2334 3112 3891 4669 5448 6227 7005 7784 8563 9341 10120 10899 11677 12456 13234 14013 14791 15570 16348 17127 17906 18684 19463 20242 21020 21799 22577 23356 24135 24914 25692 26471 27249 28028 28807 29585 30364 31143 31921 32700 33478 34256 35034 35812 36590 37368 38147 LPupilDiameter(Conv.)[mm]RPupilDiameter(Conv.)[mm] DiagramType [Class,Activity,UseCase] DiagramSize [Small,Large] LayoutQuality [good,bad] Objec�vePerformance Preference [1..5] Preference [ordering] PerceivedDifficulty [1..5] Fixa�onDura�on [ms] ScanStart [AoI] TimeNeeded [s] TestScore [0..10] BlinkRate [blinks/s] PupilDila�on [mm] Cogni�veLoad ReadingStrategy ScanPath [AoI*] LayoutClarity [1..5] Understandability [1..5] Subjec�veExperience Pointofreference/Replica�on Valida�onofpreviousstudiesbyrepea�ngthesameexperiment,with thesame(subjec�ve)measurementsonasub-sampleofpreviously applieds�muli. ModelerExper�se [novicxe,experienced] Modeler ThisPosterasPDF QAwareGmbH

×