• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
Systems Modelling and Qualitative Data

Systems Modelling and Qualitative Data



ESRC Seminar Series Modelling on the Move: Towards Transport System Transitions?

ESRC Seminar Series Modelling on the Move: Towards Transport System Transitions?



Total Views
Views on SlideShare
Embed Views



0 Embeds 0

No embeds



Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
Post Comment
Edit your comment

    Systems Modelling and Qualitative Data Systems Modelling and Qualitative Data Presentation Transcript

    • Systems Modelling and Qualitative DataDr Mike YearworthReader in Engineering Systems12th April 2013
    • !  Purpose•  Presenting an approach to grounding systemsmodelling, by…•  Describing system dynamics modelling•  Describing what I mean by qualitative data analysis –specifically grounded theory•  Bringing the two together•  Practical details – using CAQDAS and some properties of binarymatrices•  Presenting results•  Debating•  Methodology – justifying the approach using arguments ofmultimethodology212th April 2013
    • !  Origins of Dynamic Complexity•  Feedback! A causes B, B causes C, … causes A•  Negative feedback (balancing) – goal seeking, controlsystem•  Positive feedback (reinforcing) – unlimited growth untilbounded by exogenous factor(s)•  Combined: patterns of +ve or –ve loop dominance,•  Exponential, goal seeking, damped oscillation, limit cycles,chaotic behaviour•  Delays complicate behaviours e.g. in –ve feedback loops•  This is not detail complexity, system structure can bequite simple yet still produce complex dynamic behaviour312th April 2013
    • 12th March 2013!  What  is  System  Dynamics?  basic Forrester construct
    • !  System dynamics modellingelements•  Causal Loop Diagrams (CLDs)•  Used to surface mental models about the behaviour of elements(variables) of the system expressed as causal relationships andfeedback loops•  Stocks and Flows (S&F) Maps•  Describe the structure of the system in terms of flows andaccumulations of things•  System Dynamics (SD) Models•  Combined CLD+S&F which describe the dynamic behaviour of asystem•  Model boundary chart•  Endogenous/exogenous/excluded variables•  Sub-system diagrams•  Overall architecture of a model, comprising sub-systems and flows ofthings between sub-systems12th March 2013
    • !  System Dynamics modellingprocess12th March 2013STERMAN, J. D. (2000) Business dynamics : systems thinking and modeling for a complex worldReal WorldDecisions InformationFeedbackStrategy, Structure,Decision Rules Mental Models ofthe Real Worldxxx1. Problem Articulation(Boundary Selection)2. DynamicHypothesis3. Formulation4. Testing5. PolicyFormulation &Evaluation X
    • !  Causal Loop Diagrams (CLDs)•  Relationship between variablesrepresented by arrows linking them•  causal relationship•  can be positive or negative12th March 2013Investment Jobs+Cost of Fuel Car Journeys-WorkCompletionRateProjectCompletionTimeManagementComplexityNumber ofProject Staff++-+
    • !  Causal Loop Diagrams – textualanalysis•  “Increased government investment in SWRDAwill lead to the number of jobs created in Bristolgoing up”•  “Raising the tax on petrol will reduce the overallnumber of car journeys”•  Management meeting…•  Director – “Increasing the number of project staff will improveour work completion rate and improve our project completiontimes.”•  Project Manager – “But increasing the number of staff in myteam will make my job way more complex and probably leadto worse completion times.”12th March 2013
    • !  Causal links and feedbackloops12th March 2013W Z+ Read as W causes Z with +ve link polarity or mathematicallyas (∂Z/∂W>0). If the cause increases, the effect increasesabove what it would otherwise have been.A B- Read as A causes B with ,ve link polarity or mathematicallyas (∂B/∂A<0). If the cause increases, the effect decreasesbelow what it would otherwise have been.X Y+Read as X causes Y with +ve link polarity but only after somedelay.B1Label to indicate a balancing feedback loop.R1Label to indicate a reinforcing feedback loop.
    • !  Qualitative Data Analysis•  Summary of Report to the Board of Directors –“Based on the strength of the order book and the revenues thatwere being generated the CEO decided to invest in building up thesales force to generate more orders and grow revenues in line withpromises to investors. At first this worked well and revenues grew.However, you started to experience significant operationaldifficulties in meeting this growth in orders, a significant backlogbuilt up and delivery lead times began to get out of hand. Thiseventually caused problems for your sales force when word gotaround that you were getting later and later in fulfilling your orders.The sales team started to loose customers at an alarming rate,revenues fell and the CEO decided to reverse his earlier decisionand cut the sales force to reduce costs.”1012th April 2013
    • !  CLD – system behaviour12th March 2013Size of SalesForceNumber ofOrders inProcessRevenues+++OrderBacklog+SalesDifficulties-R BDeliveryLead Times++Expected OrderFulfillment Time+
    • !  Stocks and Flows (S&F) Map12th March 2013Size of SalesForceNumber ofOrders inProcessRevenues OrderBacklog+SalesDifficultiesHiring Rate Firing Rate+FulfillmentRateOrder Rate+-
    • !  System Boundary Chart12th March 2013Endogenous Exogenous ExcludedRevenues Cost of sales InventorySales difficulties Expected order fulfilment time ProductsDelivery lead times Price of productOrder backlog Delay in finding out (about problems)Work rate
    • !  System Dynamics Model12th March 2013Size of SalesForceOrders inProcessRevenuesOrderBacklog+SalesDifficultiesRBDeliveryLead Times++Expected OrderFulfillment Time+Cost of SalesOrder Rate+ FulfillmentRateWork Rate+Price of Product+-Hiring Rate+- Firing Rate+FractionInvested+Delay infinding out--+
    • !  Behaviour tests12th March 2013Selected Variables300 Orders2 Orders/Month20 People150 Orders1.45 Orders/Month14.5 People0 Orders0.9 Orders/Month9 People3333333333333333 3 3 3 3 3 3 32 2 2 2 2 2 2 2 2 2 2 2 222222222221 1 11111111111 1 1 111111110 1 2 3 4 5 6 7 8 9 10 11 12Time (Month)Orders in Process : Baseline Orders1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1Sales Difficulties : Baseline Orders/Month2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2Size of Sales Force : Baseline People3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3
    • !  Soft  Stocks  •  Examples:  confidence,  ability,  drive,  satisfaction,  alignment,  morale,  productivity,  reputation…  •  From  the  social/management  sciences  we  have  the  theory  of  scale  types  •  Nominal,  order,  interval,  ratio§  •  For  quantitative  system  dynamics  models  we  need  to  use  interval  or  ratio  measurements  for  simulations  to  work  1612th April 2013¶FOWLER, A. (2003) Systems modelling, simulation, and the dynamics of strategy. Journal of Business Research, 56, 135-144.§STEVENS, S. S. (1946) On the Theory of Scales of Measurement. Science, 103(2684), pp. 677-680.
    • !  Soft  stock  example  –  NASA’s  safety  culture¶  1712th April 2013¶http://cpmr.usra.edu/Leveson-Year1-Review.ppt
    • !  Qualitative/QuantitativeDebate•  Models can be quantitative or qualitativedepending on purpose•  Quantitative models : the normal way of usingSystem Dynamics as per method described•  Qualitative models (CLDs – only) : emphasison identifying feedback paths that produceeither balancing or reinforcing feedback whichcan be used in a learning processCoyle, G. (2000). Qualitative and quantitative modelling in system dynamics:some research questions. System Dynamics Review, 16(3), 225-244.12th March 2013
    • !  Grounded  Theory  1912th April 2013An abductive approach to theory generation•  Glaser and Strauss (1967)•  “the discovery of theory from data”•  Strauss and Corbin (1998)•  theory that is “derived from data, systematically gathered andanalyzed through the research process”•  Methodology1.  Data collection: for example interviews, transcripts, and documents2.  Procedures for interpretation and organizing dataa)  conceptualizing, reducing, elaborating and relating data; whichcollectively are referred to as codingb)  analytical procedures, such as non statistical sampling, writingof memos and diagramming3.  Output: Written and verbal reportsGLASER, B.G., STRAUSS, A.L., (1967). The Discovery of Grounded Theory: Strategies for Qualitative Research. Hawthorne: Aldine.STRAUSS, A. L. & CORBIN, J. (1998) Basics of qualitative research : techniques and procedures for developing grounded theory,Thousand Oaks ; London ; New Delhi, Sage
    • !  Grounding  Systems  Modelling    •  Background:  open  ended,  inductive  system  dynamics  modelling  projects  where  understanding  behaviour  of  complex  organisational  systems  was  a  requirement  •  Repenning  and  Sterman  (1997)  –  inductive  approach  to  modelling  dynamics  of  process  improvement  •  Morrison  (2003)  –  inductive  modelling  of  organizational  change  •  Leading  to  the  idea  of  Grounded  Theories  expressed,  or  encoded,  as  Causal  Loop  Diagrams  or  System  Dynamics  models  •  Hypotheses  about  dynamic  behaviour  •  A  modeller,  skilled  in  the  art  of  CLDs/System  Dynamics,  would  probably  argue  that  they  do  this  naturally,  models  do  not  appear  out  of  thin  air.    •  However,  by  using  the  Grounded  Theory  approach  and  using  CAQDAS  tools  provide  an  explicit  audit  trail  an  explicit  linkage  from  data  to  models  2012th April 2013MORRISON, J. B. (2003) Co‐evolution of process and content in organizational change : explaining the dynamics of start and fizzle.REPENNING, N. P. & STERMAN, J. D. (2002) Capability traps and self-confirming attribution errors in the dynamics of process improvement. AdministrativeScience Quarterly, 47, 265-295
    • !  Morrison, on his approach…2112th April 2013“Data analysis included listening to the recorded interviews and reading thetranscriptions, coupled with a review of field notes. I identified patterns ofinterest and recurring themes in the data, bounding the analysis with a focuson efforts to implement change in the first production cell. As is typical indeveloping grounded theory, I organized the data into categories, which Irepresented with variables and causal relationships between them (Glaser etal., 1967). I combined variables and causal relationships to begin identifyingcausal loops as a description of the feedback processes gradually emergingfrom this analysis. During the data analysis, I occasionally translatedportions of the emerging feedback structure into formal mathematical modelsand simulated their behavior in order to gain a richer understanding of therelationship between the feedback structure and the dynamic behavior. Theiteration between the grounded data, causal loop diagrams, and formalmathematical models led to additional insights and generated new questionsthat I could explore in the available data or pursue with my respondents.”
    • 2212th April 2013Morrison, J. B. (2003) Co-evolution of process and content in organizational change:explaining the dynamics of start and fizzle. PhD Thesis Sloan School of Management.Massachusetts Institute of Technology.
    • 2312th April 2013Repenning, N. P. & Sterman, J. D. (2002) Capability traps and self-confirming attributionerrors in the dynamics of process improvement. Administrative Science Quarterly,47(2), pp. 265-295.
    • 2412th April 2013Dunford, C. N., Yearworth, M., York, D. M. & Godfrey, P. (2012) A View of SystemsPractice: Enabling Quality in Design. Systems Engineering.Use of Taught SETechniquesTraining inSystemEngineeringNatural forgetting touse tools over timeOverall Quality ofSystems PracticeKnowledge ofSystems PracticeREvidenceNumber of projectswhere SE applied atquality per monthImprovement+-Application+Quality decayLearning+KnowledgedecayCouplingCommunicatethe Value+Awareness ofSystemsEngineering++EngineersAppreciation ofSystems PracticeEase of Tailoringthe SystemsApproach++++LogisticalComplexity--CrossLifecycleWorking+Frequency oftraining events+EnthusiasmApplicationAppreciationExpertiseRBR--
    • 2512th April 2013Yearworth, M. (2010) Inductive Modelling of an Entrepreneurial System. 28th International Conference of theSystem Dynamics Society. Seoul, Korea.Yearworth, M. & White, L. (201x) The Uses of Qualitative Data in Multimethodology: Developing Causal LoopDiagrams During the Coding Process. European Journal of Operational Research - In Review.Confidence inmanagementMethods toensure success+Entrepreneurialdrive+Cooperationbetweeninvestors+Parallelinvestment+Equity fundingsuccess+Financial outcome -Return on Equity(RoE)Ideageneration+Intellectual property,creating anddefendingProof of conceptsand prototypesSources of earlyfundingEntrepreneursequity stake+Entrepreneursrisk appetite+Investors riskappetite+Equityfunded+++MeetingcustomerneedsEvidence ofrevenue andprojections+Persuasiveness ofbusiness model++++++Portfolio offunds++R1R4R2R3-B1+R0+"Spotting Opportunities, Testing, and Validation(SOTV)""Realistic Equity Position(REP)""Scale-Up and Exit(SUE)"
    • !  Coding and an axiom•  In addition to axial coding…•  A possible relationship exists between twocodes (concepts, categories) if the twocategories code data within the samescope of the source2612th April 2013
    • !  Causality analysis2712th April 2013Rabinovich, M. & Kacen, L. (2010) Advanced Relationships Between Categories Analysisas a Qualitative Research Tool. Journal of Clinical Psychology, 66(7), pp. 698-708.Maxwell, J. A. (2004) Using Qualitative Methods for Causal Explanation. Field Methods, 16(3), pp. 243-264.
    • !  Proposition•  The value of the method is in the potentialto1.  Introduce dynamic sensibility to qualitativedata analysis§2.  Provide a more rigorous approach to theformation stage of system dynamicsmodelling¶2812th April 2013§LANE, D. C. & OLIVA, R. (1998) The greater whole: Towards a synthesis of system dynamics and soft systems methodology.European Journal of Operational Research, 107, 214-235¶Kopainsky, B. & Luna-Reyes, L. F. (2008) Closing the Loop: Promoting Synergies with Other Theory Building Approaches toImprove System Dynamics Practice. Systems Research and Behavioral Science, 25pp. 471-486.Luna-Reyes, L. F. & Andersen, D. L. (2003) Collecting and analyzing qualitative data for system dynamics: methods and models.System Dynamics Review, 19(4), pp. 271-296.
    • !  Multimethodology – theoreticalunderpinnings•  “…it seems apparent that the question is not if touse qualitative data, but when and how to use themappropriately?”[6]•  approaches such as grounded theory constitute atoolset that helps build “…relevant system dynamicsmodels, grounded in data, and with higher potentialto provide rigorous and relevant genericstructures”[7]2912th April 2013[6] Luna-Reyes, L. F. & Andersen, D. L. (2003) Collecting and analyzing qualitative data for system dynamics:methods and models. System Dynamics Review, 19(4), pp. 271-296.[7] Kopainsky, B. & Luna-Reyes, L. F. (2008) Closing the Loop: Promoting Synergies with Other Theory BuildingApproaches to Improve System Dynamics Practice. Systems Research and Behavioral Science, 25pp. 471-486.
    • !  Conclusions•  Value…•  Theoretical underpinning in multimethodology•  Tool support, both mathematical and software•  Practical examples of applicationè rigorous grounding of SD modellingè adding dynamic sensibility to grounded theory•  …and?3012th April 2013
    • !  Conclusions•  The codeàconceptàcategoryàtheory grouping andfree node/tree node (axial coding) in NVivo leadsthinking towards hierarchical structuring (arborisation)•  Matrix structures lead towards network thinking(reticulation), but much harder to do and less support foritè  work close to the data using a parallel/bridging strategywhere questions of causality are posed constantly anddynamic hypotheses generated/tested abductivelyè  requires better tool support to make this an easierprocess3112th April 2013
    • !  Questions?mike.yearworth@bristol.ac.ukhttp://www.bris.ac.uk/engineering/people/mike-yearworth/index.html3212th April 2013