IDS WORKSHOPImpact, Innovation, LearningPanel SessionValues, Learning, and SystemsA framework for critical rigourin impact...
CORE STATEMENT…. evaluation asks and answers questionsabout the quality, value, and/or importance ofthings ….If we’re not ...
The System FieldDifferentsystemstraditions whichhave shapedcontemporarysystemspracticeSome Streams of Systemic Thought(Dra...
Core System ConceptsUnderstanding interrelationships associated with asituation;Engaging with contrasting perspectives reg...
INTERRELATIONSHIPS – Dynamics, Context, Complexity• What is the structure of interrelationships within the situation (how ...
INTERRELATIONSHIPS – Relevance to Impact EvaluationEmergenceUnintended & Unanticpated ConsequencesNeed For Keeping Measure...
PERSPECTIVES – Stakeholders, Stakes, Framing
PERSPECTIVES– Relevance to Impact EvaluationUntangling map from territoryAssessing why things worked out the way they didC...
BOUNDARIES – values, power, knowledge, legitimacy
BOUNDARIES– Relevance to Impact EvaluationBroadening the range of values (utility, rights, virtues)Identifying the values ...
Types of LearningEffects InterventionValues,assumptionsRules,knowledgeSingle loop learningDouble loop learningTriple loop ...
Types of Learning and Systems ConceptsPerspectives BoundariesSingle loop learningDouble loop learningTriple loop learningI...
A systems - based framework for rigor in evaluationInterventionPerspectivesBoundariesSingle loop learningDouble loop learn...
(ii) Partial inserving somestakeholder partiesincludingpractitioners - orinterests - betterthan others(i) Partial inrepres...
1. Complexsituations orContexts ofchangeand uncertaintyAll Systems are partial in the sense of :1. representing only a sec...
Relevance to Impact Evaluation Emphasis of CAS and CriticalSystemInter-relationshipsEmergence and uncertainty of impacts =...
General• Good impact evaluation requires address to perspectives and boundaries in addition (& in relation)to complexity o...
Systems Concepts inEvaluation. Anexpertanthology(AEApublication)eds. BobWilliams & IrajImam2007Systems Approaches toManagi...
Upcoming SlideShare
Loading in …5
×

IDS Impact Innovation and Learning Workshop March 2013: Day 1, Panel Session 1 Richard Hummelbrummer, Martin Reynolds, Bob Williams

764 views
657 views

Published on

Published in: Education, Technology, Spiritual
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
764
On SlideShare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
16
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

IDS Impact Innovation and Learning Workshop March 2013: Day 1, Panel Session 1 Richard Hummelbrummer, Martin Reynolds, Bob Williams

  1. 1. IDS WORKSHOPImpact, Innovation, LearningPanel SessionValues, Learning, and SystemsA framework for critical rigourin impact evaluationsRichard HummelbrunnerÖAR Regionalberatung Graz, AustriaMartin ReynoldsSenior Lecturer, The Open University, UKBob WilliamsIndependent Consultant, Wellington NZ
  2. 2. CORE STATEMENT…. evaluation asks and answers questionsabout the quality, value, and/or importance ofthings ….If we’re not doing that, we’re not actuallydoing evaluation.Jane Davidson: Genuine Evaluation March 2013
  3. 3. The System FieldDifferentsystemstraditions whichhave shapedcontemporarysystemspracticeSome Streams of Systemic Thought(Draft update — May 2001)* * *Originated in 1996 by Dr. Eric Schwarz, Neuchâtel, Switzerland.Extended in 1998, including items from the The Story of Philosophy by Will Durant (1933).Elaborated in 2000-2001 from many sources for the International Institute for General Systems Studies.Currently a research project of the IIGSS.- - -This rendition is the property of the International Institute for General Systems Studies.All Rights Reserved.Errors and omissions in this chart are solely attributable to the IIGSS.Anti-chaos &AdaptationStuart KauffmanCriticalityPer BakSelf-organizationDissipativeSystemsIlyaPrigogineComplex DynamicalSystemsRalph AbrahamChaos TheoryM.J. FeigenbaumNonlinearSystemsSystemsBiologyRobert RosenEcodynamicsKennethBouldingGeneralSystemsTheoryMajiloMesarovicDynamic ProgrammingRichard BellmanInstrumentalPragmatismJohn DeweyPsychologicalPragmatismWilliam JamesPragmatic SemioticsCharles S. PeirceWhole SystemsDesignHarold NelsonGeneral SystemsPhilosophyErvin LaszloDialecticalIdealismGeorg HegelEvolutionaryPhilosophyHerbert SpencerNeuchatelEvolutionaryModelEric SchwarzNatural EvolutionCharles Darwin1809-1881CyberneticsNorbert WienerInformationTheoryClaudeShannonClassicalThermodynamicsClausius,BoltzmannSemantic TheoryDonald MacKayInformatics(Computer Science& Engineering)Mathematics ofFeedback &ElectrodynamicsJamesClerk-MaxwellFeedback governorJames WattSciencesof theArtificialHerbertSimonSocial SystemsTalcott ParsonsSociologicalSystemsWalter BuckleySocial Entropy TheoryKenneth D. BaileyAutopoietic SocialSystemsNiklas LuhmannManagement CyberneticsPeter SengeAppreciativeSystemsGregoryBatesonSocial EcologyEric Trist,Fred EmeryFamilySystemsTherapyWatzlewickSystems EcologyHoward T. OdumProgrammable LoomJoseph-Marie JacquardAnalytical EngineCharles BabbageTheory of ComputationAlan M. TuringRational PhilosophyAristotle384-322BCSymbolic LogicGeorge BooleSymbolic LogicAugustus de MorganAppliedMathematicsJohn von NeumannArtificialIntelligenceMarvin MinskyCellularAutomataArtificialNeural NetsDesigning SystemsBela H. BanathyTeleonicsGyuri JarosCritical SystemsMichael C. Jackson,Robert Flood,Werner UlrichTotal SystemsInterventionRobert FloodMulti-MethodologyJohn MingersMetamodelingJohn Van GigchSelf-organizedEconomicsW. Brian ArthurInfodynamicsStanley SaltheConventionalismHenri PoincaréFractal SystemsBenoitMandelbrotArtificial LifeChris LangtonMathematicalSocial ScienceAnatol RapoportChaotic SystemsEdward LorenzComplex Systems(Santa Fe Institute)Homeostatic SystemsWalter CannonQueuingTheoryHolismJan SmutsTektologyA.A. BogdanovKnowledgeSciencesBrian GainesEngineeringArchimedesSystemsDevelopmentJohn WarfieldLinkagePropositionsLen TroncaleIdealistic PhilosophyPlatoSystemic DevelopmentRichard BawdenSynergetics IIR. BuckminsterFullerRheomodalSystemsDavid BohmCybernetic SemiosisDouglas R.HofstadterTransformationalLinguisticsNaom ChomskyGeneral SemanticsAlfred Korzybski,Systems TrendsKenneth E.F. WattGeophysiology (Gaia)James E. LovelockHierarchy TheoryHoward H. PatteeDevelopmentalStructuralismJean PiagetCo-creative ProcessHector SabelliHolonomic SystemsKen WilberCommunicativeActionJürgenHabermasComplex EvolutionarySystemsPeter M. AllenKnotted SystemsLouis KauffmanInterpretive SystemologyRamses Fuenmayor,Hernan Lopez-GarayInformationalMicrodynamicsVladimir LernerSocial SemioticsFloyd MerrellSocio-CyberneticsR. Felix GeyerPansystemsWu XuemouCritical OntologyMichel FoucaultMetasystem TransitionTurchin, Heylighen, JoslynSystemic SelfnessPaul RyanHomeorheoticSystemsWilliam IrwinThompsonSystems SemioticsLuis Rocha,Howard PatteeEpistemology of ScienceSir Arthur EddingtonTimeDilationLorentzTopology ofMeaningR. Ian FlettPhilosophy of PhysicsJohn Archibald WheelerMathematical TopologyCatastropheTheoryRené ThomGeometrodynamicsAlbert EinsteinQuantum SystemsPlanck, Bohr,de Broglie, Dirac,SchrödingerSemantic PragmatismLudwig WittgensteinGeometry of Meaning& Reflexive UniverseArthur M. YoungAbstraction ofPhysical LawsGalileo GalileiObservationalAstronomyTycho BraheHeliocentric AstronomyNicolaus CopernicusSocio-TechnicalSystemsHarold LinstoneSoftwareTheorySocialPsychologyAbrahamMaslowSystemsAnthropologyMargaret MeadMathematicalAstronomyJ. KeplerStructuralismClaudeLevi-StraussProgrammingAda LovelaceEngineering DesignLeonardo da VinciHuman ScienceGiambattista VicoGeologicalSuperorganismJames HuttonBiosphereEduard SuessEarth asOrganismYevgrafKorolenkoEarth as OrganismVladimir VernadskySpecies CyberneticsLotka & VoltarraHolonicsJeffreyStampsPolitics of BenefitCiceroElectronic DigitalPrototypeAtanasoff & BerryElectronic DigitalComputingEckert & MauchlyOperating SystemsBrooks et alOperating LanguagesKenneth Iverson et alEndosymbiosisLynn MargulisFuzzySystemsLofti ZadehFuzzy LogicBart KoskoCalculus ofVariationsDialecticalMaterialismKarl MarxParadigmaticRevolutionsThomas KuhnPeriodicTableDmitryMendeleevSystematic MedicineHippocratesMedicineCelsusClinical MedicineThomas SydenhamConservation of matterAntoine LavoisierPhysical ChemistryRobert BoyleAntibacterialsAlexander FlemingElectromagneticPropagationsHeinrich HertzAntisepsisListerClinicalPsychiatryBenjamin RushConservationof EnergyJouleIntrinsicUncertaintyHeisenbergSuperstringsBrian Greene, et alEvolutionAlfred WallacePendulum Clock &Wave Theory of LightChristiaan HuygensDeterminismLaplaceScientificPhilosophyJacques MonodMathematicalNominalismGottfried von LeibnizImmaterialEmpiricismGeorgeBerkeleyEcologyC.S. HollingBio-cyberneticsManfried EigenTranscendentalismRalph WaldoEmersonNatural PhilosophyNicholas von CusaPolitical HegemonyFrederick the GreatSemioticUmweltJakob vonUexküllBio-semiosisJesperHoffmeyerComparative LinguisticsBenjamin Lee WhorfAtomisitcLinguisticsAntoine FabredOlivetSymbolic LogicWillard VanOrman QuineOperatinghardwareBlauw et alLinearProgrammingOptimalityTheorySynergyPeter A. CorningAppreciative SystemsWilliam E. SmithHolonicsArthur KoestlerMathematical PhilosophyPythagorasc.580-500BCGeometryEuclidLogical PositivismRudolf CarnapProcessHenriBergsonMeta-SkepicialEmpiricismDavid HumePolitical PragmatismJohn LockePoliticalRomanticismJ.J. RousseauCritical TranscendentalIdealismImmanuel KantFluxation& Unified OppositesHeraclitusc. 540-480 B.C.CynicPhilosophyAntisthenesCynicismDiogenesDialecticStoicismZeno of CitiumReligious StoicismEpictetusStoicismMarcus AureliusCosmological UnityGiordano BrunoPhilosophy of the WholeBaruch SpinozaNature Philosophyvon SchellingWill & Idea in NatureArthur SchopenhauerPerspectivismFriedrich NietzscheNoosphereTeilhardde JardinDecisionTheoryGeometricProgrammingUtilityTheoryLaws of FormGeorgeSpencer-BrownBlown Up SystemsShoucheng OuYangStructuredSoftwareYourdon,Constantine, DeMarco,Critical Theory ofSocietyMax Horkheimer,Theodor AdornoEcologicalCyberneticsGarrett HardinI Ching2000-1000 B.C.Social CyberneticsStuart UmplebyEco-pathologyRachel CarsonSructuralistMathematicsBourbakiKEY:whit e general syst emsred cybernet icsblack physical sciencesblue mat hemat icsmagent a comput ers & informat icsgreen biology & medicineyellow symbolic syst emsorange social syst emsolive ecologygray philosophycyan syst ems analysispurple engineeringGeneralSystemologyD.H. McNeilAnthropologicalLinguisticsEdward SapirInternational Encyclopedia ofSystems & CyberneticsCharles François1997MathematicalLogicBertrandRussellGeneral Systems TheoryYi LinClassical MathematicsDevelopmentof MentationWalter LowenPunctuated EvolutionStephen J. GouldCommunicational CyberneticsKlaus KrippendorffAtomismDemocritusSophisticPhilosophySocratesOrganizing relationsNeedham, WoodgerRationalisticDualismRené DescartesAnalysis ofComplexificationConrad H.WaddingtonRadicalConstructivismErnst vonGlasersfeldAutopoieticSystemsRod SwensonUnity & StasisParmenidesAtomismLeucippusDialecticZeno of EleaNaturalismThalesReciprocitiesAnaximanderScientificPositivismFrancis BaconScientific EmpiricismRoger BaconScientific MethodRobert GrossetestePan-ecologyJohn MuirSystemsPhilosophyMario BungeModelingJ.L. le MoigneInheritanceGregor MendelGenomicsCraig Venter,Francis CollinsSystemics ofBenefitWarren ZieglerEco-futurismHazel HendersonCriticalPhenomenologyEdmund HusserlEmpiricalSensationalismEtienne CondillacEvolutionaryOrganicismSaint-SimonEmpiricalPositivismErnst MachPositivism &SociologyAuguste ComtePositivismLucretiusPositivismEpicurusPositivismAristapippasUtilitarianismJeremy BenthamEmpiricalUtilitarianismJ.S. MillTranscendentalIdealismJohann FichtePsychologicalDeterminismThomas HobbesDeterminismVoltaireProgressivismCondorcetIntentionalityFranz BrentanoExistentialismMartin HeideggerDemographicCyberneticsThomas MalthusPolitical EconomyAdam SmithModernEconomicsJohn MaynardKeynesEconomicsPaul SamuelsonSimulationProblematiques& PlanningHasan OzbekhanAutopoieticSystemsHumbertoMaturana,FranciscoVarelaHarmony of OppositesLao Tzuc. 570-490 B.C.StatisticsKarl PearsonPagan CeltsPhilosophy ofScienceFritjof CapraMetaphysics ofScienceGary ZukavEconomicCyberneticsDavid RicardoInformationTheoryL. BrillouinLinguisticMathematicsHekki HeiskanonRelativismPostmodernismEvolutionaryPositivismRichard DawkinsDeconstructionismJacques DerridaRelativity ofRationalitesFeyerabendStructuralisticSemiologyFerdinand deSaussureSociologyEmilDurkheimNarratologyRoland BarthesLanguage &SymbolismComparativeLanguagesWilliam JonesLiterary SemioticsUmberto EcoLogical PositivismA.J. AyerScholasticicalEmpiricismThomas AquinasPersonalismRenouvierRationalismAvicennaRationalismAverroësPsychoanalysisSigmund FreudAnalyticPsychologyCarl JungScholasticicalSciencesAlbertus MagnusScientific PhilosophyPaul C.W. DaviesPhilosophy of Physics& MathematicsF. David PeatElectromagnetismAmpere, Kirchhoff,Faraday et alElectrostaticsBenjamin FranklinMagnetismGaussBiostructures(chirality)Louis PasteurMedicineMosesMaimonidesScholastical Logic,Dialectic & LinguisticsPeter AbelardScholasticismSaint AnselmNoetic SystemsWalter J. OngSemiotic CatholicityRandolph F. LumppMediaSemioticsH. MarshallMcLuhanSkepticismMichelMontaigneMaterialismDenis DiderotManagementSciencePhilosophy of RegularityPhilosophy of IrregularityRiver Map & Luo BookSociality& SelfConfuciusYinYangFu XiDivinationFormalRegularityZhu XiTessellationsRoger PenroseGeneral SystemsEvolutionErich JantschEcologicalDemographicsPaul EhrlichStabile SystemsClaude BernardParadoxicalImageryM.C. EscherAnatomicalMedicineGalenMedicineAvenzoarMedicineTaddeo AlderottiMedicineGuy de ChauliacAnatomical MedicineAnreas VesaliusChemical MedicineAureolus ParacelsusMicrobialsvon LeeuwenhoekCirculationWilliam HarveyModern Theoretical PhysicsClassical PhysicsCritical PhysicsRonald W. MosesCosmologicalPhysicsStephen HawkingEnvironmentalismDavid BrowerDialecticalNaturalismFriedrichEngelsGrand Unified TreoriesSteven WeinbergFuzzy SystemicsVladimir DimitrovGeneral System TheoryLudwig von BertalanffyOperational ResearchRussell L. Ackoff,Stafford BeerFirst Order CyberneticsSciento-politicsIan MitroffPoliticalSystemicsAnthony WildenMilitaryHegemonyAlexanderthe GreatScientificIntuitionismMichael PolanyiSelf-Reference& AutonomyControl TheoryOperations ResearchGeneral SystemsPurposeful SystemsRussell L. AckoffPhilosophyof SystemsThomasCowanExperimentalismE.A. SingerInquiring SystemsC. West ChurchmanGeneral SystemsThinkingGerald M.WeinbergManagement CyberneticsBarry ClemsonSystems CyberneticsW. Ross AshbyOrganicism & ProcessAlfred North WhiteheadGeneral SystemsModelingGeorge KlirPerceptualControlTheoryWilliam T.PowersLiving SystemsJames Grier MillerControlSystemEngineeringSystems DynamicsJay W. Forrester,Donella MeadowsDNACrick,WatsonImmunizationJonas SalkDirectiveCorrelationSommerhoffConversationTheoryGordon PaskPolitics of CynicismNiccolo MachiavelliMechanisticPhysicsIsaac NewtonCriticalEpistemologyKarl PopperLinear &NonlinearMathematicsJohn CastiHyper-mathematicsCharlesMusesSoft SystemsMethodologyPeter ChecklandRationalMechanicsApproximation-Estimation TheoryGrey SystemsSifeng Liu, Yi LinGeneral TropodynamicsSoucheng OuYang, Yi LinAlgebraAryabhataAlgebraBrahmaguptaAlgebraBhaskaraAlgebraOmar KhayyamAlgebraFibonacciRotationalInvarianceP.W. BridgemanIndianPhilosophyScientific ManagementFrederick W. TaylorRecyclical UniverseHindu MythologyCompetitiveEvolutionThomas HuxleySystems &ProceduresHeroic LegendsHomerRosetta StoneEgyptAppliedGeneral SemanticsS.I. HayakawaCyber-semioticsSøren BrierGeneral SemioticsThomas SebeokSynergeticsHermann HakenComparativeAnthropologyFranz BoasSystems EngineeringHarold Chestnut,Andrew P. SageSystems AnalysisArthur D. Hall,Van Court HareSecond OrderCyberneticsHeinz von FoersterTopology ofCyberneticsWarrenMcCullochViable SystemsStafford BeerHolographicSystemsKarl PribramConstitutionalPragmatismJames MadisonPhilosophy ofSystemsArchie BahmBehavioralPsychologyB.F. SkinnerPositivismJosefPopper-LynkeusMathematical LogicGottlob FregeMeasure TheoryGiuseppe PeanoProjectiveGeometryGeorg CantorImperial HegemonyRoman EmpireGeneticsWilliam BatesonSocial CyberneticsGeoffrey VickersDemocracyCleisthenesCuneiformSumeriansHieroglyphicsEgyptiansAlphabetPhoeniciansRegularizedLanguageIndo-EuropeansDemocratic IdealismThomas JeffersonDemocratic DevelopmentPericlesAstronomyBabylonClassical BiologyWholistic BiologyBarry CommonerFormalizedLogicSemio-PhysicsRené ThomGroup TheoryFelix KleinLinearAnalysisJ. FourierAlgorithmicComplexityTheoryG. ChaitinIdeographicsChinesePictographicsAboriginalsFormalismDavid HilbertLinearAnalysisDirichletSocialSystemsSciencesFieldTheoryEvaristeGaloisAlgebraicFunctionsBernhardRiemannAnalytic GeometryPierre de FermatAnalytic FunctionsLeonhard EulerAnalytic MechanicsJoseph-LouisLagrangeImperialHegemonyNapoleonProbability TheoryBlaise PascalAlgebraDiophantusSocial PsychologyWillis F. OvertonMathematics ofConcentric SpheresCallippusAstronomyEudoxusGeometryTheudiusLawSolonConic SectionsApolloniusAtomicWeightsJohn DaltonInorganicChemistryOrganicChemistryLiebigBio-TaxonomyCarolus LinnaeusGradual GeologicalEvolutionCharles LyellGeologicalTectonicsSymmetryA. ZeeManualTranscriptionsMonastic ScribesPrintingJohannesGutenbergObservationalAstronomyHipparchusGeocentricAstronomyPtolemyPolymath PhilosophyNicholas OresmeQuaternionsWilliamHamiltonInternetARPATelecommunications& Data BasesJames MartinApocalypticDualismZoroasterc.628-551BCNumericalAnalysisHypervisorsEmulationCollateration& HypertextTed NelsonWorld Wide WebTim Berners-LeeSystemic PerspectivismExperimentalPsychologyWilhelmWundtPsychologicalModelingClark HullClinicalPsychologyTechno-PsychologyPsycho-BiologyPsycho-PhysicsMorphogenesisDArcyThompsonGeo-ImpactEvolutionEugeneShoemakerMathematicalBiologyA.J. LotkaEigenvalues &SuperpositionDanielBernouliGeneticAlgorithmsSturm-LouivilleTheorySystemsAnalysisPerturbationTheoryGeneralizedUncertaintyJ. FordDocumentary& NarrativePhonemesRoman JakobsonHieroglyphicsMayansSocialAnthropologyBronislawMalinowskyIlluminationismSaint AugustineRationalVoluntarismJohn Duns ScotusNominalismWilliam of OckhamLogical NominalismJean BuridanInterpretivePhysicsB.J. WestMechanical CalculatorWilhelm SchickardArithmometerCharles de ColmarTabulating MachineHerman HollerithRapidMiniaturizedComputingNanotechnologyEncodingSchemesR.W. HammingPhilosophy ofNature & ScienceCarl vonWeizsäckerBifurcatedCulturesC.P. SnowEvolutionarySystematizationPierre DuhemPatternRecognitionPsycho-MetricsEcologicalEvolutionErnst HaeckelMolecularGeneticsEmpiricalMetaphysicsGeorge HenryLewesEvolutionaryPhilosophyRichard L. CorenComparativePsychologyC. Lloyd MorganEmergent ProcessSamuel AlexanderLinear AlgebraBenjamin PeirceAlgebra of LogicErnst SchröderAssociative AlgebrasWilliam CliffordStatisticalBiologyKarl PearsonModernStatisticsJerzy NeymanStatisticalMechanicsJ. WillardGibbsModel TheoryBernhardBolzanoModel TheoryAlfred TarskiModel TheoryMichael MorleyMetalogicAlonzo Church MetalogicKurt GödelIntuitionismL.E.J. BrouwerCategoryTheorySaundersMac LaneDoubleRefractionAugustinFresnelMetricUnitsAstronomicalMeasurementPierre MechainSociologicalPhenomenologyMax SchelerPhilosophy ofHistoryWilhelm DiltheyRenaissancePlatonismGiovanni PicoNumber TheoryMarin MersennePolymath PhysicsRobert HookeIndustrial SociologyG. Elton MayoPhilology, Linguistics& GrammarEthnolinguisticsWilhelm vonHumboldtEthnolinguisticsJohann HerderEtymologyJacob GrimmTagmemicsKenneth L. PikeStratificationalGrammarSydney LambGlossematicsLouis HjelmslevMorphologyHistoricismBenedetto CroceConceptualismErnst CassirerTropologyPractical MedicineEgyptiansColonialHegemonyBritish EmpireCulturalHegemonyUSAMoralPhilosophyFriedrichSchillerOrganicismEdward Stuart RussellPhilosophyof BiologyMorton O.BecknerNumber TheoryG.H. HardyPhilosophyArchelausPhilosophyAnaxagorasCognitivePsychologyAaron BeckStructural LinguisticsN.S. TrubetzkoyRealismG.E. MooreAnalyticalPhilosophyGilbertRyleSophismProtagorasGameTheoryAutomataTheorySoftwareMeasurementCapers JonesBinary NumbersChinaMeta-languages &LinguisticsU. of Nalanda, IndiaAlgorithmsDonald KnuthStructuralistPsychologyEdward TitchenerGestaltPsychologyMaxWertheimerNew CriticismJohn CroweRansomElementalismEmpedoclesDevelopmentalPsychologyCognitiveScienceAllen NewellMechanistic MaterialismJulien La MettrieComputationismJerry A. FodorLiteraryCriticismI.A. Richards StandardizedLinguisticsC.K. OgdenCritical PhilologyDesiderius ErasmusHumanismPetrarchCognitiveLinguisticsGeorge Lakoff &R.W. LangackerIntensionalLogicMontagueLinguistic LogicJerrold J. KatzPhilosophyof Art andLanguageSusanne K.LangerProblem-SovlingPropositional LogicChrysippusReligious StoicismCleanthesDiogenes of BabylonRoman StoicismPanaetiusRoman StoicismPoseidoniusRoman StoicismSenecaTectonicEngineeringJohn RoeblingIndustrialDesignCoherentismOtto NeurathPERTDelphiConditionalProbabilityThomas BayesEnlightenmentJean dAlembertEcologyEthologyKonrad LorenzObjectiveInformationT. StonierBiologicalEvolutionJean LamarckVorticlesWilliamThomsonSociobiologyEdward O. WilsonSystematicsErnst MayrInterpersonalPsychologyR.D. LaingCyberneticEpistemologyRanulphGlanvilleMathematicalEconomicsAntoine CournotIdealistic LiberalismWilliam GodwinMathematicalPsychologyR.D. LuceNeo-PlatonismPlotinusSocialPsychologyKurt LewinGestaltPsychologyWolfgangKöhlerMathematicalEconomicsVilfredo ParetoPsycholinguisticsSociolinguisticsComputationalLinguisticsLinguisticPsychoanalysisJacques LacanGroupDynamicSociometricsPoliticalSociologyMax WeberEconometricsPhilosophy ofLawHans KelsenEugenicsFrancisPrimateAnthropologyJane GoodallPrimateLinguisticsLinguistic ScienceLeonard BloomfieldPhonologyPhysics AsMetaphorRoger S. JonesAppliedTranscendentalismHenry DavidThoreauInstrumentalRationalityPeter RamusMethodizedEncyclopediasJohannHeinrichAlstedConscientizationPaolo FreireRoboticsBiodiversityUrbanEcologyLouis WirthBiotechnologyGeneticEngineeringGreenRevolutionNormanBorlaugHeredityP-L. de MaupertuisPsychoanalysisJosef Breuer
  4. 4. Core System ConceptsUnderstanding interrelationships associated with asituation;Engaging with contrasting perspectives regarding asituation, andReflecting on boundaries of such representations andinteractions
  5. 5. INTERRELATIONSHIPS – Dynamics, Context, Complexity• What is the structure of interrelationships within the situation (how arecomponents of intervention and/or effects connected)?• What is the nature of the interrelationships (e.g. strong/weak, fast/slow,direct/indirect, linear/non-linear)?• What are the patterns that emerge from the interrelationships betweenintervention and effects over time (e.g. simple, complicated, complex)?• How context sensitive are these patterns (different effects at different timesor in different contexts)?• How will this affect the ways in which effects can be associated with theintervention? Understand what really works in what circumstances?
  6. 6. INTERRELATIONSHIPS – Relevance to Impact EvaluationEmergenceUnintended & Unanticpated ConsequencesNeed For Keeping Measures Adaptable Rather Than Fixed
  7. 7. PERSPECTIVES – Stakeholders, Stakes, Framing
  8. 8. PERSPECTIVES– Relevance to Impact EvaluationUntangling map from territoryAssessing why things worked out the way they didChallenging idea of single “system”
  9. 9. BOUNDARIES – values, power, knowledge, legitimacy
  10. 10. BOUNDARIES– Relevance to Impact EvaluationBroadening the range of values (utility, rights, virtues)Identifying the values underpinning the interventionIdentifying the values underpinning the impact evaluation
  11. 11. Types of LearningEffects InterventionValues,assumptionsRules,knowledgeSingle loop learningDouble loop learningTriple loop learningAdaptive actionsUnderlying causes and consequencesMechanisms for learning, coping with problems and changes
  12. 12. Types of Learning and Systems ConceptsPerspectives BoundariesSingle loop learningDouble loop learningTriple loop learningInterrelationshipsAre we doingthings right?Are we doing the right things?Is rightness buttressed by mightiness (or vice versa)?Questions purpose, goalsPower for determining purpose and goalsIntervention Effects
  13. 13. A systems - based framework for rigor in evaluationInterventionPerspectivesBoundariesSingle loop learningDouble loop learningTriple loop learningInterrelationshipsInstrumentalvaluesCritical /PoliticalvaluesIntrinsic /personalvaluesEffects
  14. 14. (ii) Partial inserving somestakeholder partiesincludingpractitioners - orinterests - betterthan others(i) Partial inrepresenting only asection rather thanthe whole of thetotal universe ofinterrelationships inany context thatmattersAll systems are partialSystemic rigour and impact evaluationFramework 2 (triangulation): 3 systems concepts in action
  15. 15. 1. Complexsituations orContexts ofchangeand uncertaintyAll Systems are partial in the sense of :1. representing only a section rather than the totaluniverse of considerationsand2. serving some parties - or interests - better thanothersPolitical space for exploring boundary judgementsSocio-economic spacefor exploringjudgements of‘fact’Learning space forexploring valuejudgements(i) Framinginterrelationships andinterdependencies(ii) Framing engagementwith multipleperspectives3. Systems and other conceptual Tools(iii) appreciating limits onboundaries of interrelationshipsand perspectivesFramework for justifying boundaries of:(i) Framing interrelationships(ii)Framing multiple perspectives2. Peopleor stakeholders/practitionersCritical rigour and impact evaluationFramework 3 (systems thinking in practice): 3 systems concepts in action
  16. 16. Relevance to Impact Evaluation Emphasis of CAS and CriticalSystemInter-relationshipsEmergence and uncertainty of impacts = systemic effect from interactionof parts rather than single causality… whole is more than the sum of it’s partsCAS investigates the territoryCritical systems investigates theimplications of different maps of theterritoryMultiplePerspectivesComplexity of ‘impacts’ arise from interaction between differentstakeholders and stakesCAS illustrates different perspectivespredominantly from value-perspectiveof utilityCritical systems explores the valuecomponent of different perspectivesand ‘developing value’BoundaryJudgementsSystems thinking explores single, double and triple loop learning interms of not only:1. Is it being done right (efficacy, efficiency)? but also…2. Is it right thing to do (effectiveness and ethics)? and...3. Is rightness confused with mightiness (politics and powerrelations)?CAS largely focuses on efficacy,efficiency and effectivenessCritical systems focuses in additon oneffectiveness, ethics and powerCritical systemsRelevance and complexity
  17. 17. General• Good impact evaluation requires address to perspectives and boundaries in addition (& in relation)to complexity of interrelationships• How might systems concepts complement and enhance (rather than replace or merely add to)existing evaluation tools?Interrelationships• Measures of success (in a system) must adapt to change in circumstances (in situations)• In what way can systems ideas improve the rigour and relevance of impact evaluation?Perspectives• Purposeful (as against ‘purposive’) developmental evaluation requires acknowledgement of, andengagement with, ‘victims’ of intervention amongst others, in generating new value• What are the conditions under which using systems ideas in impact evaluation is feasible andvalue-able?Boundaries• Learning in impact evaluation requires address not only to utility (who learns what when?), butchanges in rights (and values) and responsibilities (power relations) ….triple loop learning• What are the conditions under which using systems ideas in impact evaluation might be worthwhile(good, right and politically just)?Challenges from a (critical) systems perspectiveQuestions for discussion
  18. 18. Systems Concepts inEvaluation. Anexpertanthology(AEApublication)eds. BobWilliams & IrajImam2007Systems Approaches toManagingChange: apractical guideeds. MartinReynolds &Sue Holwell2010Systems Concepts inAction : apractitionerstoolkiteds. BobWilliams &RichardHummelbrunner2010Richard HummelbrunnerSenior PartnerÖAR Regionalberatung GrazAustria+43 316 31 88 48 - 15http://www.oear.atemail: hummelbrunner@oear.atMartin ReynoldsSenior LecturerThe Open UniversityUK+44 (0)1908 654894http://sites.google.com/site/jjntest1/Home/people/martin-reynolds-1email: martin.reynolds@open.ac.ukBob WilliamsIndependent ConsultantWellingtonNew Zealand(64) 21 254 8983http://www.bobwilliams.co.nzemail: bob@bobwilliams.co.nzContacts...and some book publications

×