Recommended
Recommended
More Related Content
What's hot
What's hot (19)
Similar to What is a Logic Model, and How Can it Help Me Organize My Research?
Similar to What is a Logic Model, and How Can it Help Me Organize My Research? (20)
Recently uploaded
Recently uploaded (20)
What is a Logic Model, and How Can it Help Me Organize My Research?
- 1. Explore! What's a logic model, and how can it help me organize my research? Bryn Robinson, PhD Research Engagement Manager Horizon Research Services
- 2. Outline • What is a logic model? • Why would you use one in health research? • How do you build a logic model? • Examples • References and resources
- 3. Example: Vaccination Education Letters with information to parents or guardians Information for public health nurses to share with families Brochures for parents Posters in doctors’ offices and schools Social media engagement Website with information Public information sessions
- 4. What is a Logic Model? • Visualization tool for applied research • Identifies relationships: • What you plan to do • How you plan to do it • What will then happen as a result of your actions
- 5. Anatomy of the Logic Model
- 6. Activities • What are you doing to make change? • Is there a logical order to these events?
- 7. Example: Vaccine Education • Distributing information: • Information letters • Brochures • Website • Social media posts • Posters • Hosting public talks • Slides Distributing and Hosting are the activities. Clearly identify something you do, vs. what you need to do it.
- 8. Inputs • Financial • Personnel • Technology • Information • Partnerships
- 9. These are some inputs. Ask: Who is creating content? How is it being produced? What are your costs? Example: Vaccine Education • Information letters • Website • Brochures • Slides for public talks • Social media posts • Posters
- 10. Outputs vs. Outcomes • Frequent source of confusion • Outputs • “How do I know my activity happened?” • Outcomes • “How do I know my activity worked?”
- 11. Example: Vaccine Education • Information letters • Website • Brochures • Slides for public talks • Social media posts • Posters Outputs: How do I know the activities happened? Indicators could be manually or automatically generated # website visits # attendees to talk
- 12. Example: Vaccine Education • Information letters • Website • Brochures • Slides for public talks • Social media posts • Posters Outcomes: How do I know the activities worked? Did awareness increase? Did knowledge improve? Did vaccination rates change?
- 13. Assumptions • The underlying “if-then” assumptions represent the program’s theory of change: how we think a policy or program will lead to desired outcomes.
- 14. Anatomy of the Logic Model
- 15. Example: Vaccination Education Brochures sent home for parents Social media engagement Website with information Public information sessions ACTIVITIES OUTCOMES Increased parental awareness Increased parental knowledge Improved vaccination rates OUTPUTS # taken / # distributed # likes & shares # comments # sessions # attendees # visits # link shared INPUTS Graphic design Content (Evidence) Presenters Staff on social media Production costs Salary for staff Staff to get evidence
- 16. PATIENTS 500 patients newly diagnosed with cancer, eligible for this treatment Example: Applying for a Grant Three types of assays to diagnose cancer Three experimental assays to diagnose cancer Patient-reported outcomes STUDY ACTIVITIES & INDICATORS STUDY GOALS Identify strategies for best diagnosing this cancer Explore new methods of characterizing illness at molecular level Understand quality of life and how treatment impacts it Progression-free survival Overall survival # identified biomarkers # new mutations # new mechanisms Scores on quality of life measures Recruiting at the following sites… • Site 1 • Site 2 • Site 3
- 17. Example: Starting New Research Program
- 18. Why Use Them? • Identify missing resources • Gaps in theory of change • Course-correct • Check assumptions • Knowledge translation
- 19. Getting Started
- 20. Resources U Sask: Logic Models for Program Evaluation: https://teaching.usask.ca/articles/logic-models-purpose-and-parts.php U Wisconsin: Program Development and Evaluation: https://fyi.extension.wisc.edu/programdevelopment/logic-models/ CDC: Making Logic Models Work for You: https://www.cdc.gov/dhdsp/pubs/docs/cb_september_2013.pdf Community Tool Box: https://ctb.ku.edu/en Examples: • Allied Health Intervention for Older Adults in ED – Cassarino et al 2019 (10.1136/bmjopen-2019-032645) • Addressing Social Inequity through Recreation – Elsborg et al 2019 (10.1186/s12889-019-7428-3)
- 21. Questions? Bryn Robinson, PhD Research Engagement Manager, Office of Research Services ResearchServices@HorizonNB.ca @brynphd #ResearchAtHorizon #RechercheChezHorizon
Editor's Notes
- When we create a program, an activity, an intervention, it is typically with the thought that, “If we deliver X then Y will occur”. But how do we know X happened? Did X actually lead to Y – or did it lead to Z instead? What resources do you need to make all this happen? Are these the right activities to do? How do you know these happened?
- Logic models are a tool with origins in program planning and evaluation, but are valuable in clinical health research, especially as our questions become more applied in nature. We see this in the terms out there to describe research: e.g., implementation science, KT, studies on cost-effectiveness Logic models are a road map that illustrate how we think doing an activity will lead to a particular outcome (or what is known in PE parlance as “theory of change”). It links projected outcomes with processes, the underlying theoretical assumptions of the program, and the shared relationships between the different components. How a proposed program or activity is SUPPOSED to work The “what we did” and how it leads to the “so what” Can focus on any level of a policy, program or project: the entire organization, one of its component departments or programs, or just specific parts of that department or a program. Of course, the boundary between “what” and “so what” will vary accordingly. Examples later will so that. Great tool to map out the parts of a research project, especially those that have a number of sub-studies or arms, and/or those that employ a number of different methodologies (e.g., mixed methods design).
- Parts of the logic model – terms and definitions that we need to cover first They can differ across discipline and project scope, but will frequently illustrate: Inputs: The resources needed to implement the activities Activities: What the program and its staff do with those resources Outputs: Tangible products, capacities, or deliverables that result from the activities Outcomes: Changes that occur in other people or conditions because of the activities and outputs Impacts: [Sometimes] The most distal/long-term outcomes Moderators/Assumptions: Contextual factors that are out of control of the program but may help or hinder achievement of the outcomes
- Activities are the actions undertaken by the program or partners to produce outputs that then lead to desired outcomes; in other words, activities are what we do. Decide whether the activities should be ordered sequentially. If so: Think about the “logical” relationship among the activities—which may or may not be the same as how they unfold over time— and determine if some activities need to occur before others can be implemented
- Above is the various “things” that we had on our earlier slide. When we look at these, we can see that they are really referring to two types of knowledge translation activities for this education program
- What you need to do it = Inputs. Inputs are resources that go into a program or intervention; they are what we invest. They include financial, personnel, and in-kind resources from any source. Think about broad categories such as staff, equipment, data, funds, and partnerships. Ask yourself: What are the key inputs without which the program cannot be implemented?
- To identify the inputs for our activities, we would ask questions shown above. This exercise rapidly identifies where we may have made assumptions about who is doing what, or how much time is going into this program’s development and implementation. What I find here is that people will quickly learn to scale back a program to essential parts to make change based on available resources, or to identify the need to apply for grants and other sources of funding.
- Outputs are the direct, tangible results of activities; they are what we get. Essential link between what we do, and what we want to happen. Often forgotten, but always important to take a step back and ensure you have ways to capture these. If you are proposing to do X, and that your theory of X is that it leads to Y, then you have to make sure X actually happened. What are the key parts of the activity that had to take place so that the outcome could have a chance of taking place? • Outcomes are effects of the program or initiative and may be indicated as short-term, intermediate, or long-term outcomes; outcomes are what we achieve.
- What this begins to highlight is where we tend to see some challenges early in the design. For example, perhaps in this case the researcher was not planning on measuring any of these – just assuming that it would work; or, they were only going to measure at the end, not knowing what the baseline knowledge or awareness was. What we hope has happened in this case, then, is that the person came up with the strategy because of a pre-implementation level of awareness and knowledge that then needed to be addressed through said strategy.
- This is where we then uncover some truths that are fundamental to the design and success of the strategy Think of broad categories: PEST (political, economic, social, and technological) What supports and structures are you assuming are in place for this to work? Be especially conscious of key moderators without which the program cannot be implemented This is especially where the stakeholders who review your program model can help you avoid a lot of problems, remove barriers, etc. Identify what links in the program logic will be facilitated or impeded by the presence or absence of sufficient levels of the moderator. Remember moderators can facilitate or impede the ability of one activity/output to generate a successor activity/output, one activity/output to generate an outcome, a proximal outcome to generate a more distal outcome
- You’ve considered which aspects of the program to emphasize, the target population, and the type of logic model needed; it’s now time to design your logic model. While many logic models are presented in a linear diagram from left to right, logic models can be presented visually in various forms, designs, and levels of technicality. As mentioned earlier, there are some common components of logic models that are often useful in describing your program: inputs, activities, outputs, outcomes, and impacts. Have people review it Update it after your program has been running – do assumptions still hold true? Are you still doing the same activities? Which outcomes and deliverables have been achieved? Create a narrative to go with the logic model. A one-page logic model will not be able to capture all the nuances of the program. The narrative will help explain the components of the logic model and how they work together to accomplish the outcomes. The narrative should include the following: An expanded description of the activities, outcomes, and other components of the logic model Any key linkages between activities, between activities and outcomes, and between different outcomes Attribution v. contribution to outcomes, etc. Stakeholder expectations for what will be accomplished, etc. If it is important to see the link between each input and the activity it affects, then draw arrows from each input to the related activity
- A few more examples of logic models that I’ve developed for research, to show you that it’s not always one formula to produce them, or as detailed in the operations of a program as others. This one was for a grant submission, as part of a massive project with many parts: 6 separate “studies” with additional layers of PROMs. Wanted to show funders the overall gist of the proposed project, so I modified the approach to show activities with outcomes (as showing straight outputs would be too detailed for these purposes, e.g., # assays performed, etc.), and how the overall goals the team had to write for the grant are linked to different parts of the overall study.
- This one was produced for a group that was recently reviewing its programs, and wanted a way to guide their future research so that it aligns with values and priority areas identified in a recent strategic planning session. It’s also an example that your models do not have to absolutely adhere to the linear model with columns; you can start with that but then as you link the parts and consider that narrative that you’re creating, there may be other ways to better illustrate things.
- To recap, logic models can help you focus your work and clearly illustrate the linkages among investments and results that creates a theory of change. Identify missing resources: We focus more on the what we will do and the change that we think is going to have We usually start with an idea of what we want to do and how we think we will change something Which starts us later in the logic model, and then we can forget to critically look at the earlier stages It’s a bullet list – doesn’t have to be to the SKU# of the test tube you’re purchasing but if you’re designing a trial and don’t have materials for lab kit figured in, then you run the risk of missing something Show gaps in X to Y Uncover hidden assumptions about our programs or activities that we have had At the very least, one of the reasons that I would recommend using logic models for research studies, is that by mapping this information, you have to explicitly consider areas that do not have enough resources, or where assumptions are being made Can help us course-correct: Lets us see in these earlier stages when things are going off the rails. It might takes years to reach the outcome of interest (education), because there are so many changes that need to occur in the black box before that time. Logic models help to capture the contents of the black box. These are adaptable, so if we realize that we did not achieve our short-term outcomes, we need to look at our model. Can we achieve our long-term outcomes still? Is there another step or another measure that shows this change? Check that our assumptions are appropriate: Verify that our expectations for the program’s effectiveness are sound and realistic. Knowledge translation: Can demonstrate progress: Communicate information about our program to internal and external stakeholders in a reader-friendly, visual manner. It shows funders progress – if they are interested in ROI, again may not happen for years, but we can show that our program or project is still making a difference and is on the right path. Ensure that there is clarity and consensus among those who fund and deliver the program about the resources available to deliver the program, the key program activities and what they produce, and the program’s expected outcomes
- Logic models are easy to use You don’t need fancy materials Paper and coloured pens or markers Power Point Gather your information Protocol Budget justification documents Lab manuals that delineate processes Informed consent documents that outline process Websites, program brochures Data from strategic planning meetings with stakeholders Start sketching it out: Post-It notes, or paper and markers If you’re stuck, go online and get inspired: Then make it digital: PowerPoint Make sure you circulate for feedback – great gap analysis