Evidence Based Practice in the Applied Social Sciences


Published on

Oregon Library Association (OLA) 2009 Annual Conference presentation slides.

Published in: Education, Health & Medicine
1 Like
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Evidence Based Practice in the Applied Social Sciences

  1. 1. Evidence-Based Practice in the Applied Social Sciences by Robin Paynter
  2. 2. Vested Communities Academic Researchers Government Agencies Practitioners in the field Service Users Insurance Companies Evidence Based Practice Professional Organizations
  3. 3. Governmental Agencies <ul><li>United States. Department of Education. (US DOE): </li></ul><ul><li>No Child Left Behind Act of 2001 requires the use of relevant scientifically based research by educators. US DOE created the What Works Clearinghouse (see further information below) </li></ul><ul><li>Oregon. Senate Bill 267 </li></ul><ul><li>“ A unique bipartisan bill passed by the Oregon Legislature in 2003 requiring key state agencies (e.g., Department of Corrections, Oregon Youth Authority, Oregon Criminal Justice Commission, the State Commission on Children and Families and the Department of Human Services* ) to budget and implement “evidence-based programming” at incremental levels over a five year period; the legislation defined an evidence based program as “incorporates significant and relevant practices based on scientifically based research and is cost effective.” </li></ul><ul><li>From http://www.ssw.pdx.edu/focus/ebp/oregon.php. Accessed 04/01/2009 . </li></ul><ul><li>*Emphasis added </li></ul>
  4. 4. Medical vs. Social Sciences Research <ul><li>Medline = 17,000,000+ records </li></ul><ul><li>---------------------------------------------------------------------------- </li></ul><ul><li>PsycINFO = 2,600,000+ records </li></ul><ul><li>ERIC = 1,282,000+ records </li></ul><ul><li>Social Services Abstracts = 149,000+ </li></ul><ul><li>Criminal Justice Abstracts = 99,200+ </li></ul><ul><li>---------------------------------------------------------------------------- </li></ul><ul><li>▪ EVP requires a lot of research at the bottom of the pyramid to do higher level analyses - social science researchers have to go down the pyramid to find evidence as it is usually not available as RCTs, Meta-analyses, or systematic reviews </li></ul><ul><li>▪ Social science research characteristics that effect evidence-based research: </li></ul><ul><ul><li>small sample sizes </li></ul></ul><ul><ul><li>higher percentage of qualitative research, especially in some disciplines </li></ul></ul>
  5. 5. Research Skills & Strategies <ul><li>Model Query: </li></ul><ul><li>P = Patient Population </li></ul><ul><li>I = Intervention </li></ul><ul><li>C = Comparison Intervention </li></ul><ul><li>O = Outcomes </li></ul><ul><li>Application to Social </li></ul><ul><li>Science Research: </li></ul><ul><li>▪ PICO usually doesn’t work </li></ul><ul><li>▪ Find evidence farther down the pyramid </li></ul><ul><li>▪ Search Multiple Databases </li></ul><ul><li>Example : </li></ul><ul><li>P= Veterans with PTSD </li></ul><ul><li>I= Eye Movement Desensitization and Reprocessing (EMDR) therapy </li></ul><ul><li>C= ‘Classic’ Desensitization therapy </li></ul><ul><li>O= Reduce severity of symptoms </li></ul>Researcher Tool-Kit http:// www.ebm.med.ualberta.ca/ebm.htm Formulating Questions / Doing EBM Research http:// www.cebm.net/index.aspx?o =1036
  6. 6. Multidisciplinary Databases <ul><li>Campbell Collaboration: C2 SPECTR </li></ul><ul><li>Description: “The Campbell Collaboration’s Sociological, Psychological, Educational, and Criminological Trials Register (C2-SPECTR) contains over 11,600 citations to trials or possible trials. Studies are included in the register if they can be classified as a randomized control trial (RCT); cluster, place, or group randomized control trial (CRT), a possible randomized control trial (CCT), or a Systematic Review (WPH).” (Description from: </li></ul><ul><li>http://geb9101.gse.upenn.edu/RISIMAGES/c2spectrsearch.pdf Accessed July 10, 2008) </li></ul><ul><li>Availability: Free database: http://geb9101.gse.upenn.edu/RIS/RISWEB.ISA </li></ul><ul><li>Searching Tips: Keyword searching only. Citations gathered largely from other bibliographic databases, e.g. ERIC, Sociological Abstracts, Criminal Justice Abstracts, etc. </li></ul><ul><li>Campbell Collaboration: Campbell Library of Systematic Reviews </li></ul><ul><li>Description: “ The Campbell Library of Systematic Reviews provides free online access to systematic reviews in the areas of education, criminal justice and social welfare. The library is a peer-reviewed source of reliable evidence of the effects of interventions.” (Description from http:// www.campbellcollaboration.org/frontend.aspx . Accessed August 12 , 2008) </li></ul><ul><li>Availability: Free database: http://www.campbellcollaboration.org/frontend.aspx </li></ul><ul><li>Searching Tips: Reviews are listed alphabetically and also by area: Education, Crime and Justice, and Social Welfare. </li></ul><ul><li>Database of Abstracts of Review Effects (DARE) </li></ul><ul><li>Description: Contains 5000+ abstracts of quality assessed and critically appraised systematic reviews. The database focuses on the effects of interventions used in health and social care. Reviews identified by searching Medline, PsycINFO, Cochrane, etc. </li></ul><ul><li>Availability: Free database: http://www.crd.york.ac.uk/crdweb/ </li></ul><ul><li>Searching Tips: Search MeSH and keyword terms to increase search inclusiveness (e.g., ‘myocardial infarction’ retrieves different results to searching ‘heart attack’). DARE includes records from many sources, not all of which use MeSH. </li></ul><ul><li>Turning Research Into Practice (TRIP) </li></ul><ul><li>Description: Metasearch engine of evidence-based practice databases and websites, including: DARE, PubMed, Cochrane, EBP journals, and more. </li></ul><ul><li>Availability: Free database: http://www.tripdatabase.com/index.html </li></ul>
  7. 7. Psychology Resources <ul><li>PsycINFO </li></ul><ul><li>Description: PsycINFO provides citations and indexing to the published psychology literature. </li></ul><ul><li>Availability: Subscription database </li></ul><ul><li>URL: http://www.apa.org/psycinfo/ </li></ul><ul><li>Searching Tips </li></ul><ul><li>DESCRIPTORS: LIMITS: </li></ul><ul><li>Evidence Based Practice Literature </li></ul><ul><li>Treatment Effectiveness Review </li></ul><ul><li>Evaluation Systematic </li></ul><ul><li>Mental Health Program Review </li></ul><ul><li>Evaluation Meta Analysis </li></ul><ul><li>Treatment Outcomes </li></ul><ul><li>Psychotherapeutic Outcomes </li></ul><ul><li>Clinical Trials </li></ul><ul><li>Treatment Guidelines </li></ul><ul><li>Medline/PubMed </li></ul><ul><li>Description: Comprehensive database covering research literature in all areas of medicine. </li></ul><ul><li>Availability: Subscription Medline database: widely available via commercial vendors, for example: CSA, EBSCO, FirstSearch, Ovid, etc </li></ul><ul><li>URL: Free PubMed database: </li></ul><ul><li>http://www.ncbi.nlm.nih.gov/sites/entrez?db=PubMed </li></ul><ul><li>Searching Tips </li></ul><ul><li>DESCRIPTORS: LIMITS: </li></ul><ul><li>Evidence-Based Medicine Guideline </li></ul><ul><li>Decision Support Techniques Meta-Analysis </li></ul><ul><li>Practice Guidelines Randomized </li></ul><ul><li>Meta-Analysis Controlled </li></ul><ul><li>Randomized Controlled Trials Trial </li></ul><ul><li>Review </li></ul>
  8. 8. Psychology Cont’d. <ul><li>Cochrane Library </li></ul><ul><li>Description: The Cochrane Library produces and disseminates full text systematic reviews of healthcare interventions and promotes the search for evidence in the form of clinical trials and other studies of interventions. </li></ul><ul><li>Availability: Subscription database </li></ul><ul><li>URL: http://www.cochrane.org/reviews/clibaccess.htm </li></ul><ul><li>Searching Tips: </li></ul><ul><li>DESCRIPTORS: </li></ul><ul><li>Evidence-Based Medicine </li></ul><ul><li>Decision Support Techniques </li></ul><ul><li>Practice Guidelines </li></ul><ul><li>Meta-Analysis </li></ul><ul><li>Randomized Controlled Trials </li></ul><ul><li>Treatment Outcome </li></ul><ul><li>Keyword searching: best results typically achieved by using ‘Title, Abstract, or Keywords’ search. Uses MeSH (Medical Subject Headings) </li></ul>
  9. 9. Manuals, Guidelines, etc <ul><li>Essential Evidence Plus (formerly InfoPOEMs/InfoRetriever) </li></ul><ul><li>Description: “ The only database system of filtered, synopsized, evidence-based information, this integrated search engine allows you to simultaneously search, via keyword, multiple databases… EBM Guidelines is a unique, concise and easy-to-use collection of clinical guidelines for primary care combined with the best available evidence. EBM Guidelines includes 1,000 concise primary care practice guidelines, 3,000+ high quality evidence summaries and a library of 1,000 photographs and images.” Description from: https:// www.essentialevidenceplus.com/product/features_ir.cfm Accessed August 14 , 2008. </li></ul><ul><li>Availability: Subscription database </li></ul><ul><li>URL: http://www.essentialevidenceplus.com/index.cfm </li></ul><ul><li>Searching Tips: Browsing of subject guideline categories (e.g., ‘Psychiatric and Substance Abuse’) and keyword searching: within a subject guideline area or across the entire database. </li></ul><ul><li>KEYWORDS: </li></ul><ul><li>DSM disorder name (e.g., post-traumatic stress disorder or PTSD) </li></ul><ul><li>Drug name (e.g., Lexapro) </li></ul><ul><li>Treatment (e.g., cognitive behavioral therapy or CBT) </li></ul>
  10. 10. Manuals, Guidelines, etc Cont’d. <ul><li>National Guideline Clearinghouse </li></ul><ul><li>Description: &quot;A public resource for evidence-based clinical practice guidelines. NGC is an initiative of the Agency for Healthcare Research and Quality (AHRQ) , U.S. Dept. of Health and Human Services.&quot; While the primary focus of this site is currently medical guidelines, there are an increasing number of mental health/psychotherapy guidelines available. </li></ul><ul><li>Availability: Free database </li></ul><ul><li>URL: http:// www.guideline.gov / </li></ul><ul><li>Searching Tips: Mental health guidelines listing (184 guidelines as of this writing): http:// www.guideline.gov/browse/browsemode.aspx?node =16845&type=1 </li></ul><ul><li>The detailed search page offers additional ways to search, however researchers can easily and quickly specify too many elements and retrieve zero results: http://www.guideline.gov/search/detailedsearch.aspx </li></ul><ul><li>American Psychiatric Association Practice Guidelines </li></ul><ul><li>Description: Evidence-based recommendations for the assessment and treatment of psychiatric disorders. </li></ul><ul><li>Availability: Free database </li></ul><ul><li>URL: http://www.psych.org/psych_pract/treatg/pg/prac_guide.cfm </li></ul><ul><li>Searching Tips: Guidelines are arranged alphabetically by disorder (Diagnostic and Statistical Manual of Mental Disorders classification). Some links lead to subscription content on http://www.psychiatryonline.com . </li></ul>
  11. 11. Social Work Resources <ul><li>Social Services Abstracts </li></ul><ul><li>Description: Citations to research focused on social work, human services, social welfare, social policy, and community development. Major areas of coverage: community and mental health services; crisis intervention; family and social welfare; gerontology; poverty and homelessness; health policy; addiction; social work. </li></ul><ul><li>Availability: Subscription database </li></ul><ul><li>URL: </li></ul><ul><li>http://www.csa.com/factsheets/ssa-set-c.php </li></ul><ul><li>Searching Tips: </li></ul><ul><li>DESCRIPTORS KEYWORDS: </li></ul><ul><li>Evidence-based Systematic Review </li></ul><ul><li>practice </li></ul><ul><li>Treatment Outcomes Meta-analysis </li></ul><ul><li>California Evidence-Based Clearinghouse for Child Welfare </li></ul><ul><li>Description: “The California Evidence-Based Clearinghouse for Child Welfare (CEBC) is a website of evidence-based child welfare programs and practices…. Programs receive a rating based on supporting evidence, child welfare outcomes, and relevance to child welfare populations. Users can search the database by topical area, maltreatment type, scientific rating, or goals and outcomes.” </li></ul><ul><li>Availability: Free database </li></ul><ul><li>URL: </li></ul><ul><li>http://www.cachildwelfareclearinghouse.org/ </li></ul><ul><li>Searching Tips: </li></ul><ul><li>Searching by topical area or maltreatment type recommended. The advanced search page offers a number of ways to search, however searching multiple categories dramatically reduces results, often to zero. This is primarily due to the relatively small number of recommended programs and practices in the system currently. </li></ul>
  12. 12. Manuals, Guidelines, etc <ul><li>National Registry of Evidence-based Programs and Practices (NREPP) </li></ul><ul><li>Description: Searchable online registry of mental health and substance abuse interventions that have been reviewed and rated by independent reviewers. Intervention summaries include: descriptive information about the intervention and its targeted outcomes, quality of Research and Readiness for Dissemination ratings, a list of studies and materials submitted for review, and contact information for the intervention developer. Description based on http://www.nrepp.samhsa.gov/about.htm Accessed August 12, 2008 </li></ul><ul><li>Availability: Free database </li></ul><ul><li>URL: http://www.nrepp.samhsa.gov/ </li></ul><ul><li>Searching Tips: As of this writing, there are 106 interventions in the NREPP database. Researchers can use either the search form or browse the complete list to find interventions. </li></ul>
  13. 13. Social Work Evidence Pyramid <ul><li>EBP Continuum: From research-based to non research-based to promising </li></ul><ul><li>The following section describes the continuum of “evidence” that is the conventional guide for most mental health practitioners, researchers, and policy makers. These levels are developed from a wide range of sources and are meant to be a guide. They do not represent a single federal agency or academic center (Vandiver, 2006). </li></ul><ul><li>Level 1: Systematic reviews using meta-analyses or two randomized controlled clinical trials (RCT). Highest level of evidence. Where to Go: http:// www.cochrane.org /reviews (The Cochrane Collaboration) ; http:// www.campbellcollaboration.org (Campbell Collaboration - Social Welfare) </li></ul><ul><li>Level 2: No meta-analyses but one RCT or national consensus panel. Second highest level of evidence. Where to Go: American Journal of Psychiatry, Supplements; Ex. Practice Guidelines for the Treatment of Schizophrenia </li></ul><ul><li>Level 3: Quasi-experimental; uncontrolled trial or observational study; expert consensus; descriptive studies. Less rigorous research. Where to Go: http:// www.psychguides.com for Expert Consensus Guidelines </li></ul><ul><li>Level 4: Anecdotal case reports; unsystematic clinical observations; descriptive reports; single subject designs. Non-research based but practical. Where to Go: Journal: Psychiatric Services: Case Report Column </li></ul><ul><li>Level 5: Clinical opinion only; studies without comparison groups; no consistently positive measured outcomes; not research based . Where to Go: Practitioner Magazine: The Family Networker </li></ul><ul><li>Level 6: Consistently poor outcomes for a particular population; intervention may be research based but is applied to the wrong population . Where to Go: your family lawyer for legal advice on impending lawsuit </li></ul><ul><li>Level 7: Emerging Best Practices or Promising Practices are those that have not been subject to rigorous research methods but that could be undergoing research review, have strong clinical utility, are liked by clients and families and are deemed culturally competent . Where to Go: http:// www.modelprograms.samhsa.gov </li></ul><ul><li>From: http://www.ssw.pdx.edu/focus/ebp/index.php . Accessed 04/01/2009 </li></ul>
  14. 14. Criminal Justice Resources <ul><li>Criminal Justice Abstracts </li></ul><ul><li>Description: Citations and abstracts of international journals, books, reports, dissertations and unpublished papers on criminology and related disciplines. </li></ul><ul><li>Availability: Subscription database </li></ul><ul><li>URL: </li></ul><ul><li>http://www.csa.com/factsheets/cja-set-c.php </li></ul><ul><li>Searching Tips: </li></ul><ul><li>DESCRIPTORS KEYWORDS </li></ul><ul><li>Not available Evidence-based </li></ul><ul><li>Systematic review </li></ul><ul><li>Meta-analysis </li></ul><ul><li>Practice Guidelines </li></ul><ul><li>National Criminal Justice Reference Service </li></ul><ul><li>Description: Contains the full text of more than 150,000 publications on criminal justice, including federal, state, and local government reports, books, research reports, journal articles, and unpublished research. </li></ul><ul><li>Availability: Free database: </li></ul><ul><li>URL: </li></ul><ul><li>http://www.ncjrs.gov/abstractdb/search.asp </li></ul><ul><li>Searching Tips </li></ul><ul><li>DESCRIPTORS KEYWORDS </li></ul><ul><li>Literature reviews Systematic review </li></ul><ul><li>Corrections Effectiveness Meta-analysis </li></ul><ul><li>Deterrence Effectiveness Evidence-based </li></ul><ul><li>Effectiveness of Crime Randomize Prevention controlled trial </li></ul><ul><li>Juvenile Corrections Double-blind </li></ul><ul><li>Effectiveness </li></ul><ul><li>Parole Effectiveness </li></ul><ul><li>Services Effectiveness </li></ul>
  15. 15. Education Resources <ul><li>Education Full Text </li></ul><ul><li>Description: Primarily a citation database (with some full text availability), it indexes articles from ~570 English-language periodicals and yearbooks published in the United States and elsewhere. </li></ul><ul><li>Availability: Subscription database: </li></ul><ul><li>URL: </li></ul><ul><li>http://www.hwwilson.com/databases/educat.htm </li></ul><ul><li>Searching Tips </li></ul><ul><li>DESCRIPTORS: </li></ul><ul><li>Meta-analysis of research </li></ul><ul><li>Clinical trials </li></ul><ul><li>Educational Resources Information Clearinghouse (ERIC) </li></ul><ul><li>Description: Indexes research reports and journal literature in the field of education from 1966 to the present. Includes all educational levels and all aspects of education in all disciplines. </li></ul><ul><li>Availability: </li></ul><ul><li>Subscription database: available on many commercial vendor systems </li></ul><ul><li>Free database: http://www.eric.ed.gov/ERICWebPortal/Home.portal?_nfpb=true&_pageLabel=basic&Clearme=true </li></ul><ul><li>Searching Tips </li></ul><ul><li>DESCRIPTORS: </li></ul><ul><li>Literature reviews </li></ul><ul><li>Meta Analysis </li></ul><ul><li>Instructional Effectiveness </li></ul><ul><li>Counseling Effectiveness </li></ul><ul><li>Outcomes of Treatment </li></ul><ul><li>Guidelines </li></ul>
  16. 16. Manuals, Guidelines, etc <ul><li>What Works Clearinghouse </li></ul><ul><li>Description: “ The WWC aims to promote informed education decision making through a set of easily accessible databases and user-friendly reports that provide education consumers with high-quality reviews of the effectiveness of replicable educational interventions (programs, products, practices, and policies) that intend to improve student outcomes.” (Description from: http:// ies.ed.gov/ncee/wwc/overview / Accessed July 10, 2008) </li></ul><ul><li>Availability: Free web-based resource </li></ul><ul><li>URL: http://ies.ed.gov/ncee/wwc/ </li></ul><ul><li>Searching Tips: Primarily arranged by intervention topic area (e.g., early childhood education, dropout prevention, middle school math, etc): list all topic interventions, Keyword searching within topic, or sort interventions using predefined criteria. </li></ul>
  17. 17. Web Search Engines <ul><li>Google Scholar </li></ul><ul><li>Description: Google Scholar extracts records from Medline, ERIC, and other freely accessible databases in addition to institutional repositories and other high value academic and research websites. </li></ul><ul><li>Availability: Free search engine </li></ul><ul><li>URL: http://scholar.google.com </li></ul><ul><li>Searching Tips: Use high value evidence-based practice keywords (systematic review, meta-analysis, etc) for better search results. </li></ul><ul><li>Scirus </li></ul><ul><li>Description: “…the most comprehensive scientific research tool on the web. With over 450 million scientific items indexed at last count, it allows researchers to search for not only journal content but also scientists' homepages, courseware, pre-print server material, patents and institutional repository and website information.” http:// www.scirus.com / . Accessed August 13, 2008. </li></ul><ul><li>Availability: Free search engine </li></ul><ul><li>URL: http:// www.scirus.com/srsapp / </li></ul><ul><li>Searching Tips: Strengths lie more in the medicine and mental health, with less depth in some applied social science disciplines such as criminology and education. </li></ul><ul><li>Use high value evidence-based practice keywords (systematic review, meta-analysis, etc) for better search results. Results page offers additional limits by content sources (BioMed Central, ScienceDirect, etc), file types (PDF, HTML, Word), and helpful ‘refine your search’ suggestions. </li></ul>