PHPartners Meeting: New England Region National Network of Libraries of Medicine PHIA Update


Published on

Lamar Soutter Library Director Elaine Martin presents the an update of the public health initiatives, including the digital public health library.

Published in: Health & Medicine, Education
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • Bolded states were added in 2013 (mid-year 2012). (Contract year is 5/1-4/30)
  • NAL supports six information centers that provide public health information:Four of Six NAL Information Centers strongly support public health:--Food and Nutrition Information Center --Food Safety Information Center --Invasive Species Information Center – transmission of disease from animals to humans. --Rural Information Center --Technology Transfer Information Center --Water Quality Information Center --Alternative Farming Information Center --Animal Welfare Information Center
  • About 130 single titles, titles from various publishers and government journals listed alphabetically for browsing and all connected via PubMed through Linkout (e.g., Indiana’s Customized Logo) – from the digital library page – requiring no name or password.
  • EndNote provides a way to save, use and re-use information easily (complies with national initatives to improve skills and access to content). Eighteen (18 copies go to each participant at incredible pricing). Public Health Departments also have the ability to order additional copies at the same price.
  • The project promotes government US libraries that contain evidence-based information resources to support the nation’s public health.
  • Highlighted blue topics illistrate how functionality is improved beyond the search capability. Features are also covered in training.
  • In addition to core resource content related to research, e-books provide basic understanding to essential services of public health – addressing “best practices,” monitoring health and mobilizing community partnerships. These are important aspects for gaining accreditation as well as promoting improved practice. Another highly posted “best practice e-book” is Oxford Textbook of Medicine (for basic, broad understanding).Retrieved references relate to chapters, sections, and more.Joint Commission title: rates and trends against those of similar organizations through comparative databases and against best practices and scientific evidence.AmStars updates regularly:addresses federally funded Helping Young Smokers Quit (HYSQ) initiative aimed at identifying best practices for tobacco cessation in adolescents, including program components and processes.
  • Inadequate Workforce – Novick & MorrowThe public health workforce is faced with a shortage and competence challenge. On the one hand, there is a shortage of public health workers. The ratio of public health workers to U.S. residents fell from 1:457 to 1:635 between 1970 and 2000.33 A recent survey by ASTHO revealed a rapidly aging state agency workforce, high retirement eligibility rates, high vacancy rates, and high annual staff turnover rates.34Oxford Textbook of Public Health is a 3 volume set.
  • One subscription for “Data Coding and Classification Titles “ saves time, energy and money.
  • Last year we undertook an initiative to better understand resources related to PHD Labs, including testbook, journal, and database analysis. Overwhelmingly, Labs wanted access to ASM book and e-journals and unbiquitous access to CLSI.
  • This type of customization allows buy-in at various levels, including the look and feel of their own intranet. Original set up is based on a template provide and revised by the project. Also at the time newsletters come out, they are listed as a named bullet on the topic of the page and retained for 3-4 weeks.
  • Digital Libraries give PubMed a lot of press.
  • Remember the graphic only depicts use by individualphds – the total is calculated in the chart.
  • Actual numbers are on the left – only high use titles were listed on right. High use reflects about 4101 downloads. We have 23 titles in subscription; 16 are listed here. Again, this slide reinforces what is core and useful to public health. Some trial access was also granted.
  • This slide compares use during annual 2012 (12 months) with use of 8 months in 2013. Some titles were in subscription; some were granted trial access.
  • Not only are high-use, in scope titles validated, but we are finding out what titles (current in subscription) are useful to this constituency and should be added/considered. Content trends are identified. Mary Ann Liebert Titles were used 563 times in 2012.
  • The actual number of people is based on 1017 but we estimated about .87 higher based on unknowns with larger states. If you need me to clarify this further, let me know. These figures are slightly recalculated after new states come on board. 1135 is in the ball park… it’s just that is closer to 1017 right now. We know that MDPH is a high level user.
  • Hawaii was not fully functional --
  • Training includes introduction to and practice on topics of interest to Public Health Departments and National winnable issues, such as Smoking Cessation. Also includes topics that will be affected by the ACA (Affordable Care Act).
  • This is a segment now provided by leadership, epidemiologist (in Maryland’s case, it was Don Shell, MD. MPH Director of the Cancer and Chronic Disease, Bureau, Maryland Department of Health and Mental Hygiene. All content areas contain same/similar objectives. Instructors include personnel from Resource Libraries, hospital libraries, RML personnel (in some cases), Project Coordinator, and selected vendors. Training runs from 9:00 am-1:00 on day 1 (longer time alloted for EndNote) and 9:00 am - 12:30 on second day. Training is combination of demo with people following along at computer; short ppt’s for introduction followed by subsequent exercise.**What resources were chosen and why;
  • * Next to state indicates
  • This slides illustrates diversity in participants attending training. BPHC reflects difference in group (LPHD). Some trainees are nurses but may specify other titles in the sign-up sheet. Total numbers per state are documented in Training overview slide. Many attended training who had new assignments – (representing turnover)
  • Journal SurveyMonkey (recommended by Maine CDC) was adopted early as a way to collect baseline data for both the Project and PHD -- journals identified via subscription or wishlist already in existence and being used by the public health workforce.. It was also perceived that broad distribution would save money). Also collects information on libraries currently being used to find information. Pre/Post Training links– identifies what resources are familiar and used – also allows an opportunity to vent. Vendor stats illustrate use in one and all PHDs. This list of evaluation processes is not necessarily linear.
  • PHPartners Meeting: New England Region National Network of Libraries of Medicine PHIA Update

    1. 1. ―PHPartners Meeting‖ NER NN/LM PHIA Update October 2, 2013 Elaine Martin, MSLS, DA., NN/NER Director Karen H. Dahlen, MLS, Project Coordinator
    2. 2. PHIA Topics PHIA Goal  Who’s Involved in Project/Study  Relationship to National Initiatives  Resources (Subscribed/National)  ◦ Use of Selected Resources ◦ Changes Related to Document Delivery View of Digital Libraries  Training  Evaluation Plan 
    3. 3. PHIA Goal: To identify trusted library resources that are core, useful, and evidence based to improve public health practice.  Trusted library resources are made available directly through a digital library interface.  Resources are identified through use metrics, benchmarking, and input from the Public Health Workforce.  Resources not directly available are subsidized through participating National Library of Medicine Network Members. Digital Library PHD Site Direct Article Delivery via Libraries Timely Information in Support of EvidenceBased/Improved Public Health Practice
    4. 4. Public Health Department Partners PHDS Currently In the Study                Boston Public Health Commission (BPHC) Colorado Dept of Public Health and Environment (CDPHE) Connecticut Dept of Public Health (CDPH) Maine Center for Disease Control and Prevention (Maine CDC) Massachusetts Dept of Public Health (MPHD) New Hampshire Department of Health Human Services (New Hampshire DHHS) Rhode Island Dept Health (RIDH) Vermont Dept Health (VDH) Kentucky Department for Health (KYDH) Arkansas Department of Health (ADH) Wisconsin Division of Health Services (WDHS) Maryland Department of Health & Mental Hygiene Alaska Department of Health (AKDH) Hawaii Department of Health Indiana State Department of Health New PHDs Interest  Florida Department of Health  West Virginia Bureau of Public  Utah Department of Health  Washington Department of Health  Virginia Department of Health  Tennessee Department of Health  Delaware Division of Public Health
    5. 5. PHIA Public Health Partners With Library Partners State Public Heath Department Library Partners AK Alaska Division of Public Health (AKDH) Univ of Alaska (Anchorage) Medical Library Univ of Alabama Medical Sciences Library Poudre Valley Health System Library AR Arkansas Department of Health (ADH) CO CT Colorado Department of Health & Environment Connecticut Public Health Department HI IN Hawaii Department of Health Indiana State Department of Health KY Kentucky Department for Health University of Conn Maynard Stowe Library University of Hawaii Medical Library Ruth Lily Medical Library, Indiana University University of Kentucky Medical Library ME MA MA MD Maine CDC Boston Public Health Commission* Massachusetts Dept of Health Maryland Dept of Health & Mental Hygiene Maine Medical Center Library Univ of Massachusetts Soutter Library Lemuel Shattuck Hospital Library University of Maryland HS Library NH RI VT WI New Hampshire DHHS Rhode Island Department of Health Vermont Department of Health Wisconsin Dept of Health Services New Hampshire DHHS Library Rhode Island Hospital/Lifespan UVM Dana Medical Library Ebling Library of the Health Sciences
    6. 6. Digital Library Resources Available: Databases PubMed ◦ ◦ ◦ PubMed Central TOXNET Clinical Trials Global Health ◦ ◦ ◦ Enhanced Content Conference Proceedings Grey Literature CLIS Database ◦ Clinical Laboratory Standards Cochrane Library ◦ Systematic Reviews Stat!Ref—E Books ◦ Coding & Classification ◦ Connectivity to CDC ◦ Joint Commission Resources/ Trusted E-Texts (ASM Books) ASABE –Health Reports & Journals Linked to Agriculture The National Agricultural Library (Information Centers) CDC Resources Related to Literature Review
    7. 7. Resources Available Through Digital Library: Full-Text Access to Journal Articles Single Titles (14) – Societies +              Health Affairs American Journal of Tropical Medicine & Hygiene Journal of Agricultural Safety Journal of Exposure Science & Environmental Epidemiology Journal of Food Protection Journal of Public Health Policy Journal of Studies on Alcohol & Drugs Journal of Wildlife Diseases International Journal of Tuberculosis New England Journal of Medicine Pediatrics (American Academy of Pediatrics) Public Health Reports Scandinavian Journal of Work, Government Produced Journals (5) E-Journal Packages (110)  Annual Reviews (9) +Archive access to AR Public Health   American Society of Nursing (3) BJM Journals (6)  Oxford University Press (20)  Ovid --Wolters Kluwer (2)  Mary Ann Liebert (12)  Maney Publishing (3)  Ovid (LWW) Titles (6)  Sage (11)  Springer (18) + 4 (trial)  Emerging Infectious Disease  University of Chicago Press (6)  Environmental Health Perspectives  American Society of Microbiology (12)  Journal of Agricultural Health & Safety  Morbidity & Mortality Weekly Report (MMWR)  Preventing Chronic Disease
    8. 8. Functionality --Search web for full-text– symbol: paperclip --Create Groups, e.g., just vaccines --Ability to Search multiple databases, e.g., and e-books --Cite While you Write Feature With Word --Drag/Drop article into record --Input records manually or electronically --Records can be ordered by clicking on the field, e.g., year 18 copies provided as incentive to improve skills; save time; comply with national competency objectives.
    9. 9. National Resources Related to Evidence-Based Resources NLM’s PubMed, PubMed Central, Toxnet + NAL’s Information Centers     Food and Nutrition Information Center (FNIC) Food Safety Information Center (FSIC) National Invasive Species Information Center (NISIC) Water Quality Information Center (WQIC) CDC Resources Related to Literature Review      Community Guide to Preventive Services CDC Journals: MMWR, EID, PCD (alpha inclusion in journal list) Science Clips CDC Stacks Data & Statistics by Topic (NCHS; FastStats; Interview Surveys)
    10. 10. Hawaii Access to PubMed (Improved Functionality)  Filters: Review and Systematic Reviews filters allow focused retrieval.  Linkout button: provides direct access to subscribed journals.  Free full text articles are available via PubMed Central are highlighted.  Send to box: allows seamless transfer to citation mgt software.
    11. 11. Contains Global Information (1937-2012) Archive Database back to 1912 – (not subscribed) Easy to use filters and tools Alert capability Meeting and Conference Summaries; Abstracts in English NATO Conference: Vaccines: new generation, Immunological adjuvants., 1995, pp viii + 192. Access to some full text
    12. 12. ESSENTIAL SERVICES OF PUBLIC HEALTH: SELECTING E-BOOK LITERATURE USING STAT!REF Red Book: 2012 Report of the Committee on Infectious Diseases –29th Ed (2012) Bright Futures: Guidelines for Health Supervision of Infants, Children, and Adolescents, 2008 AM:STAR Adolescent Medicine: State of the Art Reviews, American Academy of Pediatrics, 2012 Diabetes Public Health: From Data to Policy (2011) Novick & Morrow’s Public Health Administration: Principles for PopulationBased Management – 3rd Ed. (2014) Kozier & Erb’s: Fundamentals of Nursing: Concepts, Process, and Practice, 9th Ed., (2012) The Joint Commission: The Source (20132003) ACP PIER®, Journal Club & AHFS DI® Essentials™
    13. 13. Content Related to Best Practices Adolescent Smoking Cessation: Promising Strategies and Evidence-based Recommendations In: ADOLESCENT MEDICINE: STATE OF THE ART REVIEWS (AM:STARs) » Advances in Health Promotion for Adolescents and Young Adults (26) Chapter 3. Joint Commission International's Infection Prevention and Control Standards and Requirements: A Detailed Study. In: BEST PRACTICES IN INFECTION PREVENTION AND CONTROL: AN INTERNATIONAL PERSPECTIVE - 2nd Ed. (2012) (9) Chapter 39. The Postpartum Family at Risk In: OLDS' MATERNAL-NEWBORN NURSING & WOMEN'S HEALTH ACROSS THE LIFESPAN - 9th Ed. (2012) (30)
    14. 14. Content: Related to Workforce Competencies NOVICK & MORROW'S PUBLIC HEALTH ADMINISTRATION: PRINCIPLES FOR POPULATION-BASED MANAGEMENT - 3rd Ed. (2014) 17 Leadership In: OXFORD TEXTBOOK OF PUBLIC HEALTH - 5th Ed. (2009) » Part 3 - The Practice of Public Health » Section 12 - Public health functions relates to the body of knowledge that underpins the field…but there is growing agreement about the core competencies required of all members of the specialist public health workforce (Council on Linkages between Academia and Public Health Practice 2006; Joint Task Group on Public Health Human Resources 2005…. Accreditation + (13)
    15. 15. STAT!Ref: Most Used Titles July 2012 – July 2013 E-Books ACP PIER, Journal Club & AHFS DI Essentials ICD-9-CM - Volumes 1, 2 & 3 (2013) Manual of Clinical Microbiology - 10th Ed. (2011) AHFS Drug Information (2013) Clinical Infectious Disease (2008) Oxford Textbook of Public Health - 5th Ed. (2009) Oxford Textbook of Medicine - 5th Ed. (2010) Red Book: 2012 Report of the Committee on Infectious Diseases - 29th Ed. (2012) ACP Medicine Emerging Infections Series Occupational and Environmental Health: Recognizing and Preventing Disease and Injury - 6th Ed. (2011) INFECTIOUS DISEASES: The Clinician's Guide to Diagnosis, Treatment, and Prevention Merck Manual Of Diagnosis and Therapy, The - 19th Ed. (2011) Textbook of Modern Toxicology, A - 4th Ed. (2010) Adolescent Medicine: State of the Art Reviews (AM:STARs) April 2013 (Subspecialty Update) Joint Commission, The: The Source (2013-2003) Human Virology - 4th Ed. (2011) CPT with RVUs Data File, INGENIX (2013) STAT!Ref: Most Used Services July 2012 – July 2013 Uses 410 315 236 220 219 169 169 Service # Uses Thomson Reuters News Alerts 410 Evidence Alerts Stedman's Medical Dictionary 28th Ed. (2006) 98 AAFP Conditions A to Z (2013) 76 82 Alert Set UP 168 154 132 105 103 96 90 89 81 70 70 High-Use STAT!Ref Titles & Services
    16. 16. Resources for Public Health Labs CLSI Documents June 2013 Total 3043 Downloads 137 311 866 376 378 572 403 QMS02-A6 Quality Management System: Development and Management of Laboratory Documents; Approved Guideline—Sixth Edition EP09-A2-IR Method Comparison and Bias Estimation Using Patient Samples; Approved Guideline -- Second Edition (Interim Revision) GP34-A Validation and Verification of Tubes for Venous and Capillary Blood Specimen Collection; Approved Guideline MM06-A2 Quantitative Molecular Methods for Infectious Diseases; Approved Guideline-Second Edition C51-A Expression of Measurement Uncertainty in Laboratory Medicine; Approved Guideline C50-A Mass Spectrometry in the Clinical Laboratory: General Principles and Guidance MM14-A2 Design of Molecular Proficiency Testing/External Quality Assessment; Approved Guideline—Second Edition
    17. 17. Arkansas Digital Library Indiana State DH Digital Library Sample Digital Library Pages Customized at Site
    18. 18. Hawaii DH Digital Library Vermont DH Digital Library Sample Digital Library Pages
    19. 19. NEJM Comparative Use: Contract Year/ Current Year NEJM Use by 15 PHDs Public Health May 2011- Jan 2013Department Apr 2012 July 2013 VDH 8000 6000 4000 2000 0 May 2011-Apr 2012; N=13,173 792 Maine CDC 715 406 BPHC 371 337 NH DHHS 553 316 CDPHE 754 223 7078 3112 RIDH 731 415 CT PHD 10000 Jan-Jul 2013; N=9823 1179 MPHD 12000 1713 1603 KY DPH 79 665 ADH 0 239 WI DHS 0 954 ISDH Maryland DHMH 0 178 0 520 HI DPH 0 63 0 13173 0 9823 AK DPHS Total
    20. 20. Total Use by PHD PHDs MDPH CT DPH KY DPH ADH MDHMH Maine CDC VDH WI DPH RIDH AK DPH CDPHE NH DHHS BPHC Hawaii DPH Indiana (ISDH) Use of Selected OUP Titles Total 1830 1256 1066 1006 741 736 660 656 572 514 486 184 178 59 0 Journal Title Clinical Infectious Diseases Total 1412 American Journal of Epidemiology Schizophrenia Bulletin JNCI: Journal of the National Cancer Institute ICES Journal of Marine Science 513 393 The Journal of Infectious Diseases International Journal of Epidemiology Journal of Antimicrobial Chemotherapy Nephrology Dialysis Transplantation Journal of Analytical Toxicology Annals of Oncology Toxicological Sciences Journal of Pediatric Psychiatry 259 Oxford University Press Journal Title Use July 2012-June 2013; N=9944 343 269 184 156 151 128 100 98 95
    21. 21. E-Journal Use: Mary Ann Liebert Titles Identifying Core/Useful Trusted Resources Jan-Dec 2012 Total=1621 Jan-Aug 2013 Total=1124 Breastfeeding Medicine Childhood Obesity Population Health… Breastfeeding Medicine Foodborne Pathogens… Journal of Women's… AIDS Patient Care and… Telemedicine and e-… Childhood Obesity Vector-Borne and… Biosecurity and… Journal of Palliative… Gaming Law Review… Population Health… LGBT Health The Journal of… Sustainability: The… Games for Health Journal Thyroid AIDS Research and… 0 Journal of Women's Health Gaming Law Review and… Vector-Borne and… Telemedicine and e-Health Foodborne Pathogens… AIDS Patient Care and… Biosecurity and… Thyroid Journal of Palliative… Election Law Journal:… Cyberpsychology, Behavio… The Journal of Alternative… 0 100 200 300 400 500 600 50 100 150 200 250
    22. 22. Mary Ann Liebert -- Titles In Subscription -- New Titles Under Consideration Total Use : Mary Ann Liebert Journals 1124 Breastfeeding Medicine Foodborne Pathogens and Disease Journals Under Consideration – Not in Subscription Telemedicine and e-Health (74) Journal of Palliative Medicine (43) Gaming Law Review & Economics (23) The Journal of Alternative & Complementary Medicine (21) LGBT Health (21) Games for Health Journal (18) Cyberpsychology, Behavior, and Social Networking (13) Journal of Caffeine Research (11) 243 145 Journal of Women's Health 126 AIDS Patient Care and STDs 76 Childhood Obesity 71 Vector-Borne and Zoonotic Diseases Biosecurity and Bioterrorism: Biodefense Strategy, Practice, and Science 67 Population Health Management 21 Thyroid 14 Viral Immunology 55 4
    23. 23. May 2011 – April 2012; Cost=$18,384 STATE Connecticut Maine Massachusetts (BPHC) STATE ILL DD Total 2 20 22 154 90 244 Massachusetts (BPHC) 9 10 19 Massachusetts (MDPH) 7 6 13 New Hampshire 0 0 0 Rhode Island 82 79 161 Vermont 54 243 297 Colorado 112 40 152 Arkansas 32 73 105 Kentucky 8 3 11 Wisconsin 0 0 0 462 593 1055 ILL DD Total 0 83 83 Connecticut 560 364 924 Maine 11 8 3 Massachusetts (MDPH) 1 1 2 New Hampshire 0 0 0 Rhode Island 47 36 83 Vermont 21 191 212 Colorado 323 156 479 960 834 1794 Total May 2012 --- April 2013; Cost=$12,241 Total Article Delivery Costs
    24. 24. Key to Resource Access: Enterprise Licensing (The 10% Rule) % Users Factors Influencing 10% Rule (Collective Data) No Users State PHD FTE Number Attending Training  Number Responding to Journal Survey Monkey  Identification of Divisions and Program Areas Utilizing Information  10 % of Total 11,355 FTE = 1,135 139 ADH 10% 55 AKDPH 525 33%  10% 2781/1390 38 BPHC* 1100/110 10% 120 CDPHE 1200 10% 40 CT DPH 800/400 10% 75 Hawaii DPH 10% 75 ISDH 780 10% 40 Kentucky 400 10% 75 MDHMH 780 10% 40 Maine CDC 395 35% 150 MDPH 15% 40 NH DHHS 279 10% 40 RIDH 400 15% 45 VDH 300 15% 45 WDPHS 400 1500/750 3000/1500
    25. 25. Pediatrics (Use by 15 PHDs) Pediatrics (15 PHDs) June 2012-July 2013 Total Use=7846 1800 1600 1400 1580 1465 1337 1200 1000 800 600 400 200 0 728 690 490 435 295 229 166 127 124 118 62 0
    26. 26. Training Incorporates National & State Public Health Topics Maryland DHMH Health Initiatives       Health Disparities(sexual minorities) Preventive Services ACA* Routine HIV screening Immigrant Health Undocumented individuals under ACA Drug Resistant Disease Impact on TB, STI (plus others) Health Care Reform and Interpersonal Violence/Domestic Violence Screening/Identification Infectious Diseases Hepatitis C Virus (HCV) HIV Infections --HIV and HCV Co-Infection *ACA –Affordable Care Act
    27. 27. Training Topics – Day 1  Welcome & Overview Public Health Leadership Training Topics – Day 2  Review of Digital Library Page**  Introduction to Cochrane Library & Systematic Reviews  Project Introduction  Why Evidence-Based Public Health*  Alternative Article Delivery Service  NLM’s PubMed Database  Introduction to Global Health  Introduction to EndNote 7  Introduction to STAT!Ref  Practicum Computer Lab Assignment  Brief Info about NIH RePORTER  Practicum Computer Lab Assignment Total of 7.0 Hours Training Two Days: Back to Back
    28. 28. PHIA Trainings STATE PHD # Trainings Attendance RI RIDH 2 30 VT VDH 3 46 ME MAINE CDC* 3 52 MA BPHC* 4 58 MA MDPH* 3 32 NH NH DHHS 2 24 CO CDPHE* 5 100 CT CT DPH* 5 101 KY AR KDPH ADH 2 2 46 WI WI DPH 2 33 AK AK DPH 2 50 HI HI DPH 2 68 IN ISDH 2 60 MD MDHMH 2 38 15 41 769 TOTAL 31
    29. 29. PUBLIC HEALTH WORKFORCE: DIVERSITY IN PARTICIPANT TRAINING Sample: By Title Maryland Indiana Hawaii BPHC Nutrition Specialist (2) State Epidemiologist Education Coordinator (2) Project Mgr (ID) Research Statistician Supervisor – Labs + (3) Asthma Coordinator/ Analyst (2) Project Coordinator EH Policy Analyst (2) Injury Prev Epidemiologist Legislative Liaison Research & Evaluation Mgr Program Manager (2) Staff Attorney Epidemiologist Project Dir (DVP) Epidemiologist Microbiologist ( 2) Research Analyst (2) Oral Health Mgr Medicare Specialist/Advisor (2) Regional Program Director Program Coordinator, SAPB Injury Prevent Dir & Coord (2) Policy Advisor (2) Director of Program Evaluation Epidemiologist (3) Asthma Prevent Dir Psychiatric VT Director of Program Develop Genetic Counselor Policy Analyst (2) Executive Ass’t Field Epi Director Tobacco Prevent Mgr ProjDirector/Chronic Deputy Secretary Access Services Mgr Informatics Analyst Director MHL Director VSA IT Supervisor Nutritionist Develop Sp (Grants)
    30. 30. EVALUATION PROCESSES Different Levels of Data Collection       Journal SurveyMonkey (baseline data for PHIA and PHD) Enterprise Licensing (cost effectiveness over time) Vendor statistical reports (validates ―use of resources‖) Monthly reports from ―library partners‖ (measures use and identify resources to develop Digital Libraries). Ongoing capture of suggested e-books, journals, and databases to enhance collection Pre-Post training links ◦ (data related to knowledge of resources) Feedback from hands-on training  Interviews with leaders and workforce  Focus group sessions (after one year; subsequently)  Quarterly/Annual Reports (submitted to NLM) 
    31. 31. NACCHO Award Brings Recognition to Project  March 2012 and 2013, the PHIA PHD Digital Library Project received a ―promising practice award‖ from NACCHO.  Article has been accepted for presentation at APHA in Boston in November 2013.  Article has been submitted to AJPH; will publish in December 2013.  Presentations have been made at national public health meetings, national and regional library meetings.
    32. 32. Testimonial ―Great recognition for this very valuable initiative. Kudos and thank you.‖ Jewel Mullen, MD, MPH, MPA, Commissioner, Connectic ut Department of Public Health