Research information management: making sense of it all
Population and Frame Development: Graduate Students and Postdoctorates in Science and Engineering Survey
1. RTI International is a trade name of Research Triangle Institutewww.rti.org
Population and Frame Development of the
Survey of Graduate Students and
Postdoctorates in Science and Engineering
(GSS)
Jordan Hudson
Eric Zwieg
2. 04/24/13
2
www.rti.org
GSS Frame Updating
• What is the GSS survey?
– An Annual Census
– National Science Foundation (NSF)/National Institutes of
Health (NIH)
– 582 Institutions (2007)
3. 04/24/13
3
www.rti.org
GSS Frame Update – Objectives
• Identify under coverage
– Research based programs
– Academically affiliated research centers
• Replicate methodology
– GSS Frame last reviewed in 2005
4. 04/24/13
4
www.rti.org
Extant Data Sources
Source Total Number of Institutions
Current GSS Frame (2007) 582
Sources – Frame Update
Integrated Postsecondary Education Data Systems (IPEDS) 537
Online Institutions 21
National Laboratories (federally funded R&D centers) 15
National Postdoctoral Association 11
Institutional membership lists from six college associations 8
Higher Education Directory (HEP) 7
NSF Expenditure Survey 4
NSF Facilities Survey 2
Association of American Universities 0
Carnegie Classification 0
Council of Graduate Schools 0
Total 605
5. 04/24/13
5
www.rti.org
Extant Data Sources
• National Center for Education Statistics (NCES)
Integrated Postsecondary Education Data System
(IPEDS)
– Comprehensive
– Includes masters and doctoral level data
– 537 possibly eligible institutions (89%)
– 236 institutions with only one GSS eligible program
6. 04/24/13
6
www.rti.org
Extant Data Sources - Limitations
• Limitations of IPEDS
– No postdoctoral data
– Cannot determine practitioner vs. research programs
• All other sources
– Merging the data
7. 04/24/13
7
www.rti.org
GSS Frame Updating - Results
• 605 institutions identified
• Graduate degree in only one GSS-eligible field
(roughly 44%)
• 68 institutions from the remaining sources
• New Institution Pilot Study
8. 04/24/13
8
www.rti.org
2008 New Institution Pilot Study - Objectives
• Determine eligibility
• Gain cooperation of the school
• Identify GSS School Coordinator (SC)
• Assist SC with data collection
9. 04/24/13
9
www.rti.org
2008 New Institution Pilot Study - Sample
• 80 institutions sampled – Characteristics:
– Traditional brick-and-mortar institutions and online
institutions
– Public and private (both nonprofit and for-profit) institutions
– Carnegie Classification
– Number of GSS-eligible programs (one vs. multiple)
– Existence of a postdoctoral office
– Geographic regions
10. 04/24/13
10
www.rti.org
2008 New Institution Pilot Study -
Methodology
• Letter to the institution President
• Email to the School Coordinator
• RTI contact of School Coordinator
• Data Collection
11. 04/24/13
11
www.rti.org
2008 New Institution Pilot Study - Results
• 80 Institutions Sampled:
– 43 eligible institutions (53.75%)
• 7 institutions reported under another institution in the pilot study
• 5 institutions reported under another institution in the GSS
– 34 ineligible institutions (42.5%)
– 3 institutions whose eligibility could not be determined
(3.75%)
12. 04/24/13
12
www.rti.org
Conclusion
• 605 potentially new institutions
• 2008 New Institution Pilot Study
• Apply lessons learned from the pilot study to the
remaining potentially new institutions
• The frame updating methodology is replicable in
subsequent GSS survey cycles
What is the GSS survey? The (GSS) is an annual census sponsored by the National Science Foundation (NSF) in collaboration with the National Institutes of Health (NIH). 2.In order for an institution to be eligible for the GSS, it must grant at least one master’s or doctoral degree in science, engineering, and health related field or have at least one postdoctoral appointee or non-faculty researcher. 3.It Collects data at the department level on the number and characteristics of postdoctorates, graduate students, and non-faculty researchers 4.The current GSS frame consists of 582 institutions and academically affiliated research centers.
Frame updating objectives Evaluate the GSS frame for eligible institutions & address any survey under coverage. Basically, we want research centric programs only. We do not want to collect data on practitioner based programs. Plus we wanted to make sure that we cover academically affiliated research centers, such as National Laboratories or medical centers associated w/ institutions. These are the type areas where Postdocatorates and non-faculty researchers are housed. --Replicate Methodology -- 2.Finally, we wanted to create a new frame updating methodology which will allow for efficient frame updating in subsequent GSS survey cycles.
--The first step was to determine which data sources were to be used in the frame update process – There are two separate totals on this table – One for the Current GSS frame (582) and another for the newly identified institutions (605). Only newly-identified schools -- i.e., those institutions not included in any previous source -- were considered for the new frame. A total of 16 sources were used From six association and memberships directories, as well as the Carnegie Classification, Higher Education Directory, and data from the other NSF surveys. As you can see the Integrated Postsecondary Education Data Systems (IPEDS) brings in the largest portion of institutions from any single source. The next largest source was the postdoc component – which was comprised from the National Postdoctoral Association database and the government list of federally funded R&D centers. This source accounted for 26 – academic institutions, and academically affiliated research centers, such as National laboratories, e.g. Ames National One surprising source in this graph comes from the online institution portion of the frame updating activity. Currently, there is only one online only institution on the GSS . The frame update process produced an additional 21 online only institutions that are possibly eligible for inclusion.
NCES – Completions Survey: As a starting point with the data sources we used the National Center for Education Statistics – Integrated Postsecondary Education Data Systems (IPEDS) survey data. It’s a mandatory survey making it one of the most comprehensive lists of master’s and doctoral degree granting institutions. . Because of this, the IPEDS data became our foundation for the frame process. IPEDS yielded 537 institutions – again, this is by far the largest source of new institutions. Of which only 236 have one GSS eligible program.
Limitations: The IPEDS data does not include information for postdocs – we overcame this limitation by using the National Postdoc Associaton database along w/ the government master list of Federally Funded Research & Development Centers. The IPEDS data does not indicate whether or not a program is a practitioner or research based. –DISCUSS WITH ERIC HOW WE OVERCAME THIS LIMITATION, IF WE EVEN DID– Another limitation was that the extant data sources used different numeric institution identifiers, because of this we created a crosswalk that allowed us to map the institutions together.
Of the 605 newly identified institutions that may be possibly eligible for the GSS – Considering that the 2007 survey universe consisted of 582 institutions this is a considerable increase. The lions share were identified via the IPEDS completions survey roughly 89% However, roughly 44% of the newly identified institutions only offer a graduate degree in one GSS eligible field. The remaining sources yielded 68 institutions – the majority of which were identified from either the online only institution list or from a postdoctoral source (National Postdoctoral Association Database – government list of Federally funded R&D centers) In order to determine the feasibility of adding the newly identified institutions - a New Institution Pilot Study was conducted. Eric Zwieg is going to discuss New Pilot study in further depth.
Determine Eligibility: We purposefully cast the net wide in building the frame. We knew potential limitations of some of the sources. Identify Potential: SC is someone who knows enough information about the programs at the institution that can help us determine eligibility Also a SC can either report data or has the authority to delegate reporting to other individuals Assist SC with data collection: We hope to find what problems the new institutions have reporting data to the GSS
Priority was given to included the top 5% (approximately 30 institutions) of the frame in terms of the total graduate student enrollment.
Letter to the inst President: Introduce them to the GSS Ask for contact info for a potential SC Email to SC: Introduce them to the GSS Confirm their role as SC & contact info Call/email RTI with questions RTI contact of SC: Help determine eligibility of institution Answer any question prior to opening of data collection Data Collection: Mailed hard copy materials just like regular gss institutions Email reminders Help desk
Eligible Institutions: Most common programs – Computer Science, Psychology, & Social Sciences, NEC Ineligible Institutions: Most common ineligible programs identified from IPEDS completions survey – Counseling Psychology, Physical Therapy, & Management Info Science Reported under another pilot institution: 8 university of phoenix programs reported under one central institution in the GSS Institutions reported under GSS inst: Mostly branch campuses or recent name changes