1.
Mining the Web to Approximate
University Rankings
Corren G. McCoy, Michael L. Nelson, Michele C. Weigle
@CorrenMcCoy @WebSciDL
JCDL-KDDL 2018, UNT Health Science Center, Fort Worth, TX
June 6, 2018

2.
@CorrenMcCoy @WebSciDL JCDL-KDDL 2018, UNT Health Science Center, Fort Worth, TX June 6, 2018@CorrenMcCoy @WebSciDL JCDL-KDDL 2018, UNT Health Science Center, Fort Worth, TX June 6, 2018
University Ranking Lists
2
 Center for World University Rankings
 Top Party Schools
 America’s Top Colleges
 QS World University Rankings
 Best Athletic Facilities
 Most Beautiful Campus
 Reuters Top 100: The World's Most Innovative Universities
 CWTS Leiden Ranking
 Kiplinger's Best College Value
 Best Campus Food

3.
@CorrenMcCoy @WebSciDL JCDL-KDDL 2018, UNT Health Science Center, Fort Worth, TX June 6, 2018@CorrenMcCoy @WebSciDL JCDL-KDDL 2018, UNT Health Science Center, Fort Worth, TX June 6, 2018
Academic Ranking Lists
Data Collected Between June and August 2016
3
• Disparate Ranking Criteria Between Lists
• Alumni Giving
• Peer Evaluation Surveys
• Value and Affordability
• Faculty Citations
• Graduation Rates
• Library Resources
• Faculty-Student Ratios

4.
@CorrenMcCoy @WebSciDL JCDL-KDDL 2018, UNT Health Science Center, Fort Worth, TX June 6, 2018@CorrenMcCoy @WebSciDL JCDL-KDDL 2018, UNT Health Science Center, Fort Worth, TX June 6, 2018
Variations Between Expert Ranking Lists
4
US News Top-10 US Universities (2016)
______________________________________
1. Harvard University
2. Stanford University
3. UC Berkeley
4. UCLA
5. Columbia University
6. University of Washington
7. Princeton University
8. University of Pennsylvania
9. Yale University
10. University of Michigan
Money Magazine Top-10 US Universities (2016)
____________________________________________
1. Princeton University
2. University of Michigan
3. Harvard University
4. Rice University
5. UC Berkeley
6. Brigham Young University
7. University of Virginia
8. Stanford University
9. Yale University
10.Texas A&M

5.
@CorrenMcCoy @WebSciDL JCDL-KDDL 2018, UNT Health Science Center, Fort Worth, TX June 6, 2018@CorrenMcCoy @WebSciDL JCDL-KDDL 2018, UNT Health Science Center, Fort Worth, TX June 6, 2018
Mining the Web
5
 Data-driven, alternative approach using only web-based metrics
 University Twitter Engagement (UTE) Score
 Traditional and “undiscovered” Twitter followers
 Popular metric for measuring influence and social reputation
 Enrollment, Endowment, and Athletic Expenditures (EEE) Score
 Measure of institutional investment in promoting school’s brand
 Adjusted Reputation Rank (ARR)
 Mean ranking across all expert lists
 Similar to Olympic Scoring
 Excludes Money Magazine’s ranking position due to its variability

6.
@CorrenMcCoy @WebSciDL JCDL-KDDL 2018, UNT Health Science Center, Fort Worth, TX June 6, 2018@CorrenMcCoy @WebSciDL JCDL-KDDL 2018, UNT Health Science Center, Fort Worth, TX June 6, 2018
Building the Web Collection
Data collected between June and August 2016
6
University Home Pages
Twitter Friends & Followers
Google Custom Search Engine
NCSE Endowment Market Values
Athletic Expenditures
Undergraduate Enrollment
NCAA Conference Membership

7.
7@CorrenMcCoy @WebSciDL JCDL-KDDL 2018, UNT Health Science Center, Fort Worth, TX June 6, 2018
Challenges to Knowledge
Discovery
7

8.
@CorrenMcCoy @WebSciDL JCDL-KDDL 2018, UNT Health Science Center, Fort Worth, TX June 6, 2018@CorrenMcCoy @WebSciDL JCDL-KDDL 2018, UNT Health Science Center, Fort Worth, TX June 6, 2018
Incomplete Social Media Directories
8
Source: https://web.archive.org/web/20150905094512/https://socialmedia.duke.edu/ (2015)

9.
@CorrenMcCoy @WebSciDL JCDL-KDDL 2018, UNT Health Science Center, Fort Worth, TX June 6, 2018@CorrenMcCoy @WebSciDL JCDL-KDDL 2018, UNT Health Science Center, Fort Worth, TX June 6, 2018
Issues with Data Consistency
9
 Inconsistent references to university names
 US News: Binghamton University—SUNY
 THE: SUNY—Binghamton
 Missing Twitter account on home page
 University of Louisville
 Ball State
 Some endowments aggregated at the system or foundation level
 The University of Texas System vs. University of Texas-Austin
 University of Minnesota & Foundations vs. University of Minnesota - Twin Cities

10.
@CorrenMcCoy @WebSciDL JCDL-KDDL 2018, UNT Health Science Center, Fort Worth, TX June 6, 2018@CorrenMcCoy @WebSciDL JCDL-KDDL 2018, UNT Health Science Center, Fort Worth, TX June 6, 2018
Web Data in PDF Format
10
Source:
https://web.archive.org/web/20180206194037/http://www.nacubo.org/Documents/EndowmentFiles/2015_NC
SE_Endowment_Market_Values.pdf

11.
@CorrenMcCoy @WebSciDL JCDL-KDDL 2018, UNT Health Science Center, Fort Worth, TX June 6, 2018@CorrenMcCoy @WebSciDL JCDL-KDDL 2018, UNT Health Science Center, Fort Worth, TX June 6, 2018
Automated Submission to Web Forms
11
Source: https://nces.ed.gov/ccd/schoolsearch/

12.
12@CorrenMcCoy @WebSciDL JCDL-KDDL 2018, UNT Health Science Center, Fort Worth, TX June 6, 2018
Applying the Web Metrics

13.
@CorrenMcCoy @WebSciDL JCDL-KDDL 2018, UNT Health Science Center, Fort Worth, TX June 6, 2018@CorrenMcCoy @WebSciDL JCDL-KDDL 2018, UNT Health Science Center, Fort Worth, TX June 6, 2018
Building the University Dataset
13
Who’s In:
• All Ivy League Schools
• Baylor University
• Citadel
• Villanova University
Who’s Out:
• Baylor College of Medicine
• Univ. of Texas Health Science Ctr
• MIT

14.
@CorrenMcCoy @WebSciDL JCDL-KDDL 2018, UNT Health Science Center, Fort Worth, TX June 6, 2018@CorrenMcCoy @WebSciDL JCDL-KDDL 2018, UNT Health Science Center, Fort Worth, TX June 6, 2018
US Schools Depend on Fundraising
14

15.
@CorrenMcCoy @WebSciDL JCDL-KDDL 2018, UNT Health Science Center, Fort Worth, TX June 6, 2018@CorrenMcCoy @WebSciDL JCDL-KDDL 2018, UNT Health Science Center, Fort Worth, TX June 6, 2018
Standardizing Rank Positions
15
Unranked schools added
to end of list

16.
@CorrenMcCoy @WebSciDL JCDL-KDDL 2018, UNT Health Science Center, Fort Worth, TX June 6, 2018@CorrenMcCoy @WebSciDL JCDL-KDDL 2018, UNT Health Science Center, Fort Worth, TX June 6, 2018
Adjusted Reputation Rank (ARR)
16
Top-10 Ranked by ARR

17.
@CorrenMcCoy @WebSciDL JCDL-KDDL 2018, UNT Health Science Center, Fort Worth, TX June 6, 2018@CorrenMcCoy @WebSciDL JCDL-KDDL 2018, UNT Health Science Center, Fort Worth, TX June 6, 2018
Enrollment, Endowment, Expenditures (EEE) Rank
17

18.
@CorrenMcCoy @WebSciDL JCDL-KDDL 2018, UNT Health Science Center, Fort Worth, TX June 6, 2018@CorrenMcCoy @WebSciDL JCDL-KDDL 2018, UNT Health Science Center, Fort Worth, TX June 6, 2018
University Twitter Engagement (UTE) Score
18
(Bi-directional linkage required)

19.
@CorrenMcCoy @WebSciDL JCDL-KDDL 2018, UNT Health Science Center, Fort Worth, TX June 6, 2018@CorrenMcCoy @WebSciDL JCDL-KDDL 2018, UNT Health Science Center, Fort Worth, TX June 6, 2018
Bi-directional Linkage Points to Official Accounts
19
duke.edu (Yes) goDuke.com (No)annualfund.duke.edu (Yes)

20.
@CorrenMcCoy @WebSciDL JCDL-KDDL 2018, UNT Health Science Center, Fort Worth, TX June 6, 2018@CorrenMcCoy @WebSciDL JCDL-KDDL 2018, UNT Health Science Center, Fort Worth, TX June 6, 2018
Rank Comparisons
20
Top-10 by ARR
1. Harvard University
2. Stanford University
3. UC-Berkeley
4. Princeton University
5. Columbia University
6. UCLA
7. Yale University
8. University of Pennsylvania
9. University of Washington
10. University of Michigan
Top-10 by EEE
1. Ohio State University
2. University of Texas
3. Penn State University
4. University of Michigan
5. Univ. of Wisconsin
6. University of Florida
7. Michigan State University
8. University of Washington
9. UCLA
10. Indiana University
Top-10 by UTE
1. Harvard University
2. Stanford University
3. Cornell University
4. Yale University
5. University of Pennsylvania
6. Arizona State
7. Columbia University
8. Texas A&M University
9. Wake Forest University
10. University of Texas

21.
@CorrenMcCoy @WebSciDL JCDL-KDDL 2018, UNT Health Science Center, Fort Worth, TX June 6, 2018@CorrenMcCoy @WebSciDL JCDL-KDDL 2018, UNT Health Science Center, Fort Worth, TX June 6, 2018
Kendall’s Tau-b Correlation Between Rankings
21

22.
@CorrenMcCoy @WebSciDL JCDL-KDDL 2018, UNT Health Science Center, Fort Worth, TX June 6, 2018@CorrenMcCoy @WebSciDL JCDL-KDDL 2018, UNT Health Science Center, Fort Worth, TX June 6, 2018
Correlation of ARR to EEE for Complete Dataset
22
EEE Rank
1 to 50
51 to100
101 to150
151 to 200
201 to 264
Large Enrollment
Big Endowments
Hefty Athletic Spending
Money Magazine Only Schools
Given ARR of 142

23.
@CorrenMcCoy @WebSciDL JCDL-KDDL 2018, UNT Health Science Center, Fort Worth, TX June 6, 2018@CorrenMcCoy @WebSciDL JCDL-KDDL 2018, UNT Health Science Center, Fort Worth, TX June 6, 2018
Correlation of ARR to UTE for Complete Dataset
23
EEE Rank
1 to 50
51 to100
101 to150
151 to 200
201 to 264
Georgia Tech
Univ. of Pittsburgh
Cornell
Wake Forest

24.
@CorrenMcCoy @WebSciDL JCDL-KDDL 2018, UNT Health Science Center, Fort Worth, TX June 6, 2018@CorrenMcCoy @WebSciDL JCDL-KDDL 2018, UNT Health Science Center, Fort Worth, TX June 6, 2018
Impact of a Celebrity Professor
Wake Forest University Loses 410K Followers From UTE Rank
24
URI: ajccenter.wfu.edu
Internet Archive:
http://web.archive.org/web/20151210173754/https:/twitter.com/M
HarrisPerry (2015)
URI: freedomontap.com
Live Web: https://twitter.com/MHarrisPerry (2018)

25.
@CorrenMcCoy @WebSciDL JCDL-KDDL 2018, UNT Health Science Center, Fort Worth, TX June 6, 2018@CorrenMcCoy @WebSciDL JCDL-KDDL 2018, UNT Health Science Center, Fort Worth, TX June 6, 2018
Correlation of EEE to UTE for Complete Dataset
25
EEE Rank
1 to 50
51 to100
101 to150
151 to 200
201 to 264
Large Enrollment
Big Endowments
Hefty Athletic Spending
Pitt
Georgia Tech
Dartmouth

26.
26@CorrenMcCoy @WebSciDL JCDL-KDDL 2018, UNT Health Science Center, Fort Worth, TX June 6, 2018
NCAA Power Five
• Richest, Most Powerful Schools in NCAA Division I
• 55 of 65 in Top-100 of ARR
• 65 of 65 in Top-100 of EEE
• 61 of 65 in Top-100 of UTE
26

27.
@CorrenMcCoy @WebSciDL JCDL-KDDL 2018, UNT Health Science Center, Fort Worth, TX June 6, 2018@CorrenMcCoy @WebSciDL JCDL-KDDL 2018, UNT Health Science Center, Fort Worth, TX June 6, 2018
Correlation of ARR to UTE
NCAA Power Five
27
NCAA Power 5
Other Schools
Louisville
https://GoCards.com
Georgia Tech
Missing Web Links
Oregon State
Missing Web LinksIvy League Schools
Not in Power Five

28.
@CorrenMcCoy @WebSciDL JCDL-KDDL 2018, UNT Health Science Center, Fort Worth, TX June 6, 2018@CorrenMcCoy @WebSciDL JCDL-KDDL 2018, UNT Health Science Center, Fort Worth, TX June 6, 2018
UTE Constrained by Domain Rules for Official Accounts
28
Full table available in Appendix A of our tech report:
Corren G. McCoy, Michael L. Nelson, and Michele C. Weigle. 2017. University Twitter Engagement: Using
Twitter Followers to Rank Universities. arXiv preprint arXiv:1708.05790.

29.
@CorrenMcCoy @WebSciDL JCDL-KDDL 2018, UNT Health Science Center, Fort Worth, TX June 6, 2018@CorrenMcCoy @WebSciDL JCDL-KDDL 2018, UNT Health Science Center, Fort Worth, TX June 6, 2018
Future Work
29
 Address bots and spam accounts
 Potentially overinflates the UTE score
 The Follower Factory – The New York Times (January 2018)
 https://www.nytimes.com/interactive/2018/01/27/technology/social-media-bots.html
 Address artificial manipulation of Twitter followers
 Campbell’s Law
 Temporal analysis using identify non-linear spikes
 Known Twitter API limitations related to historical follower counts
 Miranda Smith. 2018. Twitter Follower Count History via the Internet Archive. Blog. http://ws-
dl.blogspot.com/2018/03/2018-03-14-twitter-follower-count.html
 Expand list of official university domains
 UTE would include more secondary accounts

30.
@CorrenMcCoy @WebSciDL JCDL-KDDL 2018, UNT Health Science Center, Fort Worth, TX June 6, 2018@CorrenMcCoy @WebSciDL JCDL-KDDL 2018, UNT Health Science Center, Fort Worth, TX June 6, 2018
Conclusions
30
 Mined data from Twitter and other public, web data sources
 Raw data and 1M Twitter profiles available on GitHub
 https://github.com/oduwsdl/University-Twitter-Engagement
 Incomplete directories of social media accounts by universities
 Necessitated discovery of secondary accounts
 Methodology constrains universities to single official domain
 Potentially deflates the UTE score
 Comparable approximation to national ranking lists (ARR)
 Correlation with UTE ( τ =0.6018)
 Correlation with EEE ( τ =0.5969)

31.
@CorrenMcCoy @WebSciDL JCDL-KDDL 2018, UNT Health Science Center, Fort Worth, TX June 6, 2018@CorrenMcCoy @WebSciDL JCDL-KDDL 2018, UNT Health Science Center, Fort Worth, TX June 6, 2018
Links to Data Sources
31
 Tech report and blog post
 Corren G. McCoy, Michael L. Nelson, and Michele C. Weigle. 2017. University Twitter Engagement: Using
Twitter Followers to Rank Universities. arXiv preprint arXiv:1708.05790.
 Corren McCoy. 2017. University Twitter Engagement: Using Twitter Followers to Rank Universities. Blog.
http://ws-dl.blogspot.com/2017/08/2017-08-25-university-twitter.html
 Endowment: 2015 NCSE Endowment Market Values,
http://www.nacubo.org/Documents/EndowmentFiles/2015_NCSE_Endowment_Market_Values.p
df
 Athletic Expenditures: Equity in Athletics Data Analysis (EADA), https://ope.ed.gov/athletics/
 Undergraduate Enrollment: Integrated Postsecondary Education Data System (IPEDS),
https://nces.ed.gov/ipeds/
 Google Custom Search: https://cse.google.com/cse/
 Twitter Developer Platform: https://developer.twitter.com/content/developer-twitter/en.html
 NCAA Conference Membership: http://www.ncaa.org/about?division=d1