Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

JCDL 2018 - Mining the Web to Approximate University Rankings

430 views

Published on

This presentation is from the Knowledge Discovery From Digital Libraries workshop as part of JCDL 2018. It is based on our earlier tech report "University Twitter Engagement: Using Twitter Followers to Rank Universities" (http://arxiv.org/abs/1708.05790).

We presented an alternative methodology for approximating the academic rankings of a university using social media; specifically, the university's Twitter followers. We identified a strategy for discovering official Twitter accounts along with a comparative analysis of metrics mined from the web which could be predictors of high academic rank (e.g., athletic expenditures, undergraduate enrollment, endowment value). As expected, schools with more financial resources tend to have more Twitter followers based on larger enrollments, big endowments, and big investments in their sport programs. We also discovered that smaller schools like Wake Forest University can enhance their reputation when they employ faculty with national name recognition (e.g., Melissa Harris-Perry).

Published in: Data & Analytics
  • Be the first to comment

  • Be the first to like this

JCDL 2018 - Mining the Web to Approximate University Rankings

  1. 1. Mining the Web to Approximate University Rankings Corren G. McCoy, Michael L. Nelson, Michele C. Weigle @CorrenMcCoy @WebSciDL JCDL-KDDL 2018, UNT Health Science Center, Fort Worth, TX June 6, 2018
  2. 2. @CorrenMcCoy @WebSciDL JCDL-KDDL 2018, UNT Health Science Center, Fort Worth, TX June 6, 2018@CorrenMcCoy @WebSciDL JCDL-KDDL 2018, UNT Health Science Center, Fort Worth, TX June 6, 2018 University Ranking Lists 2  Center for World University Rankings  Top Party Schools  America’s Top Colleges  QS World University Rankings  Best Athletic Facilities  Most Beautiful Campus  Reuters Top 100: The World's Most Innovative Universities  CWTS Leiden Ranking  Kiplinger's Best College Value  Best Campus Food
  3. 3. @CorrenMcCoy @WebSciDL JCDL-KDDL 2018, UNT Health Science Center, Fort Worth, TX June 6, 2018@CorrenMcCoy @WebSciDL JCDL-KDDL 2018, UNT Health Science Center, Fort Worth, TX June 6, 2018 Academic Ranking Lists Data Collected Between June and August 2016 3 • Disparate Ranking Criteria Between Lists • Alumni Giving • Peer Evaluation Surveys • Value and Affordability • Faculty Citations • Graduation Rates • Library Resources • Faculty-Student Ratios
  4. 4. @CorrenMcCoy @WebSciDL JCDL-KDDL 2018, UNT Health Science Center, Fort Worth, TX June 6, 2018@CorrenMcCoy @WebSciDL JCDL-KDDL 2018, UNT Health Science Center, Fort Worth, TX June 6, 2018 Variations Between Expert Ranking Lists 4 US News Top-10 US Universities (2016) ______________________________________ 1. Harvard University 2. Stanford University 3. UC Berkeley 4. UCLA 5. Columbia University 6. University of Washington 7. Princeton University 8. University of Pennsylvania 9. Yale University 10. University of Michigan Money Magazine Top-10 US Universities (2016) ____________________________________________ 1. Princeton University 2. University of Michigan 3. Harvard University 4. Rice University 5. UC Berkeley 6. Brigham Young University 7. University of Virginia 8. Stanford University 9. Yale University 10.Texas A&M
  5. 5. @CorrenMcCoy @WebSciDL JCDL-KDDL 2018, UNT Health Science Center, Fort Worth, TX June 6, 2018@CorrenMcCoy @WebSciDL JCDL-KDDL 2018, UNT Health Science Center, Fort Worth, TX June 6, 2018 Mining the Web 5  Data-driven, alternative approach using only web-based metrics  University Twitter Engagement (UTE) Score  Traditional and “undiscovered” Twitter followers  Popular metric for measuring influence and social reputation  Enrollment, Endowment, and Athletic Expenditures (EEE) Score  Measure of institutional investment in promoting school’s brand  Adjusted Reputation Rank (ARR)  Mean ranking across all expert lists  Similar to Olympic Scoring  Excludes Money Magazine’s ranking position due to its variability
  6. 6. @CorrenMcCoy @WebSciDL JCDL-KDDL 2018, UNT Health Science Center, Fort Worth, TX June 6, 2018@CorrenMcCoy @WebSciDL JCDL-KDDL 2018, UNT Health Science Center, Fort Worth, TX June 6, 2018 Building the Web Collection Data collected between June and August 2016 6 University Home Pages Twitter Friends & Followers Google Custom Search Engine NCSE Endowment Market Values Athletic Expenditures Undergraduate Enrollment NCAA Conference Membership
  7. 7. 7@CorrenMcCoy @WebSciDL JCDL-KDDL 2018, UNT Health Science Center, Fort Worth, TX June 6, 2018 Challenges to Knowledge Discovery 7
  8. 8. @CorrenMcCoy @WebSciDL JCDL-KDDL 2018, UNT Health Science Center, Fort Worth, TX June 6, 2018@CorrenMcCoy @WebSciDL JCDL-KDDL 2018, UNT Health Science Center, Fort Worth, TX June 6, 2018 Incomplete Social Media Directories 8 Source: https://web.archive.org/web/20150905094512/https://socialmedia.duke.edu/ (2015)
  9. 9. @CorrenMcCoy @WebSciDL JCDL-KDDL 2018, UNT Health Science Center, Fort Worth, TX June 6, 2018@CorrenMcCoy @WebSciDL JCDL-KDDL 2018, UNT Health Science Center, Fort Worth, TX June 6, 2018 Issues with Data Consistency 9  Inconsistent references to university names  US News: Binghamton University—SUNY  THE: SUNY—Binghamton  Missing Twitter account on home page  University of Louisville  Ball State  Some endowments aggregated at the system or foundation level  The University of Texas System vs. University of Texas-Austin  University of Minnesota & Foundations vs. University of Minnesota - Twin Cities
  10. 10. @CorrenMcCoy @WebSciDL JCDL-KDDL 2018, UNT Health Science Center, Fort Worth, TX June 6, 2018@CorrenMcCoy @WebSciDL JCDL-KDDL 2018, UNT Health Science Center, Fort Worth, TX June 6, 2018 Web Data in PDF Format 10 Source: https://web.archive.org/web/20180206194037/http://www.nacubo.org/Documents/EndowmentFiles/2015_NC SE_Endowment_Market_Values.pdf
  11. 11. @CorrenMcCoy @WebSciDL JCDL-KDDL 2018, UNT Health Science Center, Fort Worth, TX June 6, 2018@CorrenMcCoy @WebSciDL JCDL-KDDL 2018, UNT Health Science Center, Fort Worth, TX June 6, 2018 Automated Submission to Web Forms 11 Source: https://nces.ed.gov/ccd/schoolsearch/
  12. 12. 12@CorrenMcCoy @WebSciDL JCDL-KDDL 2018, UNT Health Science Center, Fort Worth, TX June 6, 2018 Applying the Web Metrics
  13. 13. @CorrenMcCoy @WebSciDL JCDL-KDDL 2018, UNT Health Science Center, Fort Worth, TX June 6, 2018@CorrenMcCoy @WebSciDL JCDL-KDDL 2018, UNT Health Science Center, Fort Worth, TX June 6, 2018 Building the University Dataset 13 Who’s In: • All Ivy League Schools • Baylor University • Citadel • Villanova University Who’s Out: • Baylor College of Medicine • Univ. of Texas Health Science Ctr • MIT
  14. 14. @CorrenMcCoy @WebSciDL JCDL-KDDL 2018, UNT Health Science Center, Fort Worth, TX June 6, 2018@CorrenMcCoy @WebSciDL JCDL-KDDL 2018, UNT Health Science Center, Fort Worth, TX June 6, 2018 US Schools Depend on Fundraising 14
  15. 15. @CorrenMcCoy @WebSciDL JCDL-KDDL 2018, UNT Health Science Center, Fort Worth, TX June 6, 2018@CorrenMcCoy @WebSciDL JCDL-KDDL 2018, UNT Health Science Center, Fort Worth, TX June 6, 2018 Standardizing Rank Positions 15 Unranked schools added to end of list
  16. 16. @CorrenMcCoy @WebSciDL JCDL-KDDL 2018, UNT Health Science Center, Fort Worth, TX June 6, 2018@CorrenMcCoy @WebSciDL JCDL-KDDL 2018, UNT Health Science Center, Fort Worth, TX June 6, 2018 Adjusted Reputation Rank (ARR) 16 Top-10 Ranked by ARR
  17. 17. @CorrenMcCoy @WebSciDL JCDL-KDDL 2018, UNT Health Science Center, Fort Worth, TX June 6, 2018@CorrenMcCoy @WebSciDL JCDL-KDDL 2018, UNT Health Science Center, Fort Worth, TX June 6, 2018 Enrollment, Endowment, Expenditures (EEE) Rank 17
  18. 18. @CorrenMcCoy @WebSciDL JCDL-KDDL 2018, UNT Health Science Center, Fort Worth, TX June 6, 2018@CorrenMcCoy @WebSciDL JCDL-KDDL 2018, UNT Health Science Center, Fort Worth, TX June 6, 2018 University Twitter Engagement (UTE) Score 18 (Bi-directional linkage required)
  19. 19. @CorrenMcCoy @WebSciDL JCDL-KDDL 2018, UNT Health Science Center, Fort Worth, TX June 6, 2018@CorrenMcCoy @WebSciDL JCDL-KDDL 2018, UNT Health Science Center, Fort Worth, TX June 6, 2018 Bi-directional Linkage Points to Official Accounts 19 duke.edu (Yes) goDuke.com (No)annualfund.duke.edu (Yes)
  20. 20. @CorrenMcCoy @WebSciDL JCDL-KDDL 2018, UNT Health Science Center, Fort Worth, TX June 6, 2018@CorrenMcCoy @WebSciDL JCDL-KDDL 2018, UNT Health Science Center, Fort Worth, TX June 6, 2018 Rank Comparisons 20 Top-10 by ARR 1. Harvard University 2. Stanford University 3. UC-Berkeley 4. Princeton University 5. Columbia University 6. UCLA 7. Yale University 8. University of Pennsylvania 9. University of Washington 10. University of Michigan Top-10 by EEE 1. Ohio State University 2. University of Texas 3. Penn State University 4. University of Michigan 5. Univ. of Wisconsin 6. University of Florida 7. Michigan State University 8. University of Washington 9. UCLA 10. Indiana University Top-10 by UTE 1. Harvard University 2. Stanford University 3. Cornell University 4. Yale University 5. University of Pennsylvania 6. Arizona State 7. Columbia University 8. Texas A&M University 9. Wake Forest University 10. University of Texas
  21. 21. @CorrenMcCoy @WebSciDL JCDL-KDDL 2018, UNT Health Science Center, Fort Worth, TX June 6, 2018@CorrenMcCoy @WebSciDL JCDL-KDDL 2018, UNT Health Science Center, Fort Worth, TX June 6, 2018 Kendall’s Tau-b Correlation Between Rankings 21
  22. 22. @CorrenMcCoy @WebSciDL JCDL-KDDL 2018, UNT Health Science Center, Fort Worth, TX June 6, 2018@CorrenMcCoy @WebSciDL JCDL-KDDL 2018, UNT Health Science Center, Fort Worth, TX June 6, 2018 Correlation of ARR to EEE for Complete Dataset 22 EEE Rank 1 to 50 51 to100 101 to150 151 to 200 201 to 264 Large Enrollment Big Endowments Hefty Athletic Spending Money Magazine Only Schools Given ARR of 142
  23. 23. @CorrenMcCoy @WebSciDL JCDL-KDDL 2018, UNT Health Science Center, Fort Worth, TX June 6, 2018@CorrenMcCoy @WebSciDL JCDL-KDDL 2018, UNT Health Science Center, Fort Worth, TX June 6, 2018 Correlation of ARR to UTE for Complete Dataset 23 EEE Rank 1 to 50 51 to100 101 to150 151 to 200 201 to 264 Georgia Tech Univ. of Pittsburgh Cornell Wake Forest
  24. 24. @CorrenMcCoy @WebSciDL JCDL-KDDL 2018, UNT Health Science Center, Fort Worth, TX June 6, 2018@CorrenMcCoy @WebSciDL JCDL-KDDL 2018, UNT Health Science Center, Fort Worth, TX June 6, 2018 Impact of a Celebrity Professor Wake Forest University Loses 410K Followers From UTE Rank 24 URI: ajccenter.wfu.edu Internet Archive: http://web.archive.org/web/20151210173754/https:/twitter.com/M HarrisPerry (2015) URI: freedomontap.com Live Web: https://twitter.com/MHarrisPerry (2018)
  25. 25. @CorrenMcCoy @WebSciDL JCDL-KDDL 2018, UNT Health Science Center, Fort Worth, TX June 6, 2018@CorrenMcCoy @WebSciDL JCDL-KDDL 2018, UNT Health Science Center, Fort Worth, TX June 6, 2018 Correlation of EEE to UTE for Complete Dataset 25 EEE Rank 1 to 50 51 to100 101 to150 151 to 200 201 to 264 Large Enrollment Big Endowments Hefty Athletic Spending Pitt Georgia Tech Dartmouth
  26. 26. 26@CorrenMcCoy @WebSciDL JCDL-KDDL 2018, UNT Health Science Center, Fort Worth, TX June 6, 2018 NCAA Power Five • Richest, Most Powerful Schools in NCAA Division I • 55 of 65 in Top-100 of ARR • 65 of 65 in Top-100 of EEE • 61 of 65 in Top-100 of UTE 26
  27. 27. @CorrenMcCoy @WebSciDL JCDL-KDDL 2018, UNT Health Science Center, Fort Worth, TX June 6, 2018@CorrenMcCoy @WebSciDL JCDL-KDDL 2018, UNT Health Science Center, Fort Worth, TX June 6, 2018 Correlation of ARR to UTE NCAA Power Five 27 NCAA Power 5 Other Schools Louisville https://GoCards.com Georgia Tech Missing Web Links Oregon State Missing Web LinksIvy League Schools Not in Power Five
  28. 28. @CorrenMcCoy @WebSciDL JCDL-KDDL 2018, UNT Health Science Center, Fort Worth, TX June 6, 2018@CorrenMcCoy @WebSciDL JCDL-KDDL 2018, UNT Health Science Center, Fort Worth, TX June 6, 2018 UTE Constrained by Domain Rules for Official Accounts 28 Full table available in Appendix A of our tech report: Corren G. McCoy, Michael L. Nelson, and Michele C. Weigle. 2017. University Twitter Engagement: Using Twitter Followers to Rank Universities. arXiv preprint arXiv:1708.05790.
  29. 29. @CorrenMcCoy @WebSciDL JCDL-KDDL 2018, UNT Health Science Center, Fort Worth, TX June 6, 2018@CorrenMcCoy @WebSciDL JCDL-KDDL 2018, UNT Health Science Center, Fort Worth, TX June 6, 2018 Future Work 29  Address bots and spam accounts  Potentially overinflates the UTE score  The Follower Factory – The New York Times (January 2018)  https://www.nytimes.com/interactive/2018/01/27/technology/social-media-bots.html  Address artificial manipulation of Twitter followers  Campbell’s Law  Temporal analysis using identify non-linear spikes  Known Twitter API limitations related to historical follower counts  Miranda Smith. 2018. Twitter Follower Count History via the Internet Archive. Blog. http://ws- dl.blogspot.com/2018/03/2018-03-14-twitter-follower-count.html  Expand list of official university domains  UTE would include more secondary accounts
  30. 30. @CorrenMcCoy @WebSciDL JCDL-KDDL 2018, UNT Health Science Center, Fort Worth, TX June 6, 2018@CorrenMcCoy @WebSciDL JCDL-KDDL 2018, UNT Health Science Center, Fort Worth, TX June 6, 2018 Conclusions 30  Mined data from Twitter and other public, web data sources  Raw data and 1M Twitter profiles available on GitHub  https://github.com/oduwsdl/University-Twitter-Engagement  Incomplete directories of social media accounts by universities  Necessitated discovery of secondary accounts  Methodology constrains universities to single official domain  Potentially deflates the UTE score  Comparable approximation to national ranking lists (ARR)  Correlation with UTE ( τ =0.6018)  Correlation with EEE ( τ =0.5969)
  31. 31. @CorrenMcCoy @WebSciDL JCDL-KDDL 2018, UNT Health Science Center, Fort Worth, TX June 6, 2018@CorrenMcCoy @WebSciDL JCDL-KDDL 2018, UNT Health Science Center, Fort Worth, TX June 6, 2018 Links to Data Sources 31  Tech report and blog post  Corren G. McCoy, Michael L. Nelson, and Michele C. Weigle. 2017. University Twitter Engagement: Using Twitter Followers to Rank Universities. arXiv preprint arXiv:1708.05790.  Corren McCoy. 2017. University Twitter Engagement: Using Twitter Followers to Rank Universities. Blog. http://ws-dl.blogspot.com/2017/08/2017-08-25-university-twitter.html  Endowment: 2015 NCSE Endowment Market Values, http://www.nacubo.org/Documents/EndowmentFiles/2015_NCSE_Endowment_Market_Values.p df  Athletic Expenditures: Equity in Athletics Data Analysis (EADA), https://ope.ed.gov/athletics/  Undergraduate Enrollment: Integrated Postsecondary Education Data System (IPEDS), https://nces.ed.gov/ipeds/  Google Custom Search: https://cse.google.com/cse/  Twitter Developer Platform: https://developer.twitter.com/content/developer-twitter/en.html  NCAA Conference Membership: http://www.ncaa.org/about?division=d1

×