This document summarizes a study examining Mendeley's coverage of articles from the journal JASIST from 2001-2011 over time. It finds that Mendeley coverage of JASIST articles increased from 97.3% in 2012 to 97.7% in 2014. Readership counts also increased over time, rising from 16,436 readers in 2012 to 39,635 readers in 2014. However, readership counts fluctuated for some articles, which could be due to issues with Mendeley's attribution of publication sources or users deleting items from their Mendeley libraries. The document recommends that Mendeley provide readership counts by month to better understand interest trends over time.
3. Data collection
• All JASIST articles published between 2001 and 2011
and indexed by Web of Science were retrieved
• Info on a few missing articles was filled in manually
• Missing DOIs were supplied
• Citation data from WOS, Scopus and GS were collected
twice, once in 2012 and once in 2014
3
4. Mendeley searches
• In April 2012 and August 2013 all the searches were
conducted manually through the website
• Title searches
• Special characters excluded
• ? “ : & …
• Search results matched against full title and author
• Seemingly wrong publication source was double checked
(DOI or abstract)
• Readership counts of multiple records for a given article
were combined
4
5.
6. Mendeley searches (2)
• In 2014 we used Mike Thelwall’s Webometric Analyst
(http://lexiurl.wlv.ac.uk/) for title searches
• Carefully compared with manual searches
6
7. JASIST 2001-2011
• 1645 articles
7
# articles on Mendeley % of total (1645) Total readers
Apr-12 1,600 97.3% 16,436
Aug-13 1,540 93.6% 24,851
Apr-14 1,453 88.3% 32,968
May-14 1,607 97.7% 39,635
9. Readership counts – closer look
• For 1102 out of the 1645 articles (67%) readership
counts increased monotonously
• Only 6 articles were never located
• All six were cited more than once
• 543 non-monotonous cases
• Maximum decrease in readership count: 240
• “The link-prediction problem for social networks”
• Identical title in conference proceedings and in JASIST
• Wrong source attribution is sometimes the source of large changes in
readership counts
• Other case: found through title search in April 2014, not retrieved in May
2014, but found through URL saved in April – retrieval problems?
• All items not located in April or May 2014 were double
and triple checked 9
10. April 2012 vs. May 2014
• April 2012: 1600 out of 1645 articles
• May 2014: 1607 out of 1645 articles
• Are the same articles missing from both sets?
• Only 6 identical articles!
• The six that were never retrieved
• The 45 articles not found in April 2012:
• 320 readers in May 2014
• Most read article is from 2010 with 63 readers in May 2014
• The 38 articles not found in May 2014
• 325 readers in April 2012
• Most read article is from 2007 with 44 readers in April 2012 and
67 readers in August 2013
10
11. What happens with readership
counts over time?
• Users who bookmark the specific item and include it in
their Mendeley library
• What happens when the user deletes her account?
• What happens when the user deletes an item from her library?
• And if she later bookmarks the same item again???
• How do these actions influence the readership counts?
• The clustering process run from time to time by
Mendeley also affects readership counts
• Attribution of publication source
11
12. Two approaches
1. Once a reader always a reader
– This should result in non-decreasing readership counts
2. Only current readers are readers
– This can explain fluctuations in the counts
• A mixture of the two is also possible
– Deleted account deletes all records
– Deleted items in an existing account do not decrease reader
counts
12
13. Suggested solution
• Provide the number of readers of the item per month
• Total number of unique readers who bookmarked the item
• This would greatly increase Mendeley’s value as an
altmetric
• Easier to report the number of Mendeley readers with a
timestamp
• Allow to study interest trends, assuming that users delete items
they do not find interesting
• In any case a transparent and consistent solution is
needed
13
