Developing a Methodology for Evaluating the Cost-effectiveness of Journal Packages

726 views

Published on

The purpose of this presentation is to share our experience in developeing a methodology for evaluating the cost effectiveness of journal packages. The presentation addresses a number of important issues and provides best practices that should be followed during review. Analyzing usage statistics data, costs er subscribes and unsubscribed titles, use of subscribed and unsubscribed titles will be discussed. Practical guidance in demonstrating the value, or lack of value, of a deal will be provided.

Published in: Education, Technology, Business
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
726
On SlideShare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
6
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • Many funds mean many stakeholders.
  • Developing a Methodology for Evaluating the Cost-effectiveness of Journal Packages

    1. 1. Developing a Methodology in Evaluating Cost Effectiveness of Journal Packages Nisa Bakkalbasi Head, Electronic Collections Yale University Library ER&L 2010 February 1-3, 2010 Austin, Texas
    2. 2. About today’s talk <ul><li>Since 1990's many academic libraries signed multi-year contracts to subscribe to electronic journal packages from large publishers. </li></ul><ul><li>With recent collection budget cuts, many academic libraries are having second thoughts about journal package arrangements, which force them to spend too much money on journals they don't need and which make it difficult to pay for journals from smaller publishers and scholarly monographs. </li></ul><ul><li>In this presentation, I will share a methodology we have developed for evaluating the cost effectiveness of journal packages. </li></ul>ER&L 2010 February 1-3, 2010 Austin, Texas
    3. 3. Advantages <ul><li>Capped price increases and budgetary stability </li></ul><ul><li>Access to non-subscribed content </li></ul><ul><li>Easier to manage/maintain than title-by-title subscriptions </li></ul>
    4. 4. Organizational structure <ul><li>Decentralized library system </li></ul><ul><li>Funding is done through multiple libraries </li></ul><ul><li>The Committee for Digital General Resources makes recommendations about electronic resources that span two or more library areas and manages those resources including funding, platform choice, etc. </li></ul><ul><li>Membership of the committee represents all Collection Units. </li></ul>
    5. 5. Steps <ul><li>Identify the issue </li></ul><ul><li>Collect and analyze data </li></ul><ul><li>Review options: pros and cons </li></ul><ul><li>Discuss decision time-line and implementation </li></ul>ER&L 2010 February 1-3, 2010 Austin, Texas
    6. 6. To identify the issue, we need: <ul><li>An overview of the product </li></ul><ul><li>How funding was established? </li></ul>ER&L 2010 February 1-3, 2010 Austin, Texas
    7. 7. To collect and analyze data, we use: <ul><li>Data source: </li></ul><ul><ul><li>COUNTER-compliant usage reports </li></ul></ul><ul><ul><li>Accessible titles list from consortium or publisher </li></ul></ul><ul><ul><li>Publisher’s title-by-title list price </li></ul></ul><ul><ul><li>Subject categories assigned by the publisher and/or locally </li></ul></ul><ul><li>Data analysis technique </li></ul><ul><ul><li>Descriptive analyses using quantitative variables and qualitative variables </li></ul></ul><ul><li>Tool/software </li></ul><ul><ul><li>Excel PivotTable Report </li></ul></ul>ER&L 2010 February 1-3, 2010 Austin, Texas
    8. 8. To review options and discuss pros and cons, we need: <ul><li>Pricing model or the sales model </li></ul><ul><ul><li>Base-value </li></ul></ul><ul><ul><li>Subscribed/non-subscribed titles, if relevant </li></ul></ul><ul><ul><li>Cancellation allowance </li></ul></ul><ul><ul><li>Price cap/increase </li></ul></ul><ul><ul><li>DDP rate </li></ul></ul><ul><ul><li>Perpetual (or post-cancellation) access rights </li></ul></ul><ul><ul><li>Duration of the contract </li></ul></ul><ul><ul><li>Transfer titles </li></ul></ul><ul><ul><li>Third-party titles </li></ul></ul>ER&L 2010 February 1-3, 2010 Austin, Texas
    9. 9. Let’s get started! <ul><li>Using a case study working with anonymized data, we will walk through a step-by-step evaluation of a journal package! </li></ul>ER&L 2010 February 1-3, 2010 Austin, Texas
    10. 10. Issue <ul><li>An Academic Library has been subscribing to the ABC Publishers Premier e-Journal Collection since 2005. The initial base-value of the journal package arrangement was established based on historical print spend in 2004. </li></ul><ul><li>Reductions in collection budget and different usage patterns among subjects are reasons to consider dissolving the journal package arrangement. </li></ul><ul><li>The Library wants to gain an understanding of whether the package is worth retaining when it is up for renewal -- i.e. are we paying less in the package than we would if we broke the package and had to pay title-by-title for the journals we need to retain access to. </li></ul>ER&L 2010 February 1-3, 2010 Austin, Texas
    11. 11. Product Overview <ul><li>ABC Publishers Premier e-Journal Collection: </li></ul><ul><ul><li>Includes academic and research journals </li></ul></ul><ul><ul><li>Includes titles from learned societies </li></ul></ul><ul><ul><li>Includes titles from physical and natural sciences, medicine, social sciences, humanities, law, and mathematics </li></ul></ul><ul><ul><li>Includes 200 titles </li></ul></ul>ER&L 2010 February 1-3, 2010 Austin, Texas
    12. 12. Sales model <ul><li>Base value established in historical print spend </li></ul><ul><li>Price cap (or an agreed upon price increase) </li></ul><ul><li>Transfer titles added to and subtracted from base value </li></ul><ul><li>DDP rate </li></ul><ul><li>Post cancellation access rights for subscribed years </li></ul><ul><li>Includes access to non-subscribed content </li></ul><ul><li>Some titles are excluded from the contract </li></ul>ER&L 2010 February 1-3, 2010 Austin, Texas
    13. 13. Data Collection <ul><li>Download COUNTER Journal Report 1: Number of Successful Full-Text Article Requests by Month and Journal (preferably three-year period showing usage trend) </li></ul><ul><li>Exclude all titles that are based on separate pricing models. </li></ul><ul><li>Exclude journal archives usage if backfile purchase was a separate acquisition. </li></ul><ul><li>Obtain a list of “Accessible Titles” list, which contains a list of all titles (subscribed and non-subscribed) included in the package arrangement. </li></ul><ul><li>Obtain a list of publisher’s list prices. </li></ul><ul><li>Obtain a list of subscribed titles included in the contract. </li></ul><ul><li>Aggregate all data in a single spreadsheet. </li></ul>ER&L 2010 February 1-3, 2010 Austin, Texas
    14. 14. Producing data <ul><li>In the raw dataset each record should contain the following variables: </li></ul><ul><ul><li>Journal title </li></ul></ul><ul><ul><li>ISSNs </li></ul></ul><ul><ul><li>Usage count (YTD Total from the COUNTER JR1 report) </li></ul></ul><ul><ul><li>List price </li></ul></ul><ul><ul><li>Categorical variable indicating subscribed and non-subscribed titles </li></ul></ul><ul><ul><li>Categorical variable indicating subjects </li></ul></ul>ER&L 2010 February 1-3, 2010 Austin, Texas
    15. 15. Enhance data to facilitate decision <ul><li>Calculate cost-per-use (CPU) </li></ul><ul><ul><li>CPU= e-only price for the title /YTD Total (per year) </li></ul></ul><ul><li>Using ILL cost as a threshold, create new categorical variables to filter for possible renewals and cancellations </li></ul>ER&L 2010 February 1-3, 2010 Austin, Texas
    16. 16. Sample source data ER&L 2010 February 1-3, 2010 Austin, Texas <ul><ul><li>Let’s take a look at an anonymized data! </li></ul></ul>
    17. 17. Snapshot of title distribution ER&L 2010 February 1-3, 2010 Austin, Texas Subscribed vs. non-subscribed title counts Subject Non-sub Sub Grand Total Humanities 9 38 47 Law 4 21 25 Social Sciences 11 26 37 Science & Medicine 32 41 73 Mathematics 10 8 18 Grand Total 66 (33 %) 134 (67 %) 200
    18. 18. Snapshot of usage patterns ER&L 2010 February 1-3, 2010 Austin, Texas Subscribed vs. non-subscribed titles usage counts Subject Non-sub Sub Grand Total Humanities 1,057 7,775 8,832 Law 257 1,821 2,078 Social Sciences 1,450 4,314 5,764 Science & Medicine 10,544 60,246 70,790 Mathematics 204 1,384 1,588 Grand Total 13,512 (15 %) 75,540 (85 %) 89,052
    19. 19. Snapshot of cost distribution ER&L 2010 February 1-3, 2010 Austin, Texas Subscribed vs. non-subscribed titles cost Subject Non-sub Sub Grand Total Humanities $2,074 $10,725 $12,799 Law $1,666 $9,346 $11,012 Social Sciences $4,511 $10,363 $14,874 Science & Medicine $24,764 $42,071 $66,835 Mathematics $6,402 $9,198 $15,600 Grand Total $39,417 (33%) $81,703 (67%) $121,120
    20. 20. “ Drop & Pick-up” ER&L 2010 February 1-3, 2010 Austin, Texas CPU for Subscribed vs. Non-Subscribed Titles Non-sub Sub Grand Total CPU >= $10 29 21 50 CPU <= $10 37 113 150 Grand Total 66 134 200
    21. 21. “ Drop & Pick-up” ER&L 2010 February 1-3, 2010 Austin, Texas CPU for Subscribed vs. Non-Subscribed Titles Non-sub Sub Grand Total CPU >= $25 18 6 24 CPU <= $25 48 128 176 Grand Total 66 134 200
    22. 22. Candidates for renewal & cancellation (based on $10/ILL) ER&L 2010 February 1-3, 2010 Austin, Texas Count of possible candidates for renewal/cancellation Subject CPU >= $10 CPU <= $10 Grand Total Humanities 7 40 47 Law 13 12 25 Social Sciences 6 31 37 Science & Medicine 8 65 73 Mathematics 16 2 18 Grand Total 50 150 200
    23. 23. Candidates for renewal & cancellation (based on $25/ILL) ER&L 2010 February 1-3, 2010 Austin, Texas Count of possible candidates for renewal/cancellation Subject CPU >= $25 CPU <= $25 Grand Total Humanities 3 44 47 Law 6 19 25 Social Sciences 2 35 37 Science & Medicine 3 70 73 Mathematics 10 8 18 Grand Total 24 176 200
    24. 24. Candidates for renewal & cancellation (based on $10/ILL) ER&L 2010 February 1-3, 2010 Austin, Texas Cost of possible candidates for renewal/cancellation Subject CPU >= $10 CPU <= $10 Grand Total Humanities $1,782 $11,017 $12,799 Law $6,738 $4,274 $11,012 Social Sciences $2,581 $12,293 $14,874 Science & Medicine $7,745 $59,090 $66,835 Mathematics $15,176 $424 $15,600 Grand Total $34,022 $87,098 $121,120
    25. 25. Candidates for renewal & cancellation (based on $25/ILL) ER&L 2010 February 1-3, 2010 Austin, Texas Cost of possible candidates for renewal/cancellation Subject CPU>=$25 CPU<=$25 Grand Total Humanities $588 $12,211 $12,799 Law $3,832 $7,180 $11,012 Social Sciences $649 $14,225 $14,874 Science & Medicine $1,329 $65,506 $66,835 Mathematics $9,110 $6,490 $15,600 Grand Total $15,508 $105,612 $121,120
    26. 26. Frequency of title use ER&L 2010 February 1-3, 2010 Austin, Texas 10 % of use comes from two titles, 20 % of use comes from four titles, 50 % of use comes from fifteen titles, and so on. Cumulative Relative Frequency (%) 10% 20% 50% 80% 90% No use 1% (2) 2% (4) 8% (15) 24% (48) 40% (80) 0
    27. 27. Decision <ul><li>Depends on: </li></ul><ul><ul><li>Base value paid by the Library </li></ul></ul><ul><ul><li>Collection budget increase/reduction rate </li></ul></ul><ul><ul><li>Price cap </li></ul></ul><ul><ul><li>Quality and cost of ILL service </li></ul></ul>ER&L 2010 February 1-3, 2010 Austin, Texas
    28. 28. Closing remarks <ul><li>Methodology reveals: </li></ul><ul><ul><li>whether users are using journals </li></ul></ul><ul><ul><li>value for money </li></ul></ul><ul><ul><li>necessary but insufficient data to make strategic decisions </li></ul></ul><ul><li>Methodology does not reveal: </li></ul><ul><ul><li>The users’ experience or perception of the utility or value of the collection </li></ul></ul><ul><ul><li>Low use can occur because the product’s user interface is difficult to use or because users are unaware that the product is available </li></ul></ul>ER&L 2010 February 1-3, 2010 Austin, Texas
    29. 29. Questions? ER&L 2010 February 1-3, 2010 Austin, Texas
    30. 30. Thank you for coming! ER&L 2010 February 1-3, 2010 Austin, Texas

    ×