UKSG 2018 Breakout - Analysing value for money of journal bundle deals at the University of Strathclyde - Tinto and Bell
•Download as PPTX, PDF•
2 likes•370 views
Following successful implementation of processes for analysing the cost-per-use of single journal subscriptions, staff at the University of Strathclyde’s Andersonian Library have embarked on projects to create robust analysis for both journal bundle deals and database subscriptions. This session will review the work to date on reviewing journal bundle deals. Case studies will provide context to the discussion of the steps taken, problems encountered and solutions employed in developing a system which can be applied to various multi-title journal subscriptions. Fiona Tinto and Sally Bell University of Strathclyde
Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Similar to UKSG 2018 Breakout - Analysing value for money of journal bundle deals at the University of Strathclyde - Tinto and Bell
Similar to UKSG 2018 Breakout - Analysing value for money of journal bundle deals at the University of Strathclyde - Tinto and Bell (20)
More from UKSG: connecting the knowledge community
More from UKSG: connecting the knowledge community (20)
Recently uploaded
Recently uploaded (20)
UKSG 2018 Breakout - Analysing value for money of journal bundle deals at the University of Strathclyde - Tinto and Bell
- 1. Analysing value for money of journal bundle deals at the University of Strathclyde Fiona Tinto, E-Resources Librarian Sally Bell, Engineering Faculty Librarian
- 2. Session outline • Why did we embark on this project? • How did we decide what to do? • What have we done? • What are the outcomes so far? • What are our next steps? • Q&A
- 3. Background • Little subscription analysis done previously • Journals collections growing static • New procedure developed • 5 year rolling cost-per-use analysis • Tracking trends in usage and cost-per-use analysis • Flaws highlighted • No usage data for print titles • No comparable data for databases • Not directly applicable to bundle deals
- 4. Research / Literature Review • Literature search done in 2 main databases • Library and Information Science Abstracts (LISA) • Library, Information Science & Technology Abstracts (LISTA) • 38 articles read in total • Not overly useful • Generally lacking clarity on techniques used
- 5. Research / Literature Review • Two interesting points to work with • Highlighting titles which are high, medium or low usage within the collections helpful • No real information on how usage bands were agreed upon • Formula to calculate actual cost of a title within a bundle • Super-useful! • Jurczyk, E. and Jacobs, P. (2014) 'What's the Big Deal? Collection Evaluation at the National Level'. Portal : Libraries and the Academy, 14 (4), pp. 617-631.
- 6. What are we trying to achieve? • Crucial Questions: • How can we gather accurate and meaningful information to support informed decision making about our e-journal deals? • Can we develop a process which is manageable – which can realistically be embedded into the annual workflow with the staffing we have? • Can we apply a consistent one size fits all approach? • Work in progress: • This presentation is about the process we are trying to develop, rather than the value of the deals themselves.
- 7. Publisher 1 - approach • Early attempt at analysis (pre-literature review) • 174 subscribed titles • Publisher model changed. Collection we took discontinued; smaller subject bundles now on offer. We could afford 6 subject collections. • Analysis • Publisher’s title list of titles within each subject collection (excel spreadsheet); title list exported from Alma to indicate titles within our current deal; usage figures (JR1 minus JR1a and GOA) and turnaway statistics from JUSP. • Consolidated data from different lists using excel VLOOKUP. • Wanted to present accessible information as well as the raw data. • Focussed on usage only rather than cost per use.
- 8. Publisher 1 – subscribed and non-subscribed titles in each collection
- 9. Publisher 1 – usage and turnaway statistics
- 10. Publisher 1 – low/medium/high usage and turnaways Usage LOW 1-99 MED 100- 699 HIGH 700 and above Turnaways LOW 1-19 MED 20-99 HIGH 100 and above
- 11. Publisher 1 – stumbling blocks and outcomes • Stumbling blocks • Why is VLOOKUP not working? Formatting / hidden characters copied over from publisher’s spreadsheet – clean all this away before working with the data. • Titles not activated in Alma and therefore missing from the catalogue – impact on both analysis and discovery/usage • How do you categorize high, medium and low usage thresholds? • Outcomes • Subscribed to 6 bundles, plus 15 individual subscriptions. • Positive feedback from Faculty Librarians about this approach to analysis. • This analysis aimed to give accessible overview of usage and demand within subject collections – cost per use not included. • Achievable scale – one day’s uninterrupted work (NB relevant factors – small deal; consistent information and metadata from this publisher; did not include pricing in this analysis).
- 12. Publisher 2 - approach • Deal: • 1787 titles – 266 core titles; 1521 collection titles • Deal terms: core title prices based on capped yearly % increase on list price; pay additional fee to access bundled ‘collection’ titles. • Approach • Serials team applied same process they use to provide stats and cost per use for individual journal titles (using Microsoft Access to combine separate spreadsheets with titles, pricing and usage info) • Title list (held by serials team) • Usage stats from JUSP (JR1 minus JR1a + GOA) • 2017 list prices from publisher • Challenges when working with deals vs individual subscriptions • More titles, more changes – harder to keep track • More data sources – more scope for metadata mismatches and problems with excel matching • Determining pricing for bundled titles within a deal
- 13. ‘The Formula’ NB – VAT (+20%) included at every stage • Core titles • Capped % increase as per terms of deal • Serials team provided both list price (non-deal price) and price we paid via deal • Formula to calculate ‘notional deal price’ for individual collection titles: • Take the sum total of the list prices for the bundled collection titles • Take the price we actually paid for the collection bundle • Notional deal price for each collection title: 𝑵𝑶𝑻𝑰𝑶𝑵𝑨𝑳 𝑫𝑬𝑨𝑳 𝑷𝑹𝑰𝑪𝑬 = 𝑻𝑰𝑻𝑳𝑬 𝑳𝑰𝑺𝑻 𝑷𝑹𝑰𝑪𝑬 𝑺𝑼𝑴 𝑶𝑭 𝑳𝑰𝑺𝑻 𝑷𝑹𝑰𝑪𝑬𝑺 × 𝑶𝑼𝑹 𝑻𝑶𝑻𝑨𝑳 𝑪𝑶𝑳𝑳𝑬𝑪𝑻𝑰𝑶𝑵 𝑩𝑼𝑵𝑫𝑳𝑬 𝑷𝑹𝑰𝑪𝑬
- 14. Publisher 2
- 15. Publisher 2
- 16. Publisher 2 [average usage – mean vs median] Mean = sum of all the values in the set, divided by the number of values in the set Median = the middle point of the values (half will be above and half below)
- 17. Challenges and queries • Challenges - at best these are a delay and a nuisance, in the worst case scenario they can severely skew the figures and analysis: • Incorrect or missing information and metadata (ISSN / pricing / usage / inaccurate title lists) • Format of publishers’ list prices – PDFs; excel formatting; presentation of pricing for different countries/sizes of institution; handling of package/combination titles • Consolidating data from different sources (Print or eISSN being used) • Queries: • How to define ‘high’, ‘medium’ and ‘low’ usage/turnaways • Which usage statistics to use? We used JR1 minus JR1a+GOA to reflect paid for current content – is this the best approach? For non-JUSP publishers, additional effort to calculate. • If titles do not appear in usage statistics does this equal zero usage? • Obtaining title lists and list pricing for previous years can be difficult, making any retrospective analysis a challenge. What information do we want to collect in preparation?
- 18. Next steps • Continue looking at the queries that have been raised throughout the project. • Review our various analysis projects with other Acquisitions staff and Faculty Librarians to see which elements we think are useful or unnecessary to pursue. • Discuss with serials team to gauge what could realistically be absorbed into their workflow. • Repeat for a BIG publisher and monitor timescales more closely. • Consider whether any of the data gathering is work we would want to, or could, ask publishers to compile for us rather than trying to title lists, list prices, and usage statistics in-house. • Start using SUSHI harvesting of usage statistics into Alma (this will free up staff time currently spent gathering statistics).
- 19. Questions & Answers • Any questions? • Or (preferably) any fantastic answers to the questions we have raised, or suggestions on a better way to do this? Contact Details • Sally Bell, Engineering Faculty Librarian – sally.bell@strath.ac.uk • Fiona Tinto, E-Resources Librarian – fiona.tinto@strath.ac.uk
- 20. References • Blecic, D.D. et al. (2013) 'Deal or no deal? Evaluating big deals and their journals'. College and Research Libraries, 74 (2), pp. 178-193. • Emery, J. and Stone, G. (2013) 'Annual Review'. Library Technology Reports, 49 (2), pp. 30-34,32. • Glasser, S. (2013) 'Judging Big Deals: Challenges, Outcomes, and Advice'. Journal of Electronic Resources Librarianship, 25 (4), pp. 263-276. • Jones, M.A., Marshall, D. and Purtee, S.A. (2013) '“Big Deal” Deconstruction'. Serials Librarian, 64 (1-4), pp. 137-140. • Jurczyk, E. and Jacobs, P. (2014) 'What's the Big Deal? Collection Evaluation at the National Level'. Portal : Libraries and the Academy, 14 (4), pp. 617-631. • Rathmel, A., Currie, L. and Enoch, T. (2015) '“Big Deals” and Squeaky Wheels: Taking Stock of Your Stats'. Serials Librarian, 68 (1-4), pp. 26-37. • Shearer, B.S., Klatt, C. and Nagy, S.P. (2009) 'Development of a new academic digital library: A study of usage data of a core medical electronic journal collection'. Journal of the Medical Library Association, 97 (2), pp. 93-101. • Wilson, J. (2010) Journal Value Metrics Assessment. Available at: https://www.cdlib.org/cdlinfo/2010/03/30/journal- value-metrics-assessment/ (Accessed: 15/03/2018). • Wilson, J. and Li, C. (2012) Calculating scholarly journal value through objective metrics. Available at: https://www.cdlib.org/cdlinfo/2012/02/13/calculating-scholarly-journal-value-through-objective-metrics/ (Accessed: 15/03/2018)
Editor's Notes
- Opening [SB]
- We’re aiming to give you a broad overview of the project we are currently working on, this will cover what the motivations were for starting the work, what background reading and research we did to inform our processes, what we have done so far, and how this feeds into other pieces of work currently ongoing. It is very much a look at a work-in-progress, rather than a completed project and we are not in a position to say what is and isn’t the best way to do this kind of work. Hopefully it will give you some inspiration and also highlight some problems which we think will be universal in this kind of project.
- Sally: When I started at Strathclyde there was little analysis done of the journals portfolio – stats were gathered and sent to Faculties on an annual basis but in terms of reviewing the actual use and value of the subscriptions it was largely done on an ad-hoc basis related to how many new titles academics had requested. As budgets are largely static we are unable to take on new titles without cancelling something from the existing collection. This system didn’t work at all, and it turned out the Engineering journals portfolio had been static for multiple years, despite requests from teaching and research staff for new titles. In consultation with members of the Faculty we developed a system to look at average cost-per-use on a 5 year rolling basis, and also monitor trends in the stats; so if the usage was going up or down year on year, and if the cost-per-use were also going up or down year on year. This enabled us to review the top 10% most expensive on a cpu basis, while also keeping an eye on the trends so if the usage were going up even though the CPU were high we could consider if that title were of value or not. The system worked really well for individual subscriptions of journals, but fell down in relation to print titles, database and bundle deals. Along came Fiona and we started to look into how to fill these gaps… We have done some work on the databases and print titles, but today we’re going to talk about the bundle deals as they are such a large financial commitment and a huge percentage of our overall subscriptions collection.
- Sally: Because we are good librarians, we did a literature search to see what others have done before. I will admit it was not a systematic review but we did check LISA and LISTA and limited our search from 2010 onwards. This threw up a range of articles which sounded very promising – and we sourced full text of a total of 31 papers, this came up to 38 once we added in useful citations linked from the original 31. Sadly most of them, while interesting, didn’t prove to be of a great deal of value. The trends seemed to be for people to report on either the reason for doing analysis or the outcome of their analysis, rather than actually the processes which they followed. This proved to be incredibly frustrating.
- Sally: A few things which did crop up which we found interesting: Many studies discussed titles in terms of if they are high use or low use, but few explained what that meant in terms of usage figures or percentage of the overall use. And if they did explain those figures, they didn’t explain how they decided what was high, medium or low use. A formula for calculating the value of an individual title within a bundle deal, originally from a study by the California Digital Library which was reproduced in a 2014 paper. (see reference) Now I’ll hand over to Fiona and let her tell you a bit more about what we have actually done so far…
- Fiona: Sally has outlined our process for analysis of individual journal subscriptions. We want to have something similar in place to be able to monitor our e-journal deals and enable informed decision making around these huge subscriptions. We want a continual process which can be embedded into our regular workflows in the way that analysis of our individual subscriptions already is. For that to happen, anything we come up with has to be on a manageable scale. A process that is too big and requires additional dedicated time or staffing resource isn’t going to be useful to us other than as one-off exercises. Over the past year we have attempted to analyse journal deals from various publishers. The size and nature of the deals, and the circumstances prompting the analyses have varied in each case and so the approaches have been slightly different. I’m going to focus on our analysis of two publisher deals today and as Sally said, this is very much a work in progress and this presentation focusses more on what we’ve tried to do and how we’ve tried to do it, rather than any conclusions about the value of the deals themselves.
- Fiona: Analysis of the Publisher 1 deal happened before Sally and I had started properly on this project, so this was before we had conducted our literature review. The deal from Publisher 1 was a small collection – it contained 174 titles. The driver behind this analysis was a change to the subscription model offered by the publisher. The Collection we took was being discontinued, and the publisher was moving to offering smaller subject collections; or individual subscriptions to journals. We knew from the outset that we would be able to afford a bundle of 6 subject collections. The data I was working with was the publisher’s subject collection and title list; our own title list of the titles within our current deal exported from Alma (our library management system); and usage statistics and turnaway statistics from JUSP. What I did to combine all the data was to break down the publisher’s spreadsheet into a separate tab for each subject collection, cross-reference with our own Alma-export title list using VLOOKUP up to indicate for each title whether we did or didn’t already have access via our existing deal, and then use VLOOKUP to insert usage statistics for subscribed titles and turnaway statistics for non-subscribed titles. In this analysis I didn’t look at cost per use at all – what I was trying to do was look at usage and demand for the titles within each subject collection; and what I was very keen to do was present very accessible overall pictures, from which people could then identify areas of interest or concern before looking at the raw data for more information.
- Fiona: Firstly – this slide presents for each subject collection the number of titles we did or didn’t have access to via our existing deal. The green portion is the number of titles we did have access to, and red is titles we had no access to. You can see that our existing subscriptions were spread across the full range of subject collections – which is why this analysis was necessary – because we had to pick which of these subject collections we were going to subscribe to. (I’ve underlined some collections in green there – those were the 6 subject collections we ended up subscribing to.)
- Fiona: Then we looked at the usage and turnaway numbers within each subject collection. The green column is our usage statistics of subscribed titles; and the yellow column is turnaway figures of titles we had no access to in our existing deal. This information is helpful – you can see which collections are most heavily used – and it’s interesting to see that it’s not necessarily the biggest collections that got the highest overall usage - if you look at Collection 5 it’s got the heaviest usage by far, but it doesn’t actually contain the most journals. If we look at the previous slide there are actually four other collections where we have a greater number of subscribed titles. That could mean that the journals in this collection really are punching above their weight, or it could reflect that this collection is relevant to a subject area where we have huge class sizes – you don’t know based on this alone – you have to consider everything in context - so this is what I mean by presenting overall pictures in an accessible way which leads to specific questions, and people can then look at the data in more detail to answer those questions. If we look at this graph, for me what’s really lacking is that the high green bars look really impressive, but there’s no reflection on whether the whole collection is being heavily used, or only certain titles. For example, if you look at Collection 2 which has pretty high usage, you can’t tell from that whether all 29 titles we had access to in that collection are getting decent usage or whether a handful of titles are getting really heavily used and the rest aren’t being used at all.
- Fiona: So what I tried to do in response to that was reflect the portion of titles within each collection which were receiving high/med/low or zero usage or turnaways (first column within each pair is usage, second is turnaways). But what I really struggled with when I got to this point, was how do you categorize ‘high’, ‘medium’ and ‘low’ usage? You can see the numbers I used at the side there, but I wasn’t really happy with those thresholds. I would still like to use this approach, but to make it actually meaningful, we need a more robust way of defining those thresholds. As Sally mentioned earlier, we didn’t find anything helpful in that respect in the literature review, so if anyone has any ideas on that we’d really love to hear them.
- Fiona: I outlined already the data sources we used and the method for combining these – I did hit a few stumbling blocks with this process: initially my VLOOKUP just wasn’t working and then I realised it was due to hidden characters copied over from the publisher’s title list spreadsheet. I thought I’d cleaned out unnecessary formatting before I started but obviously not well enough – so if you’re copying and pasting data from multiple sources it’s really worth spending some time cleaning and trimming and getting rid of rogue spacing etc. thoroughly right at the start. Another hiccup was that I was using an exported title list from our library management system, Alma, to identify the titles within our current deal, but I discovered there were a handful of titles in our current deal that hadn’t been activated in Alma so they were missing from our catalogue. Because this was a small collection, it was easy to identify these titles and amend this – but that would have had an impact on the accuracy of the analysis, but also it will have affected the discovery and potentially then the usage of these titles over the past year. And also it would have been much harder and more time consuming to identify and fix this in a bigger deal. And as I’ve already mentioned – defining thresholds for high, medium and low usage (and turnaways) was a problem. In the end we subscribed to 6 subject collections, plus we took 15 individual journal subscriptions. The Faculty Librarians were positive about the way this data was presented, and it was also shared with the academics in one department to support their decision making. I don’t know whether they found it useful, but I would be interested to try and get their feedback and build on that – particularly because, as I said, this was an early experiment in presenting data in a different way. This took about 1 day’s uninterrupted work – so that’s very achievable to do on an ongoing basis. Other than the challenges listed above, from this publisher the subject collections and title list was in a helpful format (in excel and easy to manipulate) and the metadata in the different sources was accurate and consistently applied, so the excel/VLOOKUP process was quite straightforward. On the other hand, this was a small publisher deal, and we did not include pricing in this analysis which is something we would generally want to include – so both of those factors made it a smaller project.
- Fiona: Moving on to Publisher 2, this deal was a medium sized collection – just under 1,800 titles, and the nature of this deal is that the pricing for the core titles is capped at a yearly percentage increase; and then we pay an additional fee to access the bundled ‘collection’ titles. The serials team did the work collecting the initial data for us and followed the same process they would use to compare cost per use information for our individual subscriptions - which is using Microsoft Access to combine the title list, usage statistics, and pricing data into one spreadsheet; and then calculating cost per use for each title within excel. The main difference in process between this instance and the team’s regular process, is that with the individual journal subscriptions we would have pricing information data recorded, whereas in this instance we also had to obtain a price list from the publisher to get list prices for the bundled collection titles. Compared to analysis of individual journal subscriptions, with deals you’re trying to work with larger volumes of titles; things like titles transferring between publishers, journals changing names – all that becomes that bit messier on the bigger scale. It becomes more difficult to try and get definitive information, and when you’re trying to work with multiple data sources and match them using something like Excel or Access then differences in the information and the metadata can become really problematic, and very time consuming when manual checking is then required to address matching errors. And then, this was our big question – for our individual subscriptions our serials team have very precise records of our subscriptions and the relevant pricing – but how do you determine pricing for individual titles within a bundled collection deal?
- Fiona: This is where the formula comes in that Sally mentioned, which allows us to include a notional pricing for collection titles in order to calculate a notional cost per use. For our ‘core’ titles within the deal, we know what we paid. When the serials team compiled the data for us, for core titles they included both the publisher’s list price for each title – so that’s the price you would pay outwith the deal - and also the capped price for each title that we actually paid as per the terms of the deal. And I’ll just say here – for all the prices we have presented we have included the +20% VAT. But for Collection titles, the only pricing we have available at individual title level are the list prices. Because we pay a heavily discounted lump sum for access to the bundle of collection titles, these list prices aren’t in any way representative of the price we are paying for collection titles via the deal. The literature review uncovered this formula to help calculate a more representative notional price for what you’re actually paying for individual titles within a bundled collection…. - Take the sum total of the list prices for all the collection titles Take the price we actually paid for the collection bundle Then to calculate the notional price for each collection title – take the title list price, divide it by the sum total of the list prices (so that gives the value of the title as a fraction within the publisher’s collection) and then multiply that by the price we paid for the collection bundle you get your notional price (so that applies that same value ratio against the price you have paid for your collection). If you had a deal where there was no ‘core title’ component any more and everything was just a discounted fee, you could apply the same process to calculate notional pricing for all titles within the deal.
- Fiona: This is the data we were then working with – Journal Title, whether the title was Core or Collection title, ISSNs, the Non-Deal price per title; calculated Deal price per title – so that’s either the known capped core title price, or our notional collection title price, JR1 minus JR1a+GOA usage figures, Cost per use calculated based on non-deal pricing (for comparison), and CPU based on our calculated deal pricing. This is just a snapshot, but you can see here that Core titles are slightly discounted from the list price; but obviously it is in the Collection titles that you are seeing the heavily discounted pricing and cost per use – so that’s supported what we already know - that with the bundle deals we’re getting a big number of journals at a heavily discounted rate.
- Fiona: Overall if you look at the mean average price and cost per use based on being in the deal, compared to the mean averages out-with the deal those numbers look great. But we can calculate that just with using core and collection total for the deal, and total list prices – that doesn’t require any of the additional work we’ve just described. By doing the extra work to gather notional pricing and cost per use for individual titles we now have data for content within the deals that can support Sally and her team in doing the equivalent annual usage and cost per use trend monitoring that they currently do for individual journal subscriptions. And I’ll just reiterate here, what we hope to achieve from this analysis is that it can help inform and support decision making; and that you have to look at everything within an appropriate context – so whatever this type of analysis flags up, we can use that as a starting point and then say right why is that; what are the relevant factors and what do we want to do about it? To have this information means we are in a position to do that; whereas before the big deals have been to a degree exempt from the same kind of review and scrutiny as the individual subscriptions, which isn’t ideal, particularly given the sums of money we’re talking about with big journal deals.
- Fiona: In the previous slide I gave a comparison between deal and non-deal average price and average cost per use, and for that average I used the mean average. Earlier with Publisher 1 I mentioned struggling to determine thresholds for high/medium/low usage. I still want to that kind of analysis, so with Publisher 2 we tried a simplified version of just breaking it down at ‘above average’, ‘below average’ and ‘zero usage’ categories. But what is the average? We did this analysis using both the ‘mean average’ and the ‘median average’ and you can see in the two graphs the difference it makes if you alter your threshold. So if you were using the mean, then about 1300 titles in the whole deal would be reported as receiving “above average” usage; if you define average as meaning the median, then only about 800 titles in the whole deal would be classed as receiving “above average” usage. You can see what a huge difference that makes to the output, so if we want to pursue this approach of trying to present which titles get ‘high’ or ‘low’ usage, then it’s really important that we find a sound way of defining those thresholds.
- Fiona I’ll touch on some of the challenges that we hit with the Publisher 2 deal, and also with other deals we’ve looked at: There were 40 titles in the Publisher 2 deal which did not appear on the publisher’s price list document and therefore in our analysis are sitting with no price and therefore no cost per use. That’s 2% of the titles in this collection that aren’t being analysed in themselves, and this could also be skewing the overall picture. What level of additional effort do we go to track down this missing information, and at what point does this become more effort than it’s worth? When I started the same analysis for one of our big publishers, it was more like 100 titles in this category. The Publisher 2 deal contained combined or package titles. These returned VLOOKUP errors and needed manually amended to input usage and pricing information. Another complication is that different publishers handle and present combined and package titles differently in their title lists and list price documents. For Publisher 2 we don’t have a named account manager, and the helpdesk didn’t respond to a request to provide a price list in excel format – because we were under some time pressure to complete this analysis, in the end we made do with the PDF price list, which my colleague converted to excel via a method she’d found on the internet. In some other cases, although publishers do provide their list prices in excel, the tabular format used to break down pricing for different countries or institution sizes means a lot manual work is involved to isolate the relevant list prices. Other things we’re still debating – and we’d love to speak to anyone who has any thoughts on these How to define high, medium and low usage thresholds Which usage statistics are best to use – JR1, JR1 minus JR1a, or JR1 minus JR1a+GOA. I think we’ll need to look at each deal individually, certainly to see whether any archival access is included and if JR1a stats therefore need to be included. When it comes to whether or not to exclude GOA stats we’re still a bit unsure. Where titles don’t appear in usage statistics we are interpreting this as zero usage. Is it safe to assume this? How common an occurrence is this and would we have resources to investigate these individually? If we want to build this into our regular workflows then we’ll need to collect things like publisher price lists ahead of time, as they don’t stay available online indefinitely (though did obtain for one other publisher using the Internet Archive / Way Back Machine). We need to decide what information we want to collect.
- [FT]
- Sally: Not all the papers we found, but these are the main 9 references which we found most useful at the current stage of the project.