Erin Braswell, Lead Developer, Center for Open Science; The SHARE Project

1. SHARE Phase II
Judy Ruttenberg, Association of Research Libraries
Erin Braswell, Center for Open Science
Fabian von Feilitzsch, Center for Open Science
NISO Virtual Conference: October 1, 2015
Using Alerting Systems to Ensure OA Policy Compliance

2. Founded by Academic Leaders, Built
with Open Technology
Research universities are long-lived and are mission-driven to
generate, make accessible, and preserve over time new
knowledge and understanding.

3. What is SHARE?
SHARE is building a free, open data set about
research and scholarly activities across their
life cycle.

4. Research Lifecycle

5. Open Science Framework
http://osf.io

19. Using SHARE’s Search API
● API is currently a slightly restricted Elasticsearch
instance
● You can hit the API with any valid Elasticsearch query
● Going to go over some quick and sort of interesting
aggregations that are available

20. What it looks like: The query
$ curl -XPOST "https://osf.io/api/v1/share/search/" -H 'content-type: application/json' -d '{
"query": {"match_all": {}},
"size": 0,
"aggs": {
"top tags": {
"terms": {
"field": "tags"
}
}
}
}'

21. What it looks like: The response
{
"count": 2137806,
"time": 0.1,
"results": [],
"aggs": null,
"aggregations": {
"top tags": {
"buckets": [
{
"key": "ecological",
"doc_count": 20347
},
{
"key": "long",
"doc_count": 20179
},
{
"key": "term",
"doc_count": 20021
},
{
"key": "lter",
"doc_count": 18862
},
{
"key": "data",
"doc_count": 17086
}, ....
...
{
"key": "research",
"doc_count": 16539
},
{
"key": "earth",
"doc_count": 16395
},
{
"key": "water",
"doc_count": 16150
},
{
"key": "program",
"doc_count": 16098
},
{
"key": "remote",
"doc_count": 15963
}
],
"sum_other_doc_count": 1049093,
"doc_count_error_upper_bound": 3000
}
}
}

22. Kind of a pain
● We have an experimental python library to
help cut down the verbosity a bit

23. Same Example
>> from sharepa import ShareSearch
>> from sharepa.analysis import bucket_to_dataframe
>> search = ShareSearch()
>> search.aggs.bucket('top tags', 'significant_terms', field='tags')
Internal structure is:
{
"query": {
"match_all": {}
},
"aggs": {
"top tags": {
"significant_terms": {
"field": "tags"
}
}
}
}

24. Now we send the JSON blob to the SHARE search API
>> results = search.execute()
And we get back the same response.
We can then use some of our utilities to convert
the Elasticsearch response to a dataframe
(basically just a table)
>> df = bucket_to_dataframe(
'top tags',
results.aggregations['top tags']['buckets']
).sort('top tags', ascending=False)
and plot it as well:
>> df.plot(kind='bar', x='key', y=['bg_count', 'top tags'])

25. Cancer

26. Flu

27. Influenza

28. Vaccine

29. Phase II

30. Contact Us!
info@share-research.org
https://osf.io/yg3xj/