SlideShare a Scribd company logo
Crowdsourcing citation suggestions,
without needing to ask the crowd
Brian Bishop, Co-founder
@mochasteak
Brian Bishop
Business / Product stuff
Formerly VP of Platform
Development at
Springer, MBA,
14-year scientific
publishing veteran
Pawel Kowalski
Tech stuff
Tech Principal at iterativ
GmbH, full-stack dev,
MSc in Computer
Science, the whole deal
Our Team
02
Timeline*
03
Prototype goes live
November 2014
Today
January 2015
Company registered
August 2014
*timeline not to scale
We asked 1,044 authors
“What is the most
difficult or frustrating
part of writing a
paper?”
and this is what they said…
05
reading between the tags:
06
How painful is this problem?
07
836
Authors
surveyed
“How long did it take you to gather all the
references / find sources for your paper?”
25%
“More than
a week”
35%
“More than
a month”
That’s a lot of
time spent not
doing research
The average scientific paper has 35 citations
09
This is the citation
context
Each of those citations is embedded in a sentence
“Both experimental and
atomistic simulation results
show that when the
dimensions of the structures
become small then the ‘size
effect’ has significant role in
the mechanical properties
(Ruud et al. 1994).”
10
Now go the other way
Paper A
Paper B
Paper C
11
We can extract out all the citing contexts from all
the papers, and group them according to the
document being cited.
A database of contexts
A clinical implementation was described first for total mesorectal excision (TME) in the
treatment of rectal cancer [4, 5]. | The circumferential rectal margin (CRM) was
assessed according to the method of Quirke et al. [13], and a margin of < 1 mm was
considered CRM-positive. | Complete mesocolic excision (CME) with central vascular
ligation (CVL), according to the sound principles of total mesorectal excision (TME) [6,
7] for rectal cancer | A standardized routine pathology examination was performed
using the protocol of Quirke et al. [36]. | The surgical specimens were handled
according to standard clinical practice as advocated by Quirke et al. [20] and were
pathologically examined in accordance with the Tumor Node Metastasis staging
system. | For rectal cancer, the specimen was processed using the slicing technique
as described by Quirke et al. 33 | Pathologists were trained to examine the
specimens according to the protocol of Quirke et al. regarding the circumferential
resection margin (CRM), lymph nodes, and dissection plane.12 | Quirke showed a
linear correlation between the development of a local recurrence and an inadequate
resection with positive circumferential margins [14, 22]. | The pathologists were
trained to identify Circumferential Resection Margin (CRM), positive nodes, and lateral
spread of tumor according to the protocol of Quirke et al. [14]. | CRM was measured
according to the guidelines of Quirke et al.[37]. | Since 2002, all pathology
examinations for rectal cancer have been performed according to the guidelines of
Quirke et al. [15]. | Total mesorectal excision (TME) removes the primary tumor with
its surrounding mesorectum as an intact package, preventing residual tumor cells in
the mesorectum from developing into local recurrence.1,2 | Incomplete resection of
the lateral tumor margins is now considered the most important cause of local
recurrence [15–17]. | In a study by Quirke et al. [15], 83% of the patients with a
positive CRM had local tumor
12
Document X What everyone said about Document X
Example keywords
0 5 10 15 20 25
ana standardized routine pathology examination
tumor node metastasis staging system
preventing residual tumor cells
identify circumferential resection margin
complete mesocolic excision
central vascular ligation
total mesorectal excision
entire regional mesocolon
embryologic tissue planes
circumferential resection margin
radical oncologic resection
lateral tumor margins
standard clinical practice
positive circumferential margins
circumferential rectal margin
local tumor recurrence
pathology examinations
primary tumor
inadequate resection
clinical implementation
13
Where are we today?
230,000
fulltext articles
indexed
3.6 million
articles with a citation
8 million
citing contexts
14
Sources
…and you! (If you’re a publisher)
15
Now for the
really fun part
Our Service
PROCESSING
ForeCite breaks their text down
into sentences (not a trivial
task) and compares each
sentence against our database
17
UPLOAD
An author has written (or is
writing) their paper. They
upload it to ForeCite for
analysis
RECOMMENDATIONS
Finally, we provide a number of
different recommendation options
(with scoring) along with the
citing context for users to
compare against their text
ALGORITHM
There are many smart ways in
which texts can be compared. So
far we have only scratched the
surface. Hey, it’s early days.
Example Recommendations
Suggested
citations
Sentence with
suggested citations
What others wrote
when they cited the
suggested article,
inline for
comparison
purposes
What people said about Article X
One of the features
of ForeCite is that
you can look up an
individual article and
see all of the citing
contexts that we
have for this article
Benefits
20
Improves
quality
Saves
time
Enables
discovery
+44 777.173.4093
fore-cite.com
@fore_cite
brian@fore-cite.comContact Us
fore-cite.com

More Related Content

Similar to ForeCite APE Presentation

tranSMART Community Meeting 5-7 Nov 13 - Session 3: The TraIT user stories fo...
tranSMART Community Meeting 5-7 Nov 13 - Session 3: The TraIT user stories fo...tranSMART Community Meeting 5-7 Nov 13 - Session 3: The TraIT user stories fo...
tranSMART Community Meeting 5-7 Nov 13 - Session 3: The TraIT user stories fo...
David Peyruc
 
Advancing Innovation and Convergence in Cancer Research: US Federal Cancer Mo...
Advancing Innovation and Convergence in Cancer Research: US Federal Cancer Mo...Advancing Innovation and Convergence in Cancer Research: US Federal Cancer Mo...
Advancing Innovation and Convergence in Cancer Research: US Federal Cancer Mo...
Jerry Lee
 
인공지능은 의료를 어떻게 혁신할 것인가 (ver 2)
인공지능은 의료를 어떻게 혁신할 것인가 (ver 2)인공지능은 의료를 어떻게 혁신할 것인가 (ver 2)
인공지능은 의료를 어떻게 혁신할 것인가 (ver 2)
Yoon Sup Choi
 
Clinical Genomics and Medicine
Clinical Genomics and MedicineClinical Genomics and Medicine
Clinical Genomics and Medicine
Warren Kibbe
 
Advancing Convergence and Innovation in Cancer Research: Seminar at Universit...
Advancing Convergence and Innovation in Cancer Research: Seminar at Universit...Advancing Convergence and Innovation in Cancer Research: Seminar at Universit...
Advancing Convergence and Innovation in Cancer Research: Seminar at Universit...
Jerry Lee
 
ResQu: A Framework for Automatic Evaluation of Knowledge-Driven Automatic Sum...
ResQu: A Framework for Automatic Evaluation of Knowledge-Driven Automatic Sum...ResQu: A Framework for Automatic Evaluation of Knowledge-Driven Automatic Sum...
ResQu: A Framework for Automatic Evaluation of Knowledge-Driven Automatic Sum...
Nishita Jaykumar
 
Building Bridges Not Walls: Can We Develop Sustainable and Sharable Cost-Effe...
Building Bridges Not Walls: Can We Develop Sustainable and Sharable Cost-Effe...Building Bridges Not Walls: Can We Develop Sustainable and Sharable Cost-Effe...
Building Bridges Not Walls: Can We Develop Sustainable and Sharable Cost-Effe...
Office of Health Economics
 
Can SAR Database: An Overview on System, Role and Application
Can SAR Database: An Overview on System, Role and ApplicationCan SAR Database: An Overview on System, Role and Application
Can SAR Database: An Overview on System, Role and Application
inventionjournals
 
ICBO 2014, October 8, 2014
ICBO 2014, October 8, 2014ICBO 2014, October 8, 2014
ICBO 2014, October 8, 2014
Warren Kibbe
 
Performance Evaluation using Supervised Learning Algorithms for Breast Cancer...
Performance Evaluation using Supervised Learning Algorithms for Breast Cancer...Performance Evaluation using Supervised Learning Algorithms for Breast Cancer...
Performance Evaluation using Supervised Learning Algorithms for Breast Cancer...
IRJET Journal
 
Magnetom flash 55
Magnetom flash 55Magnetom flash 55
Magnetom flash 55
Jhon Arriaga Cordova
 
Role of Imprint Cytology in the diagnosis of breast tumors at Mulago Hospital
Role of Imprint Cytology in the diagnosis of breast tumors at Mulago HospitalRole of Imprint Cytology in the diagnosis of breast tumors at Mulago Hospital
Role of Imprint Cytology in the diagnosis of breast tumors at Mulago Hospital
MAK1stABMSC2019
 
What are the Responsibilities of a Product Manager by Google PM
What are the Responsibilities of a Product Manager by Google PMWhat are the Responsibilities of a Product Manager by Google PM
What are the Responsibilities of a Product Manager by Google PM
Product School
 
MCC 2011 - Slide 24
MCC 2011 - Slide 24MCC 2011 - Slide 24
MCC 2011 - Slide 24
European School of Oncology
 
Bioinformatics-R program의 실례
Bioinformatics-R program의 실례Bioinformatics-R program의 실례
Bioinformatics-R program의 실례
mothersafe
 
Clinical trial terminology
Clinical trial terminologyClinical trial terminology
Clinical trial terminology
Preeti Agarwal
 
Data supporting precision oncology fda wakibbe
Data supporting precision oncology fda wakibbeData supporting precision oncology fda wakibbe
Data supporting precision oncology fda wakibbe
Warren Kibbe
 
Logistic Regression Model for Predicting the Malignancy of Breast Cancer
Logistic Regression Model for Predicting the Malignancy of Breast CancerLogistic Regression Model for Predicting the Malignancy of Breast Cancer
Logistic Regression Model for Predicting the Malignancy of Breast Cancer
IRJET Journal
 
Federal Research & Development for the Florida system Sept 2014
Federal Research & Development for the Florida system Sept 2014 Federal Research & Development for the Florida system Sept 2014
Federal Research & Development for the Florida system Sept 2014
Warren Kibbe
 
International Journal of Biometrics and Bioinformatics(IJBB) Volume (3) Issue...
International Journal of Biometrics and Bioinformatics(IJBB) Volume (3) Issue...International Journal of Biometrics and Bioinformatics(IJBB) Volume (3) Issue...
International Journal of Biometrics and Bioinformatics(IJBB) Volume (3) Issue...
CSCJournals
 

Similar to ForeCite APE Presentation (20)

tranSMART Community Meeting 5-7 Nov 13 - Session 3: The TraIT user stories fo...
tranSMART Community Meeting 5-7 Nov 13 - Session 3: The TraIT user stories fo...tranSMART Community Meeting 5-7 Nov 13 - Session 3: The TraIT user stories fo...
tranSMART Community Meeting 5-7 Nov 13 - Session 3: The TraIT user stories fo...
 
Advancing Innovation and Convergence in Cancer Research: US Federal Cancer Mo...
Advancing Innovation and Convergence in Cancer Research: US Federal Cancer Mo...Advancing Innovation and Convergence in Cancer Research: US Federal Cancer Mo...
Advancing Innovation and Convergence in Cancer Research: US Federal Cancer Mo...
 
인공지능은 의료를 어떻게 혁신할 것인가 (ver 2)
인공지능은 의료를 어떻게 혁신할 것인가 (ver 2)인공지능은 의료를 어떻게 혁신할 것인가 (ver 2)
인공지능은 의료를 어떻게 혁신할 것인가 (ver 2)
 
Clinical Genomics and Medicine
Clinical Genomics and MedicineClinical Genomics and Medicine
Clinical Genomics and Medicine
 
Advancing Convergence and Innovation in Cancer Research: Seminar at Universit...
Advancing Convergence and Innovation in Cancer Research: Seminar at Universit...Advancing Convergence and Innovation in Cancer Research: Seminar at Universit...
Advancing Convergence and Innovation in Cancer Research: Seminar at Universit...
 
ResQu: A Framework for Automatic Evaluation of Knowledge-Driven Automatic Sum...
ResQu: A Framework for Automatic Evaluation of Knowledge-Driven Automatic Sum...ResQu: A Framework for Automatic Evaluation of Knowledge-Driven Automatic Sum...
ResQu: A Framework for Automatic Evaluation of Knowledge-Driven Automatic Sum...
 
Building Bridges Not Walls: Can We Develop Sustainable and Sharable Cost-Effe...
Building Bridges Not Walls: Can We Develop Sustainable and Sharable Cost-Effe...Building Bridges Not Walls: Can We Develop Sustainable and Sharable Cost-Effe...
Building Bridges Not Walls: Can We Develop Sustainable and Sharable Cost-Effe...
 
Can SAR Database: An Overview on System, Role and Application
Can SAR Database: An Overview on System, Role and ApplicationCan SAR Database: An Overview on System, Role and Application
Can SAR Database: An Overview on System, Role and Application
 
ICBO 2014, October 8, 2014
ICBO 2014, October 8, 2014ICBO 2014, October 8, 2014
ICBO 2014, October 8, 2014
 
Performance Evaluation using Supervised Learning Algorithms for Breast Cancer...
Performance Evaluation using Supervised Learning Algorithms for Breast Cancer...Performance Evaluation using Supervised Learning Algorithms for Breast Cancer...
Performance Evaluation using Supervised Learning Algorithms for Breast Cancer...
 
Magnetom flash 55
Magnetom flash 55Magnetom flash 55
Magnetom flash 55
 
Role of Imprint Cytology in the diagnosis of breast tumors at Mulago Hospital
Role of Imprint Cytology in the diagnosis of breast tumors at Mulago HospitalRole of Imprint Cytology in the diagnosis of breast tumors at Mulago Hospital
Role of Imprint Cytology in the diagnosis of breast tumors at Mulago Hospital
 
What are the Responsibilities of a Product Manager by Google PM
What are the Responsibilities of a Product Manager by Google PMWhat are the Responsibilities of a Product Manager by Google PM
What are the Responsibilities of a Product Manager by Google PM
 
MCC 2011 - Slide 24
MCC 2011 - Slide 24MCC 2011 - Slide 24
MCC 2011 - Slide 24
 
Bioinformatics-R program의 실례
Bioinformatics-R program의 실례Bioinformatics-R program의 실례
Bioinformatics-R program의 실례
 
Clinical trial terminology
Clinical trial terminologyClinical trial terminology
Clinical trial terminology
 
Data supporting precision oncology fda wakibbe
Data supporting precision oncology fda wakibbeData supporting precision oncology fda wakibbe
Data supporting precision oncology fda wakibbe
 
Logistic Regression Model for Predicting the Malignancy of Breast Cancer
Logistic Regression Model for Predicting the Malignancy of Breast CancerLogistic Regression Model for Predicting the Malignancy of Breast Cancer
Logistic Regression Model for Predicting the Malignancy of Breast Cancer
 
Federal Research & Development for the Florida system Sept 2014
Federal Research & Development for the Florida system Sept 2014 Federal Research & Development for the Florida system Sept 2014
Federal Research & Development for the Florida system Sept 2014
 
International Journal of Biometrics and Bioinformatics(IJBB) Volume (3) Issue...
International Journal of Biometrics and Bioinformatics(IJBB) Volume (3) Issue...International Journal of Biometrics and Bioinformatics(IJBB) Volume (3) Issue...
International Journal of Biometrics and Bioinformatics(IJBB) Volume (3) Issue...
 

ForeCite APE Presentation

  • 1. Crowdsourcing citation suggestions, without needing to ask the crowd Brian Bishop, Co-founder @mochasteak
  • 2. Brian Bishop Business / Product stuff Formerly VP of Platform Development at Springer, MBA, 14-year scientific publishing veteran Pawel Kowalski Tech stuff Tech Principal at iterativ GmbH, full-stack dev, MSc in Computer Science, the whole deal Our Team 02
  • 3. Timeline* 03 Prototype goes live November 2014 Today January 2015 Company registered August 2014 *timeline not to scale
  • 4. We asked 1,044 authors “What is the most difficult or frustrating part of writing a paper?”
  • 5. and this is what they said… 05
  • 7. How painful is this problem? 07 836 Authors surveyed “How long did it take you to gather all the references / find sources for your paper?” 25% “More than a week” 35% “More than a month”
  • 8. That’s a lot of time spent not doing research
  • 9. The average scientific paper has 35 citations 09
  • 10. This is the citation context Each of those citations is embedded in a sentence “Both experimental and atomistic simulation results show that when the dimensions of the structures become small then the ‘size effect’ has significant role in the mechanical properties (Ruud et al. 1994).” 10
  • 11. Now go the other way Paper A Paper B Paper C 11 We can extract out all the citing contexts from all the papers, and group them according to the document being cited.
  • 12. A database of contexts A clinical implementation was described first for total mesorectal excision (TME) in the treatment of rectal cancer [4, 5]. | The circumferential rectal margin (CRM) was assessed according to the method of Quirke et al. [13], and a margin of < 1 mm was considered CRM-positive. | Complete mesocolic excision (CME) with central vascular ligation (CVL), according to the sound principles of total mesorectal excision (TME) [6, 7] for rectal cancer | A standardized routine pathology examination was performed using the protocol of Quirke et al. [36]. | The surgical specimens were handled according to standard clinical practice as advocated by Quirke et al. [20] and were pathologically examined in accordance with the Tumor Node Metastasis staging system. | For rectal cancer, the specimen was processed using the slicing technique as described by Quirke et al. 33 | Pathologists were trained to examine the specimens according to the protocol of Quirke et al. regarding the circumferential resection margin (CRM), lymph nodes, and dissection plane.12 | Quirke showed a linear correlation between the development of a local recurrence and an inadequate resection with positive circumferential margins [14, 22]. | The pathologists were trained to identify Circumferential Resection Margin (CRM), positive nodes, and lateral spread of tumor according to the protocol of Quirke et al. [14]. | CRM was measured according to the guidelines of Quirke et al.[37]. | Since 2002, all pathology examinations for rectal cancer have been performed according to the guidelines of Quirke et al. [15]. | Total mesorectal excision (TME) removes the primary tumor with its surrounding mesorectum as an intact package, preventing residual tumor cells in the mesorectum from developing into local recurrence.1,2 | Incomplete resection of the lateral tumor margins is now considered the most important cause of local recurrence [15–17]. | In a study by Quirke et al. [15], 83% of the patients with a positive CRM had local tumor 12 Document X What everyone said about Document X
  • 13. Example keywords 0 5 10 15 20 25 ana standardized routine pathology examination tumor node metastasis staging system preventing residual tumor cells identify circumferential resection margin complete mesocolic excision central vascular ligation total mesorectal excision entire regional mesocolon embryologic tissue planes circumferential resection margin radical oncologic resection lateral tumor margins standard clinical practice positive circumferential margins circumferential rectal margin local tumor recurrence pathology examinations primary tumor inadequate resection clinical implementation 13
  • 14. Where are we today? 230,000 fulltext articles indexed 3.6 million articles with a citation 8 million citing contexts 14
  • 15. Sources …and you! (If you’re a publisher) 15
  • 16. Now for the really fun part
  • 17. Our Service PROCESSING ForeCite breaks their text down into sentences (not a trivial task) and compares each sentence against our database 17 UPLOAD An author has written (or is writing) their paper. They upload it to ForeCite for analysis RECOMMENDATIONS Finally, we provide a number of different recommendation options (with scoring) along with the citing context for users to compare against their text ALGORITHM There are many smart ways in which texts can be compared. So far we have only scratched the surface. Hey, it’s early days.
  • 18. Example Recommendations Suggested citations Sentence with suggested citations What others wrote when they cited the suggested article, inline for comparison purposes
  • 19. What people said about Article X One of the features of ForeCite is that you can look up an individual article and see all of the citing contexts that we have for this article