Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Open Proposals:
A Pre-Competitive Interactive
Space for Research Community
Oksana Gologorskaya, M.Sc.
Product Manager, Vir...
Learning Objectives
“Crowdsourcing” ideas and proposal
development at biomedical institution
Insights from 5 years of pr...
Harvesting Ideas in Academia
Request for Proposals
RFP Timeline
Imagine a friendly community…
Add “social” phase to the process
What is different?
Proposal Ideas are shared with community
before they are submitted
Everyone knows what everyone else ...
Open Proposal Process
Share Online
Discuss & Improve
Submit for Review
See Everyone’s Results
accelerate.ucsf.edu
First Open Forum
 18 ideas
 41 comments
accelerate.ucsf.edu
First Open Forum: Implementation
 Section on existing Drupal website
 Simple features:
 Post ideas
...
2009 UCSF CTSI Grant Renewal RFPaccelerate.ucsf.edu
 High stakes: ideas for
$112 M grant
 53 ideas
 47 comments
 We go...
Practical Insight 1: Starting Conditions
Collaborative audience
Non-$ motivations to participate
Support from the top
UCSF CTSI Pilot Awards 2012
accelerate.ucsf.edu
 28 ideas
 292 comments
Participation Increased. Why?
18 27 53
19 28
41 39
47
6
290
2009 2009 2009 2010 2012
Open Forums Participation 2009-2012
P...
Behaviors Observed:
Proposals improved
Connections made between
participants
Proposals withdrawn
Practical Insight 2: Process Matters
18 27 53
19 28
41 39
47
6
290
1 2 3 4 5
Open Forums Participation 2009-2012
Proposals...
First Non-CTSI Challenge
2012 UCSF IT Innovation Contest
Target Audience: All Staff & Faculty
Competing against other so...
Solutions Considered
Email
Salesforce Chatter
Open Proposals
UCSF Open Proposals Tool
Custom Branding for
Opportunity Sponsor
UCSF Open Proposals Tool
Standalone, easy to use space
for discussions
UCSF Open Proposals Tool
Transparent & efficient process
Open Proposals Provided:
Transparent & efficient process
Standalone space for discussions
Easy to use
Custom branding ...
IT Contest 2012 - Results
48 proposals, 5 awarded
73 people posted 216 comments
Withdrawn/merged proposals: 1
517 uniq...
A Hard Learning
Department of Medicine
Funding Patient Cohorts 2012
Funding to allocate: $200,000
Target audience: DOM F...
DOM Funding Patient Cohorts
2012 - Results
14 proposals, 2 awarded
30 people posted 66 comments
Withdrawn proposals: 1
...
Surprising Feedback
Opportunity Sponsors: “We won’t
use it again”
End Users:
– Overall experience: 2-dissatisfied / 6 –
...
Practical Insight 3: Supportive Sponsor,
Engaged Community
Make sure the audience understands the
process (no, it is not a...
Other types of opportunities that
benefited from OP
Building Community to Revamp
Medical School’s curriculum
Department ...
UCSF Open Proposals Tool
open-proposals.ucsf.edu
 Standalone Drupal Website
 Integrated with UCSF Single
Sign On
 Optio...
UCSF Open Proposals UI
open-proposals.ucsf.edu
1
3
2
5
4
6
7
8
9
10
11
UI: Forum Space
1. Sponsor branding
2. Opportunity
title/subtitle
3. Intro text – opp.
description...
1
3
2
4
UI: Proposal Page
1.Proposal title
2.Proposal view tabs
3.Proposal info
4.Proposal body
1
3
2
4
UI: Adding Proposal
1.Instructions /
format requirements
2.Title (required)
3.Taxonomy fields
4.Primary author – i...
5
7
6
8
UI: Adding Proposal
5. Proposal body
6. Multi-file attachments
7. Single file attachment
8. Post (submit) button
9...
Automation: Updates and Subscriptions
Proposal authors and
commenters are
automatically subscribed
to the updates on thei...
Key Ingredients for Success
Process
Tool
Engaged Community
Supportive Sponsor
Challenges
How to engage community?
– Intrinsic vs. extrinsic motivation for community
reviewers
Community balance
Narro...
Benefits of Open Proposals
Open, transparent process
Collaborative space
Efficiency
Better teams
Better proposals
Less Obvious Benefits
Reviewers get not only proposals
but also community’s reaction that
may inform them as well
Every ...
Winning proposals draw more attention
0 5 10 15 20 25
IT Innovations Contest
OpenSocial Gadget Contest
DOM Patient Cohorts...
Avg. # Comments, All Forums
Winning vs. Non Winning Proposals
0
1
2
3
4
5
6
Avg. comments per proposal
Winning
Non Winning
UCSF Open Proposals Today
 23 forums Since launch in 2012
 700+ proposals
 18,000+ visitors
 $ 4,000,000+ funding dist...
What’s Next?
Research
Administration,
Infrastructure,
Strategic
Planning
Collaborative
Research
Opportunities
Thank You
UCSF Open Proposals is managed by the
UCSF Clinical and Translational Science
Institute (CTSI), part of the Cli...
Special Thanks
Mini Kahlon and Clay Johnston, for
introducing the idea and making it come to
life
To Brad Bulger, for bu...
Special Thanks
To Rachael Sak, for figuring out tricky details
of the process and making it a success when
most people we...
It is literally true that you can
succeed best and quickest by
helping others to succeed.
Napoleon Hill, one of the great
...
Publication on Open Proposals
http://onlinelibrary.wiley.com/doi/10.1111/
cts.12147/full
ctsi.ucsf.edu * open-proposals.ucsf.edu
GitHub Project (ask us for access):
https://github.com/CTSIatUCSF/dreamcatcher
...
Media Credits
Noel Hidalgo, Hand made "like us on
Facebook" sign. , Flickr
Kfergos, Abstract Lights , Flickr
Sasha Golo...
Open Proposals: Pre-competitive Space for Research Community
Open Proposals: Pre-competitive Space for Research Community
Upcoming SlideShare
Loading in …5
×

Open Proposals: Pre-competitive Space for Research Community

1,022 views

Published on

Oksana Gologorskaya presents at AMIA & CTSA Tooshop 2014: “Crowdsourcing” ideas and proposal development at biomedical institution: insights from 5 years of practicing; Open Proposals collaboration tool

Published in: Science, Education, Technology
  • Be the first to comment

  • Be the first to like this

Open Proposals: Pre-competitive Space for Research Community

  1. 1. Open Proposals: A Pre-Competitive Interactive Space for Research Community Oksana Gologorskaya, M.Sc. Product Manager, Virtual Home Program, Clinical and Translational Science Institute, University of California, San Francisco
  2. 2. Learning Objectives “Crowdsourcing” ideas and proposal development at biomedical institution Insights from 5 years of practicing Open Proposals Tool Demo Invitation to collaborate
  3. 3. Harvesting Ideas in Academia Request for Proposals
  4. 4. RFP Timeline
  5. 5. Imagine a friendly community…
  6. 6. Add “social” phase to the process
  7. 7. What is different? Proposal Ideas are shared with community before they are submitted Everyone knows what everyone else is proposing Anyone in community can comment on the ideas Valuable contributors can join proposal teams Proposal teams can merge
  8. 8. Open Proposal Process Share Online Discuss & Improve Submit for Review See Everyone’s Results
  9. 9. accelerate.ucsf.edu First Open Forum  18 ideas  41 comments
  10. 10. accelerate.ucsf.edu First Open Forum: Implementation  Section on existing Drupal website  Simple features:  Post ideas  Comment  Vote  Anonymous participation ok
  11. 11. 2009 UCSF CTSI Grant Renewal RFPaccelerate.ucsf.edu  High stakes: ideas for $112 M grant  53 ideas  47 comments  We got the grant!
  12. 12. Practical Insight 1: Starting Conditions Collaborative audience Non-$ motivations to participate Support from the top
  13. 13. UCSF CTSI Pilot Awards 2012 accelerate.ucsf.edu  28 ideas  292 comments
  14. 14. Participation Increased. Why? 18 27 53 19 28 41 39 47 6 290 2009 2009 2009 2010 2012 Open Forums Participation 2009-2012 Proposals Comments
  15. 15. Behaviors Observed: Proposals improved Connections made between participants Proposals withdrawn
  16. 16. Practical Insight 2: Process Matters 18 27 53 19 28 41 39 47 6 290 1 2 3 4 5 Open Forums Participation 2009-2012 Proposals Comments Design process to encourage participation Manage the audience
  17. 17. First Non-CTSI Challenge 2012 UCSF IT Innovation Contest Target Audience: All Staff & Faculty Competing against other solutions
  18. 18. Solutions Considered Email Salesforce Chatter Open Proposals
  19. 19. UCSF Open Proposals Tool Custom Branding for Opportunity Sponsor
  20. 20. UCSF Open Proposals Tool Standalone, easy to use space for discussions
  21. 21. UCSF Open Proposals Tool Transparent & efficient process
  22. 22. Open Proposals Provided: Transparent & efficient process Standalone space for discussions Easy to use Custom branding for sponsor
  23. 23. IT Contest 2012 - Results 48 proposals, 5 awarded 73 people posted 216 comments Withdrawn/merged proposals: 1 517 unique visitors in active phases
  24. 24. A Hard Learning Department of Medicine Funding Patient Cohorts 2012 Funding to allocate: $200,000 Target audience: DOM Faculty (~620) Collaborative opportunity: reusing patient cohorts
  25. 25. DOM Funding Patient Cohorts 2012 - Results 14 proposals, 2 awarded 30 people posted 66 comments Withdrawn proposals: 1 431 unique visitors in active phases
  26. 26. Surprising Feedback Opportunity Sponsors: “We won’t use it again” End Users: – Overall experience: 2-dissatisfied / 6 – neutral / 7 - satisfied – Proposal impact: 2- improved / 7 – no change / 2- worsened – Team impact: 2- added team members
  27. 27. Practical Insight 3: Supportive Sponsor, Engaged Community Make sure the audience understands the process (no, it is not a popularity contest) Early ideas have more room for feedback than fully baked proposals The audience will not engage without Sponsor’s support
  28. 28. Other types of opportunities that benefited from OP Building Community to Revamp Medical School’s curriculum Department Strategic Planning Administrative Process Improvements
  29. 29. UCSF Open Proposals Tool open-proposals.ucsf.edu  Standalone Drupal Website  Integrated with UCSF Single Sign On  Optional IP restricted access  Linked with UCSF Profiles
  30. 30. UCSF Open Proposals UI open-proposals.ucsf.edu
  31. 31. 1 3 2 5 4 6 7 8 9 10 11 UI: Forum Space 1. Sponsor branding 2. Opportunity title/subtitle 3. Intro text – opp. description 4. Opportunity timeline 5. Proposals main feed 6. Forum menu 7. My Proposals shortcut 8. Topic filter 9. Activity feeds 10.Footer menu 11.Search
  32. 32. 1 3 2 4 UI: Proposal Page 1.Proposal title 2.Proposal view tabs 3.Proposal info 4.Proposal body
  33. 33. 1 3 2 4 UI: Adding Proposal 1.Instructions / format requirements 2.Title (required) 3.Taxonomy fields 4.Primary author – if proposal is submitted for someone else
  34. 34. 5 7 6 8 UI: Adding Proposal 5. Proposal body 6. Multi-file attachments 7. Single file attachment 8. Post (submit) button 9. Preview button 9
  35. 35. Automation: Updates and Subscriptions Proposal authors and commenters are automatically subscribed to the updates on their proposal Forum visitors can “Follow” the forum or its selected proposals
  36. 36. Key Ingredients for Success Process Tool Engaged Community Supportive Sponsor
  37. 37. Challenges How to engage community? – Intrinsic vs. extrinsic motivation for community reviewers Community balance Narrow enough to remain relevant and collaborative Wide enough to provide valuable input
  38. 38. Benefits of Open Proposals Open, transparent process Collaborative space Efficiency Better teams Better proposals
  39. 39. Less Obvious Benefits Reviewers get not only proposals but also community’s reaction that may inform them as well Every idea is captured and stays there (usually online) forever… Active participants build relationships and get noticed
  40. 40. Winning proposals draw more attention 0 5 10 15 20 25 IT Innovations Contest OpenSocial Gadget Contest DOM Patient Cohorts RFA Funding Shared Equipment RFA CHV Caring Wisely Initiative 2013 CTSI Annual Pilot Awards Funding Shared Equipment RFA… UCSF Coursera Course CFP CTSI 2016 NIH Renewal Launchpad 2013 IT Innovations Contest DOM 2013 Strategic Planning CHV Caring Wisely 2 Initiative 2014 Innovations Funding for… SOM Dean's Office Bureaucracy… Average comments for winning proposal Average comments for other proposals
  41. 41. Avg. # Comments, All Forums Winning vs. Non Winning Proposals 0 1 2 3 4 5 6 Avg. comments per proposal Winning Non Winning
  42. 42. UCSF Open Proposals Today  23 forums Since launch in 2012  700+ proposals  18,000+ visitors  $ 4,000,000+ funding distributed  Shared with 2 institutions
  43. 43. What’s Next? Research Administration, Infrastructure, Strategic Planning Collaborative Research Opportunities
  44. 44. Thank You UCSF Open Proposals is managed by the UCSF Clinical and Translational Science Institute (CTSI), part of the Clinical and Translational Science Award program funded by the National Center for Advancing Translational Sciences (Grant Number UL1 TR000004) at the National Institutes of Health (NIH).
  45. 45. Special Thanks Mini Kahlon and Clay Johnston, for introducing the idea and making it come to life To Brad Bulger, for building the tool To Cynthia Piontkowski, for creating the tool’s people friendly side To Leslie Yuan, for support and inspiration, and winning decisions
  46. 46. Special Thanks To Rachael Sak, for figuring out tricky details of the process and making it a success when most people were skeptical To Lisa Schoonerman, for help with winning hesitant new customers and for invaluable practical insight To Anirvan Chatterjee, for the most creative ideas for the tool and its introduction to the bigger world, and for huge help with this presentation
  47. 47. It is literally true that you can succeed best and quickest by helping others to succeed. Napoleon Hill, one of the great writers on success
  48. 48. Publication on Open Proposals http://onlinelibrary.wiley.com/doi/10.1111/ cts.12147/full
  49. 49. ctsi.ucsf.edu * open-proposals.ucsf.edu GitHub Project (ask us for access): https://github.com/CTSIatUCSF/dreamcatcher open.proposal@ucsf.edu oksana.gologorskaya@ucsf.edu Contact Us
  50. 50. Media Credits Noel Hidalgo, Hand made "like us on Facebook" sign. , Flickr Kfergos, Abstract Lights , Flickr Sasha Gologorskaya, Friends, Everyday Photos Ren Kuo, Dreamcatcher, Flickr

×