This slide begins a short series of slides that starts with a broad definition of innovation, then establishes a framework for thinking about innovation as a temporal sequence of connected events. Each slide in the series adds a layer of information that transforms the abstract concept of innovation into a data-driven model.
Inputs and outputs are the basis for how we measure innovation now and are a familiar reference point. Associating individual activities with documents or artifacts sets up the linkages to data sources.
From the data sources we can identify people and organizations involved, which sets up the creation of network models. This slide also introduces the notion of intermediate outcomes from the innovation perspective – things like patents and publications – that may be final outcomes from the perspective of the people working on them. Innovation is not everyone’s goal.
This introduces the idea of an activity network – the building block of innovation networks and ecosystems.
Finally we layer on the data and linkages to show how the activities connect to each other to form temporal sequences.
This slide parallels the previous slide but from the activity network to innovation network to innovation ecosystem perspective.
Where innovation metrics come from now, and more importantly, where they do not. Total elapsed time to the end of this slide is 4:00.
This slide shows the EventFlow screen with the overall data model we have constructed so far, and identifies the data sources we use.
Introduces the basics of EventFlow
This slide shows the 26 drugs that we can trace – at least partially – from research to final approval. It also frames innovation activities from the perspective of products and lists some of the important questions we can ask and important temporal metrics we can derive from analysis with EventFlow (and CoCo).
The distribution and statistics for our 26 drug sample. Measures from first patent application date to final approval date.
The distribution and statistics for all drugs we have data on. Measures from first patent application date to final approval date.
A similar look at medical devices using clinical trials and final approval dates. Introduces the measurement of gaps.
Continues the previous slide, introducing the concepts of overlaps and of visualizing patterns in the overview panel of EventFlow.
There are two general sequence patterns in the data. The images on the previous slide showed sequence patterns for which clinical trials were completed, followed by a lag, followed by FDA approval. The other pattern, shown here on the right has FDA approvals overlapping the span of clinical trials.
We prepared network models fir the Illinois Science & Technology Roadmap. One of those was for the Battery Cluster. From an academic perspective, one of the things we noticed was that the activities tended to be organized into two main components – research and industry – with a small group of activities that seemed to span these two components. We’ll call this third component the bridge – show here in collapsed form as simply a gray band.
Then we recognized similarities between the network graph and our graphic of the so-called “valley of death’. We realized that we might actually be looking at a network representation of that valley of death. If that was the case, what would we expect to find in the bridge? 1) corporate sponsored research; 2) SBIR / STTR’s; 3) intermediaries like accelerators and incubators; and 4) public-private partnerships like federal labs. When we opened up the bridge for a closer look, that is exactly what we found.
This allows us to frame a new working hypothesis about innovation ecosystems and the differences in innovation outcomes between different regions. And the new temporal metrics we are developing with NSF should allow us to test that hypothesis soon. So for us the value of University Centers is this diversity of partnerships around the tasks of developing new methods and metrics; developing new tools and solving practical economic development problems; and synthesizing those activities into new knowledge and understanding about the nature of innovation and its impact on economic growth.
As a practical matter innovation ecosystems and regional innovation clusters are the same thing. Here we show one ecosystem / cluster for regenerative medicine in Howard County Maryland, comprised of those aggregated activity networks. This small, emerging cluster does not show up in traditional cluster analysis because 1) the activity is too recent and 2) the activity is not organized according to existing NAICS codes. Thus this analysis was valuable to Howard County Economic Development. Each group includes people and organizations that are connected based on what they are working on together. The graph is organized with the largest, most connected group in the upper left and the smallest, least connected group in the lower right. It turns out that this layout is useful in helping to organize and target different types of economic development strategies to specific companies and groups so that the overall cluster strategy appropriately targets limited resources for effective economic development. The interactive network tool allows economic developers to zoom in and explore different parts of the cluster in detail. Users can also click on certain nodes to get more detailed information. This interactive network model was build using NodeXL – developed in part by the same computer scientists who created EventFlow.
14:30 – 15:30 But here is how the visualization organizes the information.
15:30 – 17:30
Nih dempwolf 20160408-v4
Modeling Drug and Medical Device Innovation
as Temporal Sequences using EventFlow
NIH and the Science of Science and Innovation Policy:
A Joint NIH-NSF Workshop
April 7 – 8
C. Scott Dempwolf, PhD
Assistant Research Professor
University of Maryland – Morgan State
Joint Center for Economic Development
Ben Shneiderman, PhD
Distinguished University Professor
University of Maryland Institute for
Advanced Computer Science (UMIACS)
(and a few networks)
Pennsylvania Innovation Networks 1990 – 2007
Emergence of Philadelphia Biopharma cluster and Pittsburgh Nuclear Cluster
Modeled with Pajek & KING
ME: “It’s cool, but…
How do I make it useful?”
“You must use NodeXL”
you are my Jedi Master”
A process of transforming knowledge and scientific
research into a new product in the marketplace.
Think of that process
as a sequence of
Research Invention Proof Commercialization Product
Each activity has inputs,
documents and artifacts
Each activity involves people
and organizations producing
The people and organizations
from each activity
create an activity network
Activities become sequences through
shared people and organizations,
citations, and other linkages
Innovation networks with embedded knowledge & resources along
with intermediaries comprise Innovation Ecosystems.
Activity networks combine to
form innovation networks.
The Regenerative Medicine cluster (ecosystem) in Howard County, MD
Combining two activities:
NSF# 1551041 and today’s presentation
NSF# 1551041 activity network
Some are based
based on inputs
based on outputs
Modeling Innovation Sequences with EventFlow
We use newly developed EventFlow
software to model innovation in drugs and
medical devices from multiple datasets:
• CTTI AACT Database
• FDA Orange Book (drugs)
• Pre-Market Approvals (PMA) (med devices)
• SBIR/STTR (pending)
• CrunchBase (pending)
• NSF (pending)
Supporting and core data sources
• NIH RePORTER
A Quick Tour of EventFlow
Each product (drug or medical device)
is a record in EventFlow
• Clinical Trials (commercialization activity)
• FDA Approval (proxy for product launch)
• Patents (invention)
Overview (Aggregation) Individual Timelines
Product-Based Innovation Metrics
How long does innovation take?
How many activities are involved?
In what sequence?
How long does each take?
Are there gaps?
Is the sequence pattern common
How long does innovation take? (drugs)
How long does innovation take? (drugs)
(884 drugs in the
FDA Orange Book)
How long does innovation take? (med devices)
Start of clinical trials
(1,225 medical devices)
How long does innovation take? (med devices)
Illinois Battery Cluster 2010 – 2014
Modeled with NodeXL
Broader applications of temporal metrics:
the Illinois Battery Cluster
Research Publication Invention Proof-of-Concept Commercialization Product
The Innovation Ecosystem and the Valley of Death
A network representation
of the valley of death
Emerging Theory & Research
Bridge What’s in
• Working Hypothesis
• Regions with denser, more connected
bridging components will be
characterized by faster innovation
sequences and more innovation
sequences leading to new products.
Measured using new
Stem cell products group
• Commercialization support
• Attract complementary
Delivery devices groups,
• Facilitate collaboration
• Niche market development
• Attract complementary firms
Regenerative Medicine &
• Develop ‘Keystones’
• Promote local sourcing
• Industry partnerships
• FDI / Business expansion
• Attraction - supply chain
• University partnerships
University groups (JHU, UMCP, UMB)
• Leads for licensing (green ties)
• Key labs (dense subgroups)
• Opportunities for faculty spin outs
• Accelerate student startups
• Corporate Partnerships
At the Cluster Level
Regenerative Medicine Cluster – Howard County, MD
Innovation-Led Economic Development
Drill-down to Company Profiles
• Click to follow link
Nascent / emerging
Howard County, Maryland - Full Innovation Network
Universities (JHU, UMCP, UMB,
• Follow-up leads for licensing or other
engagements (green ties)
• Identify key labs (dense subgroups)
and evaluate for expansion /
• Identify opportunities for faculty spin
• Identify / accelerate potential
student startups that can be seeded
in this cluster
• Build long-term sponsored research
relationships with keystone
Main Innovation Clusters
• Regenerative Medicine
• Telecom / networks / cyber
• Defense / Security / SBIR
• Research & Development
• Commercialization, acceleration,
entrepreneurial support for early
stage companies located in the
• Assistance with market Connections
to capital & cluster keystones
Business Attraction Opportunities
• Focus on early stage companies with
innovation cluster growth potential;
companies are located outside of
the county but have a HoCo
• Develop relationships and help them
plan for move to HoCo for next
• Connections to capital
• Identify & cultivate keystones in each
• Identify & cultivate capital networks
around each innovation cluster
Business Expansion & FDI
• Focus BRE on growth stage &
mature companies in innovation
• Develop keystones in the process.
• Engage MD DOC in developing FDI.
• Engage foreign-owned companies in
innovation clusters to expand their
presence in the cluster through FDI.
• Develop industry partnerships
(EARN) around innovation clusters
• Work with universities & community
colleges on talent pipeline
The ‘group-in-a-box’ layout organizes
groups from largest to smallest. This
also corresponds to a ‘strategy
gradient’ for economic development.
Research & Tech
& Workforce strategies
A few Data Issues & Needs
• Data cleaning & disambiguation
• Data matching across datasets
• RePORTER, Clinical Trials, FDA, SBIR
• Matching on full project numbers (not core)
• SBIR – More complete dates; Access to bibliographies for citation
• FDA, Clinical Trials – Basic information at the front-end
• FDA – ability to roll up drug families i.e. Adderall 10mg, 15mg, 20mg…
April 13, Wednesday 10am at NIH
Porter Building 35A, Room 610, NIH Main Campus, Bethesda, MD
Interactive Visual Discovery in Event Analytics: Electronic Health Records
May 26, Thursday at University of Maryland Human-Computer Interaction Lab
Implications for Universities: visualizing labs and research partnerships
Identify key labs (dense
subgroups) and evaluate for
expansion / enhancement
Identify opportunities for
faculty spin outs
Identify / accelerate
potential student startups
that can be seeded in
Link to Lab and researcher
pages (click to follow)
University of Maryland, College Park
Research labs, research partnerships,
and individual researchers