SlideShare a Scribd company logo
1 of 146
The Librarian
103
Interviews
Week 1:
Automated
Documentation
Week 10:
Chatbot Database
Integration
Will Papper
Product
B.S. SymSys ‘18
Sean Konz
Data Science
B.S. EE & CS ‘20
Holden Gibbons
Strategy/BD
B.S. MS&E ‘19
Anna Zeng
Software
Engineering
B.S. CS ‘18
Week 1:
Problem: Analyst
knowledge retention is
limited due to lacking
documentation tools
Solution: Develop a
software tool to automate
analyst workflow
documentation efforts
Week 10:
Problem: Useful tools are
not being used by analysts
because they are not a part
of their workflow
Solution: Develop an AI-
enhanced chatbot interface
to integrate various
software tools and
databases
0 1 2 3 4 5 6 7 8 9 10
Boiling the
ocean?
Pivotin
g
?
Develop &
Deploy
MVP:
- Recommended
readings
- Recommended
contacts
MVP:
- Profile based
search interface
- Contact database MVP:
- Chatbot integration
interface
Product Mission Fit!
Intel
Agencies
Corporate
Military
Who we interviewed
103
Challenges
Initial Public Outreach
Confidentiality Concerns
Weeks 1-3 Weeks 4-6 Weeks 7-8 Weeks 9-10
Internal Culture
The Librarian
MISSION MODEL CANVAS
Week 1
Mission Budget / Costs
Fixed:
Software development
Integration with government services
Variable:
Onboarding for analysts
System / Database integration
Mission Achievement
Reduce administrative overhead for all analysts regarding
role transitions
Bring new analysts up to speed on shared knowledge
faster
Provide a simple, intuitive experience for easy adoption
Key Partners
Administrators of ispace, rspace,
etc.
Administrators of Intellipedia
Administrators of shared drives
Analysis leadership (i.e. DA4)
across agencies, to encourage
documentation in performance
evaluations
Branch managers in the DA (i.e.
K, our sponsor)
ODNI, to implement our tool
across agencies
Enterprise HR departments, can
help onboard new employees and
aid with exit processes
Key Activities
Research processes to allow for
background documentation
Create & test mocks
Integrate multiple knowledge
sources
Develop a working prototype
Value Proposition
Reduce onboarding time for
new analysts
Improve convenience of
documentation, so analysts are
able to spend less time on
documentation
Consolidate multiple redundant
systems
Speed up analyst transition
time between projects, reducing
administrative burden from HR in
the process
Ensure a clean knowledge
transfer that is less dependent on
contractors
Reduce constant need to
consult former analysts, allowing
for complete focus on new
assignment
Buy-In / Support
Sorely needed by our sponsor;
Analysts can take a year on a 3-5
year project to get up to speed
Sponsor has tried multiple
software solutions without
success
Sponsor’s office has large degree
of autonomy to implement
prospective solutions
Beneficiaries
Primary:
New analysts can get up to
speed on tribal knowledge faster
than before
Transitioning analysts can focus
more on their work by not needing
to allot six months to transition
materials
Current analysts working on
projects enjoy a reduced
documentation load
Secondary:
Leadership can see an overall
picture of key work produced
HR is less burdened by exit
procedures
Non-sponsor Enterprise:
Applicability and potential traction
is being found outside of the
scope of our sponsor
Key Resources
Agency personnel (current and
retired) for documentation
knowledge & in-the-field
experiences
Non-government contacts
experiencing information retention
problems
R&D team contacts
Wide array of industries/job titles
available to contact.
Deployment
Pilot software with one team
within one agency
Expand to similar agencies and
share information based on
clearance level
Integrate with additional
agencies and non-government
organizations
Value Proposition
● Reduce onboarding time
● Improve convenience
● Consolidate
● Speed up analyst transition time
● Ensure a clean knowledge transfer
● Reduce constant need to consult
The Librarian
MISSION MODEL CANVAS
Week 1
Mission Budget / Costs
Fixed:
Software development
Integration with government services
Variable:
Onboarding for analysts
System / Database integration
Mission Achievement
Reduce administrative overhead for all analysts regarding
role transitions
Bring new analysts up to speed on shared knowledge
faster
Provide a simple, intuitive experience for easy adoption
Key Partners
Administrators of ispace, rspace,
etc.
Administrators of Intellipedia
Administrators of shared drives
Analysis leadership (i.e. DA4)
across agencies, to encourage
documentation in performance
evaluations
Branch managers in the DA (i.e.
K, our sponsor)
ODNI, to implement our tool
across agencies
Enterprise HR departments, can
help onboard new employees and
aid with exit processes
Key Activities
Research processes to allow for
background documentation
Create & test mocks
Integrate multiple knowledge
sources
Develop a working prototype
Value Proposition
Reduce onboarding time for
new analysts
Improve convenience of
documentation, so analysts are
able to spend less time on
documentation
Consolidate multiple redundant
systems
Speed up analyst transition
time between projects, reducing
administrative burden from HR in
the process
Ensure a clean knowledge
transfer that is less dependent on
contractors
Reduce constant need to
consult former analysts, allowing
for complete focus on new
assignment
Buy-In / Support
Sorely needed by our sponsor;
Analysts can take a year on a 3-5
year project to get up to speed
Sponsor has tried multiple
software solutions without
success
Sponsor’s office has large degree
of autonomy to implement
prospective solutions
Beneficiaries
Primary:
New analysts can get up to
speed on tribal knowledge faster
than before
Transitioning analysts can focus
more on their work by not needing
to allot six months to transition
materials
Current analysts working on
projects enjoy a reduced
documentation load
Secondary:
Leadership can see an overall
picture of key work produced
HR is less burdened by exit
procedures
Non-sponsor Enterprise:
Applicability and potential traction
is being found outside of the
scope of our sponsor
Key Resources
Agency personnel (current and
retired) for documentation
knowledge & in-the-field
experiences
Non-government contacts
experiencing information retention
problems
R&D team contacts
Wide array of industries/job titles
available to contact.
Deployment
Pilot software with one team
within one agency
Expand to similar agencies and
share information based on
clearance level
Integrate with additional
agencies and non-government
organizations
Beneficiaries
Primary:
● New analysts
● Transitioning analysts
● Current analysts
Secondary:
● Leadership
● HR
The Librarian
MISSION MODEL CANVAS
Week 1
Mission Budget / Costs
Fixed:
Software development
Integration with government services
Variable:
Onboarding for analysts
System / Database integration
Mission Achievement
Reduce administrative overhead for all analysts regarding
role transitions
Bring new analysts up to speed on shared knowledge
faster
Provide a simple, intuitive experience for easy adoption
Key Partners
Administrators of ispace, rspace,
etc.
Administrators of Intellipedia
Administrators of shared drives
Analysis leadership (i.e. DA4)
across agencies, to encourage
documentation in performance
evaluations
Branch managers in the DA (i.e.
K, our sponsor)
ODNI, to implement our tool
across agencies
Enterprise HR departments, can
help onboard new employees and
aid with exit processes
Key Activities
Research processes to allow for
background documentation
Create & test mocks
Integrate multiple knowledge
sources
Develop a working prototype
Value Proposition
Reduce onboarding time for
new analysts
Improve convenience of
documentation, so analysts are
able to spend less time on
documentation
Consolidate multiple redundant
systems
Speed up analyst transition
time between projects, reducing
administrative burden from HR in
the process
Ensure a clean knowledge
transfer that is less dependent on
contractors
Reduce constant need to
consult former analysts, allowing
for complete focus on new
assignment
Buy-In / Support
Sorely needed by our sponsor;
Analysts can take a year on a 3-5
year project to get up to speed
Sponsor has tried multiple
software solutions without
success
Sponsor’s office has large degree
of autonomy to implement
prospective solutions
Beneficiaries
Primary:
New analysts can get up to
speed on tribal knowledge faster
than before
Transitioning analysts can focus
more on their work by not needing
to allot six months to transition
materials
Current analysts working on
projects enjoy a reduced
documentation load
Secondary:
Leadership can see an overall
picture of key work produced
HR is less burdened by exit
procedures
Non-sponsor Enterprise:
Applicability and potential traction
is being found outside of the
scope of our sponsor
Key Resources
Agency personnel (current and
retired) for documentation
knowledge & in-the-field
experiences
Non-government contacts
experiencing information retention
problems
R&D team contacts
Wide array of industries/job titles
available to contact.
Deployment
Pilot software with one team
within one agency
Expand to similar agencies and
share information based on
clearance level
Integrate with additional
agencies and non-government
organizations
Mission Achievement
● Reduce administrative overhead
● Bring new analysts up to speed faster
● Provide a simple, intuitive user
experience
Boiling the Ocean
Weeks 1-3 Weeks 4-6 Weeks 7-8 Weeks 9-10
Weeks 1-3 Weeks 4-6 Weeks 7-8 Weeks 9-10
First MVP
- Categorize documents
- Recommend relevant
reports/people
- Additional featuresDoes not blend
with analyst workflow
Current Tools
WebClient
i-space, r-
space, q-
space
NGT
WIRe
Library
materials
TagIt
Common
Folders
Shared
drives
Weeks 1-3 Weeks 4-6 Weeks 7-8 Weeks 9-10
Current
Tools
Sametime
Email
WebClient
i-space, r-
space, q-
space
NGT
WIRe
Library
materials
TagIt
Common
Folders
Shared
drives
“We used to be able to
burn CDs.
They replaced it with this
steaming pile of shit
called Intellipedia.”
-Air Force Captain
Pivoting
Weeks 1-3 Weeks 4-6 Weeks 7-8 Weeks 9-10
The Shotgun Approach
Document Lists
Profiles
Contact ListsContact Lists
Document Lists
Profiles
Weeks 1-3 Weeks 4-6 Weeks 7-8 Weeks 9-10
Weeks 1-3 Weeks 4-6 Weeks 7-8 Weeks 9-10
“... it’s not part of your daily
workflow to put your work
somewhere for others to have it.”
- CIA Branch Manager
Weeks 1-3 Weeks 4-6 Weeks 7-8 Weeks 9-10
Chatbot Interface
Chatbot for analysts
Dashboard for
managers
M V P
Weeks 1-3 Weeks 4-6 Weeks 7-8 Weeks 9-10
“This would be
fantastic!”
• Branch Manager, CIA
Weeks 1-3
Weeks 4-6 Weeks 7-8 Weeks 9-10
“When can I get it?”
• USAF Operations Director
Weeks 1-3
Weeks 4-6 Weeks 7-8 Weeks 9-10
Key Partners
All-Source analysts, particularly
at CIA
Administrators of
Jive/Intellipedia
Administrators of shared drives
Branch managers in the DA to
add collaboration as a metric
Director of the Directorate of
Analysis to issue a top-down rule
regarding collaboration*
Associate Deputy Director and
Deputy Director of the
Directorate of Digital Innovation*
ODNI, to implement our tool
across agencies
In-Q-Tel, potential incubation
and funding opportunities*
Key Activities
Deeply understand analyst
workflow
-Search processes
-Sharing processes
Understand current systems in
use
-Jive
-Intellipedia
-Shared Drives
-Research procurement
practices at sponsor.
Value Proposition
Get up to speed on shared
knowledge quickly
Ease transitions to new projects
by taking documentation with you
Increase ability to accumulate
social clout for skills and
tenure.(being named a librarian or
shares/comments on own page)
See processes used to come to
a conclusion.
Ensure a clean knowledge
transfer that is less dependent on
contractors
Consolidation of databases and
tools within and across agencies
Buy-In / Support
Sorely needed by our sponsor;
Analysts can take a year on a 3-5
year project to get up to speed
Sponsor has tried multiple
software solutions without
success
Sponsor’s office has large degree
of autonomy to implement
prospective solutions
Beneficiaries
Primary:
Incoming analysts can get up to
speed on tribal knowledge faster
than before
Transitioning analysts can focus
more on their work by not needing
to allot six months to transition
materials
Current analysts working on
projects enjoy a reduced
documentation load
Secondary:
Leadership can see an overall
picture of key work produced
IT has fewer databases to
maintain
Non-sponsor Enterprise:
Applicability and potential traction
is being found outside of the
scope of our sponsor
Key Resources
All-source analysts (current and
retired) for workflow and
documentation knowledge
IT Contractors with deep
knowledge of current
databases
Branch managers/leadership to
understand culture
Wide array of industries/job titles
available to contact.
Deployment
Pilot software with one team of
analysts within CIA
Expand to multiple teams to test
collaboration capabilities.
Secure funding from sponsors
budget allocation.
Potential staged development
via In-Q-Tel.*
Roll out collaboration software
across CIA.
The Librarian
MISSION MODEL CANVAS
Mission Budget / Costs
Fixed:
Software development
Integration with government services
Variable:
Onboarding for analysts
System / Database integration
Mission Achievement
Bring new analysts up to speed on shared knowledge faster
Director of the DA: 20% fewer analysts*
Deputy Director and Associate Deputy Director of DDI: Easily access
an interoperable system that brings together multiple data sources*
Security and Legal: Perceive no additional threats to security*
Value Proposition
● Share Knowledge quickly
● Ease transitions
● Accumulate Social Clout
● Understand thought
processes
● Ensure a clean knowledge
transfer
● Consolidate databases &
Tools
Key Partners
All-Source analysts, particularly
in IC
Administrators of
Jive/Intellipedia
Administrators of shared drives
Branch managers in the DA to
add collaboration as a metric
Director of the Directorate of
Analysis to issue a top-down rule
regarding collaboration*
Deputy Director & Associate
Deputy Director of the
Directorate of Digital Innovation*
ODNI, to implement our tool
across agencies
In-Q-Tel potential incubation and
funding opportunities
Key Activities
Deeply understand analyst
effectiveness, including Search
& Sharing workflows
Understand current systems in
use, including Jive, Intellipedia,
and Shared Drives
Research procurement
practices at sponsor
Value Proposition
Quickly get up to speed on
shared knowledge
Ease transitions to new projects
by taking documentation with you
Increase ability to accumulate
social clout for skills and tenure
Visibility into what analysts are
working on, as well as processes
used to come to a conclusion
Ensure a clean knowledge
transfer that is less dependent on
contractors
Consolidation of databases and
tools within and across agencies
Buy-In / Support
Sorely needed by our sponsor;
Analysts can take a year on a 3-5
year project to get up to speed
Sponsor has tried multiple
software solutions without
success
Sponsor’s office has large degree
of autonomy to implement
prospective solutions
Beneficiaries
Primary:
Analysts can access shared
databases faster than before
Branch managers can get
visibility into what their analysts
are working on
DA Leadership can have visibility
into metrics on the performance
of the whole organization
Secondary:
Contractors see increased usage
for under-utilized applications
Security/Legal now has a single
audit log, rather than audit logs
across multiple databases
Non-sponsor Enterprise:
Applicability and potential traction
is being found outside of the
scope of our sponsor
Key Resources
All-source analysts (current and
retired) for workflow and
documentation knowledge
IT Contractors with deep
knowledge of current databases
Branch managers and leadership
to understand culture
Wide array of analysts and
related roles in industry available
to contact
Deployment
Pilot software with one team of
analysts within CIA
Expand to multiple teams to test
collaboration capabilities
Secure funding from sponsors
budget allocation
Potential staged development
via In-Q-Tel
Roll out collaboration software
across CIA
The Librarian
MISSION MODEL CANVAS
Week 10
Mission Budget / Costs
Fixed:
Software development
Integration with government services
Variable:
System / Database integration
Mission Achievement
Director of the DA: Requires 20% fewer analysts
All-source Analysts: Get up to speed on new topic areas within one
workday
Deputy Director and Associate Deputy Director of DDI: Easily
access an interoperable system that brings together multiple data
sources
Security and Legal: Perceive no additional threats to security
In other words… (3 year)
Option 1. Reduce HR budget,
level productivity.
• Work done before Librarian: 100 units
• Work done after Librarian: 100 units
• HR budget before Librarian: $100
• HR Budget after Librarian: $67 (-33%)
Option 2. Same HR budget,
expanded productivity.
• Work done before Librarian: 100 units
• Work done after Librarian: 150 units (+50%)
• HR budget before Librarian: $100
• HR Budget after Librarian: $100
Develop and Deploy
Weeks 1-3
Weeks 4-6 Weeks 7-8 Weeks 9-10
Internal Readiness
Level
Weeks 1-3
Weeks 4-6 Weeks 7-8 Weeks 9-10
MVP testing and
feature exploration
Conversational
Chatbot
Social Nudges
Group Common Folders
Document Tagging
Current Feature
Exploration
Product Feature Diagram
- Personal database
- Group contact database
- Recommended Readings
- Moderated subject Lists
- Database Integrations
- Suggested actions
- Relevant Tool Integration
- Workflow monitoring
What’s next?
• WE ARE ALL CONTINUING
• Seeking grants and funding
• Continuing to build-out the Librarian V 1.0
Weeks 1-3
Weeks 4-6 Weeks 7-8 Weeks 9-10
Acknowledgements
Sponsor: Thank you for the amazing support!
Mentor: Tom Bedecarre
TA Mentor: Darren Hau
Teaching Team: Steve Blank, Joe Felter, Pete Newell, Steve
Weinstein
TA Team, H4D mentors, everyone we interviewed
Signing Off!
Come find us at thelibrarian@lists.stanford.edu
37.4241* N 122.1661* W
What key features
should we focus on
implementing?
Weeks 1-3
Weeks 4-6 Weeks 7-8 Weeks 9-10
Key Features
• Social nudges
• Tagging
• Sorting
Weeks 1-3
Weeks 4-6 Weeks 7-8 Weeks 9-10
Social Nudges
• Social proof
• Following colleagues
• No need to change incentives
Weeks 1-3
Weeks 4-6 Weeks 7-8 Weeks 9-10
Tagging and Sorting
• Make organizational tools front and center
• Combination of push and pull mechanisms
Weeks 1-3
Weeks 4-6 Weeks 7-8 Weeks 9-10
What will the Intelligence Community
look like with The Librarian?
• Track down information more quickly than before
• Ensure that time is not wasted at critical moments.
Weeks 1-3
Weeks 4-6 Weeks 7-8 Weeks 9-10
What will the world look like with The
Librarian?
• Our ideal is similar to The Librarian in Snow Crash: We
want to allow you to learn from a bot that contains the
sum of your organization’s knowledge.
Weeks 1-3
Weeks 4-6 Weeks 7-8 Weeks 9-10
Current
Tools
Sametime
Email
WebClient
i-space, r-
space, q-
space
NGT
WIRe
Library
materials
TagIt
Common
Folders
Shared
drives
“We used to be able to
burn CDs. They replaced
it with this steaming pile of
shit called Intellipedia.” -
Air Force Captain
Beneficiary Archetype: Incoming
Analyst
● Always the “new guy”
● Reads heavily for work
● 12 months of training (or more)
● 1 Year of onsite ramp-up time “before
you know what you’re talking about or,
more importantly, [before] anyone
listens.”
Beneficiary Archetype: Outgoing
Analyst
25-35 year-old intelligent / high aptitude individual, leaving from a 3-5 year
assignment
● Reads heavily for work
● May be resistant to digital tools
● Current equilibrium: 6 Months of spool-
down, documentation time
Appendix
Competitive
Landscape
The Librarian
Knowledge transfer for analysts
Creating a single chatbot interface for the many databases and software platforms
will allow us to improve knowledge transfer for new analysts within the intelligence
community.
Will Papper Sean Konz Anna Zeng Holden
Gibbons
Product / Software Engineering Data Science
Software
Engineering Strategy
The Librarian Team
Will Papper Sean Konz Anna Zeng Holden Gibbons
Academic Program
B.S. SymSys 2018 B.S. EE & CS 2020 B.S. CS 2018 B.S. MS&E 2019
LinkedIn
https://www.linkedin.com/in/wpapper/
https://www.linkedin.com/in/sean-
k-19a327b2/?locale=
https://www.linkedin.com/in/ann
a-z-11774096
https://www.linkedin.com/in/j-holden-
gibbons-669296a3/
Subject Matter Expert?
Details available
No
(Computer Vision /
Machine Learning)
No
(Software Engineering)
Yes, Veteran
Role
Product Data Science Hacker Hustler
Experience Solving a
Problem that Seemed
Impossible
- In one month, developed a
proof-of-concept social
network for a P2P mesh
networking platform
- Worked with multiple federal
agencies
- Developed Novel
approach to eye tracking
interfacing in a populated
consumer space
- Parallel processing
experience
- Pivoted the direction of
three Microsoft products
- Coordinated the first EE
makerspace in Stanford
- Founded non-profit urban
farming initiative in the inner
city and a small real estate
development firm.
Overview
What will the world look like with The
Librarian?
• As an analyst or researcher, you had to pull from dozens of
databases (both internal and external). You would not necessarily
know where to look for information, and could miss vital info.
• With The Librarian, you can query every database simultaneously
and never miss important information.
Where did we start?
Beneficiary Archetype: Incoming Analyst
22-30 year-old intelligent / high aptitude individual, coming for a 3-5 year assignment
● Values mission recognition over pay incentives
● Strong sense of patriotism/service
● Capable of being highly interchangeable
● Often operate as ‘snowflakes’, covering down
on issues as need be
● Reads heavily for work
● 12 months of training (or more)
● 1 Year of onsite ramp-up time “before you
know what you’re talking about or, more
importantly, [before] anyone listens.”
Beneficiary Archetype: Outgoing Analyst
25-35 year-old intelligent / high aptitude individual, leaving from a 3-5 year assignment
● Values mission recognition over pay incentives
● Strong sense of patriotism/service
● Capable of being highly interchangeable
● Often operate as ‘snowflakes’, covering down
on issues as need be
● Reads heavily for work
● May be resistant to digital tools
● Current equilibrium: 6 Months of spool-
down, documentation time
Beneficiary Archetype: Agency Leadership
30-45 year-old intelligent / high aptitude individual
● Strong sense of patriotism/service
○ Sacrificed family time & higher income
elsewhere
● Served multiple rotations. some as analysts
● Has a good sense of the agency’s ecosphere
of digital tools
● Perceives organizational culture & limitations
● Mission-focused, but aware of the need for
tools to improve mission effectiveness
Customer Workflow: Life of an All-Source Analyst
NEW
ANALYST
(0-1 year)
OLD
ANALYST
(3+ years)
read
write
review
publish
document
analyst
assigned to
a new area
of expertise
analyst given
history book
and loads of
background
reading
revise
scratch work in i-space
or equivalent
executive briefing
Summary reports
Intellipediai-space or eq.
Reports & briefings
raw intel reports
boolean search feed
Project
Lifecycle
Customer Workflow: Life of an All-Source Analyst
NEW
ANALYST
(0-1 year)
OLD
ANALYST
(3+ years)
read
write
review
publish
document
analyst
assigned to
a new area
of expertise
analyst given
history book
and loads of
background
reading
revise
scratch work in i-space
or equivalent
executive briefing
Summary reports
Intellipediai-space or eq.
Reports & briefings
raw intel reports
boolean search feed
Project
Lifecycle
AUTOMATE
Customer Workflow: Life of an All-Source Analyst
NEW
ANALYST
(0-1 year)
OLD
ANALYST
(3+ years)
read
write
review
publish
&
document
analyst
assigned to
a new area
of expertise
analyst given
history book
and loads of
background
reading
revise
scratch work in i-space
or equivalent
executive briefing
Summary reports
Intellipedia
i-space or eq.
Reports & briefings
raw intel reports
boolean search feed
Project
Lifecycle
Competitive
Landscape
The Librarian
Procurement Pipeline (6 mo.)
Directorate office
creates
requirements
Information Technology
Enterprises (in DDI)
searches for solutions
If contractors are
addressing the
requirements, they
are looped in
If an existing solution
cannot be repurposed,
ITE looks in industry
LARGE COMPANY:
Digital Futures brings it
into the internal network
STARTUP:
In-Q-Tel provides
milestone-based funding
Procurement in Practice (3
mo.)
Information Technology
Enterprises (in DDI)
searches for solutions
If contractors are
addressing the
requirements, they
are looped in
If an existing solution
cannot be repurposed,
ITE looks in industry
LARGE COMPANY:
Digital Futures brings it
into the internal network
STARTUP:
In-Q-Tel provides
milestone-based funding
Directorate leadership
requests solution from
DDI leadership
Directorate leadership
requests solution from
DDI leadership
DDI leadership & ITE
create requirements
If contractors are
addressing the
requirements, they
are looped in
If an existing solution
cannot be repurposed,
ITE looks in industry
LARGE COMPANY:
Digital Futures brings it
into the internal network
STARTUP:
DDI leadership
recommends In-Q-Tel
evaluate product
Directorate leadership
requests solution from
DDI leadership
DDI leadership & ITE
create requirements
If contractors are
addressing the
requirements, they
are looped in
If an existing solution
cannot be repurposed,
ITE looks in industry
LARGE COMPANY:
Digital Futures brings it
into the internal network
STARTUP:
DDI leadership
recommends In-Q-Tel
evaluate product
Implementation & beta
testing through ITE;
DDI leadership removes
roadblocks
What individual problems are
we solving?
Key Partners
All-Source analysts, particularly
in IC
Administrators of
Jive/Intellipedia
Administrators of shared drives
Branch managers in the DA to
add collaboration as a metric
Director of the Directorate of
Analysis to issue a top-down rule
regarding collaboration*
Deputy Director & Associate
Deputy Director of the
Directorate of Digital Innovation*
ODNI, to implement our tool
across agencies
In-Q-Tel potential incubation and
funding opportunities
Key Activities
Deeply understand analyst
effectiveness, including Search
& Sharing workflows
Understand current systems in
use, including Jive, Intellipedia,
and Shared Drives
Research procurement
practices at sponsor
Value Proposition
Quickly get up to speed on
shared knowledge
Ease transitions to new projects
by taking documentation with you
Increase ability to accumulate
social clout for skills and tenure
Visibility into what analysts are
working on, as well as processes
used to come to a conclusion
Ensure a clean knowledge
transfer that is less dependent on
contractors
Consolidation of databases and
tools within and across agencies
Buy-In / Support
Sorely needed by our sponsor;
Analysts can take a year on a 3-5
year project to get up to speed
Sponsor has tried multiple
software solutions without
success
Sponsor’s office has large degree
of autonomy to implement
prospective solutions
Beneficiaries
Primary:
Analysts can access shared
databases faster than before
Branch managers can get
visibility into what their analysts
are working on
DA Leadership can have visibility
into metrics on the performance
of the whole organization
Secondary:
Contractors see increased usage
for under-utilized applications
Security/Legal now has a single
audit log, rather than audit logs
across multiple databases
Non-sponsor Enterprise:
Applicability and potential traction
is being found outside of the
scope of our sponsor
Key Resources
All-source analysts (current and
retired) for workflow and
documentation knowledge
IT Contractors with deep
knowledge of current databases
Branch managers and leadership
to understand culture
Wide array of analysts and
related roles in industry available
to contact
Deployment
Pilot software with one team of
analysts within CIA
Expand to multiple teams to test
collaboration capabilities
Secure funding from sponsors
budget allocation
Potential staged development
via In-Q-Tel
Roll out collaboration software
across CIA
The Librarian
MISSION MODEL CANVAS
Mission Budget / Costs
Fixed:
Software development
Integration with government services
Variable:
System / Database integration
Mission Achievement
Director of the DA: Requires 20% fewer analysts
All-source Analysts: Get up to speed on new topic areas
within one workday
Deputy Director and Associate Deputy Director of DDI:
Easily access an interoperable system that brings together
multiple data sources
Security and Legal: Perceive no additional threats to
security
The Librarian: Value Proposition Canvas
Products
& Services
Knowledge
management for
analysts
Bring past documentation and
work with you to new projects
Tool to reference past work and
recommendations from former
analysts (no need to call) Customer
Jobs
Get up to speed
on a new body
of knowledge
Analysts lose access to prior
work when switching roles
Existing documentation tools
are cumbersome and scattered
across systems
Incoming All-Source Analysts
Gains
Gain
Creators
Pain
Relievers
Locate documentation and
past work in one place
Track down knowledge of
analysts who have retired or
changed projects without
needing to call
Put prior work into an IC-wide
system rather than team
shared drives or email accounts
Documentation is handled by
one software solution rather
than multiple tools with
different processes
Pains
The Librarian: Value Proposition Canvas
Products
& Services
Knowledge
management for
analysts
Documentation is handled
transparently in the background
Creates a clear list of materials
to be transitioned
Customer
Jobs
Pass along
knowledge to
remaining team
members
No established procedures for
transitioning out
Two weeks is not enough time
to get materials together
Process is chaotic, informal, and
ad-hoc
Outgoing All-Source Analysts
Gains
Gain
Creators
Pain
Relievers
Documentation is handled
in the background over the
course of a project,
reducing transition time
Adds formal processes to
transition materials
Move documentation from an
active burden into a
background process
Documentation is handled by
one software solution rather
than multiple tools with
different processes
Pains
The Librarian: Value Proposition Canvas
Products
& Services
Knowledge
management for
analysts
Research processes are stored
in an organized application
rather than shared drives
Branch manager interface:
present important metrics on
analyst activities
Customer
Jobs
Synthesize
documents from
analysts for
decision-making
Finished analyses are visible, but
for controversial claims it is
difficult to see a thought process
Focus on publications creates a
lack of insight into daily work
Branch Managers
Gains
Gain
Creators
Pain
Relievers
See what documents went into a
finished product
Gain insights for what analysts are
doing (beyond publications)
Help individual analysts based on
actions
Report’s interpretation of
source material is indicated
clearly
Documentation is handled by
one software solution rather
than multiple tools with
different processes
Pains
The Librarian: Value Proposition Canvas
Products
& Services
Knowledge
management for
analysts
Librarian helps provide metrics
across the entire organization,
when there was previously no
visibility
Customer
Jobs
Ensure that
analytic
resources are
directed most
efficiently
Too many offices to keep track
of in one spot
Need to get information from
briefings, rarely able to dive
deeper
Director of the Directorate of Analysis
Gains
Gain
Creators
Pain
Relievers
Needs to direct analysts to
topic areas focused on by
policymakers
Dashboard allows Director to
monitor high-level behavior of
offices within the organization
Can independently browse
dashboard, no need to rely on
branch managers completely
Pains
The Librarian: Value Proposition Canvas
Products
& Services
Knowledge
management for
analysts
Contractors employed by org to
serve as implementation
coordinators = $$
Integration with our system for
analyst use Customer
Jobs
Deploy solutions
that meet
requirementsHave multiple systems that they need to
integrate with, which slows down
progress
Already have systems that they own
solving similar problems, and are
protective of their turf
Contractors might not want to put in the
effort to build APIs/help us integrate
Contractors
Gains
Gain
Creators
Pain
Relievers
Benefit from contracts
due to service as
implementation liaisons
Increased contractor tool
use through system
integrations
Managing a single profile could
make integration with other
software easier
If APIs present, we can
integrate without them
Pains
The Librarian: Value Proposition Canvas
Products
& Services
Knowledge
management for
analysts
Security Interface
Analyst action signaling
Customer
Jobs
Keep information
from getting
compromisedHuge database of how analysts
think/their current progress that
is a giant target for foreign
intelligence services
Another system to worry about
secure access
Security / Legal
Gains
Gain
Creators
Pain
Relievers
Immediate insight into the
actions of analysts using
signaling mechanisms
Procedure is already in place to
implement this, since this type
of database; precedent exists
Database APIs already exist
We don’t need data copies, just
links
Pains
Cultural Resistance
• Workflow
• If something isn’t in the day-to-day expectations, it won’t be used
• Need-to-know
• Your information is yours, and you shouldn’t share it
• Silos
• Incentives
Incentives: How are analysts promoted?
• Analysts are evaluated on papers produced and the decisions
that these papers affect
• Collaboration is not a great help in the promotion process, so it
is not common practice
• Who can change this?
• Branch Managers: “If it’s not shared, I’m not counting it.”
• Directorate of Analysis Leadership: Top-down requirement
Where are we now?
MVP
Chatbot for analysts
Dashboard for branch
managers/directors
MVP - Analysts
We are introducing a chatbot interface to connect new all-source
analysts with the information they need from existing databases in
the intelligence community.
Analysts will be able to reach crucial background information, identify
subject matter experts, retrieve existing reports, etc. in no time at all.
MVP - Branch Managers
Currently, branch managers do not have much insight into reports
that are in progress. Documents need to actively be moved into
shared drives, which only provides very granular insight.
By reporting aggregate queries from analysts to branch managers,
branch managers can most efficiently help analysts.
What are our next steps?
GET KEEP GROW
GET
Awareness:
- MVP testing
- Present underutilized tools
- Integrate with org chat client
Interest:
- More robust document discovery
- Ease of contacting SMEs
Consideration:
- How integration impacts workflow
- If discovering different document
pools (recommendations) would
really be beneficial
Purchase:
- Procurement process
KEEP
- Quantify usage of databases
with/without tool
- Additional tool integrations based
on user desires
- Leadership/user feedback provided
on end intel product quality
- Ensure budget for system
upgrades (re: compatibility)
- Tool should learn over time
- Better suggestions, relevant
documents based on
suggestion doc usage
GROW
Referrals:
- Introduce the tool into other
agencies (other IC, DoD, DIUx)
- Dual-use applications: additional AI
training opportunities
Cross-Sell:
- Develop machine learning to the
point of making suggestions
Next-Sell:
- Add additional internal tool
integrations
Up-Sell:
- Introduce cross-agency information
exchange / integration
Mission Achievement - Product
• Do people come back to this tool after trying it the first two times?
(Focus is on abandonment or lack thereof)
• How many times do people consult the tool per day?
• Does this tool reduce the number of times analysts access other
databases?
• How many database integrations do we have?
• Is the tool getting smarter?
Mission Achievement - Stakeholders
Party Benefit
Director of the Directorate of Analysis Requires 20% fewer analysts
All-source Analysts Get up to speed on new topic areas
within one workday, increase breadth of
relevant information presented to
analysts
Deputy Director and Associate Deputy
Director of DDI
Easily access an interoperable system
that brings together multiple data
sources
Security and Legal Perceive no additional threats to safety
Work Distribution
Will - Product: Ensure that the MVP we’re building matches
customer needs, handle User Experience
Sean - Data Science: Data Science on Database metrics,
NLP framework
Anna - Hacker: Database integrations, Software Engineering,
User Experience
Holden - Hustler: Find new customers and channels, maintain
contact with current beneficiaries
Next Steps
• Start building a chatbot!
• Integrate with an open source database API
• Begin to train the chatbot on open source data
• Test interactive demo with analysts
• Test dashboard with branch managers
We want to hear from you!
Email:
thelibrarian@lists.stanford.edu
Buy-In and Support
Analyst: Easier to get up to speed on new topics, front-ends are
consolidated to allow for simple querying.
Branch Managers: Stay current on the topics that analysts are
working on. Have more insight into ways to provide help.
Contractors: Improve usage of applications that may not fit into an
analyst’s workflow, increasing chances of renewal.
Security: Easily detect anomalous activity, since the activity will be in
one central audit log.
Expectation
Chatbot interface is new for
employees and will require a
shift in mindset.
Result
Asking other analysts questions
over IM is very common.
Analysts can now bother a bot
instead of bothering their
colleagues.
Expectation
The Librarian would be useful
for easily constructing lists of
tags
Result
There is a database for this, but
it is not used much. The
Librarian could act as a front-
end.
Expectation
There are a few crucial
databases to pull from, and
people consult those heavily.
Result
The Library has hundreds of
databases, but they are
underutilized. We can create an
interface for many databases
that are not pulled from.
Expectation
Lists of relevant documents do
not exist. The Librarian could
create these lists seamlessly.
Result
There are lists of documents
that exist in NGT, but analysts
do not realize that this feature
exists. Furthermore, it is not in
the workflow that analysts
follow.
We can add a front-end for
these lists, allowing them to be
queried easily.
MVP Link
Password: thelibrarianMVP
https://www.dropbox.com/sm/password?cont=https%3A%2F%2Fww
w.dropbox.com%2Fs%2Fpjoqo7u3chhb0v3%2FThe%2520Librarian%25
20Interface.pdf%3Fdl%3D0&content_id=ADe0DaUJAKJKJOziMFMv3QC
Vw3xR7_gAOVk
GET KEEP GROW
GET
Awareness:
- MVP testing
- Present underutilized tools
- Integrate with org chat client
Interest:
- More robust document discovery
- Ease of contacting SMEs
Consideration:
- How integration impacts workflow
- If discovering different document
pools (recommendations) would
really be beneficial
Purchase:
- Procurement process (Appendix)
KEEP
- Quantify usage of databases
with/without tool
- Additional tool integrations based
on user desires
- Leadership/user feedback provided
on end intel product quality
- Ensure budget for system
upgrades (re: compatibility)
- Tool should learn over time
- Better suggestions, relevant
documents based on
suggestion doc usage
GROW
Referrals:
- Introduce the tool into other
agencies (other IC, DoD, DIUx)
- Dual-use applications: additional AI
training opportunities
Cross-Sell:
- Develop machine learning to the
point of making suggestions
Next-Sell:
- Add additional internal tool
integrations
Up-Sell:
- Introduce cross-agency information
exchange / integration
Mission Achievement - Product
• Do people come back to this tool after trying it the first two times?
(Focus is on abandonment or lack thereof)
• How many times do people consult the tool per day?
• Does this tool reduce the number of times analysts access other
databases?
• How many database integrations do we have?
• Is the tool getting smarter?
Mission Achievement - Stakeholders
Party Effect
Director of the Directorate of Analysis Requires 20% fewer analysts
All-source Analysts Get up to speed on new topic areas
within one workday, increase breadth of
relevant information presented to
analysts
Deputy Director and Associate Deputy
Director of DDI
Easily access an interoperable system
that brings together multiple data
sources
Security and Legal Perceive no additional threats to safety
Next Steps
• Interview more analysts to understand different types of workflows
• Drill down on more specific use cases for the chatbot
• Understand which areas have the biggest efficiency improvements
• Understand which database APIs exist already
• Understand the work required to add new database integrations
Beneficiary Archetype: Incoming Analyst
Male/Female:22-30 years old
-Intelligent/high aptitude individuals.
-Values mission recognition over pay incentives.
-Strong sense of patriotism/service.
-Though capable of being highly
interchangeable, often operate as ‘snowflakes’,
covering down on issues as need be.
-Reads heavily for work.
-12 month training but can be longer with follow
along training opportunities.
- ~3-5 year assignments.
-1 Year of spool-up time. “Before you know
what you’re talking about or more
importantly, anyone listens”. -K
Beneficiary Archetype: Outgoing Analysts
Male/Female:25-35
-Intelligent/high aptitude individuals.
-Values mission recognition over pay incentives.
-Strong sense of patriotism/service.
-Though capable of being highly interchangeable,
often operate as ‘snowflakes’, covering down on
issues as need be.
-Reads heavily for work.
-12 month training but can be longer with follow
along training opportunities.
- ~3-5 year assignments.
-May or may not be hoarder/e-allergic.
-Current equilibrium: 6 Months of spool-down,
documentation time.
The Librarian: Value Proposition Canvas
Products
& Services
Knowledge
management for
analysts
- Key documents used by an
analyst are included in a
single system
-Research processes are
stored in an organized
application rather than shared
drives Customer
Jobs
Synthesize
documents from
analysts for
decision-making
-Finished analyses are visible,
but for controversial claims it is
difficult to see a thought process
-Focus on publications creates a
lack of insight into daily work
Branch Managers
Gains
Pains
Gain
Creators
Pain
Relievers
- See what documents went into a
finished product
- Get more insight into what analysts
are doing (beyond publications)
- Report’s interpretation of
source material is indicated
clearly
-Documentation is handled by
one software solution rather
than multiple tools with
different processes
Beneficiary Archetype: Team/Mis. Cntr. Managers
Male/Female: 28-40
-Responsible for bringing new analysts up to
speed.
-Intelligent/high aptitude individuals.
-Strong sense of patriotism/service manifesting in
potential sacrificed family time/higher income
elsewhere.
-Most likely served multiple mission centers 3+
years each as an analyst.
-Possesses good sense of what the ecosphere of
digital tools consists of within the agency.
-Good sense of organizational culture/limitations.
-Mission focused on day-to-day but aware of the
need for tools to be revamped to improve mission
effectiveness.
The Librarian: Value Proposition Canvas
Products
& Services
Knowledge
management for
analysts
Librarian lists help maintain
information, contacts,
glossary, prominent people
and thus reduce time spent
finding intel and more time
producing it.
Customer
Jobs
Manage cadre of
analysts and their
tools.
-Doesn’t have enough analysts
or at least the current ones
aren’t efficient enough due to
knowledge being lost at lower
levels creating redundant and
less than fully informed
conclusions.. Budget and
manpower she or he is afforded
is outside of their control. .
Director of the Directorate of
Analysis
Gains
Pains
Gain
Creators
Pain
Relievers
Able to either dig deeper in current
areas or potentially spread current
labor footprint into more/nuanced
issues for better understanding of the
world.
Automated systems for
analysts allow for less
variation in human behavior
leading to more uniformity
and more efficiency in
analysis.
Beneficiary Archetype: Director of Directorate of Analysis
-Female/Male: 40-50
-Personally responsible for allocation of analytical
manpower across array of needs.
-Ultimately responsible to Director of the agency for quality
of analytical product.
-Most likely served in several mission centers and well
versed on past and current culture. (not much changes)
-Has the authority to mandate workflow procedures such
as software utilization.
-Highly motivated and often has best in mind but has many
boiling pots to tend.
The Librarian: Value Proposition Canvas
Products
& Services
Knowledge
management for
analysts
-Single search interface would
reduce proliferation of
databases
Customer
Jobs
Implement new
tools that are
relevant to the
Agency
-Do not have proper API
integrations for existing
databases
Deputy Director and
Associate Deputy Director of
DDI
Gains
Pains
Gain
Creators
Pain
Relievers
-Would be able to consolidate multiple
databases and eliminate failed
solutions
-Focus on interoperability
could reduce cost to add new
solutions on top of our system
The Librarian: Value Proposition Canvas
Products
& Services
Knowledge
management for
analysts
-Contractors employed by org
to serve as implementation
coordinators = $$
Customer
Jobs
Deploy solutions
that meet
requirements
-Have multiple systems that
they need to integrate with,
which slows down progress
-Already have systems that they
own solving similar problems,
and are protective of their turf
Contractors
Gains
Pains
Gain
Creators
Pain
Relievers
-Benefit from contracts due to service
as implementation liaisons
-A single profile could make
integration with other
software easier
The Librarian: Value Proposition Canvas
Products
& Services
Knowledge
management for
analysts
Customer
Jobs
Keep information
from getting
compromised
-Huge database of how analysts
think/their current progress that
is a giant target for foreign
intelligence services
Security/Legal
Gains
Pains
Gain
Creators
Pain
Relievers
-Procedure is already in place
to implement this, since this
type of database. Precedent
exists.
Day in the Life
Workflow of our primary beneficiaries (analysts)
7:30AM- Boolean Search yielding 100s or 1000s of hits.
8:00AM- Triage/speed-read for use in daily brief,
construct notes in Word.
8:30AM- Team daily brief, trending items, taskings, etc.
9-Noon- Follow up with more detail.
Where our product fits in
Cultural Resistance
• Workflow
• If something isn’t in the day-to-day expectations, it
won’t be used
• Need-to-know
• Your information is yours, and you shouldn’t share
it
How do people get promoted?
• Analysts are measured on the papers you produce and
the decisions that your papers affect
• Collaboration is not part of the promotion process, so
it is not done
• Who can do this?
• Branch Managers - “If it’s not shared, I’m not
counting it.”
• Directorate of Analysis Leadership - Top-down rule
“We got an automated water cooler… [but no one] is
responsible for maintaining it. It’s about
accountability.”
- Chris, Pathfinder PM, on Intellipedia
… what if we made documentation part of an
analyst’s performance review?
Procurement Discovery
Procurement Life cycle
1.Market/internal research for tools serving a given purpose
a.Contractors generally involved in conducting this process
2. Budget analysis: Is it cheaper to build a solution, or to buy? Which is
more efficient?
3.If buying, Numerous routes: In-Q-Tel, Digital Futures within DDI
How Procurement Works in Theory (6-12mo)
1. An office within a directorate, such as the Office of Advanced Analytics
within the Directorate of Analysis, produces a set of requirements.
2. Information Technology Enterprises within the Directorate of Digital
Innovation searches for solutions to meet those requirements. If the
requirements are currently being addressed in some form by a
contractor already, the contractor is brought into the loop as well.
3. If an existing solution cannot be modified, then a commercial solution is
How Procurement Works in Theory (6-12mo)
4. If the commercial solution is from a large company, Digital Futures within
the Directorate of Digital Innovation will work to bring the company onto the
internal network.
5. If the commercial solution is from a startup, In-Q-Tel will provide
milestone-based funding to help the startup get the solution onto the
internal network. To beta test the software, a work program is initiated that
acts as a limited rollout of the software.
How Procurement Works in Practice (3mo)
1. Someone high up in a relevant office (for example, the Office of
Advanced Analytics within the Directorate of Analysis) expresses
strong support for a solution to someone high up in the Directorate of
Digital Innovation (Associate Deputy Director or Deputy Director)
2. Deputy Director of DDI works with Information Technology Enterprises
to determine requirements. May also recommend that In-Q-Tel
evaluate the solution if the company is a startup.
Procurement Life cycle con’t
Procurement through DDI:
1.Team hired by supplier to handle implementation
a.Government team holds necessary clearances, experience etc.
b.Atypical: Government creates a team specifically for implementation of
particular tools
2.Requirements set by IC (in our case, DDI)
3.Implementation team works with both org and supplier to meet
requirements
a.Will ask supplier to alter tool slightly to fit security requirements
b.Project sponsors in leadership make the connections to push
Procurement Life cycle con’t
Procurement through In-Q-Tel:
Younger companies in need of seed funding targeted
1.Companies with a desired tool are found through market research
process
2.Companies are provided funding tied to milestones. Milestones
help them progress toward getting their software ready for an IC
environment.
3.When the product is ready for use on classified networks, a work
program is initiated to beta test the software.
Scope and implementation takeaways.
-Comparable to mid-sized company in size of workforce that will be
utilizing our product
-Can be most easily implemented via one ‘top down’ directive from
Director of Directorate of Analysis.
-Can also be implemented by individual buy-in of all lower level
managers. This is not a favorable path.
Key Takeaways
“[Having leadership support] is probably one of the top five
things you need to be successful in getting a solution through”
- Interviewee 7, Integration Contractor
Having org leadership invested in the solution is a key to successful integration
Key Takeaways
“Our Government Chief knew that this was something that was
needed, so she took it upon herself to make it happen”
-Interviewee 7, Integration Contractor
If a solution is needed, leadership will recognize it is needed, and will collaborate
to make it a reality
MVP Development: Three Concepts
• Profile based: search a subject -> get back ranked profiles ->
view a timeline
• Document List
• People List
Document Based MVP
• Topics linked to Subject
Matter Experts
• Subject matter experts have
suggested readings on their
topic
• Results return documents
authored by subject expert
Search Interface MVP
-Relevancy based on
degrees of separation
- “Suggestion” panel
features subject matter
experts and top
documents
- People linked to contact
information
-Documents linked to
databases
Profile Based MVP
• Provides information on analysts and what they have viewed
• Acts as a landing page from the document and search solutions
• Could either integrate with or replace existing profile solutions
• Could pull data from HR databases to auto-populate profiles
Expectation
If an Intel product is available,
an author’s contact information
can be tracked down from their
group
Result
If the contact information is not
included in an intel product,
consumers of that product
cannot contact the authors
Contact information can be
grabbed using auditing
apparatus
Expectation
Analysts will find it helpful to
share information about what
they are working on
automatically.
Result
Teams may not find it preferable
to automatically share
potentially sensitive
information. One team does
manually collect a list of key
topics the team focuses on, but
distributes it via word document
/ email to specific collaborators.
Key Takeaways - MVP received support (with
caveats)
MVP would be “super helpful”
With caveats:
“Yes. As long as it was kept up to date and could
highlight the major themes, at the very least it would be
a starting point for a senior analyst to talk to a starting
analyst.”
-Interviewee #3, manages analysts and was an analyst
Can we do this?
Notes on how a systems like the ones we’re suggesting would fit into
the current information systems used in the IC
-Can we link directly to documents?
-Can we link directly to people?
-Can we only link documents to citations and users need to go get the
actual doc on a different database? (fatal)
-Would we be able to display search results from docs already in the
IC databases?
Next Steps
• Evaluate whether automating clustering and tagging is feasible
• Evaluate whether a top-down rule will be followed
• Bring together multiple MVPs through interview tests
• Gain access to think tank data sets (Used to build a subject
recognition and recommendation system)

More Related Content

What's hot

5 must have project management reports
5 must have project management reports5 must have project management reports
5 must have project management reportsOrangescrum
 
Agile Software Development Practices: Perceptions & Project Data
Agile Software Development Practices: Perceptions & Project DataAgile Software Development Practices: Perceptions & Project Data
Agile Software Development Practices: Perceptions & Project DataChristoph Matthies
 
Module 8 presenter notes
Module 8 presenter notesModule 8 presenter notes
Module 8 presenter notesTony
 
The Changing Landscape of Project Management in 2018
The Changing Landscape of Project Management in 2018The Changing Landscape of Project Management in 2018
The Changing Landscape of Project Management in 2018Richard Kok
 
Project Management: More Caught Than Taught
Project Management: More Caught Than TaughtProject Management: More Caught Than Taught
Project Management: More Caught Than Taughtedward_uy
 
Assessing youragility
Assessing youragilityAssessing youragility
Assessing youragilityrseniv
 
The Road to Data-Informed Agile Development Processes
The Road to Data-Informed Agile Development ProcessesThe Road to Data-Informed Agile Development Processes
The Road to Data-Informed Agile Development ProcessesChristoph Matthies
 
IT Program Assessments
IT Program AssessmentsIT Program Assessments
IT Program AssessmentsMark Archer
 
Lessons from 40 PPM Experts on Making the Transition from Project Management ...
Lessons from 40 PPM Experts on Making the Transition from Project Management ...Lessons from 40 PPM Experts on Making the Transition from Project Management ...
Lessons from 40 PPM Experts on Making the Transition from Project Management ...Ricardo Viana Vargas
 
Project management
Project managementProject management
Project managementExterro
 
PMI-SAC November Luncheon
PMI-SAC November LuncheonPMI-SAC November Luncheon
PMI-SAC November LuncheonMike Haden
 
Planning Effective Linkages
Planning Effective LinkagesPlanning Effective Linkages
Planning Effective LinkagesGhulam Qadir
 
IABC Silver Quill Case Study
IABC Silver Quill Case StudyIABC Silver Quill Case Study
IABC Silver Quill Case StudyAlina Haas
 

What's hot (20)

5 must have project management reports
5 must have project management reports5 must have project management reports
5 must have project management reports
 
Agile Software Development Practices: Perceptions & Project Data
Agile Software Development Practices: Perceptions & Project DataAgile Software Development Practices: Perceptions & Project Data
Agile Software Development Practices: Perceptions & Project Data
 
Prince 2: project managment Document Lessons learned report
Prince 2: project managment Document Lessons learned reportPrince 2: project managment Document Lessons learned report
Prince 2: project managment Document Lessons learned report
 
Module 8 presenter notes
Module 8 presenter notesModule 8 presenter notes
Module 8 presenter notes
 
The Changing Landscape of Project Management in 2018
The Changing Landscape of Project Management in 2018The Changing Landscape of Project Management in 2018
The Changing Landscape of Project Management in 2018
 
Development testing-a-new-era-in-software-quality
Development testing-a-new-era-in-software-qualityDevelopment testing-a-new-era-in-software-quality
Development testing-a-new-era-in-software-quality
 
Susan Dyl DEC
Susan Dyl DECSusan Dyl DEC
Susan Dyl DEC
 
Final Simulation
Final SimulationFinal Simulation
Final Simulation
 
Project Management: More Caught Than Taught
Project Management: More Caught Than TaughtProject Management: More Caught Than Taught
Project Management: More Caught Than Taught
 
Assessing youragility
Assessing youragilityAssessing youragility
Assessing youragility
 
The Road to Data-Informed Agile Development Processes
The Road to Data-Informed Agile Development ProcessesThe Road to Data-Informed Agile Development Processes
The Road to Data-Informed Agile Development Processes
 
Pm audit and assessment approach final
Pm audit and assessment approach   finalPm audit and assessment approach   final
Pm audit and assessment approach final
 
IT Program Assessments
IT Program AssessmentsIT Program Assessments
IT Program Assessments
 
Test Lead_Venkat Kallagunta
Test Lead_Venkat KallaguntaTest Lead_Venkat Kallagunta
Test Lead_Venkat Kallagunta
 
Lessons from 40 PPM Experts on Making the Transition from Project Management ...
Lessons from 40 PPM Experts on Making the Transition from Project Management ...Lessons from 40 PPM Experts on Making the Transition from Project Management ...
Lessons from 40 PPM Experts on Making the Transition from Project Management ...
 
Project recovery
Project recoveryProject recovery
Project recovery
 
Project management
Project managementProject management
Project management
 
PMI-SAC November Luncheon
PMI-SAC November LuncheonPMI-SAC November Luncheon
PMI-SAC November Luncheon
 
Planning Effective Linkages
Planning Effective LinkagesPlanning Effective Linkages
Planning Effective Linkages
 
IABC Silver Quill Case Study
IABC Silver Quill Case StudyIABC Silver Quill Case Study
IABC Silver Quill Case Study
 

Similar to Librarian Mission Model Canvas

Software Development Methodologies Pros, Cons, & Use Cases
Software Development Methodologies Pros, Cons, & Use CasesSoftware Development Methodologies Pros, Cons, & Use Cases
Software Development Methodologies Pros, Cons, & Use CasesPolyxer Systems
 
Project Management Overview
Project Management OverviewProject Management Overview
Project Management Overviewcford1973
 
Portfolios, programmes and projects. Delivering one version of the truth: a c...
Portfolios, programmes and projects. Delivering one version of the truth: a c...Portfolios, programmes and projects. Delivering one version of the truth: a c...
Portfolios, programmes and projects. Delivering one version of the truth: a c...Association for Project Management
 
subject.....project management.Tools and Processes Based on the pr.pdf
subject.....project management.Tools and Processes Based on the pr.pdfsubject.....project management.Tools and Processes Based on the pr.pdf
subject.....project management.Tools and Processes Based on the pr.pdfSALES97
 
Postmortemanalysis 120520033844-phpapp02
Postmortemanalysis 120520033844-phpapp02Postmortemanalysis 120520033844-phpapp02
Postmortemanalysis 120520033844-phpapp02Loriebel Manabat
 
Microsoft Project in Manufacturing and Resources - Presented by Atidan
Microsoft Project in Manufacturing and Resources - Presented by AtidanMicrosoft Project in Manufacturing and Resources - Presented by Atidan
Microsoft Project in Manufacturing and Resources - Presented by AtidanDavid J Rosenthal
 
Building a CI platform
Building a CI platformBuilding a CI platform
Building a CI platformMarcel Brussee
 
Writing Successful Project Proposal
Writing Successful Project ProposalWriting Successful Project Proposal
Writing Successful Project ProposalGhulam Qadir
 
Software Development Methodologies Pros, Cons, & Use Cases.pdf
Software Development Methodologies Pros, Cons, & Use Cases.pdfSoftware Development Methodologies Pros, Cons, & Use Cases.pdf
Software Development Methodologies Pros, Cons, & Use Cases.pdfPolyxer Systems
 
Time to Money- Software Development for a Regulated Medical World
Time to Money- Software Development for a Regulated Medical WorldTime to Money- Software Development for a Regulated Medical World
Time to Money- Software Development for a Regulated Medical WorldDina Sifri
 
Excellent Estimating – the key to Happy Clients
Excellent Estimating – the key to Happy ClientsExcellent Estimating – the key to Happy Clients
Excellent Estimating – the key to Happy ClientsRobert McGeachy
 
(Fall2016)Lecture1.pptx
(Fall2016)Lecture1.pptx(Fall2016)Lecture1.pptx
(Fall2016)Lecture1.pptxgarkapifye
 
Pps Fall2010 Presentation Final
Pps Fall2010 Presentation FinalPps Fall2010 Presentation Final
Pps Fall2010 Presentation Finalahollander7
 
Changing Mindsets to Change the Future in Business Process Services Industry
Changing Mindsets to Change the Future in Business Process Services IndustryChanging Mindsets to Change the Future in Business Process Services Industry
Changing Mindsets to Change the Future in Business Process Services IndustryDebi Prasad Dash
 
Microsoft PPM tool (Project Online / Project Server) Case Study by epmsolutio...
Microsoft PPM tool (Project Online / Project Server) Case Study by epmsolutio...Microsoft PPM tool (Project Online / Project Server) Case Study by epmsolutio...
Microsoft PPM tool (Project Online / Project Server) Case Study by epmsolutio...Sophia Zhou
 
Microsoft Project Online - EPM Solutions
Microsoft Project Online - EPM SolutionsMicrosoft Project Online - EPM Solutions
Microsoft Project Online - EPM SolutionsEPM Solutions
 
Projects Management
Projects ManagementProjects Management
Projects ManagementNirmal Shah
 

Similar to Librarian Mission Model Canvas (20)

Software Development Methodologies Pros, Cons, & Use Cases
Software Development Methodologies Pros, Cons, & Use CasesSoftware Development Methodologies Pros, Cons, & Use Cases
Software Development Methodologies Pros, Cons, & Use Cases
 
Project Management Overview
Project Management OverviewProject Management Overview
Project Management Overview
 
Portfolios, programmes and projects. Delivering one version of the truth: a c...
Portfolios, programmes and projects. Delivering one version of the truth: a c...Portfolios, programmes and projects. Delivering one version of the truth: a c...
Portfolios, programmes and projects. Delivering one version of the truth: a c...
 
subject.....project management.Tools and Processes Based on the pr.pdf
subject.....project management.Tools and Processes Based on the pr.pdfsubject.....project management.Tools and Processes Based on the pr.pdf
subject.....project management.Tools and Processes Based on the pr.pdf
 
Postmortemanalysis 120520033844-phpapp02
Postmortemanalysis 120520033844-phpapp02Postmortemanalysis 120520033844-phpapp02
Postmortemanalysis 120520033844-phpapp02
 
Microsoft Project in Manufacturing and Resources - Presented by Atidan
Microsoft Project in Manufacturing and Resources - Presented by AtidanMicrosoft Project in Manufacturing and Resources - Presented by Atidan
Microsoft Project in Manufacturing and Resources - Presented by Atidan
 
TerrySeekinsBA3
TerrySeekinsBA3TerrySeekinsBA3
TerrySeekinsBA3
 
Building a CI platform
Building a CI platformBuilding a CI platform
Building a CI platform
 
Writing Successful Project Proposal
Writing Successful Project ProposalWriting Successful Project Proposal
Writing Successful Project Proposal
 
Software Development Methodologies Pros, Cons, & Use Cases.pdf
Software Development Methodologies Pros, Cons, & Use Cases.pdfSoftware Development Methodologies Pros, Cons, & Use Cases.pdf
Software Development Methodologies Pros, Cons, & Use Cases.pdf
 
project management essentials
project management essentialsproject management essentials
project management essentials
 
Time to Money- Software Development for a Regulated Medical World
Time to Money- Software Development for a Regulated Medical WorldTime to Money- Software Development for a Regulated Medical World
Time to Money- Software Development for a Regulated Medical World
 
Cs244 w10ch1
Cs244 w10ch1Cs244 w10ch1
Cs244 w10ch1
 
Excellent Estimating – the key to Happy Clients
Excellent Estimating – the key to Happy ClientsExcellent Estimating – the key to Happy Clients
Excellent Estimating – the key to Happy Clients
 
(Fall2016)Lecture1.pptx
(Fall2016)Lecture1.pptx(Fall2016)Lecture1.pptx
(Fall2016)Lecture1.pptx
 
Pps Fall2010 Presentation Final
Pps Fall2010 Presentation FinalPps Fall2010 Presentation Final
Pps Fall2010 Presentation Final
 
Changing Mindsets to Change the Future in Business Process Services Industry
Changing Mindsets to Change the Future in Business Process Services IndustryChanging Mindsets to Change the Future in Business Process Services Industry
Changing Mindsets to Change the Future in Business Process Services Industry
 
Microsoft PPM tool (Project Online / Project Server) Case Study by epmsolutio...
Microsoft PPM tool (Project Online / Project Server) Case Study by epmsolutio...Microsoft PPM tool (Project Online / Project Server) Case Study by epmsolutio...
Microsoft PPM tool (Project Online / Project Server) Case Study by epmsolutio...
 
Microsoft Project Online - EPM Solutions
Microsoft Project Online - EPM SolutionsMicrosoft Project Online - EPM Solutions
Microsoft Project Online - EPM Solutions
 
Projects Management
Projects ManagementProjects Management
Projects Management
 

More from H4Diadmin

gARGoyle H4D Final Project Video
gARGoyle H4D Final Project VideogARGoyle H4D Final Project Video
gARGoyle H4D Final Project VideoH4Diadmin
 
James Madison University Final Presentation
James Madison University Final PresentationJames Madison University Final Presentation
James Madison University Final PresentationH4Diadmin
 
Austra Lumina
Austra LuminaAustra Lumina
Austra LuminaH4Diadmin
 
Team Surgency
Team SurgencyTeam Surgency
Team SurgencyH4Diadmin
 
Xplomo final presentation
Xplomo final presentationXplomo final presentation
Xplomo final presentationH4Diadmin
 
Technology Threat Prediction
Technology Threat PredictionTechnology Threat Prediction
Technology Threat PredictionH4Diadmin
 
Modern Day Minutemen
Modern Day MinutemenModern Day Minutemen
Modern Day MinutemenH4Diadmin
 
Light Tactical All-terrain Vehicle (LTATV) Crew Augmentation
Light Tactical All-terrain Vehicle (LTATV) Crew AugmentationLight Tactical All-terrain Vehicle (LTATV) Crew Augmentation
Light Tactical All-terrain Vehicle (LTATV) Crew AugmentationH4Diadmin
 
Big Crypto for Little Things
Big Crypto for Little ThingsBig Crypto for Little Things
Big Crypto for Little ThingsH4Diadmin
 
Deep Clearance Situational Awareness
Deep Clearance Situational AwarenessDeep Clearance Situational Awareness
Deep Clearance Situational AwarenessH4Diadmin
 
Digital Overlay
Digital OverlayDigital Overlay
Digital OverlayH4Diadmin
 
Drone detection.pptx
Drone detection.pptxDrone detection.pptx
Drone detection.pptxH4Diadmin
 
Team Spyglass Final Presentation
Team Spyglass Final PresentationTeam Spyglass Final Presentation
Team Spyglass Final PresentationH4Diadmin
 
Lessons Learned
Lessons LearnedLessons Learned
Lessons LearnedH4Diadmin
 
H4D Week 12 Final Presentation
H4D Week 12 Final PresentationH4D Week 12 Final Presentation
H4D Week 12 Final PresentationH4Diadmin
 
Blue and Gray Berets H4D Final Presentation
Blue and Gray Berets H4D Final PresentationBlue and Gray Berets H4D Final Presentation
Blue and Gray Berets H4D Final PresentationH4Diadmin
 

More from H4Diadmin (20)

gARGoyle H4D Final Project Video
gARGoyle H4D Final Project VideogARGoyle H4D Final Project Video
gARGoyle H4D Final Project Video
 
James Madison University Final Presentation
James Madison University Final PresentationJames Madison University Final Presentation
James Madison University Final Presentation
 
Frogman
FrogmanFrogman
Frogman
 
Austra Lumina
Austra LuminaAustra Lumina
Austra Lumina
 
Broadcom
BroadcomBroadcom
Broadcom
 
Seacurity
Seacurity Seacurity
Seacurity
 
Team Surgency
Team SurgencyTeam Surgency
Team Surgency
 
Xplomo final presentation
Xplomo final presentationXplomo final presentation
Xplomo final presentation
 
Technology Threat Prediction
Technology Threat PredictionTechnology Threat Prediction
Technology Threat Prediction
 
Modern Day Minutemen
Modern Day MinutemenModern Day Minutemen
Modern Day Minutemen
 
Light Tactical All-terrain Vehicle (LTATV) Crew Augmentation
Light Tactical All-terrain Vehicle (LTATV) Crew AugmentationLight Tactical All-terrain Vehicle (LTATV) Crew Augmentation
Light Tactical All-terrain Vehicle (LTATV) Crew Augmentation
 
Big Crypto for Little Things
Big Crypto for Little ThingsBig Crypto for Little Things
Big Crypto for Little Things
 
Deep Clearance Situational Awareness
Deep Clearance Situational AwarenessDeep Clearance Situational Awareness
Deep Clearance Situational Awareness
 
Digital Overlay
Digital OverlayDigital Overlay
Digital Overlay
 
Drone detection.pptx
Drone detection.pptxDrone detection.pptx
Drone detection.pptx
 
Vital
VitalVital
Vital
 
Team Spyglass Final Presentation
Team Spyglass Final PresentationTeam Spyglass Final Presentation
Team Spyglass Final Presentation
 
Lessons Learned
Lessons LearnedLessons Learned
Lessons Learned
 
H4D Week 12 Final Presentation
H4D Week 12 Final PresentationH4D Week 12 Final Presentation
H4D Week 12 Final Presentation
 
Blue and Gray Berets H4D Final Presentation
Blue and Gray Berets H4D Final PresentationBlue and Gray Berets H4D Final Presentation
Blue and Gray Berets H4D Final Presentation
 

Recently uploaded

The Most Attractive Pune Call Girls Handewadi Road 8250192130 Will You Miss T...
The Most Attractive Pune Call Girls Handewadi Road 8250192130 Will You Miss T...The Most Attractive Pune Call Girls Handewadi Road 8250192130 Will You Miss T...
The Most Attractive Pune Call Girls Handewadi Road 8250192130 Will You Miss T...ranjana rawat
 
(PRIYA) Call Girls Rajgurunagar ( 7001035870 ) HI-Fi Pune Escorts Service
(PRIYA) Call Girls Rajgurunagar ( 7001035870 ) HI-Fi Pune Escorts Service(PRIYA) Call Girls Rajgurunagar ( 7001035870 ) HI-Fi Pune Escorts Service
(PRIYA) Call Girls Rajgurunagar ( 7001035870 ) HI-Fi Pune Escorts Serviceranjana rawat
 
VIP Call Girls Service Bikaner Aishwarya 8250192130 Independent Escort Servic...
VIP Call Girls Service Bikaner Aishwarya 8250192130 Independent Escort Servic...VIP Call Girls Service Bikaner Aishwarya 8250192130 Independent Escort Servic...
VIP Call Girls Service Bikaner Aishwarya 8250192130 Independent Escort Servic...Suhani Kapoor
 
Climate change and occupational safety and health.
Climate change and occupational safety and health.Climate change and occupational safety and health.
Climate change and occupational safety and health.Christina Parmionova
 
Item # 4 - 231 Encino Ave (Significance Only).pdf
Item # 4 - 231 Encino Ave (Significance Only).pdfItem # 4 - 231 Encino Ave (Significance Only).pdf
Item # 4 - 231 Encino Ave (Significance Only).pdfahcitycouncil
 
Global debate on climate change and occupational safety and health.
Global debate on climate change and occupational safety and health.Global debate on climate change and occupational safety and health.
Global debate on climate change and occupational safety and health.Christina Parmionova
 
(TARA) Call Girls Chakan ( 7001035870 ) HI-Fi Pune Escorts Service
(TARA) Call Girls Chakan ( 7001035870 ) HI-Fi Pune Escorts Service(TARA) Call Girls Chakan ( 7001035870 ) HI-Fi Pune Escorts Service
(TARA) Call Girls Chakan ( 7001035870 ) HI-Fi Pune Escorts Serviceranjana rawat
 
Call Girls Chakan Call Me 7737669865 Budget Friendly No Advance Booking
Call Girls Chakan Call Me 7737669865 Budget Friendly No Advance BookingCall Girls Chakan Call Me 7737669865 Budget Friendly No Advance Booking
Call Girls Chakan Call Me 7737669865 Budget Friendly No Advance Bookingroncy bisnoi
 
Fair Trash Reduction - West Hartford, CT
Fair Trash Reduction - West Hartford, CTFair Trash Reduction - West Hartford, CT
Fair Trash Reduction - West Hartford, CTaccounts329278
 
VIP Russian Call Girls in Indore Ishita 💚😋 9256729539 🚀 Indore Escorts
VIP Russian Call Girls in Indore Ishita 💚😋  9256729539 🚀 Indore EscortsVIP Russian Call Girls in Indore Ishita 💚😋  9256729539 🚀 Indore Escorts
VIP Russian Call Girls in Indore Ishita 💚😋 9256729539 🚀 Indore Escortsaditipandeya
 
Building the Commons: Community Archiving & Decentralized Storage
Building the Commons: Community Archiving & Decentralized StorageBuilding the Commons: Community Archiving & Decentralized Storage
Building the Commons: Community Archiving & Decentralized StorageTechSoup
 
Night 7k to 12k Call Girls Service In Navi Mumbai 👉 BOOK NOW 9833363713 👈 ♀️...
Night 7k to 12k  Call Girls Service In Navi Mumbai 👉 BOOK NOW 9833363713 👈 ♀️...Night 7k to 12k  Call Girls Service In Navi Mumbai 👉 BOOK NOW 9833363713 👈 ♀️...
Night 7k to 12k Call Girls Service In Navi Mumbai 👉 BOOK NOW 9833363713 👈 ♀️...aartirawatdelhi
 
VIP High Class Call Girls Amravati Anushka 8250192130 Independent Escort Serv...
VIP High Class Call Girls Amravati Anushka 8250192130 Independent Escort Serv...VIP High Class Call Girls Amravati Anushka 8250192130 Independent Escort Serv...
VIP High Class Call Girls Amravati Anushka 8250192130 Independent Escort Serv...Suhani Kapoor
 
The U.S. Budget and Economic Outlook (Presentation)
The U.S. Budget and Economic Outlook (Presentation)The U.S. Budget and Economic Outlook (Presentation)
The U.S. Budget and Economic Outlook (Presentation)Congressional Budget Office
 
Call On 6297143586 Viman Nagar Call Girls In All Pune 24/7 Provide Call With...
Call On 6297143586  Viman Nagar Call Girls In All Pune 24/7 Provide Call With...Call On 6297143586  Viman Nagar Call Girls In All Pune 24/7 Provide Call With...
Call On 6297143586 Viman Nagar Call Girls In All Pune 24/7 Provide Call With...tanu pandey
 
PPT Item # 4 - 231 Encino Ave (Significance Only)
PPT Item # 4 - 231 Encino Ave (Significance Only)PPT Item # 4 - 231 Encino Ave (Significance Only)
PPT Item # 4 - 231 Encino Ave (Significance Only)ahcitycouncil
 

Recently uploaded (20)

The Most Attractive Pune Call Girls Handewadi Road 8250192130 Will You Miss T...
The Most Attractive Pune Call Girls Handewadi Road 8250192130 Will You Miss T...The Most Attractive Pune Call Girls Handewadi Road 8250192130 Will You Miss T...
The Most Attractive Pune Call Girls Handewadi Road 8250192130 Will You Miss T...
 
How to Save a Place: 12 Tips To Research & Know the Threat
How to Save a Place: 12 Tips To Research & Know the ThreatHow to Save a Place: 12 Tips To Research & Know the Threat
How to Save a Place: 12 Tips To Research & Know the Threat
 
(PRIYA) Call Girls Rajgurunagar ( 7001035870 ) HI-Fi Pune Escorts Service
(PRIYA) Call Girls Rajgurunagar ( 7001035870 ) HI-Fi Pune Escorts Service(PRIYA) Call Girls Rajgurunagar ( 7001035870 ) HI-Fi Pune Escorts Service
(PRIYA) Call Girls Rajgurunagar ( 7001035870 ) HI-Fi Pune Escorts Service
 
VIP Call Girls Service Bikaner Aishwarya 8250192130 Independent Escort Servic...
VIP Call Girls Service Bikaner Aishwarya 8250192130 Independent Escort Servic...VIP Call Girls Service Bikaner Aishwarya 8250192130 Independent Escort Servic...
VIP Call Girls Service Bikaner Aishwarya 8250192130 Independent Escort Servic...
 
Climate change and occupational safety and health.
Climate change and occupational safety and health.Climate change and occupational safety and health.
Climate change and occupational safety and health.
 
Item # 4 - 231 Encino Ave (Significance Only).pdf
Item # 4 - 231 Encino Ave (Significance Only).pdfItem # 4 - 231 Encino Ave (Significance Only).pdf
Item # 4 - 231 Encino Ave (Significance Only).pdf
 
The Federal Budget and Health Care Policy
The Federal Budget and Health Care PolicyThe Federal Budget and Health Care Policy
The Federal Budget and Health Care Policy
 
Global debate on climate change and occupational safety and health.
Global debate on climate change and occupational safety and health.Global debate on climate change and occupational safety and health.
Global debate on climate change and occupational safety and health.
 
(TARA) Call Girls Chakan ( 7001035870 ) HI-Fi Pune Escorts Service
(TARA) Call Girls Chakan ( 7001035870 ) HI-Fi Pune Escorts Service(TARA) Call Girls Chakan ( 7001035870 ) HI-Fi Pune Escorts Service
(TARA) Call Girls Chakan ( 7001035870 ) HI-Fi Pune Escorts Service
 
Delhi Russian Call Girls In Connaught Place ➡️9999965857 India's Finest Model...
Delhi Russian Call Girls In Connaught Place ➡️9999965857 India's Finest Model...Delhi Russian Call Girls In Connaught Place ➡️9999965857 India's Finest Model...
Delhi Russian Call Girls In Connaught Place ➡️9999965857 India's Finest Model...
 
Call Girls Chakan Call Me 7737669865 Budget Friendly No Advance Booking
Call Girls Chakan Call Me 7737669865 Budget Friendly No Advance BookingCall Girls Chakan Call Me 7737669865 Budget Friendly No Advance Booking
Call Girls Chakan Call Me 7737669865 Budget Friendly No Advance Booking
 
Fair Trash Reduction - West Hartford, CT
Fair Trash Reduction - West Hartford, CTFair Trash Reduction - West Hartford, CT
Fair Trash Reduction - West Hartford, CT
 
VIP Russian Call Girls in Indore Ishita 💚😋 9256729539 🚀 Indore Escorts
VIP Russian Call Girls in Indore Ishita 💚😋  9256729539 🚀 Indore EscortsVIP Russian Call Girls in Indore Ishita 💚😋  9256729539 🚀 Indore Escorts
VIP Russian Call Girls in Indore Ishita 💚😋 9256729539 🚀 Indore Escorts
 
Call Girls In Rohini ꧁❤ 🔝 9953056974🔝❤꧂ Escort ServiCe
Call Girls In  Rohini ꧁❤ 🔝 9953056974🔝❤꧂ Escort ServiCeCall Girls In  Rohini ꧁❤ 🔝 9953056974🔝❤꧂ Escort ServiCe
Call Girls In Rohini ꧁❤ 🔝 9953056974🔝❤꧂ Escort ServiCe
 
Building the Commons: Community Archiving & Decentralized Storage
Building the Commons: Community Archiving & Decentralized StorageBuilding the Commons: Community Archiving & Decentralized Storage
Building the Commons: Community Archiving & Decentralized Storage
 
Night 7k to 12k Call Girls Service In Navi Mumbai 👉 BOOK NOW 9833363713 👈 ♀️...
Night 7k to 12k  Call Girls Service In Navi Mumbai 👉 BOOK NOW 9833363713 👈 ♀️...Night 7k to 12k  Call Girls Service In Navi Mumbai 👉 BOOK NOW 9833363713 👈 ♀️...
Night 7k to 12k Call Girls Service In Navi Mumbai 👉 BOOK NOW 9833363713 👈 ♀️...
 
VIP High Class Call Girls Amravati Anushka 8250192130 Independent Escort Serv...
VIP High Class Call Girls Amravati Anushka 8250192130 Independent Escort Serv...VIP High Class Call Girls Amravati Anushka 8250192130 Independent Escort Serv...
VIP High Class Call Girls Amravati Anushka 8250192130 Independent Escort Serv...
 
The U.S. Budget and Economic Outlook (Presentation)
The U.S. Budget and Economic Outlook (Presentation)The U.S. Budget and Economic Outlook (Presentation)
The U.S. Budget and Economic Outlook (Presentation)
 
Call On 6297143586 Viman Nagar Call Girls In All Pune 24/7 Provide Call With...
Call On 6297143586  Viman Nagar Call Girls In All Pune 24/7 Provide Call With...Call On 6297143586  Viman Nagar Call Girls In All Pune 24/7 Provide Call With...
Call On 6297143586 Viman Nagar Call Girls In All Pune 24/7 Provide Call With...
 
PPT Item # 4 - 231 Encino Ave (Significance Only)
PPT Item # 4 - 231 Encino Ave (Significance Only)PPT Item # 4 - 231 Encino Ave (Significance Only)
PPT Item # 4 - 231 Encino Ave (Significance Only)
 

Librarian Mission Model Canvas

  • 2. Will Papper Product B.S. SymSys ‘18 Sean Konz Data Science B.S. EE & CS ‘20 Holden Gibbons Strategy/BD B.S. MS&E ‘19 Anna Zeng Software Engineering B.S. CS ‘18
  • 3. Week 1: Problem: Analyst knowledge retention is limited due to lacking documentation tools Solution: Develop a software tool to automate analyst workflow documentation efforts Week 10: Problem: Useful tools are not being used by analysts because they are not a part of their workflow Solution: Develop an AI- enhanced chatbot interface to integrate various software tools and databases
  • 4. 0 1 2 3 4 5 6 7 8 9 10 Boiling the ocean? Pivotin g ? Develop & Deploy MVP: - Recommended readings - Recommended contacts MVP: - Profile based search interface - Contact database MVP: - Chatbot integration interface Product Mission Fit!
  • 6. Challenges Initial Public Outreach Confidentiality Concerns Weeks 1-3 Weeks 4-6 Weeks 7-8 Weeks 9-10 Internal Culture
  • 7. The Librarian MISSION MODEL CANVAS Week 1 Mission Budget / Costs Fixed: Software development Integration with government services Variable: Onboarding for analysts System / Database integration Mission Achievement Reduce administrative overhead for all analysts regarding role transitions Bring new analysts up to speed on shared knowledge faster Provide a simple, intuitive experience for easy adoption Key Partners Administrators of ispace, rspace, etc. Administrators of Intellipedia Administrators of shared drives Analysis leadership (i.e. DA4) across agencies, to encourage documentation in performance evaluations Branch managers in the DA (i.e. K, our sponsor) ODNI, to implement our tool across agencies Enterprise HR departments, can help onboard new employees and aid with exit processes Key Activities Research processes to allow for background documentation Create & test mocks Integrate multiple knowledge sources Develop a working prototype Value Proposition Reduce onboarding time for new analysts Improve convenience of documentation, so analysts are able to spend less time on documentation Consolidate multiple redundant systems Speed up analyst transition time between projects, reducing administrative burden from HR in the process Ensure a clean knowledge transfer that is less dependent on contractors Reduce constant need to consult former analysts, allowing for complete focus on new assignment Buy-In / Support Sorely needed by our sponsor; Analysts can take a year on a 3-5 year project to get up to speed Sponsor has tried multiple software solutions without success Sponsor’s office has large degree of autonomy to implement prospective solutions Beneficiaries Primary: New analysts can get up to speed on tribal knowledge faster than before Transitioning analysts can focus more on their work by not needing to allot six months to transition materials Current analysts working on projects enjoy a reduced documentation load Secondary: Leadership can see an overall picture of key work produced HR is less burdened by exit procedures Non-sponsor Enterprise: Applicability and potential traction is being found outside of the scope of our sponsor Key Resources Agency personnel (current and retired) for documentation knowledge & in-the-field experiences Non-government contacts experiencing information retention problems R&D team contacts Wide array of industries/job titles available to contact. Deployment Pilot software with one team within one agency Expand to similar agencies and share information based on clearance level Integrate with additional agencies and non-government organizations Value Proposition ● Reduce onboarding time ● Improve convenience ● Consolidate ● Speed up analyst transition time ● Ensure a clean knowledge transfer ● Reduce constant need to consult
  • 8. The Librarian MISSION MODEL CANVAS Week 1 Mission Budget / Costs Fixed: Software development Integration with government services Variable: Onboarding for analysts System / Database integration Mission Achievement Reduce administrative overhead for all analysts regarding role transitions Bring new analysts up to speed on shared knowledge faster Provide a simple, intuitive experience for easy adoption Key Partners Administrators of ispace, rspace, etc. Administrators of Intellipedia Administrators of shared drives Analysis leadership (i.e. DA4) across agencies, to encourage documentation in performance evaluations Branch managers in the DA (i.e. K, our sponsor) ODNI, to implement our tool across agencies Enterprise HR departments, can help onboard new employees and aid with exit processes Key Activities Research processes to allow for background documentation Create & test mocks Integrate multiple knowledge sources Develop a working prototype Value Proposition Reduce onboarding time for new analysts Improve convenience of documentation, so analysts are able to spend less time on documentation Consolidate multiple redundant systems Speed up analyst transition time between projects, reducing administrative burden from HR in the process Ensure a clean knowledge transfer that is less dependent on contractors Reduce constant need to consult former analysts, allowing for complete focus on new assignment Buy-In / Support Sorely needed by our sponsor; Analysts can take a year on a 3-5 year project to get up to speed Sponsor has tried multiple software solutions without success Sponsor’s office has large degree of autonomy to implement prospective solutions Beneficiaries Primary: New analysts can get up to speed on tribal knowledge faster than before Transitioning analysts can focus more on their work by not needing to allot six months to transition materials Current analysts working on projects enjoy a reduced documentation load Secondary: Leadership can see an overall picture of key work produced HR is less burdened by exit procedures Non-sponsor Enterprise: Applicability and potential traction is being found outside of the scope of our sponsor Key Resources Agency personnel (current and retired) for documentation knowledge & in-the-field experiences Non-government contacts experiencing information retention problems R&D team contacts Wide array of industries/job titles available to contact. Deployment Pilot software with one team within one agency Expand to similar agencies and share information based on clearance level Integrate with additional agencies and non-government organizations Beneficiaries Primary: ● New analysts ● Transitioning analysts ● Current analysts Secondary: ● Leadership ● HR
  • 9. The Librarian MISSION MODEL CANVAS Week 1 Mission Budget / Costs Fixed: Software development Integration with government services Variable: Onboarding for analysts System / Database integration Mission Achievement Reduce administrative overhead for all analysts regarding role transitions Bring new analysts up to speed on shared knowledge faster Provide a simple, intuitive experience for easy adoption Key Partners Administrators of ispace, rspace, etc. Administrators of Intellipedia Administrators of shared drives Analysis leadership (i.e. DA4) across agencies, to encourage documentation in performance evaluations Branch managers in the DA (i.e. K, our sponsor) ODNI, to implement our tool across agencies Enterprise HR departments, can help onboard new employees and aid with exit processes Key Activities Research processes to allow for background documentation Create & test mocks Integrate multiple knowledge sources Develop a working prototype Value Proposition Reduce onboarding time for new analysts Improve convenience of documentation, so analysts are able to spend less time on documentation Consolidate multiple redundant systems Speed up analyst transition time between projects, reducing administrative burden from HR in the process Ensure a clean knowledge transfer that is less dependent on contractors Reduce constant need to consult former analysts, allowing for complete focus on new assignment Buy-In / Support Sorely needed by our sponsor; Analysts can take a year on a 3-5 year project to get up to speed Sponsor has tried multiple software solutions without success Sponsor’s office has large degree of autonomy to implement prospective solutions Beneficiaries Primary: New analysts can get up to speed on tribal knowledge faster than before Transitioning analysts can focus more on their work by not needing to allot six months to transition materials Current analysts working on projects enjoy a reduced documentation load Secondary: Leadership can see an overall picture of key work produced HR is less burdened by exit procedures Non-sponsor Enterprise: Applicability and potential traction is being found outside of the scope of our sponsor Key Resources Agency personnel (current and retired) for documentation knowledge & in-the-field experiences Non-government contacts experiencing information retention problems R&D team contacts Wide array of industries/job titles available to contact. Deployment Pilot software with one team within one agency Expand to similar agencies and share information based on clearance level Integrate with additional agencies and non-government organizations Mission Achievement ● Reduce administrative overhead ● Bring new analysts up to speed faster ● Provide a simple, intuitive user experience
  • 10. Boiling the Ocean Weeks 1-3 Weeks 4-6 Weeks 7-8 Weeks 9-10
  • 11. Weeks 1-3 Weeks 4-6 Weeks 7-8 Weeks 9-10 First MVP - Categorize documents - Recommend relevant reports/people - Additional featuresDoes not blend with analyst workflow
  • 12. Current Tools WebClient i-space, r- space, q- space NGT WIRe Library materials TagIt Common Folders Shared drives Weeks 1-3 Weeks 4-6 Weeks 7-8 Weeks 9-10
  • 13. Current Tools Sametime Email WebClient i-space, r- space, q- space NGT WIRe Library materials TagIt Common Folders Shared drives “We used to be able to burn CDs. They replaced it with this steaming pile of shit called Intellipedia.” -Air Force Captain
  • 14. Pivoting Weeks 1-3 Weeks 4-6 Weeks 7-8 Weeks 9-10
  • 15. The Shotgun Approach Document Lists Profiles Contact ListsContact Lists Document Lists Profiles Weeks 1-3 Weeks 4-6 Weeks 7-8 Weeks 9-10
  • 16.
  • 17.
  • 18. Weeks 1-3 Weeks 4-6 Weeks 7-8 Weeks 9-10 “... it’s not part of your daily workflow to put your work somewhere for others to have it.” - CIA Branch Manager
  • 19. Weeks 1-3 Weeks 4-6 Weeks 7-8 Weeks 9-10 Chatbot Interface
  • 20. Chatbot for analysts Dashboard for managers M V P Weeks 1-3 Weeks 4-6 Weeks 7-8 Weeks 9-10
  • 21.
  • 22. “This would be fantastic!” • Branch Manager, CIA Weeks 1-3 Weeks 4-6 Weeks 7-8 Weeks 9-10
  • 23.
  • 24. “When can I get it?” • USAF Operations Director Weeks 1-3 Weeks 4-6 Weeks 7-8 Weeks 9-10
  • 25. Key Partners All-Source analysts, particularly at CIA Administrators of Jive/Intellipedia Administrators of shared drives Branch managers in the DA to add collaboration as a metric Director of the Directorate of Analysis to issue a top-down rule regarding collaboration* Associate Deputy Director and Deputy Director of the Directorate of Digital Innovation* ODNI, to implement our tool across agencies In-Q-Tel, potential incubation and funding opportunities* Key Activities Deeply understand analyst workflow -Search processes -Sharing processes Understand current systems in use -Jive -Intellipedia -Shared Drives -Research procurement practices at sponsor. Value Proposition Get up to speed on shared knowledge quickly Ease transitions to new projects by taking documentation with you Increase ability to accumulate social clout for skills and tenure.(being named a librarian or shares/comments on own page) See processes used to come to a conclusion. Ensure a clean knowledge transfer that is less dependent on contractors Consolidation of databases and tools within and across agencies Buy-In / Support Sorely needed by our sponsor; Analysts can take a year on a 3-5 year project to get up to speed Sponsor has tried multiple software solutions without success Sponsor’s office has large degree of autonomy to implement prospective solutions Beneficiaries Primary: Incoming analysts can get up to speed on tribal knowledge faster than before Transitioning analysts can focus more on their work by not needing to allot six months to transition materials Current analysts working on projects enjoy a reduced documentation load Secondary: Leadership can see an overall picture of key work produced IT has fewer databases to maintain Non-sponsor Enterprise: Applicability and potential traction is being found outside of the scope of our sponsor Key Resources All-source analysts (current and retired) for workflow and documentation knowledge IT Contractors with deep knowledge of current databases Branch managers/leadership to understand culture Wide array of industries/job titles available to contact. Deployment Pilot software with one team of analysts within CIA Expand to multiple teams to test collaboration capabilities. Secure funding from sponsors budget allocation. Potential staged development via In-Q-Tel.* Roll out collaboration software across CIA. The Librarian MISSION MODEL CANVAS Mission Budget / Costs Fixed: Software development Integration with government services Variable: Onboarding for analysts System / Database integration Mission Achievement Bring new analysts up to speed on shared knowledge faster Director of the DA: 20% fewer analysts* Deputy Director and Associate Deputy Director of DDI: Easily access an interoperable system that brings together multiple data sources* Security and Legal: Perceive no additional threats to security* Value Proposition ● Share Knowledge quickly ● Ease transitions ● Accumulate Social Clout ● Understand thought processes ● Ensure a clean knowledge transfer ● Consolidate databases & Tools
  • 26. Key Partners All-Source analysts, particularly in IC Administrators of Jive/Intellipedia Administrators of shared drives Branch managers in the DA to add collaboration as a metric Director of the Directorate of Analysis to issue a top-down rule regarding collaboration* Deputy Director & Associate Deputy Director of the Directorate of Digital Innovation* ODNI, to implement our tool across agencies In-Q-Tel potential incubation and funding opportunities Key Activities Deeply understand analyst effectiveness, including Search & Sharing workflows Understand current systems in use, including Jive, Intellipedia, and Shared Drives Research procurement practices at sponsor Value Proposition Quickly get up to speed on shared knowledge Ease transitions to new projects by taking documentation with you Increase ability to accumulate social clout for skills and tenure Visibility into what analysts are working on, as well as processes used to come to a conclusion Ensure a clean knowledge transfer that is less dependent on contractors Consolidation of databases and tools within and across agencies Buy-In / Support Sorely needed by our sponsor; Analysts can take a year on a 3-5 year project to get up to speed Sponsor has tried multiple software solutions without success Sponsor’s office has large degree of autonomy to implement prospective solutions Beneficiaries Primary: Analysts can access shared databases faster than before Branch managers can get visibility into what their analysts are working on DA Leadership can have visibility into metrics on the performance of the whole organization Secondary: Contractors see increased usage for under-utilized applications Security/Legal now has a single audit log, rather than audit logs across multiple databases Non-sponsor Enterprise: Applicability and potential traction is being found outside of the scope of our sponsor Key Resources All-source analysts (current and retired) for workflow and documentation knowledge IT Contractors with deep knowledge of current databases Branch managers and leadership to understand culture Wide array of analysts and related roles in industry available to contact Deployment Pilot software with one team of analysts within CIA Expand to multiple teams to test collaboration capabilities Secure funding from sponsors budget allocation Potential staged development via In-Q-Tel Roll out collaboration software across CIA The Librarian MISSION MODEL CANVAS Week 10 Mission Budget / Costs Fixed: Software development Integration with government services Variable: System / Database integration Mission Achievement Director of the DA: Requires 20% fewer analysts All-source Analysts: Get up to speed on new topic areas within one workday Deputy Director and Associate Deputy Director of DDI: Easily access an interoperable system that brings together multiple data sources Security and Legal: Perceive no additional threats to security
  • 27.
  • 28.
  • 29. In other words… (3 year) Option 1. Reduce HR budget, level productivity. • Work done before Librarian: 100 units • Work done after Librarian: 100 units • HR budget before Librarian: $100 • HR Budget after Librarian: $67 (-33%) Option 2. Same HR budget, expanded productivity. • Work done before Librarian: 100 units • Work done after Librarian: 150 units (+50%) • HR budget before Librarian: $100 • HR Budget after Librarian: $100
  • 30. Develop and Deploy Weeks 1-3 Weeks 4-6 Weeks 7-8 Weeks 9-10
  • 31. Internal Readiness Level Weeks 1-3 Weeks 4-6 Weeks 7-8 Weeks 9-10 MVP testing and feature exploration
  • 32. Conversational Chatbot Social Nudges Group Common Folders Document Tagging Current Feature Exploration Product Feature Diagram - Personal database - Group contact database - Recommended Readings - Moderated subject Lists - Database Integrations - Suggested actions - Relevant Tool Integration - Workflow monitoring
  • 33. What’s next? • WE ARE ALL CONTINUING • Seeking grants and funding • Continuing to build-out the Librarian V 1.0 Weeks 1-3 Weeks 4-6 Weeks 7-8 Weeks 9-10
  • 34. Acknowledgements Sponsor: Thank you for the amazing support! Mentor: Tom Bedecarre TA Mentor: Darren Hau Teaching Team: Steve Blank, Joe Felter, Pete Newell, Steve Weinstein TA Team, H4D mentors, everyone we interviewed
  • 35. Signing Off! Come find us at thelibrarian@lists.stanford.edu 37.4241* N 122.1661* W
  • 36. What key features should we focus on implementing? Weeks 1-3 Weeks 4-6 Weeks 7-8 Weeks 9-10
  • 37. Key Features • Social nudges • Tagging • Sorting Weeks 1-3 Weeks 4-6 Weeks 7-8 Weeks 9-10
  • 38. Social Nudges • Social proof • Following colleagues • No need to change incentives Weeks 1-3 Weeks 4-6 Weeks 7-8 Weeks 9-10
  • 39. Tagging and Sorting • Make organizational tools front and center • Combination of push and pull mechanisms Weeks 1-3 Weeks 4-6 Weeks 7-8 Weeks 9-10
  • 40. What will the Intelligence Community look like with The Librarian? • Track down information more quickly than before • Ensure that time is not wasted at critical moments. Weeks 1-3 Weeks 4-6 Weeks 7-8 Weeks 9-10
  • 41. What will the world look like with The Librarian? • Our ideal is similar to The Librarian in Snow Crash: We want to allow you to learn from a bot that contains the sum of your organization’s knowledge. Weeks 1-3 Weeks 4-6 Weeks 7-8 Weeks 9-10
  • 42. Current Tools Sametime Email WebClient i-space, r- space, q- space NGT WIRe Library materials TagIt Common Folders Shared drives “We used to be able to burn CDs. They replaced it with this steaming pile of shit called Intellipedia.” - Air Force Captain
  • 43. Beneficiary Archetype: Incoming Analyst ● Always the “new guy” ● Reads heavily for work ● 12 months of training (or more) ● 1 Year of onsite ramp-up time “before you know what you’re talking about or, more importantly, [before] anyone listens.”
  • 44. Beneficiary Archetype: Outgoing Analyst 25-35 year-old intelligent / high aptitude individual, leaving from a 3-5 year assignment ● Reads heavily for work ● May be resistant to digital tools ● Current equilibrium: 6 Months of spool- down, documentation time
  • 47. The Librarian Knowledge transfer for analysts Creating a single chatbot interface for the many databases and software platforms will allow us to improve knowledge transfer for new analysts within the intelligence community. Will Papper Sean Konz Anna Zeng Holden Gibbons Product / Software Engineering Data Science Software Engineering Strategy
  • 48. The Librarian Team Will Papper Sean Konz Anna Zeng Holden Gibbons Academic Program B.S. SymSys 2018 B.S. EE & CS 2020 B.S. CS 2018 B.S. MS&E 2019 LinkedIn https://www.linkedin.com/in/wpapper/ https://www.linkedin.com/in/sean- k-19a327b2/?locale= https://www.linkedin.com/in/ann a-z-11774096 https://www.linkedin.com/in/j-holden- gibbons-669296a3/ Subject Matter Expert? Details available No (Computer Vision / Machine Learning) No (Software Engineering) Yes, Veteran Role Product Data Science Hacker Hustler Experience Solving a Problem that Seemed Impossible - In one month, developed a proof-of-concept social network for a P2P mesh networking platform - Worked with multiple federal agencies - Developed Novel approach to eye tracking interfacing in a populated consumer space - Parallel processing experience - Pivoted the direction of three Microsoft products - Coordinated the first EE makerspace in Stanford - Founded non-profit urban farming initiative in the inner city and a small real estate development firm.
  • 50. What will the world look like with The Librarian? • As an analyst or researcher, you had to pull from dozens of databases (both internal and external). You would not necessarily know where to look for information, and could miss vital info. • With The Librarian, you can query every database simultaneously and never miss important information.
  • 51. Where did we start?
  • 52. Beneficiary Archetype: Incoming Analyst 22-30 year-old intelligent / high aptitude individual, coming for a 3-5 year assignment ● Values mission recognition over pay incentives ● Strong sense of patriotism/service ● Capable of being highly interchangeable ● Often operate as ‘snowflakes’, covering down on issues as need be ● Reads heavily for work ● 12 months of training (or more) ● 1 Year of onsite ramp-up time “before you know what you’re talking about or, more importantly, [before] anyone listens.”
  • 53. Beneficiary Archetype: Outgoing Analyst 25-35 year-old intelligent / high aptitude individual, leaving from a 3-5 year assignment ● Values mission recognition over pay incentives ● Strong sense of patriotism/service ● Capable of being highly interchangeable ● Often operate as ‘snowflakes’, covering down on issues as need be ● Reads heavily for work ● May be resistant to digital tools ● Current equilibrium: 6 Months of spool- down, documentation time
  • 54. Beneficiary Archetype: Agency Leadership 30-45 year-old intelligent / high aptitude individual ● Strong sense of patriotism/service ○ Sacrificed family time & higher income elsewhere ● Served multiple rotations. some as analysts ● Has a good sense of the agency’s ecosphere of digital tools ● Perceives organizational culture & limitations ● Mission-focused, but aware of the need for tools to improve mission effectiveness
  • 55. Customer Workflow: Life of an All-Source Analyst NEW ANALYST (0-1 year) OLD ANALYST (3+ years) read write review publish document analyst assigned to a new area of expertise analyst given history book and loads of background reading revise scratch work in i-space or equivalent executive briefing Summary reports Intellipediai-space or eq. Reports & briefings raw intel reports boolean search feed Project Lifecycle
  • 56. Customer Workflow: Life of an All-Source Analyst NEW ANALYST (0-1 year) OLD ANALYST (3+ years) read write review publish document analyst assigned to a new area of expertise analyst given history book and loads of background reading revise scratch work in i-space or equivalent executive briefing Summary reports Intellipediai-space or eq. Reports & briefings raw intel reports boolean search feed Project Lifecycle AUTOMATE
  • 57. Customer Workflow: Life of an All-Source Analyst NEW ANALYST (0-1 year) OLD ANALYST (3+ years) read write review publish & document analyst assigned to a new area of expertise analyst given history book and loads of background reading revise scratch work in i-space or equivalent executive briefing Summary reports Intellipedia i-space or eq. Reports & briefings raw intel reports boolean search feed Project Lifecycle
  • 60. Directorate office creates requirements Information Technology Enterprises (in DDI) searches for solutions If contractors are addressing the requirements, they are looped in If an existing solution cannot be repurposed, ITE looks in industry LARGE COMPANY: Digital Futures brings it into the internal network STARTUP: In-Q-Tel provides milestone-based funding
  • 62. Information Technology Enterprises (in DDI) searches for solutions If contractors are addressing the requirements, they are looped in If an existing solution cannot be repurposed, ITE looks in industry LARGE COMPANY: Digital Futures brings it into the internal network STARTUP: In-Q-Tel provides milestone-based funding Directorate leadership requests solution from DDI leadership
  • 63. Directorate leadership requests solution from DDI leadership DDI leadership & ITE create requirements If contractors are addressing the requirements, they are looped in If an existing solution cannot be repurposed, ITE looks in industry LARGE COMPANY: Digital Futures brings it into the internal network STARTUP: DDI leadership recommends In-Q-Tel evaluate product
  • 64. Directorate leadership requests solution from DDI leadership DDI leadership & ITE create requirements If contractors are addressing the requirements, they are looped in If an existing solution cannot be repurposed, ITE looks in industry LARGE COMPANY: Digital Futures brings it into the internal network STARTUP: DDI leadership recommends In-Q-Tel evaluate product Implementation & beta testing through ITE; DDI leadership removes roadblocks
  • 65. What individual problems are we solving?
  • 66. Key Partners All-Source analysts, particularly in IC Administrators of Jive/Intellipedia Administrators of shared drives Branch managers in the DA to add collaboration as a metric Director of the Directorate of Analysis to issue a top-down rule regarding collaboration* Deputy Director & Associate Deputy Director of the Directorate of Digital Innovation* ODNI, to implement our tool across agencies In-Q-Tel potential incubation and funding opportunities Key Activities Deeply understand analyst effectiveness, including Search & Sharing workflows Understand current systems in use, including Jive, Intellipedia, and Shared Drives Research procurement practices at sponsor Value Proposition Quickly get up to speed on shared knowledge Ease transitions to new projects by taking documentation with you Increase ability to accumulate social clout for skills and tenure Visibility into what analysts are working on, as well as processes used to come to a conclusion Ensure a clean knowledge transfer that is less dependent on contractors Consolidation of databases and tools within and across agencies Buy-In / Support Sorely needed by our sponsor; Analysts can take a year on a 3-5 year project to get up to speed Sponsor has tried multiple software solutions without success Sponsor’s office has large degree of autonomy to implement prospective solutions Beneficiaries Primary: Analysts can access shared databases faster than before Branch managers can get visibility into what their analysts are working on DA Leadership can have visibility into metrics on the performance of the whole organization Secondary: Contractors see increased usage for under-utilized applications Security/Legal now has a single audit log, rather than audit logs across multiple databases Non-sponsor Enterprise: Applicability and potential traction is being found outside of the scope of our sponsor Key Resources All-source analysts (current and retired) for workflow and documentation knowledge IT Contractors with deep knowledge of current databases Branch managers and leadership to understand culture Wide array of analysts and related roles in industry available to contact Deployment Pilot software with one team of analysts within CIA Expand to multiple teams to test collaboration capabilities Secure funding from sponsors budget allocation Potential staged development via In-Q-Tel Roll out collaboration software across CIA The Librarian MISSION MODEL CANVAS Mission Budget / Costs Fixed: Software development Integration with government services Variable: System / Database integration Mission Achievement Director of the DA: Requires 20% fewer analysts All-source Analysts: Get up to speed on new topic areas within one workday Deputy Director and Associate Deputy Director of DDI: Easily access an interoperable system that brings together multiple data sources Security and Legal: Perceive no additional threats to security
  • 67. The Librarian: Value Proposition Canvas Products & Services Knowledge management for analysts Bring past documentation and work with you to new projects Tool to reference past work and recommendations from former analysts (no need to call) Customer Jobs Get up to speed on a new body of knowledge Analysts lose access to prior work when switching roles Existing documentation tools are cumbersome and scattered across systems Incoming All-Source Analysts Gains Gain Creators Pain Relievers Locate documentation and past work in one place Track down knowledge of analysts who have retired or changed projects without needing to call Put prior work into an IC-wide system rather than team shared drives or email accounts Documentation is handled by one software solution rather than multiple tools with different processes Pains
  • 68. The Librarian: Value Proposition Canvas Products & Services Knowledge management for analysts Documentation is handled transparently in the background Creates a clear list of materials to be transitioned Customer Jobs Pass along knowledge to remaining team members No established procedures for transitioning out Two weeks is not enough time to get materials together Process is chaotic, informal, and ad-hoc Outgoing All-Source Analysts Gains Gain Creators Pain Relievers Documentation is handled in the background over the course of a project, reducing transition time Adds formal processes to transition materials Move documentation from an active burden into a background process Documentation is handled by one software solution rather than multiple tools with different processes Pains
  • 69. The Librarian: Value Proposition Canvas Products & Services Knowledge management for analysts Research processes are stored in an organized application rather than shared drives Branch manager interface: present important metrics on analyst activities Customer Jobs Synthesize documents from analysts for decision-making Finished analyses are visible, but for controversial claims it is difficult to see a thought process Focus on publications creates a lack of insight into daily work Branch Managers Gains Gain Creators Pain Relievers See what documents went into a finished product Gain insights for what analysts are doing (beyond publications) Help individual analysts based on actions Report’s interpretation of source material is indicated clearly Documentation is handled by one software solution rather than multiple tools with different processes Pains
  • 70. The Librarian: Value Proposition Canvas Products & Services Knowledge management for analysts Librarian helps provide metrics across the entire organization, when there was previously no visibility Customer Jobs Ensure that analytic resources are directed most efficiently Too many offices to keep track of in one spot Need to get information from briefings, rarely able to dive deeper Director of the Directorate of Analysis Gains Gain Creators Pain Relievers Needs to direct analysts to topic areas focused on by policymakers Dashboard allows Director to monitor high-level behavior of offices within the organization Can independently browse dashboard, no need to rely on branch managers completely Pains
  • 71. The Librarian: Value Proposition Canvas Products & Services Knowledge management for analysts Contractors employed by org to serve as implementation coordinators = $$ Integration with our system for analyst use Customer Jobs Deploy solutions that meet requirementsHave multiple systems that they need to integrate with, which slows down progress Already have systems that they own solving similar problems, and are protective of their turf Contractors might not want to put in the effort to build APIs/help us integrate Contractors Gains Gain Creators Pain Relievers Benefit from contracts due to service as implementation liaisons Increased contractor tool use through system integrations Managing a single profile could make integration with other software easier If APIs present, we can integrate without them Pains
  • 72. The Librarian: Value Proposition Canvas Products & Services Knowledge management for analysts Security Interface Analyst action signaling Customer Jobs Keep information from getting compromisedHuge database of how analysts think/their current progress that is a giant target for foreign intelligence services Another system to worry about secure access Security / Legal Gains Gain Creators Pain Relievers Immediate insight into the actions of analysts using signaling mechanisms Procedure is already in place to implement this, since this type of database; precedent exists Database APIs already exist We don’t need data copies, just links Pains
  • 73. Cultural Resistance • Workflow • If something isn’t in the day-to-day expectations, it won’t be used • Need-to-know • Your information is yours, and you shouldn’t share it • Silos • Incentives
  • 74. Incentives: How are analysts promoted? • Analysts are evaluated on papers produced and the decisions that these papers affect • Collaboration is not a great help in the promotion process, so it is not common practice • Who can change this? • Branch Managers: “If it’s not shared, I’m not counting it.” • Directorate of Analysis Leadership: Top-down requirement
  • 75. Where are we now?
  • 76. MVP Chatbot for analysts Dashboard for branch managers/directors
  • 77. MVP - Analysts We are introducing a chatbot interface to connect new all-source analysts with the information they need from existing databases in the intelligence community. Analysts will be able to reach crucial background information, identify subject matter experts, retrieve existing reports, etc. in no time at all.
  • 78. MVP - Branch Managers Currently, branch managers do not have much insight into reports that are in progress. Documents need to actively be moved into shared drives, which only provides very granular insight. By reporting aggregate queries from analysts to branch managers, branch managers can most efficiently help analysts.
  • 79.
  • 80. What are our next steps?
  • 81. GET KEEP GROW GET Awareness: - MVP testing - Present underutilized tools - Integrate with org chat client Interest: - More robust document discovery - Ease of contacting SMEs Consideration: - How integration impacts workflow - If discovering different document pools (recommendations) would really be beneficial Purchase: - Procurement process KEEP - Quantify usage of databases with/without tool - Additional tool integrations based on user desires - Leadership/user feedback provided on end intel product quality - Ensure budget for system upgrades (re: compatibility) - Tool should learn over time - Better suggestions, relevant documents based on suggestion doc usage GROW Referrals: - Introduce the tool into other agencies (other IC, DoD, DIUx) - Dual-use applications: additional AI training opportunities Cross-Sell: - Develop machine learning to the point of making suggestions Next-Sell: - Add additional internal tool integrations Up-Sell: - Introduce cross-agency information exchange / integration
  • 82. Mission Achievement - Product • Do people come back to this tool after trying it the first two times? (Focus is on abandonment or lack thereof) • How many times do people consult the tool per day? • Does this tool reduce the number of times analysts access other databases? • How many database integrations do we have? • Is the tool getting smarter?
  • 83. Mission Achievement - Stakeholders Party Benefit Director of the Directorate of Analysis Requires 20% fewer analysts All-source Analysts Get up to speed on new topic areas within one workday, increase breadth of relevant information presented to analysts Deputy Director and Associate Deputy Director of DDI Easily access an interoperable system that brings together multiple data sources Security and Legal Perceive no additional threats to safety
  • 84. Work Distribution Will - Product: Ensure that the MVP we’re building matches customer needs, handle User Experience Sean - Data Science: Data Science on Database metrics, NLP framework Anna - Hacker: Database integrations, Software Engineering, User Experience Holden - Hustler: Find new customers and channels, maintain contact with current beneficiaries
  • 85. Next Steps • Start building a chatbot! • Integrate with an open source database API • Begin to train the chatbot on open source data • Test interactive demo with analysts • Test dashboard with branch managers
  • 86. We want to hear from you! Email: thelibrarian@lists.stanford.edu
  • 87.
  • 88.
  • 89.
  • 90.
  • 91.
  • 92.
  • 93.
  • 94.
  • 95.
  • 96.
  • 97.
  • 98.
  • 99.
  • 100.
  • 101.
  • 102.
  • 103.
  • 104.
  • 105. Buy-In and Support Analyst: Easier to get up to speed on new topics, front-ends are consolidated to allow for simple querying. Branch Managers: Stay current on the topics that analysts are working on. Have more insight into ways to provide help. Contractors: Improve usage of applications that may not fit into an analyst’s workflow, increasing chances of renewal. Security: Easily detect anomalous activity, since the activity will be in one central audit log.
  • 106. Expectation Chatbot interface is new for employees and will require a shift in mindset. Result Asking other analysts questions over IM is very common. Analysts can now bother a bot instead of bothering their colleagues.
  • 107. Expectation The Librarian would be useful for easily constructing lists of tags Result There is a database for this, but it is not used much. The Librarian could act as a front- end.
  • 108. Expectation There are a few crucial databases to pull from, and people consult those heavily. Result The Library has hundreds of databases, but they are underutilized. We can create an interface for many databases that are not pulled from.
  • 109. Expectation Lists of relevant documents do not exist. The Librarian could create these lists seamlessly. Result There are lists of documents that exist in NGT, but analysts do not realize that this feature exists. Furthermore, it is not in the workflow that analysts follow. We can add a front-end for these lists, allowing them to be queried easily.
  • 111. GET KEEP GROW GET Awareness: - MVP testing - Present underutilized tools - Integrate with org chat client Interest: - More robust document discovery - Ease of contacting SMEs Consideration: - How integration impacts workflow - If discovering different document pools (recommendations) would really be beneficial Purchase: - Procurement process (Appendix) KEEP - Quantify usage of databases with/without tool - Additional tool integrations based on user desires - Leadership/user feedback provided on end intel product quality - Ensure budget for system upgrades (re: compatibility) - Tool should learn over time - Better suggestions, relevant documents based on suggestion doc usage GROW Referrals: - Introduce the tool into other agencies (other IC, DoD, DIUx) - Dual-use applications: additional AI training opportunities Cross-Sell: - Develop machine learning to the point of making suggestions Next-Sell: - Add additional internal tool integrations Up-Sell: - Introduce cross-agency information exchange / integration
  • 112. Mission Achievement - Product • Do people come back to this tool after trying it the first two times? (Focus is on abandonment or lack thereof) • How many times do people consult the tool per day? • Does this tool reduce the number of times analysts access other databases? • How many database integrations do we have? • Is the tool getting smarter?
  • 113. Mission Achievement - Stakeholders Party Effect Director of the Directorate of Analysis Requires 20% fewer analysts All-source Analysts Get up to speed on new topic areas within one workday, increase breadth of relevant information presented to analysts Deputy Director and Associate Deputy Director of DDI Easily access an interoperable system that brings together multiple data sources Security and Legal Perceive no additional threats to safety
  • 114. Next Steps • Interview more analysts to understand different types of workflows • Drill down on more specific use cases for the chatbot • Understand which areas have the biggest efficiency improvements • Understand which database APIs exist already • Understand the work required to add new database integrations
  • 115. Beneficiary Archetype: Incoming Analyst Male/Female:22-30 years old -Intelligent/high aptitude individuals. -Values mission recognition over pay incentives. -Strong sense of patriotism/service. -Though capable of being highly interchangeable, often operate as ‘snowflakes’, covering down on issues as need be. -Reads heavily for work. -12 month training but can be longer with follow along training opportunities. - ~3-5 year assignments. -1 Year of spool-up time. “Before you know what you’re talking about or more importantly, anyone listens”. -K
  • 116. Beneficiary Archetype: Outgoing Analysts Male/Female:25-35 -Intelligent/high aptitude individuals. -Values mission recognition over pay incentives. -Strong sense of patriotism/service. -Though capable of being highly interchangeable, often operate as ‘snowflakes’, covering down on issues as need be. -Reads heavily for work. -12 month training but can be longer with follow along training opportunities. - ~3-5 year assignments. -May or may not be hoarder/e-allergic. -Current equilibrium: 6 Months of spool-down, documentation time.
  • 117. The Librarian: Value Proposition Canvas Products & Services Knowledge management for analysts - Key documents used by an analyst are included in a single system -Research processes are stored in an organized application rather than shared drives Customer Jobs Synthesize documents from analysts for decision-making -Finished analyses are visible, but for controversial claims it is difficult to see a thought process -Focus on publications creates a lack of insight into daily work Branch Managers Gains Pains Gain Creators Pain Relievers - See what documents went into a finished product - Get more insight into what analysts are doing (beyond publications) - Report’s interpretation of source material is indicated clearly -Documentation is handled by one software solution rather than multiple tools with different processes
  • 118. Beneficiary Archetype: Team/Mis. Cntr. Managers Male/Female: 28-40 -Responsible for bringing new analysts up to speed. -Intelligent/high aptitude individuals. -Strong sense of patriotism/service manifesting in potential sacrificed family time/higher income elsewhere. -Most likely served multiple mission centers 3+ years each as an analyst. -Possesses good sense of what the ecosphere of digital tools consists of within the agency. -Good sense of organizational culture/limitations. -Mission focused on day-to-day but aware of the need for tools to be revamped to improve mission effectiveness.
  • 119. The Librarian: Value Proposition Canvas Products & Services Knowledge management for analysts Librarian lists help maintain information, contacts, glossary, prominent people and thus reduce time spent finding intel and more time producing it. Customer Jobs Manage cadre of analysts and their tools. -Doesn’t have enough analysts or at least the current ones aren’t efficient enough due to knowledge being lost at lower levels creating redundant and less than fully informed conclusions.. Budget and manpower she or he is afforded is outside of their control. . Director of the Directorate of Analysis Gains Pains Gain Creators Pain Relievers Able to either dig deeper in current areas or potentially spread current labor footprint into more/nuanced issues for better understanding of the world. Automated systems for analysts allow for less variation in human behavior leading to more uniformity and more efficiency in analysis.
  • 120. Beneficiary Archetype: Director of Directorate of Analysis -Female/Male: 40-50 -Personally responsible for allocation of analytical manpower across array of needs. -Ultimately responsible to Director of the agency for quality of analytical product. -Most likely served in several mission centers and well versed on past and current culture. (not much changes) -Has the authority to mandate workflow procedures such as software utilization. -Highly motivated and often has best in mind but has many boiling pots to tend.
  • 121. The Librarian: Value Proposition Canvas Products & Services Knowledge management for analysts -Single search interface would reduce proliferation of databases Customer Jobs Implement new tools that are relevant to the Agency -Do not have proper API integrations for existing databases Deputy Director and Associate Deputy Director of DDI Gains Pains Gain Creators Pain Relievers -Would be able to consolidate multiple databases and eliminate failed solutions -Focus on interoperability could reduce cost to add new solutions on top of our system
  • 122. The Librarian: Value Proposition Canvas Products & Services Knowledge management for analysts -Contractors employed by org to serve as implementation coordinators = $$ Customer Jobs Deploy solutions that meet requirements -Have multiple systems that they need to integrate with, which slows down progress -Already have systems that they own solving similar problems, and are protective of their turf Contractors Gains Pains Gain Creators Pain Relievers -Benefit from contracts due to service as implementation liaisons -A single profile could make integration with other software easier
  • 123. The Librarian: Value Proposition Canvas Products & Services Knowledge management for analysts Customer Jobs Keep information from getting compromised -Huge database of how analysts think/their current progress that is a giant target for foreign intelligence services Security/Legal Gains Pains Gain Creators Pain Relievers -Procedure is already in place to implement this, since this type of database. Precedent exists.
  • 124. Day in the Life Workflow of our primary beneficiaries (analysts) 7:30AM- Boolean Search yielding 100s or 1000s of hits. 8:00AM- Triage/speed-read for use in daily brief, construct notes in Word. 8:30AM- Team daily brief, trending items, taskings, etc. 9-Noon- Follow up with more detail. Where our product fits in
  • 125. Cultural Resistance • Workflow • If something isn’t in the day-to-day expectations, it won’t be used • Need-to-know • Your information is yours, and you shouldn’t share it
  • 126. How do people get promoted? • Analysts are measured on the papers you produce and the decisions that your papers affect • Collaboration is not part of the promotion process, so it is not done • Who can do this? • Branch Managers - “If it’s not shared, I’m not counting it.” • Directorate of Analysis Leadership - Top-down rule
  • 127. “We got an automated water cooler… [but no one] is responsible for maintaining it. It’s about accountability.” - Chris, Pathfinder PM, on Intellipedia … what if we made documentation part of an analyst’s performance review?
  • 129. Procurement Life cycle 1.Market/internal research for tools serving a given purpose a.Contractors generally involved in conducting this process 2. Budget analysis: Is it cheaper to build a solution, or to buy? Which is more efficient? 3.If buying, Numerous routes: In-Q-Tel, Digital Futures within DDI
  • 130. How Procurement Works in Theory (6-12mo) 1. An office within a directorate, such as the Office of Advanced Analytics within the Directorate of Analysis, produces a set of requirements. 2. Information Technology Enterprises within the Directorate of Digital Innovation searches for solutions to meet those requirements. If the requirements are currently being addressed in some form by a contractor already, the contractor is brought into the loop as well. 3. If an existing solution cannot be modified, then a commercial solution is
  • 131. How Procurement Works in Theory (6-12mo) 4. If the commercial solution is from a large company, Digital Futures within the Directorate of Digital Innovation will work to bring the company onto the internal network. 5. If the commercial solution is from a startup, In-Q-Tel will provide milestone-based funding to help the startup get the solution onto the internal network. To beta test the software, a work program is initiated that acts as a limited rollout of the software.
  • 132. How Procurement Works in Practice (3mo) 1. Someone high up in a relevant office (for example, the Office of Advanced Analytics within the Directorate of Analysis) expresses strong support for a solution to someone high up in the Directorate of Digital Innovation (Associate Deputy Director or Deputy Director) 2. Deputy Director of DDI works with Information Technology Enterprises to determine requirements. May also recommend that In-Q-Tel evaluate the solution if the company is a startup.
  • 133. Procurement Life cycle con’t Procurement through DDI: 1.Team hired by supplier to handle implementation a.Government team holds necessary clearances, experience etc. b.Atypical: Government creates a team specifically for implementation of particular tools 2.Requirements set by IC (in our case, DDI) 3.Implementation team works with both org and supplier to meet requirements a.Will ask supplier to alter tool slightly to fit security requirements b.Project sponsors in leadership make the connections to push
  • 134. Procurement Life cycle con’t Procurement through In-Q-Tel: Younger companies in need of seed funding targeted 1.Companies with a desired tool are found through market research process 2.Companies are provided funding tied to milestones. Milestones help them progress toward getting their software ready for an IC environment. 3.When the product is ready for use on classified networks, a work program is initiated to beta test the software.
  • 135. Scope and implementation takeaways. -Comparable to mid-sized company in size of workforce that will be utilizing our product -Can be most easily implemented via one ‘top down’ directive from Director of Directorate of Analysis. -Can also be implemented by individual buy-in of all lower level managers. This is not a favorable path.
  • 136. Key Takeaways “[Having leadership support] is probably one of the top five things you need to be successful in getting a solution through” - Interviewee 7, Integration Contractor Having org leadership invested in the solution is a key to successful integration
  • 137. Key Takeaways “Our Government Chief knew that this was something that was needed, so she took it upon herself to make it happen” -Interviewee 7, Integration Contractor If a solution is needed, leadership will recognize it is needed, and will collaborate to make it a reality
  • 138. MVP Development: Three Concepts • Profile based: search a subject -> get back ranked profiles -> view a timeline • Document List • People List
  • 139. Document Based MVP • Topics linked to Subject Matter Experts • Subject matter experts have suggested readings on their topic • Results return documents authored by subject expert
  • 140. Search Interface MVP -Relevancy based on degrees of separation - “Suggestion” panel features subject matter experts and top documents - People linked to contact information -Documents linked to databases
  • 141. Profile Based MVP • Provides information on analysts and what they have viewed • Acts as a landing page from the document and search solutions • Could either integrate with or replace existing profile solutions • Could pull data from HR databases to auto-populate profiles
  • 142. Expectation If an Intel product is available, an author’s contact information can be tracked down from their group Result If the contact information is not included in an intel product, consumers of that product cannot contact the authors Contact information can be grabbed using auditing apparatus
  • 143. Expectation Analysts will find it helpful to share information about what they are working on automatically. Result Teams may not find it preferable to automatically share potentially sensitive information. One team does manually collect a list of key topics the team focuses on, but distributes it via word document / email to specific collaborators.
  • 144. Key Takeaways - MVP received support (with caveats) MVP would be “super helpful” With caveats: “Yes. As long as it was kept up to date and could highlight the major themes, at the very least it would be a starting point for a senior analyst to talk to a starting analyst.” -Interviewee #3, manages analysts and was an analyst
  • 145. Can we do this? Notes on how a systems like the ones we’re suggesting would fit into the current information systems used in the IC -Can we link directly to documents? -Can we link directly to people? -Can we only link documents to citations and users need to go get the actual doc on a different database? (fatal) -Would we be able to display search results from docs already in the IC databases?
  • 146. Next Steps • Evaluate whether automating clustering and tagging is feasible • Evaluate whether a top-down rule will be followed • Bring together multiple MVPs through interview tests • Gain access to think tank data sets (Used to build a subject recognition and recommendation system)