Fundamental of intelligence (decision-support) analysis -- rarely done well by anyone.

1. ®
All-Source Analysis:
Making Magic
Robert David Steele
Intelligence Coach
bear@oss.net

2. ®
Big Change #1: Multi-Cultural Bottom-Up
TIME
IMPACT
SHORT
TIME
IMPACT
LONG
SINGLE-CULTURE
SINGLE-ORGANIZATION
EQUITIES
MULTI-CULTURAL &
TRANS-NATIONAL
EQUITIES
LEADERS
DECIDE
PEOPLE
DECIDE
TOP-DOWN
COMMAND & CONTROL
SECRET
SOURCES & METHODS
OPEN
SOURCES & METHODS
BOTTOM-UP
INFORMATION-SHARING
OBVIOUS DETAIL
OBSCURE DETAIL
OLD
NEW

3. ®
Big Change #2: The Really Big Gap
The New
Intelligence
Gap: the
difference
between what
you can know
and what you
can use!
I
N
F
O
R
M
A
T
I
O
N
TIME
Actionable
Intelligence
Available
Information
Open source information is more complex than secrets…

4. ®
Big Change #3:
Threat Changes with Levels of Analysis
Over time and space
Channels & Borders
Of strategic value
Quantities & Distribution
Internally available for use
Volatility of sectors
Training & Maintenance
Mobility implications
Cohesion & Effectiveness
STRATEGIC
Integrated
Application
OPERATIONAL
Selection of
Time and Place
TACTICAL
Application of
Finite Resources
TECHNICAL
Isolated
Capabilities
Military Sustainability
Geographic Location
Civil Allies
Military Systems One by One
Climate Manipulation
Civil Power, Transport,
Communications,
Finance
Military Availability
Geographic Resources
Civil Stability
Military Reliability
Geographic Terrain
Civil Psychology
Military Lethality
Geographic Atmosphere
Civil Infrastructure

5. ®
Who is your historian?
• History is vital to the
“framing” of a
problem.
• History from different
national & cultural
perspectives helps
“map” the issues and
the differences.
• History matters!

6. ®
Who is your librarian?
• Spies and soldiers are
not good at doing
research.
• Librarians, especially
the “new” librarians
that focus on external
discovery instead of
internal archiving, are
essential to your
success.

7. ®
Internet Competency
No longer a toy--now a serious source
• “All-source” means all
sources--the Internet is
now a major source
• Search engines vary.
• Find images and maps.
• Find experts and groups.
• Limit to 1-hour efforts.
• Need an Internet
specialist on call.

8. ®
Commercial Online Competency
100X more important than Internet
• Factiva.com
• DIALOG
• European Services
• Value-added is
enormous--reputable
sources, editorial
selection, structured
storage and retrieval
• Need a specialist.
Italian intelligence chief puts Al-
Qa'idah's assets at 5bn dollars
BBC Monitoring, 05/16/2002, 142
words.
CIA 'Probably' Helps Italian
Subversive Groups.
Xinhua News Agency, 03/20/2001,
268 words.

9. ®
Gray Literature Competency
Limited edition, must know to ask
• Pre-prints, technical
reports, company
telephone books,
university yearbooks,
“niche” references.
• Generally requires
human access and
special knowledge of
availability.
• Unique and useful.

10. ®
Primary Research Competency
Knowing Who Knows, Direct Contacts
• Citation Analysis is
key to finding top
experts across
different nations.
• Using the telephone
(and the Internet) to
reach top experts
yields powerful
results.

11. ®
Citation Analysis Example
DIALOG, SSCI, $1000 = Savings
• DIALOG access to
Social Science
Citation Index
• Use OSS methodology
• $500 in access charges
+ $500 in analyst time
= list of top experts on
any country or topic
• Then you call them...

12. ®
Analytic Toolkit Competency
Software can be a curse or a help
• Digital conversion,
storage, visualization,
and retrieval tools
• Geospatial tools
• Structured analysis
and detection tools
• Multi-media
publication and
presentation tools

13. ®
Geospatial Competency
Maps & images make a difference
• Commercial imagery
cheaper than ignorance
• Russian military maps
of Third World vital
• Post-processing support
from private sector
• Desktop tools for
plotting information in
time & space context

14. ®
Analytic Tradecraft
Emerging appreciation for its value
• CIA University
breaking new ground
• Moving away from
“cutting and pasting”
• Moving away from
hard-copy files
• Focus on learning how
to think, and how to
structure digital data

15. ®
Creating an OSINT Cell
Central discovery, distributed exploitation
• Six people can
leverage global
OSINT for an entire
Ministry or Service or
Command
• This eliminates need
for duplicate open
source infrastructure
• Also saves money
Senior All-Source
Collection Manager
Internet
Specialist
Commercial
Online Expert
Primary
Research
External
Contracts
All-Source Analyst/
Presentation Manager

16. ®
Analytic Tradecraft #1:
Trade-offs analysis in client context
• Greatest value of
analysis is strategic--
helping evaluate trade-offs
in policy and
acquisition and
operational costs
• Analysis must be done
in client context but
with external views
carefully represented

17. ®
Analytic Tradecraft #2
Nine evaluation criteria for credibility
• Relevance to policy
• Analytic sophistication
• Unique information
• Timely and concise
• Facts/sources are stated
• Conclusions are offered
• Assumptions are stated
• Options are outlined

18. ®
Analytic Tradecraft #3
Assumptions must be specified
• Separate “box” for
assumptions gains
consumer confidence
• List variables and
relationships--some are
factual, others not
• Obtain second views
• Question assumptions

19. ®
Analytic Tradecraft #4
Alternative outcomes must be specified
• What are the
alternative outcomes?
• If they occur, what
signposts or events
might be triggers?
• In choosing one
outcome over others,
what judgements were
made?

20. ®
Analytic Tradecraft #5
Facts, opinions, sources
• Distinguish between
direct & indirect sources
• Be precise about
sourcing of information
• Understand policy
sensitivities
• Respect complexity
• Account for deception
• Label judgements

21. ®
Analytic Tradecraft #6
Harnessing internal and external expertise
• Must rise above the
competition--and use
external experts
• Demonstrate research,
scholarship, database
access, ground truth,
teamwork, experience
• Customize intelligence
for specific individuals

22. ®
Analytic Tradecraft #7
The art of distillation and summarization
• 30-second lead buys
you 3 minutes for
summary which might
get them to read it all
• Focus on what’s new
• Make it easy to read
• A few points, clearly
• Role play the audience
Lead
Summary
Analyst

23. ®
Analytic Tradecraft #8
Deception, counterintelligence, timeliness
• Account for denial &
disinformation
• Gaps in collection?
• Odd contradictions?
• Odd confirmations?
• Use textbox to show
reader that steps were
taken to consider &
defeat deception

24. ®
The Analyst as Manager
of Customer Relations
• Analyst is a manager of
personal relations:
– top policymaker
– gatekeeper staff
– heads of bureaus
– policy analysts
– embassy counselors
– visiting dignitaries
– assigned liaison officers

25. ®
The Analyst as Manager
of Money for Open Sources
• Analyst is a manager
of money to fund:
– External source
discovery & purchase
– External source
processing/evaluation
– Individual expert
consultants
– Ad hoc team endeavors

26. ®
The Analyst as Manager
of External Experts
• Analyst is a manager
of external experts:
– Experts in other
branches of own
government
– Experts in other
governments & NGOs
– Business sector experts
– Academic experts
– Media experts
Branch
Chief
Analyst

27. ®
The Analyst as Manager
of Classified Collection Management
• Analyst is a manager of
classified capabilities:
– Tasking attaches
– Tasking spies
– Tasking technical
collection
– Tasking NATO or USA
or other classified assets
– Suggesting new secret
capabilities are needed

28. ®
Making Magic
People, Sources, Tools, Timeframes, Cultures
• Analyst makes magic
by mixing:
– People with special
skills and insights
– Secret & other sources
– Tools for thinking
– Timeframes--taking
longer views
– Cultures--taking
different views

29. ®
Challenges for Analysis
Forecasting, Generalizations, Two Levels Down
• Analyst is sometimes
the only one willing to
think clearly:
– about the future of the
region and the world
– about strategic
generalizations
– about details two levels
down that impact on
more general policies