® 
Processing 
Make the Most of What You Know 
Robert David Steele 
Intelligence Coach 
bear@oss.net
® 
Bangalore, India 
No computers, no bus passes 
• If you think you have it 
bad, consider 
Bangalore, India. 
• 99 people have to 
depend on one jeep, five 
motorcycles, and a 
scooter. 
• No bus passes, no 
training, no computers.
® 
National Drug Intelligence Center 
Processing Makes A Strategic Difference 
• Telling a story: DEA 
900 files, FBI 90 files 
• Case file benefits: 
visualizing the data for 
deep understanding 
• Inter-agency benefits: 
providing an incentive 
for sharing leads
® 
Processing Objectives 
Pattern analysis & anomaly detection 
• Automated processing is 
vital to making sense of 
vast quantities of 
information. 
• Pattern analysis & 
anomaly detection are 
the primary objectives-- 
the human mind will 
provide the 
understanding.
® 
Collection Management 
Can’t do CM without processing 
• If you don’t know 
what you already 
know, and if you don’t 
know what your 
critical information 
gaps are, you cannot 
do effective collection 
management or case 
development.
® 
Analysis 
Can’t do analysis without processing 
• Our first mistake is to 
treat people as “free” 
goods. 
• Our second mistake is 
to assume that an 
analyst can make 
sense of information 
simply by reading and 
thinking. 
THE NAKED ANALYST
® 
Processing Pre-Requisite #1 
Data handling standards 
• Digitization of data at 
the point of 
acquisition makes 
everything else easier. 
• Agreeing on common 
data handling 
standards compatible 
with web-based 
information sharing is 
helpful.
® 
Processing Pre-requisite #2 
Geospatial attributes 
• Adding geospatial 
attributes to your data 
will increase its 
processing value by 
100X to 1000X! 
• Geospatial and time 
attributes are what 
make automated fusion 
and pattern detection 
possible.
® 
Processing Pre-requisite #3 
Interoperability 
• Interoperability of 
digital data is vital. 
– Among the seven 
national tribes 
– Among regional 
military commands 
– Among military and 
non-governmental 
organizations including 
foreign businesses
® 
The Big Picture 
Four processing quadrants 
III - External Information 
EXTERNAL 
Data 
Visualization 
Expert Hires 
“Just Enough, 
Just in Time” 
Business 
Intelligence 
Institutionalized 
Local 
Knowledge 
Environmental Monitoring 
Technology Monitoring 
Customer Monitoring 
TECHNICAL HUMAN 
Vendor Reporting 
INTERNAL 
Organizational 
Memory System 
Heterogeneous 
Search & Retrieval 
Data 
Conversion 
Churning 
(Rotationals) 
Trip 
Reports 
Internal Reporting 
Government Monitoring 
CHUNKS 
(Intellectual Property) 
PERSONALITY 
(Insight/Intuition) 
Patents, Etc. 
Trade Secrets 
Meta-Data 
Knowledge Capital™ 
Rolodexes/E-Mail 
Personal Brand 
Out-Sourcing of 
Information 
Processing 
Project/Group 
Management 
Training 
E-Commerce 
Automated Analysis 
Cell # 
IV - Organizational Intelligence 
I - Knowledge Management II - Collaborative Work
® 
Processing Quadrant #1 
Knowledge Management 
• Know what you know 
• Do not lose data, 
insights, links 
• Integrate people, 
projects, vendors, 
times, places, objects 
• Optimize application 
of technology to 
internal information 
INTERNAL 
• Internal Reporting 
• Vendor Reporting 
• Project Management 
• Data Conversion 
• Automated Analysis
® 
Processing Quadrant #2 
Collaborative Work 
• Human Capital 
• Inherent in People 
• Who They Know 
• How They Know 
• When They Know 
• What They Do 
• Who They Tell 
• How They Feel 
INSIGHT 
• Employee BrandNames 
• Rolodexes 
• E-Mail Directories 
• Cell Telephone Networks 
• Trip Reports 
• Rotationals 
• Training
® 
Processing Quadrant #3 
External Information Acquisition 
• Peter Drucker says 
this is the next 50 
years of innovation 
• OLD: spend on 
technology 
• NEW: spend on 
external information in 
all languages, from all 
sources, all the time 
EXTERNAL 
• Local knowledge 
• Expert hires “just 
enough, just in time” 
• Customer monitoring 
• Government monitoring 
• Technology monitoring 
• Environment monitoring
® 
Processing Quadrant #4 
Organizational Intelligence 
• Data standards 
• Data entry mandated 
• Storage & retrieval 
• Historical access 
without legacy system 
training 
• Employee shoeboxes 
integrated/not lost 
MEMORY 
• Intellectual Property 
• Organizational Memory 
• Meta-Data 
• History of Information 
• Electronic/Human Links 
• Survive Human Turnover
® 
Finished Intelligence and Reporting 
Revision Tracking 
and Realtime Group 
Review 
Publishing and 
Word Processing 
Collaborative 
Work 
Interactive Search 
and Retrieval of 
Data 
Clustering and 
Linking of 
Related Data 
Conversion of 
Paper Documents 
to Digital Form 
Production of 
Graphics, Videos and 
Online Briefings 
Structured 
Argument 
Analysis 
Modeling and 
Simulations 
Detection of Alert 
Situations 
Notetaking and 
Organizing Ideas 
Desktop 
Graphic and Map- 
Based Visualization 
Detection of 
Changing Trends 
Automated Extraction 
of Data Elements From 
Text and Images 
Standardizing 
and Converting 
Data Formats 
of Data 
Statistical Analysis 
to Reveal 
Anomalies 
Processing Images, 
Video, Audio, 
Signal Data 
Automated 
Foreign Language 
Translation 
Open Literature Non-Text Data Restricted Information 
A 
B 
C 
Processing 
Desktop: 
Generic Analytic 
Functionalities
® 
Processing Desktop 
No Easy Solutions 
PPllaann 
EExxtteenndd 
Analyst Notebook 
**Plumtree Corporate Portal 
EDGE? 
RReeppoorrtt 
SShhaarree 
OSALAT 
Excaliber 
**Powerize 
Copernic CCoolllleecctt 
Topic 
Analyst Workbench 
Identifier, SIFT, 
OnTopic, Labrador 
GMS, Athens 
Aerotext 
Intelligent Miner 
For Text 
AAnnaallyyzzee 
Content Extractor 
Information Portal, 
Comprendium 
**Strategy! 
ClearForest Suite 
CrimeLink 
**C-4-U Scout 
**CI Spider 
**Knowledge.Works 
**Market Signal Analyzer 
**E-Sense 
**Corporate Intelligence Service 
**TextAnalyst 
**Wincite 
**WisdomBuilder 
Groove? 
** Previously reviewed in the Fuld & Co. Software Report 
MindMap? 
SYSTRAN+? 
Other 
CATALYST 
Elements?
® 
Emerging Technologies #1 
Digitization of documents 
• Battlefield digitization 
is no longer an issue 
• Battlefield translation 
is no longer an issue 
• What is missing is the 
leadership willingness 
to link troops to both 
processing and expert 
translators.
® 
Emerging Technologies #2 
Visualization of links in text 
• Visualization of links 
among people, vehicles, 
weapons, and bank 
accounts is no longer an 
issue. 
• What is missing is the 
leadership commitment 
to arming troops with 
intelligence
® 
Emerging Technologies #3 
Peer-to-peer computing & communications 
• Networked “side to side” 
intelligence is vastly 
more effective than up 
and down chain of 
command intelligence. 
• What is lacking is a 
leadership commitment 
to training and then 
trusting the troops.
® 
Processing Solution #1 
Internet as common operating environment 
• War and operations 
other than war are now 
a “come as you are” 
situation, with ad hoc 
allies that cannot be 
anticipated. 
• Only the Internet offers 
a global C4I solution for 
mix and match people, 
equipment, and data.
® 
Processing Solution #2 
Open source software & security 
• European Community 
has the right idea-- 
open source software 
is the wave of the 
future 
• Security must be in the 
software, not the 
hardware or the 
physical controls
® 
Processing Solution #3 
24/7 “Plots” at every level 
• Need “plots” at the 
township, city, 
province, and national 
levels, as well as 
special regional 
intelligence centers 
• Sources, geospatial 
processing, and 
analysts must merge
® 
Recommendation #1 
National “skunkworks” for seven tribes 
• Neither the government 
nor the business world 
will solve the processing 
problem alone. 
• Need a national 
“skunkworks” where 
open sources and open 
software can be safely 
integrated
® 
Recommendation #2 
Push NATO and USA for web-based sharing 
• Help the European 
Community focus on 
the urgency of insisting 
that NATO and the 
USA migrate to a web-based 
approach to 
sharing all information, 
at every classification 
level. This is a Native 
American ethic.
® 
Recommendation #3 
Internet, Wireless, Spectrum 
• Extend the Internet to 
every street corner 
• Go wireless, with 
encryption, quickly 
• Free up as much 
spectrum as possible-- 
South Korea is the 
leader, not the USA
® 
Creating a Smart Nation 
Connect Content Coordinate C-ecurity 
• Connectivity everywhere 
• Content is digitized 
• Coordination of 
standards and 
investments 
• C4 Security across all 
seven tribes--public 
safety at same level as 
safety of secrets

2004 05 intelligence processing seminar

  • 1.
    ® Processing Makethe Most of What You Know Robert David Steele Intelligence Coach bear@oss.net
  • 2.
    ® Bangalore, India No computers, no bus passes • If you think you have it bad, consider Bangalore, India. • 99 people have to depend on one jeep, five motorcycles, and a scooter. • No bus passes, no training, no computers.
  • 3.
    ® National DrugIntelligence Center Processing Makes A Strategic Difference • Telling a story: DEA 900 files, FBI 90 files • Case file benefits: visualizing the data for deep understanding • Inter-agency benefits: providing an incentive for sharing leads
  • 4.
    ® Processing Objectives Pattern analysis & anomaly detection • Automated processing is vital to making sense of vast quantities of information. • Pattern analysis & anomaly detection are the primary objectives-- the human mind will provide the understanding.
  • 5.
    ® Collection Management Can’t do CM without processing • If you don’t know what you already know, and if you don’t know what your critical information gaps are, you cannot do effective collection management or case development.
  • 6.
    ® Analysis Can’tdo analysis without processing • Our first mistake is to treat people as “free” goods. • Our second mistake is to assume that an analyst can make sense of information simply by reading and thinking. THE NAKED ANALYST
  • 7.
    ® Processing Pre-Requisite#1 Data handling standards • Digitization of data at the point of acquisition makes everything else easier. • Agreeing on common data handling standards compatible with web-based information sharing is helpful.
  • 8.
    ® Processing Pre-requisite#2 Geospatial attributes • Adding geospatial attributes to your data will increase its processing value by 100X to 1000X! • Geospatial and time attributes are what make automated fusion and pattern detection possible.
  • 9.
    ® Processing Pre-requisite#3 Interoperability • Interoperability of digital data is vital. – Among the seven national tribes – Among regional military commands – Among military and non-governmental organizations including foreign businesses
  • 10.
    ® The BigPicture Four processing quadrants III - External Information EXTERNAL Data Visualization Expert Hires “Just Enough, Just in Time” Business Intelligence Institutionalized Local Knowledge Environmental Monitoring Technology Monitoring Customer Monitoring TECHNICAL HUMAN Vendor Reporting INTERNAL Organizational Memory System Heterogeneous Search & Retrieval Data Conversion Churning (Rotationals) Trip Reports Internal Reporting Government Monitoring CHUNKS (Intellectual Property) PERSONALITY (Insight/Intuition) Patents, Etc. Trade Secrets Meta-Data Knowledge Capital™ Rolodexes/E-Mail Personal Brand Out-Sourcing of Information Processing Project/Group Management Training E-Commerce Automated Analysis Cell # IV - Organizational Intelligence I - Knowledge Management II - Collaborative Work
  • 11.
    ® Processing Quadrant#1 Knowledge Management • Know what you know • Do not lose data, insights, links • Integrate people, projects, vendors, times, places, objects • Optimize application of technology to internal information INTERNAL • Internal Reporting • Vendor Reporting • Project Management • Data Conversion • Automated Analysis
  • 12.
    ® Processing Quadrant#2 Collaborative Work • Human Capital • Inherent in People • Who They Know • How They Know • When They Know • What They Do • Who They Tell • How They Feel INSIGHT • Employee BrandNames • Rolodexes • E-Mail Directories • Cell Telephone Networks • Trip Reports • Rotationals • Training
  • 13.
    ® Processing Quadrant#3 External Information Acquisition • Peter Drucker says this is the next 50 years of innovation • OLD: spend on technology • NEW: spend on external information in all languages, from all sources, all the time EXTERNAL • Local knowledge • Expert hires “just enough, just in time” • Customer monitoring • Government monitoring • Technology monitoring • Environment monitoring
  • 14.
    ® Processing Quadrant#4 Organizational Intelligence • Data standards • Data entry mandated • Storage & retrieval • Historical access without legacy system training • Employee shoeboxes integrated/not lost MEMORY • Intellectual Property • Organizational Memory • Meta-Data • History of Information • Electronic/Human Links • Survive Human Turnover
  • 15.
    ® Finished Intelligenceand Reporting Revision Tracking and Realtime Group Review Publishing and Word Processing Collaborative Work Interactive Search and Retrieval of Data Clustering and Linking of Related Data Conversion of Paper Documents to Digital Form Production of Graphics, Videos and Online Briefings Structured Argument Analysis Modeling and Simulations Detection of Alert Situations Notetaking and Organizing Ideas Desktop Graphic and Map- Based Visualization Detection of Changing Trends Automated Extraction of Data Elements From Text and Images Standardizing and Converting Data Formats of Data Statistical Analysis to Reveal Anomalies Processing Images, Video, Audio, Signal Data Automated Foreign Language Translation Open Literature Non-Text Data Restricted Information A B C Processing Desktop: Generic Analytic Functionalities
  • 16.
    ® Processing Desktop No Easy Solutions PPllaann EExxtteenndd Analyst Notebook **Plumtree Corporate Portal EDGE? RReeppoorrtt SShhaarree OSALAT Excaliber **Powerize Copernic CCoolllleecctt Topic Analyst Workbench Identifier, SIFT, OnTopic, Labrador GMS, Athens Aerotext Intelligent Miner For Text AAnnaallyyzzee Content Extractor Information Portal, Comprendium **Strategy! ClearForest Suite CrimeLink **C-4-U Scout **CI Spider **Knowledge.Works **Market Signal Analyzer **E-Sense **Corporate Intelligence Service **TextAnalyst **Wincite **WisdomBuilder Groove? ** Previously reviewed in the Fuld & Co. Software Report MindMap? SYSTRAN+? Other CATALYST Elements?
  • 17.
    ® Emerging Technologies#1 Digitization of documents • Battlefield digitization is no longer an issue • Battlefield translation is no longer an issue • What is missing is the leadership willingness to link troops to both processing and expert translators.
  • 18.
    ® Emerging Technologies#2 Visualization of links in text • Visualization of links among people, vehicles, weapons, and bank accounts is no longer an issue. • What is missing is the leadership commitment to arming troops with intelligence
  • 19.
    ® Emerging Technologies#3 Peer-to-peer computing & communications • Networked “side to side” intelligence is vastly more effective than up and down chain of command intelligence. • What is lacking is a leadership commitment to training and then trusting the troops.
  • 20.
    ® Processing Solution#1 Internet as common operating environment • War and operations other than war are now a “come as you are” situation, with ad hoc allies that cannot be anticipated. • Only the Internet offers a global C4I solution for mix and match people, equipment, and data.
  • 21.
    ® Processing Solution#2 Open source software & security • European Community has the right idea-- open source software is the wave of the future • Security must be in the software, not the hardware or the physical controls
  • 22.
    ® Processing Solution#3 24/7 “Plots” at every level • Need “plots” at the township, city, province, and national levels, as well as special regional intelligence centers • Sources, geospatial processing, and analysts must merge
  • 23.
    ® Recommendation #1 National “skunkworks” for seven tribes • Neither the government nor the business world will solve the processing problem alone. • Need a national “skunkworks” where open sources and open software can be safely integrated
  • 24.
    ® Recommendation #2 Push NATO and USA for web-based sharing • Help the European Community focus on the urgency of insisting that NATO and the USA migrate to a web-based approach to sharing all information, at every classification level. This is a Native American ethic.
  • 25.
    ® Recommendation #3 Internet, Wireless, Spectrum • Extend the Internet to every street corner • Go wireless, with encryption, quickly • Free up as much spectrum as possible-- South Korea is the leader, not the USA
  • 26.
    ® Creating aSmart Nation Connect Content Coordinate C-ecurity • Connectivity everywhere • Content is digitized • Coordination of standards and investments • C4 Security across all seven tribes--public safety at same level as safety of secrets

Editor's Notes

  • #11 I’m going to go slowly with this chart. I want to make four key points here. First, knowledge management is elementary. We need to make the most of what we already know, and that means that we need an architecture that can handle both secrets and non-secrets while permitting the sharing of information across organizational lines. We can’t do that today within DoD, much less the rest of USG. Second, collaborative work, the ability to find, work with, and co-produce knowledge with anyone, anywhere in the world, is the real challenge facing us today. We are spending too much money on proprietary high-end compartmented solutions. The Internet and nomadic wireless computing are well-established, let’s get on with it, starting with funding for direct analyst access to the open sources that can help them identify expert peers in the private sector. Third, and Peter Drucker emphasized this in August 1998, we have to stop obsessing on information technology and we have to get serious about accessing external information. We can apply the “T” here, but we need to put the “I” horse in front of the “T” cart. Drucker says the third quadrant, External Information, is where the next revolution in IT must happen I have been saying this since 1988. Lastly, our architecture must provide for how we turn these first three quadrants into Organizational Intelligence, that is, knowledge that remains available to future generations in the face of rotations, retirements, deaths, and other human passages.
  • #16 Here is another way of looking at where we need to focus intelligence technology--on desktop processing, rather than on satellite collection! CIA--Diane Webb in particular--defined these functionalities in 1986 and they still are not available to any intelligence analyst today for two reasons: First, because we spend all our money in highly compartmented programs where cooperation is viewed as one step short of treason; and Second, and I don’t have an answer, we have failed to focus on the importance of establishing Application Program Interface (API) standards that permit plug and play among third party softwares, and are stable or at least easily updatable. We desperately need analytic tools, and I would very much like to see General Kernan at JFCOM take on these 18 functionalities as part of the Future Intelligence Collaborative Environment (FICE), and in the process, working with the National Institute of Standards and Technology, devise an approach to the API issue. We need to establish a cross-government council on analytic tools, with JFCOM and NSA as the heavy-duty performers, and CIA as the top client. If we establish XML data standards, open encryption, and stable API’s, we can have all of this on every analyst’s desktop within five years, at a reasonable cost.
  • #17 This slide was created by Claudia Porter, a senior manager at Austin Info Systems in Texas, for her presentation to my conference last May. I asked her to start with the existing survey of business intelligence software by Leonard Fuld, a very competent practitioner. The four items in red are my addition after the fact. This is a mish-mash and most of the products do not meet customer expectations. None of them work well with one another, and all of them assume that the data will already be digital and in some structured form. The “T” must provide for stable and robust unstructured data entry and data conversion portals, for plug and play handshakes, and for idiot-proof online training embedded within each program…and good documentation of code.