Librarians in the Intelligence Process


Published on

Presented at the "Understanding and Improving Intelligence Analysis: Learning from Other Disciplines" Workshop, University of Mississippi, Oxford, MS, July 15, 2013.

Published in: Career, Education, Technology
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Librarians in the Intelligence Process

  1. 1. 1
  2. 2. Bios: Dr. Hock specializes in writing and teaching about advanced information retrieval skills, and his clients include the intelligence agencies of three countries. He also teaches a course in actionable intelligence as part of our library science curriculum at the Catholic University of America. Blane Dessy for a number of years ran the library and information services system for the US Department of Justice, which is of course one of the sixteen Intelligence Community agencies. Currently he leads Fedlink, the umbrella organization for libraries throughout the Federal government. I might add that as part of his current responsibility for coordinating various activities of libraries across the Federal government, Blane oversees a coordinating council of Intelligence Community librarians. For my part, I ran library and information services at the MITRE Corporation, which over the years has worked closely with many if not all the IC agencies under long term Federally funded research and development center programs. 2
  3. 3. But my colleagues and I wanted to participate in this workshop not just to represent ourselves. Instead, we’re here to represent, to the best of our ability, the profession of librarianship. When we saw the call for proposals, we all felt that librarians should be part of this conversation. It’s our thesis that in fact librarians play a significant, if sometimes overlooked, role in the intelligence process. Further, we believe that changes taking place in our profession at present will enable librarians to be increasingly valuable contributors in the years ahead. 3
  4. 4. One depiction of the classic intelligence process is the one shown here. It’s seen as involving five major phases: Planning, Collection, Processing, Analysis and Production, and Dissemination. ( whatwedo.html ) Both the traditional and emerging activities of librarians map into this process. Librarians have often been involved with managing the information repositories used by Analysts. They acquire, organize, manage, preserve, and disseminate information. Much of the information they deal with is open source, or OSINT. This includes regular, commercially published documents of all kinds, as well as what’s known as “Grey literature” – the whitepapers, conference reports, government studies, brochures, and other materials that are unclassified and openly distributed, but not included in the mainstream publication system and thus not as easy to discover and obtain. Note that these activities apply to both the legacy world of print publications and the dominant digital, web-based world. In fact, the age of digital content has made this work more challenging than ever in some respects. For example, it has created the illusion of free and easy access where in fact the deep web allows information to hide in plain sight. In fact it’s the shift to a digital information world that has both enabled and required librarians to take on new roles that increase their value. Three of these new roles are advising, analyzing, and teaching. 4
  5. 5. stalk-Twitter-and-Facebook.html Probably the easiest role to identify is analysis. We even have an open source example that discusses it. In November, 2011, the Associated Press put out a story about the CIA’s Open Source Center – the unit set up in 2005 to make better use of open sources in the intelligence process. It’s staffed heavily by librarians – the so- called “vengeful librarians” of this title. 5
  6. 6. are-tracking-twitter-and-facebook-as-they-should/ In fact, this little story is an interesting example of the characteristics of open source material. You’ll note the shift in terminology in the Forbes article from “vengeful” to “ninja”. The story was originally put out with the “vengeful” title, but apparently someone thought that was too strong – I’ve heard that it was in fact a mis-quote – and so in later versions the word was changed to “ninja”. The story no longer exists on the AP site, but you can find both “vengeful” and “ninja” in various sources around the web. 6
  7. 7. 7 Anyway, the point is, “vengeful” or “ninja”, librarians have become integral to the planning of open source collection, the analysis of open source material, and I would add, teaching other analysts about open sources and methods, or tradecraft if you will. These new tasks have been added on to their traditional roles in collecting and curating open source, processing it for future use, and disseminating it. Librarians have moved from helping analysts turn information into intelligence, to partnering in the intelligence analysis process.
  8. 8. 8 Let’s turn to the competencies that enable librarians to fulfill these roles. There are five that I’d like to highlight: OSINT collection; data and metadata management; knowledge management; adapting our work to the realities of human information behavior; and finally instructional design.
  9. 9. 9 Again, OSINT collection is our traditional area of competency. It’s an extension of the work in developing collections of published resources to meet the information needs of any given librarian’s community. In the intelligence community it also encompasses the acquisition of gray literature, and nonpublic information. For example, when I was at the MITRE Corporation, one of the activities I managed was the collection and stewardship of internal documentation, both classified and unclassified.
  10. 10. I was in Florida last month soon after the Edward Snowden story broke. At my hotel, I got a copy of USA Today one morning, which I wouldn’t see otherwise. When I opened it up, I was amazed to see that the word “metadata” was in a mass media news headline. That is something I had never imagined I would see. Amid all the other ramifications of this case, it’s had the effect of putting “metadata” into the public consciousness. 10
  11. 11. So, we already knew that “Big data” had become a huge buzzword in the past couple years, and that metadata was an essential element of the big data phenomenon. The exploitation of “big data” plays to traditional strengths of librarians, which are beginning to be leveraged in this new area. One reason for that is because one of the keys to the exploitation of large, heterogeneous collections of structured data is metadata – the description of the contents of the data repository. Librarians have been managing metadata for generations – after all, that’s what our library catalogs are. Furthermore, we recognize the power of metadata, because in fact, it’s only a bit of an exaggeration to say as this quotation does that “everything’s metadata for something else.” The quotation, by the way, comes from a news report on a recent conference of scholarly publishers, in which a central topic of discussion was the interconnectedness of data. For example, as librarians and analysts have understood for a long time, documents are metadata for people: the people whose knowledge can only ever be partially represented by the documents they produce. So if you really want to know something, you use the document to get to the person. It’s metadata. Outside the IC, librarians are applying their traditional competency in metadata management to the new world of big data. They’re being written into biological and health sciences research grants as data managers. They’re taking the lead in standing up institutional data repositories in higher education and government agencies. I don’t know if they’re performing these same functions in the IC, but I’d hope so. 11
  12. 12. Librarians are the natural knowledge managers in any organization. They are good at “knowing what we don’t know”. As the information scientist Marcia Bates has said, they follow the “red thread” of information through all human activity. Because librarians are interested in the information dimensions of any activity, they recognize and analyze needs, even those that are unexpressed, or “visceral” as termed by the late Robert S. Taylor. Thus they can plan, develop, and execute strategies to meet these needs, both in the immediate context and for the longer term. Bringing in their knowledge of sources, they are able to contribute to the integrated use of diverse sources to meet the information needs of analytical projects. 12
  13. 13. 13 There is a rich body of research in human information behavior that informs the practice of librarians. The competence to apply the knowledge from this research is essential. Like analysts, librarians have learned that many factors affect the role of information in decision-making. In part, this realization is informed by the work of cognitive scientists, led by Amos Tversky and Daniel Kahneman, who have documented the heuristics we all use in making sense of the information we encounter, and the biases that result. In part, it’s informed by scholars like Max Bazerman, who have studied these principles as they apply to decision-making processes. But there are also principles arising within the field of information science. One is the Principle of Least Effort, first articulated as it relates to the economy of language by George Zipf. In Information Science, we take it to mean that as information seekers we economize our efforts, preferring the most convenient sources and methods over the highest quality. (I describe it to my students as the principle that we are all lazy.) Related to the Principle of Least Effort is the notion of “satisficing” – of settling for “good enough”. In practice of course we must all satisfice, as no project of information collection and analysis is ever perfect, nor can be. The question is, at what level do we satisfice? We’ve learned that for the average undergraduate, it’s a pretty low level. If the prof wants ten citations in the research paper, the undergrad will grab the first ten plausible citations from Google, and that’s good enough. The
  14. 14. question for librarians, and for analysts, is “at what level do you satisfice?” What’s the highest level of quality and completeness you can achieve, and still meet the practical constraints on your effort? Beyond the Principle of Least Effort is Mooers’“Law”, not a law, really but an insight by Calvin Mooers that information is not always wanted – not even convenient information. There are times and situations when decision-makers do not want information. This is a hard principle for librarians to swallow, as information is our stock in trade. We believe information is good, and therefore more information must be even better. It is a hard lesson that every successful librarian needs to learn. Finally, we’ve learned from all our research about the primacy of HUMINT in information seeking – if you define HUMINT broadly as asking other people. In study after study of all sorts of demographic groups, this principle emerges. As the librarian Jessamyn West has been quoted, “when people have an information need, they’ll always ask people they know…” (Lankes, p. 83) For librarians, therefore, the key to effective exploitation of the best OSINT and other information sources is to make sure that, as West continues, “librarians are some of the people they know.” 13
  15. 15. These observations about human information behavior lead us directly to learning theory and instructional design. Under various names, “information fluency”, “information literacy”, “learning how to learn”, our society is increasingly alert to the importance of the abilities to articulate questions, seek and find information, analyze it, and use it effectively and ethically. These are widely recognized as essential life skills for the 21st century. Employers are demanding them. Educators are incorporating them into learning goals at all levels of our educational system. Simply put, these skills are what librarians do. So it’s natural that in a growing number of institutions, librarians are taking the lead in integrating these skills into the curriculum. Similarly, in the IC, librarians have the ability to infuse these competencies into the intelligence process. However, that means developing our ability to create and deliver instruction effectively. Increasingly, that’s just what we are doing. 14
  16. 16. 15 Now let’s turn to the third and last section of the presentation. There are a few key principles that form the foundation of the role of librarians in the intelligence process. As we begin this, I want to differentiate this section from the section on competencies. In the previous section, my coauthors and I have proposed some distinctive characteristics that librarians possess. In this section, the emphasis is more on shared principles that enable them to collaborate effectively with others in the intelligence process.
  17. 17. 16 The first is information quality. Information is a key element for both analysts and librarians. We recognize the slippery nature of “truth” and the challenges in assessing the value and quality of our information – yet the critical importance of making that assessment in the process of developing “actionable intelligence”.
  18. 18. 17 Thus we’ve developed the determinants of value and clues to quality that you see here. I won’t go into these in detail here, except to say that for librarians, the assessment of information quality has taken on new complexity and importance in recent years. As noted earlier, our domain of collection has moved from commercial and official publications to gray literature, internal documentation, and social media. For the Open Source Center, the entire open web is their collection. As we’ve done that, our old shortcuts to assess quality no longer work. We used to go by reputation: of publishers, editors, and authors. We used to go by reviews contributed by other professionals. These things now cover a much lower proportion of our resource base, and so our approach to assessing quality, as a profession, has moved much closer to that of the intelligence analyst: assessing motives, seeking corroboration, and so forth.
  19. 19. A second principle is that of added value. Tom Davenport is a scholar of management information, affiliated with Babson College and Harvard University. A generation ago, in a paper co-authored with Laurence Prusak, an information scientist and librarian, he advised “blowing up the corporate library”. He wasn’t talking about explosives, of course – he was talking about dispersing the librarians throughout the enterprise. In the ensuing years, many smart organizations have done that. Most recently, Davenport has been writing about the importance of analytics, as shown in this quotation from a Wall Street Journal blog posting. Information is not enough; analysis is key. Smart, effective librarians understand this. Added value through better analysis of the best available information is essential for all participants in the intelligence process. 18
  20. 20. Still, as our understanding of human information behavior tells us, sound analysis is worthless without effective communication. In our information-abundant organizations, attention is the scarce resource. So our principle must be to package intelligence so that the intended audience will, literally, pay attention to it. 19
  21. 21. Last and perhaps most important of all is the mission focus, because the principle of concentrating on the mission provides the motivation to learn the skills and perform the activities that we’ve presented today. Librarians’ understanding of their mission is undergoing a profound evolution. In his recent book, The Atlas of New Librarianship, Dr. David Lankes of Syracuse University proposes this mission statement: “The mission of librarians is to improve society by facilitating knowledge creation in their communities.” My coauthors and I believe that Lankes’ statement can be easily customized for librarians in the IC, as you see here. Our mission is to “improve its effectiveness by facilitating the creation and application of knowledge.” 20
  22. 22. So, I’ll conclude by returning to that article about “vengeful” librarians or “ninja” librarians or whatever they were. The article quotes Doug Naquin, then-director of the OSC, as citing the importance of a degree in library science for open source officers. We agree, and we hope that in this presentation we’ve been able to highlight some important ways in which librarianship is aligned with the intelligence process, and is becoming more relevant to it in our current environment. Thanks. 21