FBI Virtual Case File project (a case study)


Published on

Published in: Technology, Business
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

FBI Virtual Case File project (a case study)

  1. 1. Joseph Howerton IS 430, Section 210 – David Bixby Summer I - 2009-10 Case Study 1: FBI Virtual Case File (VCF) project
  2. 2. Introduction to the FBI’s VCF project failure The FBI’s Trilogy IT modernization program was intended to upgrade/modernize the IT Enterprise Architecture/Infrastructure of the FBI; by providing a high-speed network linking the offices of the FBI, modern workstations and software within each office for every FBI employee, and a user application known as the Virtual Case File to enhance the ability of agents to organize, access, and analyze information. From A Review of the FBI’s Trilogy Information Technology Modernization Program Initial VCF project scope It was the User Applications Component, which would ultimately become the VCF that staked out the most ambitious goals. First, it was to make the five most heavily used investigative applications—the Automated Case Support system, IntelPlus, the Criminal Law Enforcement Application, the Integrated Intelligence Information Application, and the Telephone Application—accessible via a point-and-click Web interface. Next, it would rebuild the FBI's intranet. Finally, it was supposed to identify a way to replace the FBI's 40-odd investigative software applications, including ACS. From A Review of the FBI’s Trilogy Information Technology Modernization Program Executive Summary The back-story for the notorious FBI Virtual Case File (VCF) project failure really began in the early 1990’s, when FBI Special Agent, Larry Depew, identified the need for a database program that would help him (and other agents) to organize the reams of evidence collected and compiled during their investigation(s); and from wiretaps, interviews, and financial transactions, over the course of two and a half years investigating mafia involvement in skimming off millions of dollars in federal and New Jersey state gasoline and diesel taxes. The problem that emerged was one of an inability of FBI Special Agents to use a software component to connect the dots, both on Agent Depew’s case specifically, and any new dots that might come along. There was no doubt that a new system was needed, which would become the Virtual Case File (VCF) system project; to replace the existing system, the archaic Automated Case Support (ACS) system, as it was extremely cumbersome, inefficient, and limited in its capabilities. As it turned out, Depew ended up writing a database program on his own, that he used to trace relationships between telephone calls, meetings, surveillance, and interviews; but he still could not import information from other investigations that would assist him in his own investigation(s). That inability created the need for a new system which would result in the formal creation of the FBI’s VCF project. Officially, the FBI’s VCF project began in 2000 when the FBI began to deal with its outdated IT systems infrastructure. On July 17, 2000, the FBI hired Bob E. Dies to create a plan for this IT transformation. Dies, who was a former executive with IBM, basically replaced Louis J. Freeh, who did not have the IT skills or competencies required to complete such a task. We will find that the FBI’s lack of IT skills, a technical ignorance of technology, combined with their incompetence in delivering IT systems infrastructure, is a common thread that binds this epic and unnecessary IT project failure. And Dies was just the first of five ―officials‖ who, struggled over four years to lead the FBI’s sprawling and antiquated information systems, and get the VCF project under way.
  3. 3. In the fall of 2002, the only early warning sign came from Matthew Patton, a security expert working for SAIC on the VCF development team. Patton unequivocally bashed the non-existent technical expertise on the side of the FBI, as well as SAIC management practices, declaring both organization’s incompetent. In the end of that scenario, Patton would not be granted security clearance with the FBI, could not take a position with SAIC due to their inability to base him in Chicago, and maybe most importantly, Patton’s declarations and objections were summarily dismissed, thus ignored. In 2005, Glenn A. Fine, the U.S Department of Justice’s inspector general, submitted an 81-page audit of the FBI’s VCF project. In it he described eight factors that contributed to the VCF’s failure. Among the factors included were poorly defined and slowly evolving design requirements; overly ambitious schedules; and the lack of a strategic plan to guide hardware purchases, network deployments, and software development for the bureau. Fine’s conclusion was submitted in the light of the September 11th , 2001, terrorist attacks on the World Trade Center; he surmised that four years after 9/11, the FBI still did not have the software it needed to ―connect the dots‖ with the data in their case files. The project was killed (before completion) in May 2005. The FBI spent $170 million on the VCF project, and in the end they are going with a customizable, off the shelf system. The solution is not to keep throwing money at the project, but to recognize the serious issues surrounding the project’s failure. It is this researcher’s opinion that, with so much turnover at the highest ranks of the FBI, and the inability to document lessons learned as part of an IT Governance strategy, the FBI didn’t, and couldn’t have learned much. In the end, this IT project, like so many IT projects, suffered from a total misunderstanding by the FBI on how to define and implement Enterprise-wide architecture, based upon user expectations. Every step of this project would have benefitted from defining and designing around Usability Engineering tasks, activities, deliverables, and communication vehicles. It is imperative that these practices occur from the beginning to the end of the project design and development lifecycle. Research Approach Using IEEE Spectrum’s article on the VCF project failure as an anchor, I created a sort of outline of the pertinent details that works for me. Then I went online to find further analysis on the topic. I found that (because) the IEEE Spectrum article was (it seemed to me) the first to ―break the story‖ in great detail to the general public, I felt it was used as the base anchor reference for (basically) all of the analysis briefs out in the ether; so I figured that my analysis will be added to the many. As I read through the IEEE analysis, Goldstein pointed to another report of interest - A Review of the FBI’s Trilogy Information Technology Modernization Program report on the FBI trilogy Project, which changed my viewpoint. I found that this report is actually the most comprehensive and complete report that I found on the failure of the VCF project. It details the issues across the board, and puts forth detailed recommendations for the project based on their exhaustive analysis.
  4. 4. From Anatomy of an IT disaster: How the FBI blew it
  5. 5. From here, I analyzed each of the other reference documents in deference to the expertise and experience of the authors bringing it to the fore, having the core ideas resonate in a new way, to capture their thoughts on an idea, when golden, and (of course) to find any differences or discrepancies from opinion to opinion. The general consensus (I feel) in the research is one of acknowledging precepts put forth in the IEEE Spectrum report that this enterprise project fell into the most basic traps of software development, from poor planning to bad communications throughout the lifecycle of the project. Key Issues The first key IT issue was the 800-page requirements document put forth by the FBI. This document was totally unusable from the start, and many analysts assert that with such a poorly defined approach to a project, left most to wonder, how could it possibly achieve success? Further, the FBI’s understanding of the requirements was not strong at all. There was a seeming disconnect between the FBI and SAIC from the beginning. Add to the mirth, a constant turnover of FBI Directors, the FBI’s lack of internal expertise, and a lack of really any credible IT management and technical expertise internal to the FBI. Of course the results would be (and were) catastrophic for the VCF project. The project demonstrated a systematic failure of software engineering practices which include, from Wikipedia: Virtual Case File  Lack of a strong blueprint from the outset led to poor architectural decisions.  Repeated changes in specification.  Repeated turnover of management, which contributed to the specification problem.  Micromanagement of software developers.  The inclusion of many FBI Personnel who had little or no formal training in computer science as managers and even engineers on the project.  Scope creep as the requirements were continually added to the system even as it was falling behind schedule.  Code bloat due to changing specifications and scope creep. At one point it was estimated the software had over 700,000 lines of code.  Violating Brooks' law by adding people and resources to the project as it was falling behind, slowing it further.  Planned use of a flash cutover deployment, which made it difficult to adopt the system until it was perfected. Because of the bureaucracy (at least) inside the FBI (I think), as well as inconsistent responses and communications from SAIC, there was no cogent, cohesive blueprint laid out for achieving success. There was no enterprise architecture (blueprint) to describe how an organization operates currently, how it wants to operate in the future, and includes a road map – a transition plan – for getting there….leaving no detailed description of the FBI’s processes and IT infrastructure as a guideline.
  6. 6. The National Research Council pointed out that without this blueprint/road map the bureau could not make coherent or consistent operational or technical decisions about linking databases, creating policies and methods for sharing data, and making trade-offs between information access and security. Key IT Impacts According to the National Research Council’s (NRC’s) Report Review Committee in their report A Review of the FBI’s Trilogy Information Technology Modernization Program, the specific factors that would end up in an implosion of the VCF project include, • Enterprise architecture - the committee concluded that the FBI’s efforts and results in the area of enterprise architecture are late and limited, and fall far short of what is required. • System design - The design process was well under way prior to the expansion of the intelligence mission, and the requirements for the processes supporting the intelligence mission were not included in the VCF design. For this reason, and because of the significant differences in IT requirements between systems supporting investigation and those supporting intelligence, the committee strongly recommends that the FBI refrain from using the VCF as the foundation on which to build its analytical and data management capabilities for the intelligence processes supporting the counterterrorism mission. • Program and contract management - In practice, it is essentially impossible for even the most operationally experienced IT applications developers to be able to anticipate in detail and in advance all of the requirements and specifications. Therefore, internal development plans, and the development contracts with supporting organizations, should call for an approach that is based on a process of extensive prototyping and usability testing with real users. Doing so allows iterative development with strong user feedback and involvement, thus increasing the chances that what is ultimately delivered to the end users meets their needs. • Skills, resources, and external factors - the FBI lacks a human resource and skill base adequate to deal with the bureau’s IT modernization program. Specifically, the FBI is extremely short on experienced program managers and contract managers and senior IT management team members with good communications skills. Final Analysis It’s been a challenge just to figure out how to approach the analysis of the FBI’s failed VCF project. It’s difficult to truly understand, with so many resources, and supposed expertise that one would expect at their disposal, how this project could fail. I had to simplify how I was looking at it. Upon reflection, my opinion is simply that the primary stakeholders on both sides of the project turned out to be a bunch of people that didn’t have the guts to speak up and speak out about the gross incompetence at any and every juncture of this project. So, I approached my analysis of this colossal IT project failure wearing two hats; usability engineer and project manager. My opinion paper begins and ends with Usability Engineering (UE), as it is my opinion that the VCF project was a classic case for the need of insertion of UE and User Experience Design (UXD) addressed across the project lifecycle to inform and speak for
  7. 7. the users. I just don’t see how they could execute this project so poorly. I know from my own experiences that 2000-2005 were very tumultuous times for IT Governance boards, specifically with the implosion (and consequences) of the internet bubble. I can imagine that SAIC was going through a lot of not-so positive or pleasant experiences during those times as well. In the final analysis, I assert that there is equal ―blame‖ here for both parties – the FBI, and SAIC – to accept responsibility for the VCF project failure. I should say that I don’t really like the term blame, except to say that when something goes wrong, per David Bixby, the IT Project Manager should step out in front to shield the project team from said blame; I’m pretty confident that this didn’t happen on either the side of the FBI or SAIC. Any IT project is going to have changes, and requires constant communications, which I don’t feel was the case in this project. There were issues from the very beginning that SAIC could have stopped the project in its tracks to perform better, more comprehensive, up-front strategy. The VCF has become a primary cautionary tale for IT professionals of all types. From the perspective of a Usability Engineer, the lack of an appropriate created requirements document left this project for dead from the beginning. In the end, I feel we are left wondering about too many aspects of this IT project failure. As a result, along the way, along with major scope creep, the FBI went through five different CIO’s, 10 Project Managers, and 36 contract changes. As well as reports from the Government Accountability Office, the DOJ’s inspector general, and the National Research Council, all submitted reports acknowledging the ongoing problems with the project, as a result of not having an appropriate blueprint/roadmap to success. I don’t feel that representatives from either side, the FBI or SAIC, stepped up and took any responsibility for the ―comedy of errors‖ that the VCF became; and with the practice of Usability Engineering taking root in IT development methodologies, a little up-front inspection, and with checkpoint across the development lifecycle, this project could have been saved. Instead, what the FBI and SAIC engaged the taxpayer in, was a project that was a roller coaster of inefficiencies, riddled with change requests, constantly changing priorities in the guise of an actual plan, schedule slippages, integration delays, the FBI’s sloppy inventories of existing networks, underestimates throughout the development lifecycle, and significant management turbulence. Throwing good money after bad to committee upon committee, and never realizing the source of the problem, which was they employed no constructive usability methodologies to appropriately engage the design and development of such an advanced Enterprise Architecture. In the end, after spending at least $170 million for the VCF piece of the FBI’s Trilogy project, the FBI is going to go with an off the shelf solution to address the requirements left unresolved in the wake of the VCF project failure. From a project management perspective, the lessons of the VCF cautionary tool are many. In the light of day, the role of project manager from both sides of the project, was not integrated into the project with any kind of authority.
  8. 8. Writing this brief has been very frustrating because the details of this project are enough to frustrate any taxpayer alone, but also as an IT professional, with a focus in User Experience Design and Project Management. In my opinion, it speaks to the ongoing corruption of government; maybe especially if it’s a corruption of confidence; because that confidence has slipped dramatically. At every step of the way, when the FBI asked, Congress approved more and more money for the Trilogy project. Where did all the money go for the VCF? The whole thing offends me quite frankly. And I’m not asserting that I could have managed this, such a complex project. I am simply saying that I received my BS in HCI in November 1999; so if this project started in 2000, why weren’t there usability professionals assigned as resources to this project? It’s not clear to us if there were or not, but I assert here that they could have saved this project at the beginning, by rejecting the requirements documentation as put forth by the FBI. The reason I’m being so vehement here is simply because if things can go this wrong when the FBI is working on an IT project, what other projects are out there that are top secret, and we know nothing of the waste and corruption, save what we hear from other government agencies via committees and task forces after internal audits. It’s a bit nerve wracking when so much is just unknown.
  9. 9. References Who Killed the Virtual Case File? http://spectrum.ieee.org/computing/software/who-killed-the-virtual-case-file A Review of the FBI’s Trilogy Information Technology Modernization Program http://www.enterprise-architecture.info/Images/Documents/CSTB- FBI%20Project%20Review%2006-2004.pdf Congressional Testimony of US DOJ inspector general Glenn A. Fine, February 2005 http://www.usdoj.gov/oig/testimony/0502/final.pdf Matthew Patton's October 24, 2002 posting on InfoSec News about VCF http://www.infosecnews.org/hypermail/0210/6688.html The Failure Of The FBI’s Virtual Case File Project http://strategicppm.wordpress.com/2010/04/05/the-fbis-virtual-case-file-project-and-project- failure/ The FBI's Upgrade That Wasn't http://www.washingtonpost.com/wp-dyn/content/article/2006/08/17/AR2006081701485.html Anatomy of an IT disaster: How the FBI blew it http://www.infoworld.com/d/developer-world/anatomy-it-disaster-how-fbi-blew-it-243 FBI's Virtual Case File Flops http://www.internetnews.com/ent-news/article.php/3459921/FBIs-Virtual-Case-File-Flops.htm Wikipedia: Virtual Case File http://en.wikipedia.org/wiki/Virtual_Case_File GAO: Questionable fees paid on FBI Trilogy project http://gcn.com/articles/2006/05/08/gao-questionable-fees-paid-on-fbi-trilogy-project.aspx Report: FBI wasted millions on 'Virtual Case File' http://www.cnn.com/2005/US/02/03/fbi.computers/