Systems Engineering Tools@MITRE


Published on

1 Like
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Systems Engineering Tools@MITRE

  1. 1. Volume 4 - Number 1 March 2006 Systems Engineering Tools@MITRE Contents MITRE applies a variety of systems engineering tools to the work it does. Systems Systems Engineering Tools@MITRE 1 engineering tools can aid in modeling and managing enterprise architec- How Much Is tures, understanding complex systems, and assessing and managing risk, Your Architecture Worth? 1 to name a few areas. There are many tools out there: some are commer- Evolution of a MITRE Tool 2 cial products that can be applied to a broad range of jobs, while others Welcome to the Enterprise are home grown to fit the needs of a specific task or program. With all the Technology Laboratory (ETLab) 2 choices, you may have difficulty selecting a tool that meets your needs. This Using the Netica Software Tool for Terrorist Risk Assessment 3 issue highlights some of the tools that various Centers use. You’ll find articles that A Perspective on offer insight into the tools’ ease of use and effectiveness as well as the value these tools Popkin System Architect 4 add to the programs in which they are employed. Architects Cannot Live by Tools Alone: EA Must Speak to Business Users 5 The Other EA Tools 6 Using Tools in How Much Is Your the Context of EA Efforts 8 change will have on the people who are IT Investment Management and Architecture Worth? performing or interacting with a particular Enterprise Architecture 10 USMC DRPM AAA Program Benefits Owning an enterprise architecture (EA) is like aspect of the business. from Establishing an EA Program 12 owning a car. Most organizations have one. To understand or measure what your GroupSystems II: Yours may be a luxury model with all the EA is worth, you must first determine Asynchronous Collaborative bells and whistles or a bare-bones economy if the EA clearly articulates what your Collection of Requirements 12 model. It may even be an ugly junker that gets organization does to carry out its business. Want to Learn More? 14 you where you want to go, or a lemon that n Does your EA show what functions Selecting Software Tools 15 doesn’t work. But what you always want to your organization performs? Attention EA Practitioners! 15 know is: how much is your EA worth to your n Does the EA indicate how well these System Engineering Process Toolkits Available on SEPO’s Public Web Site 16 organization? functions need to be performed? The answer may be found by determin- n Does it identify what constituen- ing whether you use th EA. To carry the cies you must satisfy by function, and car metaphor further, is it on the road? whether they have the same needs as Highlights of the parked in the garage? or up on blocks? the business? next issue of Determining its value or worth is an The worth of an EA is also determined Collaborations . . . exercise in checking the incorporation of by how well the EA can manage change In our next issue, we will look at the chal- information that you use to develop the and support decisions that need to be lenges in creating systems engineering plans EA into the actions and decisions that made in the course of the day-to-day (SEPs) at MITRE. What are SEPs? How are they written at MITRE and by whom? What your organization makes and the direction business of the organization. does the customer expect to be able to do it will take. Value is also determined by n Does the EA show who is responsible with them? Who ultimately uses the SEP? how the EA aids the approach an organi- for performing specific functions in the We will present SEPs produced by various zation uses in making changes to the way business? Centers and offer hints on creating them it currently does business, or it may be n Does it show what constituencies these as well as the value that they can add to determined by the potential impact that a functions serve? programs. Architecture, continued on page 5
  2. 2. Evolution of a MITRE Tool — From Rapid Prototype to Commercial Product MITRE applies rapid System Development supporting over one billion MITRE is in the process prototyping to develop (CAASD), and the Center dollars worth of government of licensing its RiskNav concepts and solutions in for Air Force Command and programs. technology. Companies who a timely and cost-effective Control Systems (CAFC2S). Success of this MITRE- license the RiskNav software manner. A well-designed RiskNav is a risk manage- developed risk management may extend its functionality prototype may lead to either ment tool that MITRE tool led to frequent requests to build new and improved further improvements for the developed to facilitate the for RiskNav demonstra- versions for commercial as initial sponsor, or reuse by a program risk assessment tion/evaluation copies from well as government users. new sponsor. They may also incorpo- The evolution of RiskNav® The evolution of RiskNav is an example rate elements of RiskNav is an example of how this of how this process culminated in the technology into their existing process culminated in the product line. This transfer is creation of a new commercial risk tool. an excellent example of how creation of a new commer- cial risk tool. The technol- MITRE serves the public ogy transfer occurred after interest by making afford- process and help program government, academia, indus- able technology available to many incremental improve- managers manage their try, and NATO. New issues for-profit businesses. Govern- ments of the initial concept. risk space. RiskNav theory emerged related to support ment users can obtain a free Concurrently, RiskNav usage started with the support of and distribution as RiskNav copy by contacting SEPO@ expanded into many govern- an Air Force program. In evolved from a prototype to a ment agencies represented 2000, the Center for Enter- product. For more information by several MITRE Centers, prise Modernization (CEM) RiskNav is a Web-based about RiskNav®, contact including the Washington demonstrated this theory application with server and Norman Bram, 781-271- Command, Control and in a software prototype that database components. Instal- 8822, Communications (WC3) provided features lacking lation of RiskNav is more Center, the Center for in commercially available difficult than installation of Integrated Intelligence risk management tools. By a simple executable or MS Systems (CIIS), the Center April of 2003, RiskNav was Excel-based tool. for Advanced Aviation Welcome to the Enterprise Technology Laboratory (ETLab) CEM’s Enterprise Technology Laboratory (ETLab) is n Erwin v4.1. This AllFusion ERwin Data Modeler helps a hands-on systems engineering lab that also serves as create and maintain databases, data warehouses, and a showcase for sponsor demonstrations. The ETLab enterprise data models. includes collaborative lab space suitable for use by small n Extend v6.5. A flexible, extendable simulation tool that working groups, as well as Classified and Sensitive But models every aspect of an organization at all levels of Unclassified (SBU) lab facilities. expertise—from manager to engineer/scientist and The ETLab offers conferencing resources with the from novice to professional modeler. usual complement of audiovisual capabilities (VTC, n Metis v3.4.7. The EA modeling tool Metis creates, vi- plasma displays, VCR, DVD, and DSS) and a variety of sualizes, changes, shares, and manages visual enterprise UNIX and Windows-based systems. For most people, the models. lab’s real draw is its growing portfolio of software, much of n NetViz v6.0. A unique visualization tool that helps you it consisting of EA resources such as: understand and manage your information technology and business systems. 2 ________________________________________________________________________ SEPO Collaborations — Volume 4 Issue 1
  3. 3. Using the Netica Software Tool for Terrorist Risk Assessment A MITRE research project used Netica from Norsys Software Corp. (www.norsys. com) to develop a methodology for improv- ing counterterrorism analysis. The project, Indications and Warning (IW) for Countering Terrorism, employed Netica to support Bayesian Inference Networks (BINs) development in performing terrorist risk assessment. The BIN approach provides a probabilis- tic framework for analyzing terrorist attack timelines, such as the sequence of terror- ist activities that led to the September 11 attacks. Associated precursor activities serve as indicators of possible terrorist actions at various stages: intent, planning, preparation, and execution. At the planning stage, for Figure 1. Simple Bayesian Network of Terrorist Activities example, we might expect to see meetings, funds transfers, and visa applications. Capturing these precursor activities in BINs provides a At the preparation stage we would anticipate finding means of integrating disparate and uncertain indicators of surveillance of target areas, videotaping, or suspicious individ- suspicious activity and possible terrorist plans. This informa- uals seeking sensitive information or participating in question- tion can also be used to update the relative probabilities of able activities such as flight training. At the execution stage a range of associated terrorist attacks. BINs can establish we could be wary of suspicious individuals purchasing airline probabilistic links between possible terrorist actions and their tickets, of signal intelligence (goodbyes), and funds transfers necessary preparatory activities, which can then be used as (in the opposite direction). Netica, continued on page 4 n Popkin System Architect v10.0. A comprehensive These products may be used for direct project support. modeling solution that provides all the tools necessary Additional licenses for some of these tools, such as System for the development of successful EA models. Architect, Metis, and Prosight, have been set up as a n Prosight v5.0. A portfolio management tool that helps special capability in the ETLab’s Tools Testbed. While corporations make well-informed business decisions. the testbed focuses on tools of interest to CEM, it is n Rational Suite. This suite, consisting of Rational Rose available to everyone in MITRE for test and evalua- and Requisite Pro, provides the tools and processes tion. The tools testbed can be used in support of internal necessary for developing and deploying software for research programs and continuing education, and as a e-business, infrastructure and embedded systems means of collaborating and information sharing across companies. Centers. In addition, the CEM ETLab supports several data- The ETLab is located at MITRE’s Rappahannock bases and related tools such as DB2, Oracle 9i Enterprise Building on West Branch Road, McLean, VA. New tools Edition, Crystal Reports 9.0, the ERSI GIS Products are added and upgraded periodically, so to stay updated, (ArcGIS, ArcView, ArcEditor, ArcInfo, and ArcIMS), contact Jill Becker, 703-983-6720, XMLSpy 2004, the Inxight v5.0 search engine, and more. SEPO Collaborations — Volume 4 Issue 1 _______________________________________________________________________ 3
  4. 4. A Perspective on Popkin System Architect To manage the transformation of a n Defense Information Systems Agency to sponsors directly from the tool, SA business, modeling is essential. Although Information Technology Contracting gives you the option of publishing your an extensive requirements document Office Enterprise Systems Modern- models as Hypertext Markup Language could potentially cover all the neces- ization program. MITRE recom- (HTML) files so that anyone who does sary details, it’s unlikely that it would be mended the architecture components not have the tool can view and navigate read, much less understood. Enterprise to be developed for this program. We through the models. customized the meta-model as well models provide the ability to view your Many tools are available for EA. as the frameworks using SA. The De- enterprise from different perspectives Choosing the right one to perform a task partment of Defense (DoD) Archi- through a graphical representation. For tecture Framework was customized is always a challenge when considering enterprise models that require levels of to include artifacts from the Zach- cost, training, and the learning curve. SA decomposition and need to be linked to mann framework, which captured is affordable, stable, and (because it is other models, you must use a tool that information that best represents the DoDAF compliant) it’s used by many of includes a repository for architectural scope and goals of the architecture our customers. SA is currently working elements, their definitions, and the effort. on developing Core Architecture Data relationships among them. Linkages and Model (CADM) eXtensible Markup navigation capabilities allow the archi- Many tools are available Language (XML) to integrate the tool tect to see how a particular object (i.e., for EA. Choosing the with the DoD Architecture Repository activity) impacts or is impacted by other right one to perform System (DARS). DARS is a National objects in the repository. Information Infrastructure effort to Most modeling tools include an a task is always integrate architectures from multiple integrated data dictionary that contains a challenge when architecture modeling tools. information about all the graphical considering cost, training, Some complain that SA is not a representations in the repository. Other user-friendly tool and that you must and the learning curve. tools require the architect to build these be an expert in order to use it. Yes, data dictionary capabilities from scratch. training is required. The tool assumes Either way, this is a valued capability for n Business Management Moderniza- that the architect understands model- enterprise modeling, since it helps to tion Program (BMMP). BMMP ing notations (i.e., Business Process illustrate the alignment of business and is using SA for the DoD Business Modeling Notation (BPMN), Unified information technology (IT) through an Enterprise Architecture. In our inde- Modeling Language (UML), and integrated architecture model. pendent verification and validation Integration Definition for Function The Popkin System Architect (SA) role, MITRE identified gaps between Modeling (IDEF) methodology), since it V10.1 is a popular enterprise modeling artifacts and missing data with the constrains the model construction based tool that can be used to build integrated SA reporting capabilities. We also on the chosen notation or methodology. enterprise architectures. It stores the performed semantic checks and Architects also need to use the tool to object information in its integrated data identified unconformities to modeling stay current and to understand how to standards using the model integrity dictionary and allows it to be shared optimize the use of its capabilities. verification capabilities. across architectural views. This promotes For more information about the re-use of objects and eliminates the Popkin SA can be licensed as a MITRE’s use of SA, contact Andrean need to propagate information manually stand-alone tool validated on your local Colón-Padilla, 703-983-7366, to all the instances of an object. Popkin machine, or as a network tool where a SA also comes with various frameworks license manager controls the number of that serve as guides to determine what concurrent users based on the licensing artifacts should be developed to repre- agreement. With the network license sent the goals and objectives of your EA. manager, however, you can use the I’ve used SA in the following application only on the network. While programs: this means you cannot show the models 4 ________________________________________________________________________ SEPO Collaborations — Volume 4 Issue 1
  5. 5. Architects Cannot Live by Tools Alone: country can be found where cheaper, more comfortable EA Must Speak to Business Users facilities might be procured Creating a perfect technology budgeting decision makers. further their business goals for government use, you solution for a service-oriented Any lingering doubts will and needs. If this scenario wouldn’t recommend that architecture (SOA) some- surely be offset by the comes close to describ- your agency move to one of times may not be enough for projected budget windfall, ing your EA-development those sites, no matter what your customer. Even though which can only be realized if project, the deficiency repre- the projected facility’s ROI the ideal SOA covers the full spending approval is granted. sents a serious risk to your is. First, you’d give serious range of IT capabilities, it client’s success, and it needs consideration to the impact must also address the busi- “What’s wrong with of such a move on the organi- to be quickly addressed. ness goals of your client. this picture?” Federal EA projects should zation’s ability to perform Suppose that your SOA That’s the question your CIO not be driven by IT econom- its mission. Indeed, under- comes with a technical view will likely face from seasoned ics, but by a strategic vision standing an organization’s of the processes and industry business leaders who may of the business, which aims goals and objectives should standards that are needed to think, “Though you may to streamline and improve be the starting point for any implement viable instances have found a perfect answer the delivery of government significant change initiative, of the architecture. Let’s say for replacing our current services. regardless of the urgency of that the SOA also features IT systems, how does your Like the physical facili- the mandate behind it. The a logical data model that proposed solution add value ties occupied by government best place to start is at the top reflects the data entities, to our business?” To business workers and the real estate of an agency, with a thorough data exchange packages, and leaders of your agency, it upon which those facilities review of its published data repositories required to might seem that nothing in are built, IT represents just mission statement, its goals manage moving data into and the scenario described above one of the many costs of and objectives, and its latest out of the enterprise. Finally, suggests that the “perfect” doing business. Although strategic plan.1 presume that all applicable SOA was developed to many sites around the Architects, continued on page 3 security and privacy require- ments have been more than adequately met, thus confirm- ing that your SOA is as “good as it can get.” To top off this scenario, envision analytical results “proving” that for a one-time cost of $25 million the new SOA will reduce IT mainte- nance and support costs $5 million a year for the next 7 to 10 years. That is, the projected return on invest- ment (ROI) for your SOA is at least $10 million. With some luck, it might turn out to be quite a lot more. Your agency’s chief informa- tion officer (CIO) is at once ecstatic and impatient. She can hardly wait to showcase Figure 2. Federal Case Management Needs for Law Enforcement Purposes your SOA, which will be irresistible (she thinks) to SEPO Collaborations — Volume 4 Issue 1 _______________________________________________________________________ 5
  6. 6. The Other EA Tools CACI SIMPROCESS code for Web services from SIMPROCESS is a business process definitions created in Even though developing the artifacts within a particular EA ProVision. process modeling and process framework may seem a daunting task, we still need to do more. simulation tool that supports ProModel Solutions: The information within any EA framework is only a static process analysis, reengineer- Process Simulator representation. We need to perform dynamic analyses to see ing, process monitoring, Process Simulator quickly if the performance we defined is feasible and is likely to occur business performance predic- and easily adds simulation within given constraints. tion, and continual process capabilities and analysis to But even static and dynamic analysis is not enough. The management. It combines Microsoft Visio flowcharts, analysis information must be integrated in two ways with the discrete event simulation, value stream maps, and enterprise’s decision environment or the use of the EA will process mapping, flow- workflow diagrams. Process be limited. Enterprise managers must make decisions in charting, and activity-based Simulator installs as a relation to performance trade-offs (both organizational and costing. SIMPROCESS plug-in to Visio, allowing technical), project achievability, and funding. In addition to spans simulation and process you to seamlessly create the static analysis framework tools, such as Troux’s Metis, modeling and supports EA and run simulation models and Pokin’s SA, we need dynamic analysis tools and decision initiatives. inside Visio. As a result, analysis tools. ARIS Simulation you can build “as-is” and ARIS Simulation performs “to-be” workflow models and Dynamic to validate a static analysis dynamic analysis of business quickly understand time- Analysis Tools but to integrate the static processes. ARIS Simula- based quantitative impacts Dynamic analysis focuses on and dynamic analysis, so tion is fully integrated in the of changes. This simulator observing the execution of a that agency managers can ARIS Toolset, enabling use predicts resource require- model of the enterprise and make appropriate decisions of the data related to the ments, capital equipment making inferences based on given potential changes in the processes (recorded in the investments, process times, these observations. Dynamic enterprise’s environment. The ARIS Toolset) as a basis for and service levels. analysis uses various kinds 2004 MITRE Sponsored the simulation of business Decision of simulation models. The Research project “Executable processes. This supports Analysis Tools advantage of dynamic analysis Architecture Methodology for analysis of the executability In a holistic sense, EA is that it provides the ability Analysis” offers an example of processes, process weak analysis is an optimiza- to review actual results and of linking static and dynamic points, resource bottlenecks, tion problem. Given the the influences of changes in analyses. Examples of simula- and different alternative enterprise’s stated goals and variables on the results rather tion tools for enterprise archi- processes. environment constraints, how than drawing broad conclu- tecture dynamic analysis are: Proforma: does it structure its organiza- sions as the static models ProVision Enterprise tional and systems resources PIKOS: Bonapart do. The general limitation of to attain the goals? Enterprise Forty-five standard analyses ProVision Enterprise dynamic analysis is that the architects use decision analy- maintain model consistency contains 23 integrated model may not have high sis techniques to solve various and quality. They also answer strategy, process, and system fidelity with the real world parts of this overall problem. common questions on topics modelers, along with Monte and so may not represent Both the static and dynamic such as direct and indirect Carlo and discrete event what will actually occur. EA analyses provide informa- costs of people, resources simulators, model inter- Within the EA domain, the tion to support the decision and data processing systems, preters, a Web-enabled drive is to develop execut- analysis. Three key types of time series analysis, manage- publishing facility, a spell- able architectures that are decision analyses are perfor- ment scope, communication ing/completeness checker, near-direct instantiations of mance trade-offs, achievabil- structures, etc. An additional and model comparison the static artifacts to increase ity, and portfolio funding. 35 user-defined analyses facility. A Business Process real-world fidelity. n Performance trade-offs are provided and are easily Execution Language interface However, the drive to identify changes to perfor- modified to create any desired module is also available that executable architecture- mance measures to attain analysis. supports the generation of based simulation is not just 6 ________________________________________________________________________ SEPO Collaborations — Volume 4 Issue 1
  7. 7. improved performance Lumina Decision ian fragments, which can be ProSight Portfolios given various constraints. Systems: Analytica quickly combined to support ProSight provides a gover- n Achievability decides on Analytica is an influence future analysis in related nance framework for EA organizational policies and diagram-based, visual domains. and capital planning and directions that increase environment for creating, investment control (CPIC) Norsys Software the likelihood of attaining analyzing, and communicat- processes and compliance the desired overall perfor- Corp: Netica ing probabilistic models for Netica is a powerful program within the federal govern- mance, given the current ment, helping agencies business, risk, and decision that implements Bayes- defined organizational, analysis. It has a user-friendly, ian networks and influence integrate the EA and CPIC system, and acquisition intuitive interface that diagrams. Users can specify processes. It provides a performance. includes hierarchical submod- the interaction of variables in common, easily understood n Portfolio funding decides els and a variety of graphs. terms of conditional proba- reference point for dynamic on the most desirable over- Analytica is especially good bilities, equations, or data and collaborative portfolio all performance subject to the funds that are available when a problem requires files of observations. (The management, aggregating and now and in the future. modeling both continuous conditional probability distri- interfacing many data sources and discrete variables. butions will be learned from to consolidate the informa- Enterprise architects IET: Quiddity Suite these observations.) Netica tion in one location. support these three decision analyses as they perform the Quiddity is a Bayesian is very fast and comes with PlanView Enterprise static and dynamic EA analy- modeling tool that supports a multi-language application PlanView Enterprise ses. These decision analyses the modeling of complex programmer’s interface (API) combines a comprehensive must be integrated with the multi-entity inference situa- library that allows software IT portfolio management EA analyses in two ways: tions such as understand- development around Netica. solution, real-time portfolio first, allowing the impact of ing multi-organization Portfolio Analysis analytics, and best-of-breed alternatives to be understood, performance. It has a simple Which combination of resource management and and second, understanding conceptual structure based on systems provides the best processes. Working together, how a planned change to the multi-entity Bayesian model- benefits to the organiza- these elements give custom- EA may strengthen or invali- ers total IT visibility, fewer date a previous decision. The information within any EA framework redundancies, increased Many tools support decision is only a static representation. We need efficiency, and the ability analysis; I’ve grouped them to focus limited resources to perform dynamic analyses to see if the on higher value work. By into inference modeling and portfolio analysis. performance we defined is feasible and is integrating analytics with Inference Modeling likely to occur within given constraints. root-cause analysis, PlanView Enterprise enables optimiza- Performance trade-offs tion, yielding even greater such as organizational ing using first-order logic tion, given their respective cost savings. process performance and and object-oriented analysis. performance and cost? You For more information achievability analyses have these tools, click on the tool This structure allows devel- could use an inference tool influences, utility, value, and opment of Bayesian network name and it will take you to uncertainty associated with such as Analytica or one of fragments and merges the the following packages to find the vendors’ site (soft copy them. Influence diagrams fragments into a situation- only). Or contact William and Bayesian networks out. In addition, MITRE’s specific Bayesian network Center for Acquisition and Bunting, 703-983-1919, are generally used for model for analysis when such decisions. Analytica, Systems Analysis group required. The Quiddity tool has developed the Portfolio Quiddity, and Netica (see supports the development of article on page 3) are three Analysis Machine (PALMA) a reusable library of Bayes- tool for this purpose. tools to consider. SEPO Collaborations — Volume 4 Issue 1 _______________________________________________________________________
  8. 8. Using Tools in the management processes don’t automati- and what might need to be addressed cally demand an automated tool, they as it ages). Context of EA Efforts can be hampered by the lack of such a All the above arguments easily Why do we need EA tools? EA tools tool. It’s possible to use standardized translate into generic requirements for are complex business and technology office support products such as word an EA support tool. These products have programs that incorporate a great range processing, spreadsheets, databases, weaknesses, which differ from product of information, from business policies and content management systems to to product, but do not detract from to the description of processes and hold the EA’s artifacts. Creation of the the importance of having such a tool. organizations that support the policies, initial architectural elements will not Weaknesses are in the area of non-archi- through the technologies that support be impeded by those tools. However, tect access to information; management and enable those business processes. You the following activities will be impeded, level presentations and reporting; tool are just as likely to find some of the text or must be augmented by significant learning curves; tool adaptation and of an executive order in an EA as the manual work: ramp-up efforts to align with an enter- network protocols, routers, and switches n Linking EA artifacts within a domain prise’s requirements; software version that are used in the communications (higher to lower levels) updates, and control of the tool. network for the enterprise. n Linking EA artifacts between do- mains (such as tying an IT applica- Where Do Tools Fit in the This large and complex compendium Processes Supporting EA? of information must be researched, tion system to a function in a business process) The process of creating and maintaining documented, maintained, and, n Correlating EA artifacts to a frame- EAs is demanding, and requires coordi- especially, analyzed and reviewed to work or FEA reference model nation across an enterprise. The EA tool be of value in the EA. An EA exists to will be a vital component in all these answer questions and to solve business n Correlating EA artifacts to a capital investment project steps. The following diagram, created problems. If the data that support the by Charlie Martinez (WC3), shows a answers are hidden, then the EA has no n Finding any particular artifact in an generic approach to the EA process. value. EA The chart shows examples of how an An EA can contain text, flowcharts n Maintaining EA artifacts and objects EA tool is used in each high-level step of (many different flowcharting protocols (knowing what is in the EA to update the process. may exist in a single EA), tables, images, video, and data elements from relational databases. Today, the EA will also be made available via the Web to its stakeholders, and will need to commu- nicate via common proto- cols (especially XML) with outside parties such as the Office of Manage- ment and Budget (OMB). Benefits of Having EA Tools at the Enterprise’s Disposal Automated tools are becoming a required item in an EA. While the EA and other capital invest- ment governance and Figure 3. A Generic Approach to the EA Process 8 ________________________________________________________________________ SEPO Collaborations — Volume 4 Issue 1
  9. 9. Determine intended use for the Analyzing and storing (if not already in the tool) the business drivers for the effort will help to focus architecture the effort. Documenting in the tool will allow later artifacts of the EA to be aligned with the business drivers and OMB business areas/lines (a key component in obtaining capital project funding) Determine the scope of the archi- This is also an alignment-sensitive task. It does not require extensive tool interactions. However, the tectural effort categorization of EA data with scope-bounded problems will help later efforts stay within bounds of the problem set. Without a tool it’s easier to build a mountain out of a molehill. By no means, though, does the tool alone ensure a properly limited scope. Determine data required to The rigor of the tool, and the framework or artifacts chosen for this scoped effort, will now set the support architecture development “how do I collect it” bounds. The tool will help frame the questions and type of data needed. Collect, organize, correlate, and This is where the tool will be the constant companion of the architects. All data must be input or store architecture data accessed and potentially amended in the tool. The greatest volume of tool usage is during this phase. Conduct analyses in support of This is one of the two phases for which the tool produces the most obvious value, in support of. architecture objectives Questions for the EA such as “how many,” “when,” and “where” should be obtainable from the tool. Overlaps (accessing the same system, replacing the same function, etc.) can be detected. In many cases, modeling tools can be tied into the data stored in the EA tool. Document results aligned with In some cases what used to be a demanding extra step can now be rolled into a “tune the tool” or architecture frameworks “how to use the tool” step and eliminates the duplicate effort that a manual approach requires. Tools can generate output and store information in alignment with standard frameworks and increasingly generate output in alignment with the FEA reference model needs. Tools can also be configured to support a customized framework, although that requires significant additional upfront effort. Determine intended it” bounds. The tool will help frame the into a “tune the tool” or “how to use use for the architecture questions and type of data needed. the tool” step and thus eliminate the Analyzing and storing the business Collect, organize, correlate, duplicate effort that a manual approach drivers for the effort (if not already in and store architecture data requires. Tools can generate output and the tool) will help to focus the effort. This is where the tool will be the store information in alignment with Documenting them in the tool will allow constant companion of the architects. standard frameworks and increasingly later artifacts of the EA to be aligned All data must be input or accessed and generate output in alignment with the with the business drivers and OMB potentially amended in the tool. The FEA reference model needs. Tools can business areas/lines (a key component in greatest volume of tool usage is during also be configured to support a custom- obtaining capital project funding). this phase. ized framework although that requires significant additional up-front effort. Determine scope of the architectural Conduct analyses in support of effort The tools will support more ties than architecture objectives the EA process itself. The CPIC process This is also an alignment-sensitive task. This is one of the two phases in support can automatically tie to the EA tool. It does not require extensive tool interac- of the tool which produces the most Each capital investment must align with tions. However, the categorization of EA obvious value. Questions for the EA the EA-automated linkages between data with scope-bounded problems will such as “how many,” “when,” “where” major vendors to allow that alignment help later efforts stay within the bounds of should be obtainable from the tool. to be extracted from the tool itself. The the problem set. Without a tool it’s easier Overlaps (accessing the same system, subsequent Exhibit 300 generation (in to build a mountain out of a molehill. replacing the same function, etc.) can be an XML format) can then pull align- By no means, though, can the tool alone detected. In many cases, modeling tools ment with FEA reference models that ensure a properly limited scope. can be tied into the data stored in the have cascaded from the business lines Determine data required EA tool. in the EA tool. FEA reference models to support architecture development Document results aligned should have most components repre- The rigor of the tool, and the frame- with architecture frameworks sented in the architecture and therefore, work or artifacts chosen for this scoped In some cases what used to be a in the EA tool where the information effort, will now set the “how do I collect demanding extra step can now be rolled can be generated for reports. Using Tools, continued on page 4 SEPO Collaborations — Volume 4 Issue 1 _______________________________________________________________________
  10. 10. IT Investment Management Based on the Clinger–Cohen Act of 1996, and its “Select, Control, Evaluate” management paradigm (Figure 4), the and Enterprise Architecture ITIM Maturity Model reflects GAO’s and OMB’s ongoing 4 “Select, Control, Evaluate” Management Paradigm efforts to GAO’s ITIM Maturity Model ensure that IT projects are properly aligned with each agency’s Based on its experience in assisting agencies to improve the EA and that IT projects are properly managed to ensure that management of their information technology (IT) portfo- risks are mitigated and the expected return on investment is lios, the Government Accountability Office (GAO) decided achieved. to update the Exposure Draft, Version 1, of its Information Putting aside changes in detail, the most fundamental Technology Investment Management: A Framework for Assessing difference between the two versions of the document is that and Improving Process Maturity (GAO/AIMD-10.1.23), by the exposure draft focused on Process whereas Version 1.1 publishing Version 1.1 (GAO-04-394G) in March 2004. focuses on Outcomes. Figure 5 shows not only the greater The GAO document provides a framework to analyze an emphasis on performance measurement but also the criteria agency’s IT investment management processes through the use used to define the individual maturity stages. of a: As one example, Version 1.1 is very specific in GAO’s n Rigorous, standardized tool for internal and external evalu- ations of these processes minimum expectations of how an agency should select invest- ments and manage its portfolio (Stage 2 and Stage 3). By n Consistent and understandable mechanism for reporting contrast, the Exposure Draft provided only vague definitions the results of assessments and terminology: n Road map that agencies can follow in improving their n Repeatable investment control techniques are in place and processes. key foundation capabilities have been implemented (Stage 2) The GAO Information Technology Investment Manage- n Comprehensive IT investment portfolio selection and con- ment (ITIM) Maturity Model is based on similar Capability trol techniques are in place that incorporate benefit and risk Maturity Models (CMM) developed by the System Engineer- criteria linked to mission goals and strategies (Stage 3). ing Institute (SEI), e.g., CMM for software and software GAO emphasizes that the purpose of its maturity model is acquisition, and the framework developed by GAO for the not to prescribe the “how” of achieving increased control over an assessment of EAs. agency’s IT portfolio, but rather to describe the characteristics Version 1.1 both refines and differentiates from the GAO that identify increased effectiveness in managing IT assets. ITIM Maturity Model Version 1 with the clear understand- According to GAO, Stages 2 and 3 have been the primary ing that “one size DOES NOT fit all.” The document states beneficiaries of the lessons learned from the use of the frame- repeatedly that each agency must determine how best to meet work since its publication and comments received in response the specified maturity criteria in a manner appropriate to its to the Exposure Draft. Stage 2 avoids the previous ambiguity specific environment (mission, size, complexity) and the nature to ensure more consistent interpretation by specifically spelling and value of its IT portfolio. out criteria for project selection and by adding the requirement that IT investment boards be not only established but also aligned at this stage rather than at Stage 3. Experience shows that the proper alignment of IT investment boards is critical to avoid conflicts of authority and scope. Based on the significant work that has been done in recent years, GAO also points out that Stage 3 was enhanced to better explain IT portfolio management. Additionally, GAO added post implemen- tation reviews (PIRs) as a Figure 4. “Select, Control, Evaluate” Management Paradigm 10 _______________________________________________________________________ SEPO Collaborations — Volume 4 Issue 1
  11. 11. qualifier for Stage 3 maturity. (Originally they were not required investments, GAO states that it will be useful if at a minimum until Stage 4). Consequently, Stage 3 agencies now have to the organization’s EA is used as a repository for investment demonstrate full control over their IT investments, following the information. This helps decision makers to avoid duplication or Clinger–Cohen “Select, Control, Evaluate” model. overlap in approving new investments, ensures that each invest- In the context of the Engineering Archi- ment is evaluated as part of an invest- tecture Body of Knowledge (EABOK), the ment portfolio, and provides specific . . . the ITIM Maturity revised GAO model recognizes that there value to the organization. can be no disciplined ITIM decision-making Model reflects GAO’s and Only at Stage 4 and Stage 5 are without reference to the agency’s EA (to OMB’s ongoing efforts to agencies expected to use their EAs as a the extent that one exists). The impact of ensure that IT projects are major tool to improve portfolio manage- this line of thinking becomes clear in the ment and the ability of IT to shape recent changes by OMB to the requirements properly aligned with each strategic outcomes. for Exhibit 300, which must be submitted agency’s EA . . . Stage 4 emphasizes evaluation annually by all agencies under OMB Circular techniques to improve the performance A-11: Preparation, Submission, and Execu- of both individual IT projects and the tion of the Budget, July 2004, as part of their documentation for portfolio as a whole. This stage uses the organization’s EA as a budget requests relating to major IT investments5. control mechanism to: When completing Exhibit 300, Part II.A, each agency must n Add value over the typical project-centric gathering of les- not only demonstrate compliance of its major IT investments sons learned through PIRs with its EA but also state whether the agency’s EA review n Enable agencies to look across PIRs to identify recommen- board has reviewed the investment. The agency also must dations for changing and improving the overall IT invest- demonstrate how the investment maps to the EA reference ment process. models developed by OMB’s Federal Enterprise Architecture At Stage 5, the EA itself is expected to become the object Project Management Office (FEAPMO). When reviewing of business performance-oriented improvement efforts, as the submissions, OMB uses this information to score agencies agencies seek to become more agile and look to “best-in-class” on EA compliance, resulting in direct and possibly negative organizations for benchmarks and breakthrough technologies. budgetary consequences for the agency. MITRE is positioned to help government executives If anything, the GAO ITIM maturity model may be more understand the actions they need to progress toward leverag- forgiving than OMB, since most agencies are still in the process ing IT for strategic outcomes. For more information, contact of developing their EA. While OMB appears to set the bar quite Matthias Senger, Ph. D., 703-983-6353, high in its expectations of EA compliance when considering Figure 5. Performance Measurement and Maturity Stages SEPO Collaborations — Volume 4 Issue 1 ______________________________________________________________________ 11
  12. 12. USMC DRPM AAA Program Benefits from Establishing an EA Program The Direct Reporting Program overall program effectiveness rather result indicated a convergence on Management Advanced Amphibi- than to meet federal mandates. This the processes that are key to current ous Assault (DRPM AAA) Vehicle focus has yielded successful program- program objectives and long term Program is the United States Marine wide support and cooperation in estab- objectives. These processes were priori- Corps (USMC) top priority ground lishing the EA program and provided tized based on Six Sigma6 guidance. weapon system acquisition program. more immediate benefits. The critical processes serve as input for The DRPM AAA program delivers building the enterprise-wide archi- battlefield capabilities to the Navy and tecture. By examining touch points to USMC for amphibious operations in Information management is existing command, control, commu- support of expeditionary maneuver used to converge complex, nications, computers, and intelligence warfare. Information management dependent work packages (C4I) operations, the architecture is used to create convergence among helps address information flow from and activities to achieve complex, dependent work packages program management, to battlefield, to and activities to achieve mission mission success, moving logistics support. success, moving the program towards the program towards The impact of business process net-centricity to attain information net-centricity to attain development throughout the program superiority. paved the way for the EA team to The EA program helps program information superiority. build essential components for manag- management relate the program’s ing and institutionalizing EA practices vision and mission to the enterprise’s in the DRPM AAA program. This business, operations, systems, and The EA approach began with inter- includes establishing a review board, infrastructure. EA provides decision views of each directorate to capture steering committee, program plan, support for budget justification and individual views of critical success governance plan, policies, and the like. acquisition planning. DRPM AAA business processes. The processes This approach established executive is using the EA program to improve were analyzed as a collective set. The USMC, continued on page 6 GroupSystems II: Asynchronous Collaborative Collection of Requirements If the definition of a capability is a positive high-level statement deadlines for new business transformation areas. of an outcome for a business process, service, or development The EA tool GroupSystems II adds, revises, and deletes program that leads to specific functional requirements, then capabilities and requirements instantaneously among the the line of sight between capabilities and requirements begs proper team users, allowing the iterative enterprise life the ability for any large federal organization to update, change, cycle management wheel to grind on with no appreciable validate, or expand. stoppage.The enterprise life cycle governance team respon- Some organizations attempt to update this connec- sible for the quarterly or yearly changes of either capabilities tion on a quarterly basis if their missions or businesses are or their requirements sends out the list for review across the in a constant state of flux. Others only collect and report enterprise to the appropriate users. changed capabilities/requirements annually. Whether quarter- All of this can be done synchronously or asynchronously ly or yearly, within a given enterprise life cycle of projects, and sent back to the originating team to begin the process changing capabilities and subsequent requirements must be of updating, revising, or deleting capabilities or requirements collected, discussed, and validated with stakeholders in a that have changed over the course of the project’s life. timely and user-friendly manner in order to meet submission At the right is a sample of the GroupSystems II EA tool 12 _______________________________________________________________________ SEPO Collaborations — Volume 4 Issue 1
  13. 13. Architects, continued from page 5paper, Post-It notes, masking An Illustrated management needs for all When Your Tool Bag tape, whiteboards, and magic Concept of Federal case managers. Almost im- Is Empty, Techniques markers. Indeed my experi- Case Management mediately, we discovered Can Save the Day ence suggests that the more To assess the communica- important differences in case management require- For an EA initiative to start at primitive the tools, the easier bility of a single diagram ments traceable to the way the top, however, your team it is to connect with business (below), which aims to convey different communities of needs to have the wherewithal leaders in such a setting. a great deal of information interest manage their cases to perform effectively at a Once this launch phase of about the federal vision2 for and interact with case files. strategic level, where even an EA initiative is success- business, processes needed to n Specialized business com- the fullest kit of EA tools is fully negotiated, aided by the support the Case Manage- munities energized: by soon rendered all but useless. liberal use of envisioneering, ment line of business3, and focusing on the early de- It also means that your team the law-enforcement-related velopment of this concept the next most important must help your client formu- domains that need to apply at the top level, we were techniques to be intro- late or refine a vision of the those processes. Developed also able to quickly recog- duced are communication, future organization. In many by CEM for the Department nize different stakeholder communication, and commu- communities, which have instances, you must sketch of Justice (DOJ), the concept a new paradigm for future nications. Even the sharp- since been empowered depicted in Figure 2, page 5 business operations. At this est business vision, if not to address their distinc- has given rise to a number early stage of the initiative, effectively conveyed, will have tive sets of case manage- of important consequences, innovative techniques are limited impact on the way ment requirements more including: future business is conducted. intently. needed more than tools. n Separate procurement You’ll also need thinkers Here again, my own experi- initiatives for each of the n Enterprise-level strategic who can creatively wander ence suggests that the simpler (three) depicted sets of decision support: OMB the medium, the more widely case management needs: and DOJ have also been beyond traditional boundar- this decision alone proved able to do a better job of ies and established role defini- the message is likely to be to be a major milestone setting implementation tions. The idea is to engage disseminated. That’s why a priorities, which help to the client in imagineering PowerPoint slide or two can for federal law enforce- ment officials. During inform both near-term exercises, not engineering be more effective than the and long-term budget activities. The finest tools to some 20 years prior to output of the most powerful considerations. this concept, DOJ had be brought to bear on these EA tool when it comes to failed to complete four For more information, work sessions may be nothing communicating a visionary separate initiatives aimed contact Ken Mullins, 703-983- more elegant than chart concept to business users. at defining a set of case 3349, voting form that allows users to choose from a list of Number of capabilities. The voting form can be tailored to show multiple People Who Polling Method:Yes/No Polled Each choices, check boxes, or other custom designs. Additionally, Value GroupSystems II gives contributing stakeholders the luxury Ballot Items Yes No of adding new capabilities asynchronously, along with words in support of their additions in preparation for the face-to- CAP 001 Ability to set the time to 0 0 generate the number 1 face discussions of the delta since the last updates. The value of this tool is immeasurable. It allows an CAP 002 Ability to set the time to 0 0 generate the number 2 organization to quickly expand content-rich capabilities into CAP 003 Ability to set the time to 0 0 useful requirements, granting users the line of sight needed generate the number 3 to trace their business needs to the project/lifecycle level. CAP 004 Ability to set the time to 0 0 This tool should not be taken lightly by large enterprises, for generate the number 4 it is a collaborative, organizationally safe method of getting real work done. CAP 005 Ability to set the time to 0 0 generate the number 5 SEPO Collaborations — Volume 4 Issue 1 ______________________________________________________________________ 13
  14. 14. Netica, continued from page 3 Using Tools, continued from page Tool Vendor Challenges IWs. In particular, individual IWs The systems development process The immature state of the EA tool can be associated with multiple possible should also benefit from accessing the market presents challenges to the terrorist actions. An analyst can use any EA tool; business requirements can architect and the enterprise. Many tools subset of observed IWs to update the be derived directly from the business are descended from system develop- probabilities associated with possible drivers and business flows in the ment tools and lack the sophistication terrorist actions. architecture. They can be drawn into for tracking things at an “enterprise” The figure shows a simple Bayesian requirements tool suites for consider- level. And because vendors of tools (EA, network of terrorist activities with some ation. Similarly, the technically oriented portfolio management, systems develop- activities highlighted as observed (i.e., and application-oriented needs can be ment, and regulatory) are not universally 100 percent “True”). These observa- drawn into systems development tools connected, it takes considerable effort tions (IWs) can then be used to assess or reviewed by IT staff when needed. to pull data from one tool into another. the relative likelihood of two potential Tools in a The consolidations in today’s market terrorist attacks – in this case a Septem- Federated Environment (Troux/Computas; Telelogic/Popkin) ber 11-type attack and a nuclear plant also show that we cannot guarantee that Federated enterprises, such as cabinet- attack (via explosives carried on a small one particular approach to EA tools level departments (Health and Human airplane). will remain constant over time. These Services, Treasury, Department of Uses of BINs include: challenges are not to be taken lightly, but Homeland Security) where individual n Ranking possible terrorist actions they do not mar the basic EA tool value architectures must be analyzed, lever- by either probability of occurrence, proposition. expected value to a terrorist, or aged, and rolled up into department-level architectures, present a stronger case for Just as systems development tool impact on the United States EA tool automation. Looking at multiple sets have undergone significant evolu- n Assessing the potential impact of architectures reveals the need to use tion over time, EA tools will also change proactive countermeasures on significantly. Today, however, the EA detecting IWs, thus reducing risk systems to help correlate high volumes of information. If the tools in use are process can benefit from applying these of successful terrorist attack consistent across the federated enter- tools to support the intended use of the n Providing insight into the most prise, rollup assistance from the product architecture. effective allocation of is likely. If not, interfaces and manual For more information, contact counterterrorism resources. analysis will make the rollup harder to Bruce Gordon, 703-983-1024, For more information, contact accomplish. A federated enterprise can Richard Moynihan, 781-271-6227, institute standards that will benefit all in its community for the long term. Want to Learn More? BP Trends (http://www. n CACI’s SIMPROCESS n Proforma Corporation’s We recommend that has n Holocentric’s Holocen- ProVision you periodically visit the published a report on tric Modeler n ProModel Solutions’ Enterprise Architecture enterprise architecture n IDS Scheer’s ARIS Process Simulator Solutions page to see new tools that might interest n iGrafx’s iGrafx n BML Innovations’ links and other resources. you. It is titled, The 2005 xBPM Innovation You may access the page by n MEGA’s MEGA Suite Enterprise Architecture, The report culminates using Fast Jump to EAS, or Process Modeling Simula- n Popkin Software’s Sys- go directly to http://cem/ in a nice matrix compar- tem Architect tion Tools Report, and ing these tools. You may sites/V400/V410/eas/ contains information on n Process Wizard Ltd.’s default.aspx. download the report for the following tools: ProcessWizard free, but you must register at the site to do so. 14 _______________________________________________________________________ SEPO Collaborations — Volume 4 Issue 1
  15. 15. Selecting Software Tools each detailed requirement within each category, and assign a weight to each category that reflects its importance. MITRE is often asked to help our clients select the best n Identify your evaluation team. This team will attend all software product to perform a certain task. Frequently, many vendor presentations and demonstrations and evaluate each candidate products might be suitable: but how do you identify product. the best product for your client? The following steps are a n Contact vendors and schedule demonstrations. It can be guide for selecting software products. very time consuming to coordinate all evaluation team n Identify tool requirements at a very high level. Understand- members’ schedules and facilities for the vendor demonstra- ing a tool’s functionality at a high level helps to identify tions. stakeholders better and perform detailed requirements collection. n Evaluate each product. Rate how well the product meets every detailed requirement on a fixed scale. n Identify your stakeholders. Determine the primary and secondary users and each user’s role. Once the scores have been computed and the tool that best n Perform a detailed literature search. Make use of available meets the requirements is identified, there may be other factors information on software tools within MITRE, on the Inter- to consider. For example, technical evaluation may uncover a net, and from other sources such as Gartner, but remember requirement that is not acceptable to the acquiring organiza- that some information may not be accurate or current. tion. The version of the tool evaluated may not be currently n Collect detailed requirements. Identify the precise func- available. The vendor may not be able to support the tool in a tional requirements for the tool. timely manner. These and other considerations may be consid- n Identify technical evaluation criteria: although not impor- ered so significant that the tool that best meets the require- tant to the end users, these criteria may be very critical to ments may not be the best tool to acquire. The organization the support organization. Some technical criteria for an may consider these other factors and select a tool with less EA tool are: system architecture and infrastructure, system functionality, but with immediate availability. performance and capability, and system support. For more detailed information about selecting software n Categorize, prioritize, and weight the detailed requirements. tools, contact Carole Meals,, 703-983-1536. Group the detailed requirements into categories, prioritize Architecure, continued from page To understand or measure n Does it show what services and Attention information need to be provided to what your EA is worth, you perform a specific function? must first determine if the EA Practitioners! n Does the EA identify the performance EA clearly articulates what If you are supporting an enterprise or quality that needs to be achieved as these services are performed? your organization does to architecture project for MITRE The most important determinant of carry out its business. or one of its clients, or if you have the worth of the EA is how well the EA some expertise in the field that you can support the future direction of the that, if included in the EA, drive both are willing to share, please regis- organization. understanding and knowledge about the n Does the EA reflect the current organization, while supporting technology ter your contact information on strategic direction of the organization? investment decisions and organizational the survey form on the Enterprise n Does it identify the functions that change. The worth of the EA is a function Architecture Solutions site. You support the organizational strategies of how well the EA reflects the organiza- tion and how the organization uses the can access this site via FastJump at directly versus those functions that provide indirect support or those that EA. Just like your car, the EA’s worth is CONTACTS4EA, or go directly do not support the strategic direction? determined by how effectively it gets you to it at: http://cem/sites/V400/ n Does the EA identify the functional to your destination. V410/eas/Lists/EA%20Contacts/ gaps, i.e., functions that are not For more information, contact Dr. performed by the organization? Mary T. Ayala-Bush, 03-83-460, overview.aspx. These questions point to key elements SEPO Collaborations — Volume 4 Issue 1 ______________________________________________________________________ 15