1. Building a Requirements Management System for Software Requirements
Kevin Ryan
Honolulu Community College
Rob Nelson, Peter Konohia, Des Iorgova
Akimeka, LLC
2. • Akimeka is a software development firm that specializes in IT
solutions focused for the Department of Defense.
Services Include:
Enterprise Architecture
Cloud Infrastructure
Virtualization
Web-Based Technologies
Database Management
Software Development
Network Engineering
Information Security Assurance
Akimeka LLC
Software Company
1305 N Holopono St #3, Kihei, HI 96753
Phone: (808) 442-7100
Akimeka, LLC
3. Project Context
• Akimeka for many years has maintained, improved, and developed
specialized computer systems and software for the DoD.
• Medical Record Database Directory
• User/Mobile friendly website
• The development team enhances the software per requirements
defined by the DoD.
• Requirement has to be verifiable, clear, and concise.
• Example Requirement: The input form must produce validated JSON and
send output to the name server.
EXCEL
Example Requirement
4. Project Context
The input form must produce validated JSON and send output to
the name server.
• That requirement needs to be properly documented in a
Requirements Management System
• The current system is a hybrid of MS Excel sheets and a MS Word document
• Difficult to analyze requirements
• Traceability is an issue
• Potential for inconsistency and duplication.
EXCEL
Example Requirement
MS Word
Example Requirement
5. Project Objective
• The objective of the Requirements Management System is to store
baselined software requirements from the DoD.
• By doing this we will improve on…
• Requirements analysis
• Data accuracy (No duplicates)
• Search capability (Query displays all pertaining records)
• Traceability
• Consistency
6. My Project
• My project was to create a Extract, Transform, & Load application
that completely and accurately moved data from MS Excel
spreadsheets into Oracle database tables.
• The ETL Process will structure the data and baseline software
requirements into the Requirements Management System.
• ETL Goals:
• All records are migrated to the database.
• Values that are “N/A” will be replaced to NULL.
• Uses SQL Scripts to compare tables so that no duplicate data will be
baselined or processed.
8. Custom Java vs Pentaho Data Integration
Criteria
Custom Java
Code
Pentaho Data
Integration
(Kettle)
The solution must be able to extract all data from the excel spreadsheet. Yes Yes
The solution must map data to the specific key value pairs in the data dictionary
for JSON output.
Yes Yes
The solution must map data to the specific column name in the data dictionary
for SQL output.
Yes Yes
The solution must be able to write output as JSON. Yes Yes
The solution must be able to write output as SQL. Yes Yes
The solution must be able to adapt to changes quickly in the event that datatypes
change.
No Yes
Can we plan design, build, implement, test, and document the solution within 5
weeks?
No Yes
9. Custom Java vs Pentaho Data Integration
Criteria
Custom Java
Code
Pentaho Data
Integration
(Kettle)
The solution must be able to extract all data from the excel spreadsheet. Yes Yes
The solution must map data to the specific key value pairs in the data dictionary
for JSON output.
Yes Yes
The solution must map data to the specific column name in the data dictionary
for SQL output.
Yes Yes
The solution must be able to write output as JSON. Yes Yes
The solution must be able to write output as SQL. Yes Yes
The solution must be able to adapt to changes quickly in the event that datatypes
change.
No Yes
Can we plan design, build, implement, test, and document the solution within 5
weeks?
No Yes
10. Custom Java vs Pentaho Data Integration
Criteria
Custom Java
Code
Pentaho Data
Integration
(Kettle)
The solution must be able to extract all data from the excel spreadsheet. Yes Yes
The solution must map data to the specific key value pairs in the data dictionary
for JSON output.
Yes Yes
The solution must map data to the specific column name in the data dictionary
for SQL output.
Yes Yes
The solution must be able to write output as JSON. Yes Yes
The solution must be able to write output as SQL. Yes Yes
The solution must be able to adapt to changes quickly in the event that datatypes
change.
No Yes
Can we plan design, build, implement, test, and document the solution within 5
weeks?
No Yes
13. Flow Chart of the Data for ETL processing
EXCEL
EXCEL
EXCEL
14. Flow Chart of the Data for ETL processing
EXCEL
Staging
Table
EXCEL
EXCEL
Staging
Table
Staging
Table
15. Flow Chart of the Data for ETL processing
EXCEL
Staging
Table
EXCEL
EXCEL
Staging
Table
Staging
Table
Data Cleanup
and
Transform
Draft
If No Management Review
16. Flow Chart of the Data for ETL processing
EXCEL
Staging
Table
EXCEL
EXCEL
Staging
Table
Staging
Table
Data Cleanup
and
Transform
Draft
If No Management Review
Final
If Yes
Oracle
Database
17. Load raw data into staging table
EXCEL
Staging
Table
EXCEL
EXCEL
Staging
Table
Staging
Table
Data Cleanup
and
Transform
Draft
If No Management Review
Final
If Yes
Oracle
Database
20. After Evaluating data from the Staging Table
• As expected, a SQL SELECT Query will not reference the correct
amount of records.
• SELECT * FROM PR.STG WHERE PROJECT_NAME=‘Medical Situational
Awareness In Theater (MSAT)’
• PROJECT_NAME
• Medical Situational Awareness In Theater
• Medical Situational Awareness In Theater (MSAT)
• Medical Situation Awareness In Theater (MSAT)
21. Create a Transform: Find and Replace Method
STAGING
TABLE
REFERENCE TABLE
(MAP_COLLECTED_
TO_TARGET)
DRAFT TABLESQL Script SQL Script
SELECTS
DISTINCT
VALUES
FIND AND
REPLACE
23. MAP_COLLECTED_TO_TARGET Table
• Medical Situational Awareness In Theater
• Medical Situational Awareness In Theater (MSAT)
• Medical Situation Awareness In Theater (MSAT)
RCS_COLUMN_NAME COLLECTED_VALUE TARGET_VALUE
PROJECT_NAME Medical Situational Awareness In Theater Medical Situational Awareness In Theater (MSAT)
PROJECT_NAME Medical Situational Awareness In Theater
(MSAT)
Medical Situational Awareness In Theater (MSAT)
PROJECT_NAME Medical Situation Awareness In Theater Medical Situational Awareness In Theater (MSAT)
RELEASE_NUMBER One.7.5.4 1.7.5.4
25. Flow Chart of the Data for ETL processing
EXCEL
Staging
Table
EXCEL
EXCEL
Staging
Table
Staging
Table
Data Cleanup
and
Transform
Draft
If No Management Review
Final
If Yes
Oracle
Database
27. Results
• Ingested 5608 rows of data, each requirement is unique and baselined.
• Structured
• Accurate
• Searchable
• SELECT COUNT (*) FROM RM.REQUIREMENTS
• All records are migrated to the database.
• SELECT * FROM RM.REQUIREMENTS WHERE COLUMN_NAME=‘N/A’
• Values that were “N/A” was replaced to NULL.
• Input validation/exception handling
• Utilizes SQL Scripts to compare tables so that no duplicate data will be
baselined or processed.
28. Acknowledgments
Akamai is led and managed by the Institute for Scientist & Engineer Educators at the University of
California Santa Cruz, in partnership with the University of Hawaii Institute for Astronomy. Funding
for the 2015 Akamai Internship and Mentor Program is provided by: Thirty Meter Telescope
International Observatory, THINK Fund at the Hawaii Community Foundation, University of Hawaii
System, University of Hawaii at Hilo, National Science Foundation (AST#1347767), and National Solar
Observatory.
Akimeka
Rob Nelson
Peter Konohia
Des Iorgova
Akamai Workforce Initiative
Lisa Hunter
Austin Barnes
Jerome Shaw
David Harrington
Mike Nassir
