1.
Running Head: NASA INTERNSHIP PROJECT PLAN 1
NASA Internship Project Plan:
ICARTT Data Merge Configuration
Austin T. Schaffer
NASA Langley Research Center

2.
NASA INTERNSHIP PROJECT PLAN 2
Abstract
The primary focus of this internship is to improve the ICARTT (International Consortium
for Atmospheric Research on Transport and Transformation) data merge process, a process that
joins a series of ICARTT data files into a single ICARTT data file, which aligns all
measurements at different time scale onto a common time stamp. The current data merge process
uses a tool that was developed in the mid 1990s. Also, this process has to be configured by hand,
is prone to error, and often has to be run overnight, due to its inefficiencies. This project is
intended to streamline and automate the data merge process as much as possible.
The primary goal of this internship project is to build an application that will allow users
to configure the data merge process. A replacement to the current data merge process is being
developed in collaboration with other student developers. This application will be a C# Windows
Forms application, which will allow users to generate configuration files for the data merge
process much easier than before. This will be achieved through grouping as many actions
together as possible, based on similarities between files that contain data of the same species.
This internship will also develop a specification for configuration files that can be used by the
replacement data merge process.

3.
NASA INTERNSHIP PROJECT PLAN 3
NASA Internship Project Plan:
ICARTT Data Merge Configuration
Communication with Mentor
Two co­mentors will oversee the products of this internship. Face­to­face communication
with the mentors will take place on a weekly basis at minimum to provide progress reports,
solicit feedback, and resolve any issues that arise. More frequent communication will take place
via email, and in addition, some works in progress (such as flow diagrams or code) can be
discussed using collaboration tools like Google Documents or GitHub.
ICARTT Merge Process
The data merge process takes a collection of data files from a single mission and
combines them to create a collection of larger data files. Each data file from a mission only
includes data from a single PI (principal investigator) using a single instrument or set of
instruments on a single day. As a result, missions will typically create over a dozen data files for
each day of the mission. The data merge process will create a new data file for each day of the
mission. Each of these new files will absorb all of the mission data files that were created on the
day that the new file represents. These new data files are useful for creating and verifying
models, since they include all relevant data points generated by the mission.
Existing Process
The current merge process was developed to perform merges on data with the GTE file
format. ICARTT data files have to be converted to and from GTE before and after each merge.
Users must configure the existing process by populating configuration files by hand, a slow and
tedious process.

4.
NASA INTERNSHIP PROJECT PLAN 4
Replacement Process
The merge process in development will be designed to work specifically with the
ICARTT data format, tailored to user requirements. Whenever the ICARTT format is updated,
the new process should also be able to be easily updated to accommodate the changes. The new
process will automate most of the merge configuration sub­processes. The new process will also
generate the configuration file automatically, so that no data files will be written by hand.
Plan of Attack
Below is a rough outline of the phases that will lead to the completion of the final
product. Each phase will be prevalent throughout the duration of the internship, but will show
peak activity in the order described.
Phase 1​. Work with data merge scientists to understand the mechanics of the current
merge process. Identify areas of strength and weakness with the original program. Begin to
understand the structure of ICARTT data files at an intuitive level. Begin to formulate a flow for
the replacement process.
Phase 2.​ Develop utilities for parsing and interpreting ICARTT data files. Develop
utilities for loading a directory of ICARTT files. Develop utilities that will generate log files,
allowing users to find and fix errors in the replacement process as well as the input data files.
Phase 3.​ Formulate processes that will allow users to easily and visually execute group
actions on a set of data files.
Phase 4.​ Design a replacement schema for the configuration files that will be used to
configure the replacement merge program.
Phase 5.​ Design, build, and test GUI application.

5.
NASA INTERNSHIP PROJECT PLAN 5
Phase 6.​ Build utilities that will allow users to edit previously generated configuration
files.
Phase 7.​ Test replacement merge process against ICARTT files that were produced using
the existing merge process.
Technologies
ICARTT File Format
The ICARTT file format standards were developed to fulfill the data management needs
for the International Consortium for Atmospheric Research on Transport and
Transformation (ICARTT) campaign in 2004. The ICARTT study consisted of eleven
highly coordinated individual field experiments with over 300 government­agency and
university participants from five countries, i.e., US, Canada, UK, Germany, and France.
A common and simple­to­use data file format, ICARTT file format was established for
this study to primarily facilitate data exchange and to promote collaborations among the
science teams for achieving the ICARTT science objectives. The ICARTT file format is
text­based and composed of a header section (metadata) with critical data description
information (e.g., data source, uncertainties, contact information, and brief overview of
measurement technique), and a data section. Although it was primarily designed for
airborne data, the ICARTT format proved to be practical for other mobile and
ground­based studies and various data types. Upon the success of the ICARTT study, the
ICARTT file format has since been widely accepted in the atmospheric composition field
study community and used in recent major airborne studies sponsored by NASA, NSF,
NOAA and international partners. (ICARTT File Format, 2013, p. 1)

6.
NASA INTERNSHIP PROJECT PLAN 6
Above is the abstract from the ICARTT File Format Standards. At the time of writing this, the
most recent version of the ICARTT Format Specification is version 1.1 (one point one). ICARTT
files are designed with a headers and data sections that are readable by humans. This allows them
to be editable, portable, and sharable without the need of specialized programs. This also is what
leads to some of the issues with the data type, since different PIs may have different
interpretations of how to create an ideal data file.
Replacement Merge Process
The ICARTT data merge process is being built by another intern using the C++
programming language. C++ was chosen in order to support the servers that will use the
replacement data merge process in the future. As a result, the merge process can only be operated
by Windows operating systems.
C# Windows Forms Application
The application used to generate configuration files for the replacement merge process
will be built on the C# programming language. As a result, it can only be operated by Windows
operating systems. C# was chosen for its GUI development resources and to match the system
requirements of the replacement merge process. Also, Microsoft Visual Studio has utilities for
streamlining GUI application development processes. In the event that the application needs
adjustment, users should be able to easily debug the source code.
ICARTT Merge Configuration File Schema
The replacement data merge process will use configuration files, generated according to
an XML schema. The output of the C# Windows Forms Application will be a single
configuration file that will allow the replacement merge program to successfully execute all

7.
NASA INTERNSHIP PROJECT PLAN 7
normal merges for a particular mission. The configuration file schema will also have the capacity
to be used to configure custom merges to fulfill specific customer orders.