This document proposes an Intelligent Data Entry and Acquisition (IDEA) system to help with on-site highway maintenance and construction. It describes an architecture using wearable computers and sensors to collect asset data in the field, process it using pattern recognition, and upload it to centralized databases. Field workers could use tools like digital notepads, cameras, and GPS to gather location-tagged images, notes and condition reports on assets, which the IDEA system would then analyze and integrate into maintenance planning databases back at the office. The goal is to streamline data collection and improve safety, productivity and data quality for tasks like infrastructure inspections.

1. Smart and Secure Data Access
for
On-Site Highway Maintenance and Construction
Intelligent Cooperative
Data Acquisition and Identification
Dr. Martin Dudziak
Parikh Advanced Systems, Inc.
September, 1998

2. Presentation Outline
System and Enterprise Issues
IDEA (Intelligent Data Entry and
Acquisition) Architecture
Practical Implementation Scenario
10/10/12 Parikh Advanced Systems, Inc.
2

3. Data Collection and Currency is more than a
Network and Systems Performance Issue
10/10/12 Parikh Advanced Systems, Inc.
3

4. Enterprise-Level Premises and Constraints
Field-based operations and management vs.
Centralized control structures
Integration of Computer Technology with
Nuts-and-Bolts Technology
Must be oriented to more than the IT-
proficient users of the departments and
divisions
10/10/12 Parikh Advanced Systems, Inc.
4

5. Platform Independence and Versatility
“I need a hands-free PC for this job”
“This is not the belt-pack wearable PC I used last
week but it will do”
“This job needs pictures and video and all we have
is this serial-port camera interface”
“I’m going to be in the field for all day and need a
longer-life extra battery”
“I can’t be bothered with a display panel; where’s
the headset?”
10/10/12 Parikh Advanced Systems, Inc.
5

6. Hands-free and Wearable - Why?
Full-power for computing and communication
Desktop capabilities for not much more $
Speech interface that fits Everyman
Displays that fit the task - headset, flat-panel,
flexi-wrap
Expandable and modular
10/10/12 Parikh Advanced Systems, Inc.
6

7. Two Exemplary Products
ViA II
MENTIS
10/10/12 Parikh Advanced Systems, Inc.
7

8. 3 Areas of Integration and Improvement
Speech, Paper Notes, Sketches, and Vision
Security and Simplicity of the
SmartCard
Assisted Intelligence & Intelligent Assistance
10/10/12 Parikh Advanced Systems, Inc.
8

9. Generic Inventory-Oriented Data Collection
Van-based Extra-
Video vehicular
Recording Video/Still
Recording
Location-Indexed
Image/Loc Recs
Interactive
ID/Condition
Resolution
System
Maintenance
Database
Entries
10/10/12 Parikh Advanced Systems, Inc.
9

10. Early Version of IDEA -- Projected IDAAT Work Flow
Field (Mobile) Base Station
Field Collection Units RDBMS
DATA SERVER
10/10/12 Parikh Advanced Systems, Inc.
10

11. Projected IDAAT Data Flow (cont.)
GIS,
Inventory,
Maintenance,
Financial,
Planning
Databases and
Decision
Field (Mobile) Support
Collection Unit Field (Mobile) Base Station
Systems
Data Cards
10/10/12 Parikh Advanced Systems, Inc.
11

12. IDAAT Network Overview
Maintenance
Training
Workstation
Asset Counting
Unit (BarCode)
GIS, Spatial,
Address Data
Asset Resources
Inventory
Unit (PDA)
Asset Counting IDAAT Relational/Object-
Unit MagStrip Oriented
Reader
Base Station
Data Server
Portable
GPS Field CAD/Map
Unit Publishing Server
In the Field
Pen-based
Portable Digital Body-
Notepad Wearable In Regional and Central
PC Offices
 Local/manual connectivity   High-speed LAN/WAN (ATM) Network

10/10/12 Parikh Advanced Systems, Inc.
12

13. IDAAT Base Station Architecture
PCMCIA
(PC Card)
Reader or
equivalent
PDA Docking
Station or Direct Network
equivalent Pentium Workstation
(Desktop Minitower)
64 MB RAM, 5 GB disk,
CDROM, modem, spare Bays,
2 serial ports)
Connectivity Dial-Up Modem
Magstrip/
Barcode Reader
Docking Port
Land-Line Wireless
Standard PC
I/O Ports and
Video Board Devices
with Camera
Port(s) UPS power backup for mobile
(vehicle-based) operations
10/10/12 Parikh Advanced Systems, Inc.
13

14. PDA/HPC Connectivity and Data Transfer
Cradle/Docking Station with
standard Synchronization software
PC Card or Docking Station connectivity
Field (Mobile) Base Station
PC Card exchange or
internet connectivity
Internet or LAN connectivity
10/10/12 Parikh Advanced Systems, Inc.
14

15. Base Station to Network Connectivity
District network, sharing of incident
and asset status multimedia data
Central data server(s) linking
to IDAAT and other central systems
with GIS and full asset data acquisition
data sets collected in the field
Field (Mobile) Base Station
Financial and accounting system,
providing task and timekeeping data
10/10/12 Parikh Advanced Systems, Inc.
15

16. Intelligent Data Entry and Acquisition
(IDEA)
Van-based
Recording Extra-vehicular
Recording
Asset-Targeted
Image/Loc recs
Automated
Asset ID and
Condition
Rejections
Assessment
Interactive
ID/Condition
Acceptances Resolution
Resolutions
IDAAT Database
Entry
10/10/12 Parikh Advanced Systems, Inc.
16

17. IDEA System Architecture and Interfaces
Wireless
Internet Video/
GPS Still Cam
Tracking
Body- Voice
Linear wearable Input
Ref Data PC
Instrumentation Van
Docking Station
IDAAT Data Collection
Post- Processing
10/10/12 Parikh Advanced Systems, Inc.
17

18. IDEA in Action - Collection
Target Acquisition and Identification
Image/video (optional)
GPS location (automatic)
Asset characterization
Notepad sketches (optional)
Voice input (optional)
10/10/12 Parikh Advanced Systems, Inc.
18

19. Intelligent Data Entry and Acquisition System Integrated Maintenance Management
System and Databases
(typically Oracle 8 on RS/6000
or similar platform)
District Field Office
PC 2
(Base Station) Task dataset loaded onto
WorkCard in Base Station PC
WORKCA 1 Card Reader
RD
6
WorkCard returned to Base 3
Station PC for upload
Wearable PC
WORKCA with CardReader
RD built-in or as plug-in
WORKCA
RD GPS
Keyboardinput
5
4
Work completed and WORKCARD
time-stamped and ready for upload In-field data collection process; data Camera or
through BASE station processed on PC and stored on video
WORKCARD
Internet
access
Voice
input
10/10/12 Parikh Advanced Systems, Inc.
19

20. NTriplicate and the Digital Pad
10/10/12 Parikh Advanced Systems, Inc.
20

21. From Pad to Form to Database
Ntriplicate Form Scanner – Rayzer’s Red Cross Blood Bank Drive
File View Navigate Scan Options Help Report Pane
Open Form (Notebook) *8/12/98, 3:29:10PM:
Open Pad TR12 form with 13
of 14 fields processed
*TR10 form with 22
Upload Form (Notebook) of 22 fields processed
Upload Pad *TR12 form with 14
of 14 fields processed
Exit
Transferring form 5 of 24 forms to
http://www.redcross.org/bloodbank/
CANCEL
5
24
Preliminary NTriplicate Form Scanner User Interface
10/10/12 Parikh Advanced Systems, Inc.
21

22. IDEA Internal Data Model
PTR (Primary Transaction Record)
FIELD VALUE
Unique key alphanumeric string
User-ID alphanumeric string
TCR Field List alphanumeric string; field pointers
separated by delimiters
Jobstart date/time
Jobend date/time
optional other task-defining fields (optional)
TCR (Transaction Content Record)
FIELD VALUE CONTENT
PTR key alphanumeric string pointer to the associated PTR record
file list linked list; e.g. (see below) (link/field ID + file pointer + locator in file)
--- link/field1 text memo in MEMO.XXX, loc 001
--- link/field2 still photo in PHOTO1.YYY
--- link/field3 text memo in MEMO.XXX, loc 002
--- link/field4 sketch in DRAW01.ZZZ
--- link/field (n) video clip in VID01.XXX
10/10/12 Parikh Advanced Systems, Inc.
22

23. IDEA On-Board Intelligent Processing
Assimilation and Normalization
Feature Extraction - Building the
Representation Set
Classification - Matching Examples
Resolution - Interaction by the Expert
Database Entry and QA/QC
10/10/12 Parikh Advanced Systems, Inc.
23

24. Adaptive Pattern Recognition -
Like Humans Do It
Beginning of recognition process Later stage of recognition process
10/10/12 Parikh Advanced Systems, Inc.
24

25. Another Example - Cutting Through the
Noise
Beginning of recognition process Later stage of recognition process
10/10/12 Parikh Advanced Systems, Inc.
25

26. Other Aspects of Asset Identification,
Assessment, and Planning ...
10/10/12 Parikh Advanced Systems, Inc.
26

27. Extending IDAAT and the IDEA Model
to Many Highway Engineering Problems
Non-Destructive Testing
Bridges and Metal Structures
Tanks and Containers
Hard-to-reach, Hard-to-inspect Structures
Traffic Incident / Density Planning and
Emergency Response
Motorist Safety and Security
Emergency Weather Response
10/10/12 Parikh Advanced Systems, Inc.
27

28. One Interesting New Technology for NDT,
One More Plug-In for the IDEA System
Magneto-Optic Defect Inspection and
Imaging
Easy to use
Safe and “hassle-free”
Images, not abstract data (e.g., ultrasonics)
Can be automated
Relatively inexpensive
Easy training, interpretation
10/10/12 Parikh Advanced Systems, Inc.
28

29. Magnetic Imaging for Stress and Defect
Detection in Bridges and Poles
Microcracks
Anomaly/Occlusion
10/10/12 Parikh Advanced Systems, Inc.
29

30. More Images Pertaining to
Highway Structure Safety
Surface
Interior
Crack hidden by ice
10/10/12 Parikh Advanced Systems, Inc.
30

31. And A Few More Still...
Soft Steel with artificially
created pit defects
Soft Steel with
artificially created
gouges and pits
10/10/12 Parikh Advanced Systems, Inc.
31

32. Acknowledgements
Ray Reaux, PARIKH
NTriplicate architecture and development
Ray Lindquist, PARIKH
Enterprise information technology analysis
Dan Widner, VDOT
GIS Systems Engineering
Robin Bresley, VDOT
ICAS Project Manager
Murali Rao, VDOT
IMMP Technology Manager
10/10/12 Parikh Advanced Systems, Inc.
32