1. BNSF Machine Vision Project Update Matthew Greve, Tristan Rickett, Anirudh Vemula, and Lingfei Zhang

2. Outline Project Background Wayside Installation System in Sibley, MO Train Monitoring System (TMS) Automatic Equipment Identification (AEI) Data Mini Universal Machine Language Equipment Register (UMLER) Database Train Scoring System Current Progress Future Work

3. Improving the energy efficiency of intermodal freight trains Intermodal (IM) freight is one of the largest sources of revenue for the freight railroads in North America However, IM freight is the least energy efficient in comparison to other types of freight High speeds are necessary to compete with trucks Large gaps between loads result in poor aerodynamics

4. Current practice in intermodal freight train loading Slot Utilization is metric used to measure the percentage of the spaces (a.k.a. slots) on intermodal cars that are used for loads However, this metric does not account for the size of the slot and the size of the load 8 loads / 10 slots = 80% Slot Utilization 10 loads / 10 slots = 100% Slot Utilization Y.C. Lai 2008

5. Slot efficiency methodology Slot Efficiency: comparison of the difference between the actual and ideal loading configuration This metric is similar to slot utilization except that it also considers the energy efficiency of the load-slot combination 53’ Well

6. Evaluating the energy efficiency of intermodal freight trains Lai et al. designed a machine vision system to evaluate train efficiency based on its loading and gap lengths Depending on the train configuration, an IM train can save as much as 1 gallon of fuel per mile BNSF is currently installing an intermodal efficiency measurement system at a location on their principal intermodal corridor from Chicago to Los Angeles

7. Wayside installation at Sibley, MO Automation System Video Acquisition System Train Monitoring System Train Scoring System Communication System

8. Train Monitoring System (TMS) Analyzes the videos to determine the intermodal train’s loading using machine vision algorithms Removes the background and creates mosaic image of the entire train Determines the train’s container/trailer loading configuration by detecting the edges of the container/trailer and assigning it a distance and time value using Determine the gap length between loads

9. Video analysis using TMS Train videos are tested using the TMS to generate gap lengths and to help identify any problems within the code and if they concur with TSS If output does not concur with TSS, it caused by errors such as poor acquisition conditions or problems within the code Common problems with video acquisition include Large motion of trees on windy days Poor Lighting Conditions

10. Timestamps Breakdown Load Stop Load Start TMS data AEI data Axle Times Axle Times

11. AEI data Automatic Equipment Identification (AEI) data is collected by a wayside AEI reader that reads the AEI tags on side of the railcars AEI data gives the time at which each axle on a train crosses a certain detection point At each point, the ordinal number, the car manufacturer, and the car number are also recorded The information from the AEI data is important because it identifies the railcars in the train consist

12. Mini-UMLER Database The Universal Machine Language Equipment Register (UMLER) is an electronic database of North American transportation equipment, including railroad rolling stock It contains the following information for intermodal railcars: Car type Compatible load types Loading capacities The Train Scoring System uses this database to determine “ideal loads” for each car

13. Mini-UMLER Database We use a condensed version of the database, dubbed “Mini-UMLER”, which only contains entries for intermodal (Type F, P, S, and Q) cars Because our algorithms aren’t applicable to non-intermodal rolling stock, this allows us access to all the information we need without adding excess data The Train Scoring System uses this database to determine “ideal loads” for each car

14. Train Scoring System (TSS) The purpose of the train scoring system is to evaluate an intermodal train’s slot efficiency and provide an aerodynamic coefficient to estimate fuel consumption The results from the TSS can aid terminal managers in creating more fuel-efficient trains

15. Flowchart of the Train Scoring System

16. Load Placement Method For each car type, best load is determined using the mini UMLER database. If the number of wells is invalid (i.e. 0 or 9) or car not found in UMLER, then the car is ignored and corresponding axles are skipped. Start and Stop times for a gap are extracted from the TMS file and are compared with the Axle Start time If Stop time exceeds Axle Start time (implying that the car was skipped earlier) then do not place the load on that car.

17. Load Placement Method If current axle lies within the first half of the car [i.e. axle start time is less than (Load Start + Load Stop)/2 ] and next axle lies within the second half, then the load has been ‘matched’ Total number of axles should be either: 4 times the number of wells 2 times the number of wells + 2 otherwise, report error and skip the axle Once the load is matched, determine Cargo Length and include it to the train score.

18. Load Placement Method Load Start Load Stop X ✔ ✔ ✔ ✔ Load Stop > Axle Time?

19. Current progress Artificial File Creation to test TSS Data Management Intermodal Verification Module AEI reader installed at Sibley Testing new results from Sibley

20. Artificial file creation Artificial files were created in Excel, based on dimensions from 1997 Car & Locomotive Cyclopedia and the TTX Loading Capabilities Guide Exported to text files to test in TSS Used to gain insight into parameters required by TSS: Both files need to be in complete agreement TMS data is offset from AEI data by .3 seconds. Given correct data, TSS can, in fact, interpret single-stack containers and trailers

23. Intermodal Verification Module (IVM) This subroutine identifies whether the given train-consist contains an intermodal car. Car information is extracted from the Axle Timestamp data (i.e. AEI file). An SQL query is used to check whether the car exists in the UMLER database. This process is iterated for each car until a match is found. If none of the cars are in the UMLER database, then the train does not contain intermodal cars.

24. Intermodal Verification Module (IVM) It can also be run directly from command prompt (blind-mode) using appropriate command line arguments (providing both AEI and UMLER file paths). This option is very useful for automating the verification process.

25. AEI reader installed at Sibley In July, an AEI reader was installed at the Sibley installation The AEI files are automatically transferred to the computer and are labeled with the date and time that it was captured to allow easy comparison to train videos With the AEI data, we can now start analyzing and scoring trains at Sibley

26. Troubleshooting Sibley data Removed locomotive axle timestamps from AEI to match TMS that does not identify locomotives as units like in the dummy files Modify TMS files to have edge the timestamp and distance begin at the first load edge AEI axle timestamps need pre-processing to match the timestamps of TMS Updating Mini-UMLER database to include other possible railcar loading configurations

27. Future work Continue to test new train videos from Sibley site Integrate version 4 of the Aerodynamic Subroutine to TSS Correct AEI reader to eliminate data pre-processing Integrate the intermodal verification module, the AEI reader and data, and the revised TSS to the wayside automation system Investigate how implementing slot efficiency for intermodal train loading would affect intermodal terminal operations

28. Acknowledgements Special Thanks to BNSF Railway and BNSF’s Technical Research and Development department for sponsoring this research project Mark Stehly, Larry Milhon, Paul Gabler, and Josh McBain Leonard Nettles and Kevin Clarke for their assistance in constructing the installation in Sibley John M. Hart and Avinash Kumar from the Computer Vision and Robotics Laboratory at the Beckman Institute for Advanced Science and Technology

29. Questions?

30. Wayside Automation System

31. TMS output file

32. AEI data

33. Load Placement Method Axle Start Axle End Gap Start Gap Stop Gap Stop > Axle Start (2.899 > 0.26924) (Gap Start + Gap Stop)/2 = (0.41 + 2.899)/2 = 1.6545 > 0.26924 (AxleStart) (Gap Start + Gap Stop)/2 = (0.41 + 2.899)/2 = 1.6545 < 2.96588 (AxleEnd) => LOAD MATCHED.

34. Load Placement Method Axle Start Axle End Gap Start Gap Stop Gap Stop > Axle Start (5.80095 > 3.24784) (Gap Start + Gap Stop)/2 = (3.28333 + 5.80095)/2 = 4.54214 > 3.24784 (AxleStart) (Gap Start + Gap Stop)/2 = (3.28333 + 5.80095)/2 = 4.54214 < 5.96356 (AxleEnd) => LOAD MATCHED.

35. TSS Output & Report Files