Abstract. Recent artificial intelligence (AI) technology is capable to generate new texts (so that it makes new stories) by learning order of words in partic-ular datasets. This paper reports an experiment of generation of imaginal in-cident reports by AI text generator, which learn from real aviation incident reports. AI tries to synthesize new texts to maintain similarity of word order pattern to training data of real reports, while it may generate brand-new sto-ries due to randomness in its algorithm. We may find hidden risk of human error incidents among those imaginal reports. Although theoretical capability of AI text generation is widely approved, it is still a practically hard problem, which requires large costs and massive amount of data. Therefore, the main contribution of this paper would be not only qualitative evaluation of made imaginal reports, but also quantitative evaluation of its feasibility. Keywords: Incident report · Human Error · Human Factors· Neural net-works · Text synthesis · Natural Language Processing · Artificial Intelligence

1. Generation of plausible
incident stories by using
recurrent neural networks
Toru Nakata
National Institute of Advanced Industrial Science
and Technology
1
LSTM
Incident
Story made
by me

2. Research Problem:
Can we predict future incidents?
Type of incident Recorded Data Prediction Tool
Mechanical/Electric
al Failures
Numerical. (Failure
rate, etc)
Statistics, Reliability
Engineering
Human Error
Text (Reports,
Narratives, etc) NLP?
Incidents by
complex causes
Various ?
2
Text data are crucial for major incident prevention, but we
cannot use math for them.
Natural Language Processsing (NLP) may solve it.

3. LSTM: Generation of Stories
•LSTM (“Long Short-Term Memory”) is the most
popular (and almost de facto standard of)
recurrent neural networks that can learn
patterns of sequences.
•Pro:
• High quality of output. Generation of sequence of
words with almost perfect grammar.
•Cons:
• Huge amount of teaching data, calculation time, cost
are required.
3

4. Imaginal Stories for Prediction
•LSTM can generate plausible incident reports
by learning the sequences of real incident
reports.
4
Real
Report
Real
Report
Real
Report
Real
Report
Real
Report
Real
Report LSTM
Unreal but
learned
sequence
pattern
Learn sequence
patterns of words
Generate
sequence of
words
The words are plausible in respect to
the teaching data statics. Is it a
candidate of future incident?
Teaching Data

5. LSTM learns patterns of sequential data
5
wi-s wi
… wi+1
LSTM
Lean
and
Guess
A given word sequence of a real incident report
wi-s+1
LSTM
• Guess the statistically most likely(*) word to follow given
phrase.
• *: Randomness of word selection can be adjusted manually.
• Repeating the guesses, LSTM generates a plausible (but may
not be real) report.

6. wi-s wi
…wi+1
LSTM
Lean
and
Guess
Learning in reverse direction
6
wi-s+1
LSTM
• LSTM can lean backward.
• From given ending of an incident report, LSTM guess the
beginning situation of the report.
A given word sequence of a real incident report

7. Teaching Data
• NASA’s Aviation Safety Reporting System (ASRS) Database
• Narrative Part Texts
• Report written by pilots or other agents involved in incidents.
Described from point of view of first-person.
7
Period of incidents dates Jan 2013 to Dec 2018
Number of reports 31,285
Number of characters 54,880,919
Number of words 11,008,384
Number of kinds of words (+ 2
auxiliary marks)
38,228 (+2)
The bigger data, the better quality of LSTM learning.

8. LSTM (19,762,176 param.)
Drop out (prob. = 10%)
Dense (4,926,690 param.)
Activation
Sequence of 30 Words
(vocabulary = 9,098)
Prediction of Next Word
256 dim. vector
256 dim. vector
19170 dim. vector
To choose a plausible word
depending on probability.
To calculate probabilities
of all words.
To learn sequence
patterns.
To erase data randomly
to avoid over-learning.
8
Days are required for learning
on my desktop PC.
Adopted some famous text generation
programs with Python, Tensor Flow,
and Keras posted at Github for free. No
need to code by yourself.

9. Generation Example
• <Mark of Start of Report> I had a pilot of the flight to zzz2. I was
flying the aircraft to fly the aircraft and that I was going to fly the
aircraft in the vicinity of the airport. I was able to get the aircraft to the
east and the aircraft was on the ILS to Runway 16 at the same time. I
requested a heading of 150 and told the controller that we were not
going to be on the arrival. I asked the controller if we were on the
approach and told me to maintain visual separation. I told him to
maintain visual separation. I then turned Aircraft X to 10,500 feet to
avoid the traffic. I then told the controller that I had to climb to 11,000
feet to 3,000 feet. I could not see the traffic at the time, I was not able
to see the aircraft. I was not sure if I had the radios and I had to do so.
I was trying to get the airplane out of the way down the runway. I was
not sure where the airplane was on and that I had a chance to do the
job. …
• Vicinity of airport  Ready to land with ILS (Instrumental Landing
System)  Negotiation with Controller  Visual Separation 
Hidden Traffic
• Randomness = 0.4
9

10. Generation with specific initial phrase
• I commanded LSTM start with phrase of ”And then I found a
strange …”
• {And then I found a strange} smell. I told the captain that the
battery was off the door and the lavatory was in the galley. I
was told that there was a passenger in the cockpit and that the
door was still closed. I asked the fa flight attendant to hear me
and that she was leaving the lavatory door and said she was not
injured. I told her that she was injured and she was not going to
declare an emergency. I also told her that we were returning to
[departure airport].
• Strange  Smell  Lavatory  Returning to departure airport
• LSTM generated as a typical word sequence after the initial
phrase.
10

11. Example: Reverse direction with specific
ending of Hard-Landing
• I notified dispatch of the weather in weather conditions, and
return to our destination. We notified ATC and directed us to
divert to ZZZ. We ran the before takeoff checklist. We
completed the overweight landing checklist. We completed the
QRH and after completing the landing checklist. We were asked
if we needed to return to ZZZ. We replied that we were at FL330,
and were given a descent to 4,000 feet. We were cleared for the
approach. We were cleared for the approach, and we were
cleared for a visual approach. However, we were cleared for the
approach. We briefed the approach and we were cleared for the
approach. On the final approach, {we made a hard landing. We
made a hard landing. We made a hard landing.}
• Before takeoff checklist  Overweight landing checklist 
Quick Reference Handbook (of Engine, QRH)  Hard Landing
11

12. Output Example failing consistency
• <Mark of Start of Report> After landing on the ground, we
were cleared to taxi to the ramp and cleared us to taxi to the
gate. I was concerned about the situation with the aircraft in the
gate and the aircraft was towed to the gate. The ramp agent
called the fire trucks and they informed us that the gear was still
down, but they did not have the gear to retract the gear. I then
called for flaps 20. He then called for flaps 30 and then
proceeded to complete the approach. The aircraft was cleared
for the ILS.
• Under Randomness = 0.5.
• After Landed  Taxi  Need to tell us gear down?
• Too much random to keep context persistent.
12

13. Example of too low randomness
• I was the pilot flying and I was flying pilot flying. I was the pilot
flying (PF) and I was flying the airplane and I was flying the
aircraft. I was the pilot flying and I was flying the aircraft. I was
flying a visual approach and landing. I was the pilot flying and I
had the first officer and I was flying the aircraft.
• Randomness = 0.3
• Repeating the loop of “I was the pilot flying”, one of the most
frequently appearing sequence.
• After days of leaning using huge text data, LSTM still becomes
silly, but sometimes it generates good outputs (if we train LSTM
more).
13

14. Conclusion
• LSTM can generate
• (perhaps) unreal incident stories
• with specific beginning or ending
• that might tell us possibilities of unheard-of incidents in the
future
• Adjusting randomness was difficult.
• Too low randomness results in loops of frequently used words.
• Too high randomness breaks persistency of context.
• Many know-hows are very important to improve the quality.
• The hope is that more data and more large LSTM model
will solve such problem of quality.
14