Software fails 2016

© 2017 Tricentis. All rights reserved.
Software
Fail Watch:
2016
in Review

How It Works
The Big Picture
Software Fails by Industry
One Year Rewind
Software Type
Type of Software Fail
Software Fails in the Stock Market
Government Software Fails
Retail Software Fails
Entertainment Software Fails
Transportation Software Fails
Service Software Fails
Finance Software Fails
The Software Fail Awards
Biggest Hacks of 2016
Biggest Fails of 2016 That Went
Un-noticed (Almost)
Most Shocking Fails of 2016
Conclusion
Resources
The Software Fail Watch: 2016 in Review is a sobering reminder
of how even a single software bug can cripple an enterprise.
With 4.4 billion people and 1.1 trillion in assets impacted by
software failures in 2016, it’s hard to argue that “more of the
same” is the best path forward for software development
professionals.
As the demand for the latest and greatest in technology and
convenience grows, so does the need for software testers to
protect their users and their brand from the potential influx of
software failures. Our goal at Tricentis is to help testers succeed
in this role—enabling fast, efficient, comprehensive testing that’s
designed to support Continuous Testing, Agile, and DevOps.
Ultimately, we want to ensure that the inevitable software bugs
are found by your testers, not your customers.
Preface

The Software Fail Watch is a
collection of software bugs found in
a year’s worth of English language
news articles. To find the stories, we
set up a Google account with an
alert for phrases such as “software
glitch” and “software bug”.
Then we manually sorted through
each of the alerts, picking out
promising headlines, reading the
articles for relevance, and noting
down any specific details of interest.
If the article reported a software
bug, we logged it into an Excel
sheet (which you can download
here), and extracted as much
information as possible.
What industry does this fall into?
Does the article say how much the
affected software cost to
implement?
Does it mention how many cars
were recalled? How long was the
system down? Is the associated
company public, private, or a
government contractor? You get the
idea.
No exact numbers were recorded
unless they were explicitly stated in
the article itself.
Stories that appeared in multiple
news alerts and were covered by
multiple reputable sources were
marked as having a “high level” of
brand damage.
In the end we categorized all the
stories into 6 broad industry
categories: Entertainment, Finance,
Government, Retail, Services, and
Transportation.
How It Works
1159
363
Stories
548 Software Fails
Companies

Statistically, there is a very good chance
that you have been personally impacted by
a software failure this year – perhaps even
in the last quarter, or week.

The Big Picture
To find the total amount of people affected, we added all the numbers we had
pulled: i.e., cars recalled, mobile phones with malware, paychecks undelivered,
accounts hacked, etc. and assumed that each item directly affected one person.
That means that in reality, our end sum is on the conservative side. It is far more
likely that for every car recalled, a larger total number of people were affected,
with further implications that rippled outwards in a software “butterfly effect”.
7.4 bn
4.4 bn
People Affected
Amounts to over 50% of
the world‘s population
0
10
60
50
40
30
20
70
Jan. Feb. Mar. April
Software fails by month, 2016
May June July Aug. Sept. Oct. Nov. Dec.

315 years,
6 months,
2 weeks,
6 days,
16 hours,
& 26 minutes.
Accumulated time lost
$1,062,106,142,949
Assets affected (USD)
17,701
Gulf Stream G-550jets
(valued at $60,000,000 USD)
The Big Picture, Pt.2

Software Fails by Industry
stories picked up by
multiple news outlets472
Software fails by industry, 2016
A high-level view reveals clear patterns
in where and how these software fails
occur. Government-related software
fails dominate the charts, with an
average of 15 fails per month. Retail
and Transportation are tied in second
place, both clocking in an average of 9
fails per month.
Many trends observed in last year’s
Software Fail Watch continued this
year. For example, Transportation’s
software fails peaked in late spring,
while Retail’s software fails rose steadily
in the months leading up to the
Christmas holidays. The Finance and
Entertainment industries kept a fairly
low profile over the course of the year,
both averaging just 2 software fails per
month.
The wild card in 2016 is the Services
representing internet,
and telecom, etc. The
of software fails jumped
industry,
electricity,
numbers
erratically from month
with
to month,
11 recordedpeaking in May
software fails.
0
5
10
15
20
25
Jan. June July Aug. Sept. Oct. Nov. Dec.Feb. Mar.
Entertainment
April May
Government Finance Retail Services Transportation

There are far more software bugs in the
world than we will likely ever know about.

Software Fails
by Industry, Pt. 2
12%
yearly increase
in software fails
Comparing 2016 data vs. 2015 throws the picture into
even sharper relief. If anything, the need for better
software testing is only growing.
Ever the buggiest industry - Government - clocked in an
additional 42 fails in 2016 over 2015. Retail,
Transportation, and Service exceeded 2015’s numbers,
while the Entertainment and Finance industries managed
a year-on-year decrease.
Finance’s numbers decreased by nearly 35%, indicating
that either the testing or the public relations in this
industry have improved over the pastyear.
0
20
40
60
80
100
120
140
160
180
200
Entertainment Finance Government
Software fails by industry, 2015 and 2016
Retail Services Transportation
Year 2015 Year 2016

One Year Rewind
4.3 billion
People affected
vs. 4.4 billion
People affected
239
Companies
vs. 363
Companies
4.2 billion
Assets affected
vs. 1.1 trillion
Assets affected
2015 2016
0
10
20
30
40
50
60
70
80
90
100
Jan. Feb. Mar. April May June July
Software fails by month, 2015 and 2016
Oct.Aug. Sept.
Total 2015
Nov. Dec.
Total 2016

Software Type
190
Mobile/Cloud
“Mobile/Cloud” encompasses all web
or app-based software. If a website
went down or an app failed, it went
into this category. While each industry
was well represented within this
category (showing how even industries
that traditionally run legacy software
have jumped on the mobile-boat),
Retail predictably took the lead with
72 stories. Of those 72 stories, 44
featured hardware and smart phone
manufacturers such as Apple, Google,
or Samsung.
217
On Premise
“On Premise” encompasses all software that requires installation
in a specific location. If an organization’s internal system, such as
an ERP (Enterprise Resource Program) or accounting software
crashed, it went into this category. On Premise software exists
within every industry. However Government overwhelmingly
dominates this list, making up 59% of the stories in this category.
141
Embedded
“Embedded” includes all software that
is pre-installed on a device or piece of
hardware. If a casino’s slot machine
experienced a glitch, or a car’s airbag
sensor malfunctioned, it was placed
into this category. Transportation
makes up the majority of this category,
141 stories. This is
given that most
with 67 of
unsurprising
Transportation related software fails
stem from a problem with the vehicle
itself, be it car, plane, or train. Fails by software type, 2016

Type of Software Fail
432 Software Bugs
38 Usability Glitches
78 SecurityVulnerabilities
In recording the stories, we identified three main types of software
failures. The first, and most common, is a software bug: an instance in
which a software application does not work as designed. The second
type is a usability glitch: a design flaw that decreases the usability of
the product or application. The third is a security vulnerability: a flaw
that attackers can exploit to alter a system’s behavior.
Type of fail by industry, 2016
Type of fail by month, 2016
0
10
20
30
40
50
60
70
Aug. Sept. Oct. Nov. Dec.Jan. Feb. Mar. April
Software Bug
May June July
Usability Glitch Security Vulnerability
0
20
40
60
80
100
120
140
160
180
Entertainment Finance Government Retail
Software Bug Usability Glitch
Services Transportation
Security Vulnerability

395
390
400
405
410
415
420
425
430
435
18. Jul 19. Jul 20. Jul 21. Jul 22. Jul
Approximately 40% of the companies hit by software fails in 2016 were public companies. While it
is not always possible to trace the effects of a software fail in the rise and fall of a company‘s stock
prices, there are times when the correlation isunmistakable.
British Airways’ parent company, International Consolidated Airlines Group, S.A (IAG), is one such
example. British Airways implemented a new global check-in system in 2016, which quickly
became a source of frustration and embarrassment. British Airways faced 5 major computer
outages between May and September, resulting in thousands of flights canceled or delayed, and a
cumulative stock market loss of 10.54% or £92.9 billion. The graph above shows a snapshot of
IAG’s stocks following the July 17th outage, in which the stock prices dipped 2.28% in the first day
alone, for a loss of £20.8 billion.
Software Fails
in the
Stock Market
148
Public Companies
Affected
Upon news of the
software fail, the
stock prices dropped
-2.28%, for a loss of
-£20,776,000,000 in
market capitalization.
By July 22nd, a combination
of factors (including
Brexit), had caused IAG’s
stock price to fall 25
points since July 18th.Stock markets
open Monday, July
18, after a severe
British Airways
software fail over
the weekend.

"Every company is a software company.
You’re building software that is going to
deliver business outcomes and software is
the differentiator for your business.”
https://www.skytap.com/blog/theresa-lanowitz-on-solving-age-old-problems-in-the-enterprise/
Theresa Lanowitz, voke

Government organizations undeniably top
the charts when it comes to software fails.
The cause is multifaceted. Government
organizations typically rely on contracted
developers to custom build software that
can fulfill complex operations and handle
massive amounts of data. Furthermore,
many organizations do not have in-house
resources for
implementation.
revealed during
decision makers
testing or software
Even if defects are
implementation, many
are pressured to move
ahead since they have already invested
massive amounts of public funds into the
project. This also explains why so many
government organizations are still using
legacy software – the resources, time, and
money required for a software upgrade is
simply too high for many to justify.
Government Software Fails
$5,703,579,938
Averaged knowncost
of failed government software (USD)
can
buy
Government fail by sector, 2016
30,193
Houses in the UnitedStates
(median price of a house in 2016: $188,900 USD)
0
5 10 15 20 25 30 35 40 45 50
Transport
Taxes
Space Exploration
School
Justice
Healthcare
Emergencies
Elections
Admin

Retail Software Fails
The “Internet of Things” (IoT) has expanded the reach of software within our homes and across even
mundane activities. Connecting your tea kettle, thermostat, and interior lighting to the internet is
opening up new realms of possibility that the retail technology sector - and hackers - are just
beginning to explore. The vast majority of the “security vulnerability” software fails recorded in 2016
were linked to the IoT in some way. Even the most harmless IoT hacks highlight both the weaknesses
in our current IoT devices, and the increasing need for security as our lives become more web-
dependent. However, many of the software fails featured hacks with dire implications – including a
“botnet” DDOS attack that took down large portions of America’s internet for most of October 21st.
2,666,600 , 0 0 0
Mobile Phones
Infected with Malware
20
18
16
14
12
10
8
6
4
2
0
Retail software fails by month, 2015 and 2016
Year 2015 Year 2016

A significant portion of the entertainment-related
software fails in 2016 were discovered as part of Bug
Bounty programs. Bug Bounty Hunting has become
increasingly popular in the last few years, with big-
name enterprises like Apple, Google, and Facebook
offering cash rewards for newly discovered bugs.
Even government organizations have joined the fray,
such as the United State’s “Hack the Pentagon”
initiative announced this year. The increase in Bug
Bounty Hunting reveals two interesting trends: the
widespread acceptance of crowd-sourcing quality
assurance, and the mainstreaming of the white or
gray-hat hackers. One of the notable stories from
2016 was Facebook’s $10,000 USD award to a 10-
year-old Finnish hacker-in-the-making. Though
$10,000 is not the highest award recorded, a 10-
year-old boy certainly is the youngest person to be
awarded.
Entertainment Software Fails
0
2
4
6
8
10
12
14
Jan. Feb. Mar. April May June July
Entertainment software fails by month, 2015 and 2016
Aug. Sept. Oct.
Year 2015
Nov. Dec.
Year 2016

Transportation Software Fails
21,228,066 Cars recalled
8,831 Planes grounded
22,712,987 People affected
Self-driving cars made the headlines a few times in 2016, as key players such as
Google and Tesla sought to mainstream their use. In early January, Google released a
report recounting the numbers of times a human took control of the self-driving car
during tests (354 times within 15 months). In July, it was revealed that Tesla would
likely be facing serious consequences after an Autopilot failure resulted in one man’s
death. In October, the Singapore-based nuTonomy reported an accident while
conducting testing for their flagship self-driving car service. We expect the headlines
related to self-driving cars will only increase in the coming years. This showcases yet
again that comprehensive risk-based software testing is now a necessity in all
industries.
12
10
8
6
4
2
0
Jan. Feb. Mar. April May
Transportation software fails by month and mode oftravel
June July Aug. Sept. Oct. Nov. Dec.
Road Air Rail

Service Software Fails
The service industry plays a significant role in underpinning our every day lives. Whether
it be within healthcare, internet, or telecom, a service-oriented company‘s influence
interweaves into other industries like government, retail, or transportation. Roughly 50%
of the service software fails from 2016 fell within the internet or telecom sectors. Many
dramatic service-related stories made the news in 2016, one of the most notable being
Yahoo’s admission of a cyber-breach that stole data from a staggering 1.5 billion
accounts. Another big story detailed how a recently discovered bug could negate 15
years worth of MRI findings and research.
27%
Of the stories
explained the
cause of the
software fail
18
16
14
12
10
8
6
4
2
0
Service software fails by month, 2015 and 2016
Year 2015 Year 2016

Finance Software Fails
1.3years
Average time lost
$521million
Biggest single loss
In our experience, software fails in
the finance industry rarely hit the
headlines. It is not that the industry
does not have software fails –
rather, they simply seem to be
reported less. As opposed to some
other industries, much of a financial
institution’s software lies behind the
scenes. This makes it easier for bugs
to be patched quickly before their
repercussions can snowball into a
major news story. The finance
stories that do come to light either
take place in public-facing venues
(such as a stock market), or feature
bugs so catastrophic that the story
cannot be buried. It seems that
financial institutions are extremely
vigilant when it comes to
maintaining
When we
2014 and
fails, many
their public image.
recently reviewed our
2015 finance software
of the original reports
were removed from news sites and
scrubbed from search results.
0
2
4
6
8
10
12
14
16
18
Jan. Feb. Mar. April May June
Finance software fails by month, 2015 and 2016
July Aug. Sept. Oct.
Year 2015
Nov. Dec.
Year 2016

We’ve said it before and we’ll say it again:
software may come and go, but software
testing is here to stay.

Scariest Fail
Multiple incidents have been
reported of creeps hacking into
internet-connected baby monitors.
Parents have discovered hackers
taking advantage of a security
vulnerability in the baby monitor to
control the camera, watch the
room, play creepy music, and speak
to the babies in the night.
The
Software
Fail Awards
Biggest “WTF?“ Fail
A US-based IP mapping firm
accidentally mapped over 600
million “unknown” IP Addresses
onto a single address in Kansas.
The unfortunate family living at the
address has dealt with the
consequences for the past 10
years: police and FBI showing up
regularly, having “followed” an IP
address to the house looking for
kidnappers, child pornographers,
drug cartels, thieves, and more.
Funniest Fail
An English programmer purchased
a Wi-Fi enabled, voice-activated tea
kettle. He then spent the next 11
hours attempting to make a cup of
tea. An entire day’s worth of
reprogramming later, the poor man
finally got his morning cup of tea.

Biggest Hacks of 2016
politicians, celebrities,
“The Panama Papers” hack leaked
11.5 million documents and 2.6TB
of data from the Mossack Fonseca
law firm based in Panama – a
known tax haven. The leak revealed
the illicit financial dealings of
and
dignitaries worldwide.
The CIA formally accused Russia of
attempting to influence the United
State’s presidential elections in
favor of the Republican Party by
releasing private emails from the
Democratic National Committee.
While the hack itself certainly
occurred, whether Russia was
responsible is still being debated.
Multiple banks in India were
affected by a massive financial data
breach, compromising 3.2 million
debit cards. The source of the
breach was traced back to malware
in Hitachi’s Payment Services,
allowing hackers to collect sensitive
banking data and steal funds.
Hackers managed to steal $81
million from a bank in Bangladesh
by exploiting a vulnerability in
SWIFT, a financial transfer system.
The dramatic story made headlines
worldwide, and brought to light
accounts of smaller bank heists
exploiting the same vulnerability in
recent years.
Yahoo admitted to two damaging
hacks that occurred in 2013 and
2014, resulting in data stolen from
1.5 billion accounts. It is unclear
why this incident only came to light
in 2016, however it does not bode
well for Yahoo’s future.

Biggest Fails of 2016 That Went Un-noticed (Almost)
CGTechnology, a sports gambling company,
paid a fine of $1.5 million USD and fired its
CEO after willfully ignoring a software glitch
that underpaid bettors $700,000 USD in
winnings. The company was not only accused
of purposefully avoiding fixing the bug, but
also of interfering with the Nevada Gaming
Control Board‘s investigations into the matter.
Worldpay, a UK-based payment processing
firm that processes over 36 million payments
per day, experienced a software fail that
crippled their services for over three weeks.
The bug was traced back to a server software
update that resulted in an overload of error
messages. The failure comes as a particular
embarrassment after the enterprise had
recently invested over $500 million USD in
updating the system’s software.
A secretive database containing 2.2 million
names of people suspected of terrorism and
organized crime was leaked into the public
domain in July 2016. The database, which is
managed by Thomson Reuters and used by
banks and intelligence agencies alike, was not
hacked (as might be expected), but simply
the public sphere after an
“database software error”
dropped into
unexplained
occurred.
A teenager in Arizona faced three felony
charges after creating a bug that took out
911 emergency phone services in
old created a
an attempt to
three
“non-
win a
states. The 18 year
harmful” iOS bug in
bounty from Apple’s
The bug, which he
bug-hunting program.
shared via a link on
Twitter, caused iPhones to call 911 on repeat
until the phone was shut off. The link was
reportedly opened thousands of times,
swamping local 911 emergency services with
hundreds of hang-up calls per minute.
The DAO, an investment fund containing
Ethereum, Bitcoin’s rival crypto-currency, was
robbed of over $50 million USD worth of
“Ether” in late June. Not only did the hackers
manage to successfully exploit the fund’s
security vulnerabilities, but the necessary
patch was so complicated that developers
could not fix the vulnerability for several days
after the theft. During that delay, a half-dozen
smaller copy cat heists occurred.

Most Shocking Fails of 2016
UK’s National Health Service admitted to mis-
prescribing medication to over 300,000 heart
patients due to a software error. SystmOne,
the software used to calculate the risk of a
heart attack, has reportedly produced
incorrect results since 2009. As a result,
some patients suffered from otherwise
preventable heart attacks or strokes. Others
needlessly dealt with the serious side-effects
of taking unnecessary medication.
Fiat Chrysler recalled over 1 million vehicles
after a government investigation revealed
that a gear-shift flaw resulted in 266
accidents, 68 injuries, and at least 1 death (of
a celebrity, no less). A software update was
issued to address the flaw, but the update
reportedly failed to fix at least 29,000 of the
recalled vehicles.
A security researcher disclosed a firmware
bug found in CCTV point-of-sale security
cameras sold by over 70 different vendors.
The bug, nicknamed ”Backoff”, is suspected
to be a contributing factor to the spate of
credit card breaches that occurred in major
retailers in the past years. The U.S.
Department of Homeland Security has
warned that up to 1,000 US businesses may
be infected with the bug.
Thanks to
system,
a faulty automated software
the
Unemployment
state of Michigan’s
Insurance Agency (UIA)
mistakenly accused over 20,000 innocent
people of fraud. Midas, the data analysis
system used, was wrongly accusing claimants
of fraud in a whopping 93% of cases,
resulting in unemployed persons losing their
benefits and facing fines of up to $100,000
USD. The system, implemented in 2013, had
been operating unchecked until 2015, when
Michigan’s auditor general issued a scathing
report on the system’s shortcomings.An Australian hospital suffered a string of
software outages, the longest lasting for 10
hours in early November. The software fail
reportedly
cutting off
medication,
admit and
threw the hospital in chaos,
access to patient records and
and forcing staff to manually
track patients by hand. The
hospital administration had reportedly
complained several times in the previous
months that the new software (implemented
in June) would eventually have “fatal
consequences” for patients if not fixed soon.

At one time (long, long ago), software was just a way of getting things
done. It was nothing more than the convenience of using a calculator
instead of doing your sums by hand. Those days are gone, however. The
average enterprise software landscape contains 52 interconnected
systems.* The average person’s software landscape is far vaster – as the
saying goes, “no man is an island”, and the bridges between ourselves
and the world are increasingly built with software.
Logically, this means that we could only expect the numbers we collect for
the annual Software Fail Watch to increase year-by-year. That is not,
however, the future Tricentis sees for software. Software testing methods
will not simply plateau while software development and innovation
climbs. The mainstreaming of DevOps and Agile has already done much
to fulfill today’s demands for modern software. The future of software
testing, however, belongs to Continuous Testing.
Conclusion
Love details?
Download the full list of software bugs collected in 2016.

Software fails 2016

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Software fails 2016

Similar to Software fails 2016 (20)

Recently uploaded

Recently uploaded (20)

Software fails 2016