D3 data driven development in practice - the AirPortal for Schiphol and Transavia Airlines

D3 - Data Driven Development in practice - The AirPortal case.doc Page 1
Data First
D3: Data Driven Development in action
Developing the AirPortal for
Transavia Airlines & Schiphol

D3 - Data Driven Development in practice - The AirPortal case.doc page 2
D3
- Data Driven Development in practice.
Developing the AirPortal for Schiphol & Transavia Airlines
Version Management
#
Date Author Description
0.1 21 Sep 2014 Kasev, Laan (vd) Document [dump] @ Hackathon
0.11 22 Sep Laan (vd) Structure document
0:2 25 Sep Laan (vd) 1st
draft for Testing blog
0:21 1 Oct Laan (vd) Document “case”
0:22 23 Oct Sluis (vd) Testimony specific aspects
0:25 15 Nov Laan (vd) Cleaned-up for TestNet publication
0:30 6 Jan 2015 Laan (vd)/Sluis (vd) Updated after internal presentations to team
0:31 6 July Laan After visit Schiphol (rules 7 and 8 – details added)
Related information
Title
# Date Author Description
Harmony Users Guide 3.4 2 Nov 2014 Kasev,Laan
history and decision support, workflow, UI proto v0.1,
rules, business events. Limited testcases.
Harmony concepts & design guide 0.2 July 2014 Laan
Published/distributions
#
Channel Date By Remarks
0.2 gDOCS, team 16 Jul ‘14 Nanno
0:25 www/ Testnet 16 Nov
0:30 www 8 Jan ‘15

D3
Table of Contents
Version Management .........................................................................................................................2
Related information............................................................................................................................2
Published/distributions.......................................................................................................................2
Table of figures ...................................................................................................................................4
Introduction ............................................................................................................................................5
About complex IT systems? ................................................................................................................5
Data First Driven Development (D3
) at a glance .................................................................................5
D3
versus Use Cases ............................................................................................................................5
D3
and tooling: introducing Harmony and Testimony ........................................................................5
Testimony, what it does..................................................................................................................6
Harmony, what it does....................................................................................................................6
Liquid Sequence’s product development strategy .............................................................................6
Disclaimer............................................................................................................................................6
Introduction ............................................................................................................................................7
Data First Driven Development in practice at the Dutch Open Hackathon............................................7
The case: Transavia’s challenges.........................................................................................................7
A new online booking system: the AirPortal for Schiphol and Transavia Airlines..............................7
The agile D3
approach: Harmony & Testimony..................................................................................8
Spreadsheet as “the source” – some advantages...........................................................................9
Test stories are specifications: describing the booking system.....................................................9
Define test cases: structure + data = specification.......................................................................10
Creating (developing) the actual AirPortal [system].........................................................................11
Testing: automated, regression........................................................................................................12
How to deal with Production Acceptance Testing (PAT) ..............................................................13
Concluding ........................................................................................................................................13
Notes.....................................................................................................................................................14
Sample Use Case...............................................................................................................................14

D3
Table of figures
Figure 1 Data Driven Developments: structure defined (input, expectations and outcomes) ..............8
Figure 2 Create the data model - define relationships through descriptive text .................................12

D3
Introduction
The purpose of this document is to familiarize (future) users with D3
, which stands for Data Driven
Development, an evolutionary process for developing and maintaining (complex) IT systems. When
we speak of “developing” we refer to the design, development, deployment and test of applications.
This whitepaper describes the merits of D3
; in terms of benefits that other methods simply don’t
deliver when setting out to develop complex IT systems. We do this by using tooling provided by
Testimony and Harmony – using the Google DOCS platform.
About complex IT systems?
Complex IT systems are applications that perform database processing, using workflows to support
one or more business processes, with business rules and of course, a user interface to support
various groups (roles) of users. Expressions, such as calculations, concatenations, are generally
present in these types of application. Within the context of this whitepaper we’d like to state that
business rules should preferably be implemented by decision tables; since these provide the
necessary documentation to govern proper execution of data and process.
Data First Driven Development (D3
) at a glance
D3
is an agile software development process aimed at domain experts using data as the primary
specification mechanism. It bears similarities to TDD, which is, however, aimed at developers.
What makes D3
special is that it supports the analysis and design phases: by defining sufficient test
cases, using “input” and “expectation” sections, a “data driven” specification comes to live. With
this, system designers can design & implement, as Harmony [power] users can implement/configure
the proposed solution. Essentially the inputs and expectations are purely about data specifications:
given certain inputs, what should the system “do”. Note that in the D3
design phase we don’t focus,
in UI terms, how the system should “present” outcomes/results.
D3
versus Use Cases
Use Case driven development is the dominant method in the majority of software development
projects. Use Cases lack data definitions – which is where D3
kicks in. In a typical use case there’s
little notion of data, whereas D3
emphasizes on data. In general it’s fair to state that the D3
test story
is the same as the UC [name], the big difference is that the detail process is described in term of test
cases, using data, whereas with use case these a mere textual descriptions.
D3
and tooling: introducing Harmony and Testimony
Harmony and Testimony are productivity platforms for developing browser, mobile web and voice
applications. Any type of system can be developed in a very short time frame, from simple “contact
form” web apps to enterprise systems. Implemented as a Platform-as-a-Service (PaaS), users are
freed from typical IT set-up tasks; developing apps only requires a few “clicks”. Both are Do It
Yourself IT, aimed at:
1. Domain (subject matter) experts – business professionals with in-depth industry knowledge
of business rules and process details. Within this category we identify
a. business or systems analysts, who use Harmony to turn flowcharts/process models
into executable code
b. expert users, like employees, who, instead of writing text documents, or being
interviewed, document their knowledge using Harmony & TestimonyT . This
‘configuration-style-documentation’ approach turns business rules / decision logic –
and workflows – into a ready to execute format (i.e. an application).

D3
2. Spreadsheet power users – persons with a solid understanding of the power of
spreadsheets that want to develop their own business applications
3. IT professionals looking to develop easy-to-maintain high performance, secure cloud
applications. This encompasses a wide spectrum of IT experts identified by:
a. unfamiliarity with the risks involved with “the cloud”
b. developers realizing that they have to moving away from traditional programming
languages such as COBOL, RPG, Java, PHP etc
c. developers who like to take advantage of modern IT concepts such as business rule
driven development
Testimony, what it does
Testimony provides a spreadsheet framework for defining stories and test cases and support for
testing applications using these very same test cases. Spreadsheets offer the flexibility at almost no
learning curve. Testimony provides templates to structure the specifications and the back-end
support for automated testing.
Harmony, what it does
Harmony uses the same spreadsheet framework to configure business applications, instead of
coding applications – freeing users from worrying about functional semantic issues (are the
specifications correctly defined, are they developed correctly?) and technical issues – such as
performance, security and scalability.
Harmony also provides guidance for a reference architecture as well as application integration.
Liquid Sequence’s product development strategy
We adapt to the latest open standards, using technologies and concepts that have been, and are still
being, developed by industry leaders such as Google. Our development path is agile, evolutionary,
delivering new functions as when these are needed.
The development sequence is practice driven; new features being implemented as standard
patterns emerge and fulfilling our customer’s needs. We monitor industry trends and follow industry
leaders; we implement standards provided that these contribute to user productivity and “ease of
use”. Failing to meet these criteria, which is the case with DMN decision table standards, user
friendly features are implemented (DMN style decision tables are complex to understand;
Harmony’s implementation results in much easier to read & define decision tables).
Disclaimer
It should be noted that not one single method provides the best in class for all cases; all methods
(techniques, methodologies) have merits and demerits.

D3
Introduction
This concepts and design guide provides an overview on how to implement Agile IT development
using the Data First Driven Development paradigm
Data First Driven Development in practice at the Dutch Open Hackathon.
In an Agile development environment Test Driven Development (TDD) is often chosen as the standard
technique for application development and testing. However, TDD falls short when it comes to
developing business critical systems, which emphasize on supporting business processes.
Data Driven Development is a more effective approach, partly because it lowers the threshold for
"users" to actively participate in the development process. The goals of D3
are defining specifications
and verify using these same specs through testing. The definition is a data format, for test cases, as a
more concrete part of test stories, and this same test cases are used for verification. In this article we
describe this approach, using two cloud platform solutions: Testimony for the specification and
testing and Harmony as the application development and "execution" platform. The contents of this
document are practice driven, describing the experience with D3
on the Dutch Open Hackathon,
where the 3 person Liquid Sequence team, delivered the AirPortal solution, solving the IT issues for
Transavia and Schiphol.
The case: Transavia’s challenges
Transavia Airlines wants to help its passengers with a variety of service information and solutions for
finding and booking tickets, check-in comfort, tax-free shopping in-flight entertainment etc. New
features make demands on the current Transavia systems; demands great difficulty can be resolved
due, partly caused by the outdated technology. Transavia strongly believes in the boundless
creativity of a new generation of developers who are active both inside and outside the aviation
sector. In the context of the Dutch Open Hackathon, Liquid Sequence has developed a new search
and book system which addressing their imminent needs, and at the same time, being the
foundation for their “wishes and ideas” as described above. Flexibility = extensibility; meaning that
yet unknown, unidentified, requirements should be added easily in the future. This was seen as a
major non-functional requirement, fulfilled by the Liquid Sequence team.
A new online booking system: the AirPortal for Schiphol and Transavia Airlines
The new booking system is designed using a “portal” concept: the scope is not limited to an airline’s
bookings; instead the “context” of an airline is used to define & (partially0 implement the booking &
other services. This offers significant advantages in terms of flexibility: changing or adding new
features are relevant easy-to-do tasks, given the fact that Harmony and Testimony are the
productivity platforms supporting this.
In technical terms, the AirPortal web and UI services support both "internal" business users (airlines,
Schiphol) and "external" consumers, such as travellers. The AirPortal is set-up by the principle of
identifying joint “consumers” of IT services.

D3
The AirPortal has implemented 2 APIs that communicate with external systems: one with the
Transavia availability & pricing system, the other provides the connection to the Schiphol system for
real-time flight information.
The choice has been made to store all [ticketing] data within the AirPortal [database] – thus offering
significant improvement possibilities. ; The technical implementation is based on a so-called “Triple
Store”, a database optimized for storage and retrieval of subject-predicate-object, such as a flight to
Barcelona. Implementing a Triple Store is an excellent example of applying the latest [open source]
technologies in a business critical environment. The 24 x7 availability requirements are implemented
by Harmony’s "build-to-break", multi-node, architecture. Harmony runs on Google Compute Engine,
Linode.com and Amazon, linear scalability is achieved.
The agile D3
approach: Harmony & Testimony
The AirPortal has been developed using Harmony and Testimony. The 1st phase of the new system is
search and find [tickets] using the D3
approach. As explained before, D3
is a user friendly version,
aimed at users, other than TDD, which focuses on developers. Both Data Driven Development and
Test Driven Development are governed by a uniform data structures; formed by three data blocks:
Figure 1 Data Driven Developments: structure defined (input, expectations and outcomes)
Unlike TDD, D3
focuses on functional description of data and desired outcomes, and is ideally suited
for "domain experts"; domain experts are users who know their business well and can use their
knowledge to describe, in concrete terms, what the system must “do”. The “to do” is described in
the data entry and how it must respond to these inputs – which are described by so-called
"expectations”.

D3
Spreadsheet as “the source” – some advantages
The most efficient and flexible format to describe specifications is spreadsheets; which provide a
structure, by means of templates, that can easily be extended in adding more inputs (columns) or
test cases (test specifications are in rows). Spreadsheets offer other significant advantages; data can
be defined (forget about complex SQL like instructions to create tables) and spreadsheets are the
optimal mechanism for defining decision tables1
!
This then must immediately ask ... what tools we can use for this? After all, saving test cases should
be converted to an "executable" format. Spreadsheets Productivity and quality go hand-in-hand;
why we support this with Testimony, developed by our test platform, which is based entirely on
spreadsheets.
Testimony spreadsheets are templates providing the "three-block" structure in the image above. An
important goal in D3
is to minimize unstructured specifications; a perfect example is a system
description with 100% of the functional specifications defined in spreadsheets - not, as with most
projects, in text documents.
Testimony runs on top of Google DOCS; one of the many nice features provided by DOCS is “sharing”
– Testimony sheets can easily be shared with colleagues and/or experts. Google DOCS provides
share and chat, which are particularly useful when additional information is needed to complete
specifications. Providing information on specifications by email or text documents always requires
translation into a format which can then be executed by a tool or program. Hence the optimum
format is the spreadsheet supporting collaborative worker processes Our experiences to-date
indicate a significant increase in productivity and quality at a much lower cost, of which the latter is
due to working in the cloud = supporting remote workers.
Test stories are specifications: describing the booking system
The advantage of airline booking systems is that the specifications are easily traceable; simply check
the booking sites (reservation systems) used by KLM, Schiphol or Transavia - and a specification is in
the making – without interviewing users. An additional benefit to the readability of this article is that
most of us have booked flights using an online reservation system.
The main functions of the new system are grouped into test stories, lines of text describing what the
features are. We structure this by first defining the capabilities of the system describing Examples of
test stories are all test stories:
1. standard bookings for adults
2. same but travelling with children
3. same but travelling with babies
4. same but travelling with children and babies
1
Decision tables are the best format/structure to capture complex business logic.

D3
Business rules and D3
stories
Next we add test stories that describe the limitations of the system. This is interesting ... an example
of a limitation of the current system:
5. Transavia's [CURRENT] system does not accept bookings for minors (under 12 years)
travelling without adults
This is restriction, a business rule, of the current system – which is advised to specify as test story.
Since Transavia wants to provide better services, by providing new services to attract passengers, a
new functionality is described as a test story. I.e. the extensions for the new system are defined:
6. Transavia's NEW system will accept bookings for only minors (under 12 years) – in which
case the "Unaccompanied Minor Service” must be used
Remarks on scope and context
The above groups of test stories are typical examples of how key users would define stories; the
focus is clearly on the current booking system – most notably the limitations of it. However, if we
take into account that the new AirPortal has to be the "heart", the foundation, of all future IT
systems, it is desirable (a "should do" in project management/business case terms) that the context,
and perhaps even the scope of the new system are widened. Let’s define two test stories – of which
the latter doesn’t apply to Transavia (since they don’t fly to Bali):
7. Peter 74 years, flying from Amsterdam to Barcelona, and can’t walk long distances ... he’s
more or less disabled ... needs transportation to the gate
This will involve the need for other services to be implemented. In the detailed / test case
we’ll add which services can be called-upon/booked. As such the requirement + specification
becomes immediately clear.
8. Roy flies to Bali. Upon making the booking it turns out that his passport validity date doesn’t
meet Indonesian regulations. The business rule is: departure [date] from Bali - passport
expiry date > 6 months
Again a nice example of how facilitating external [immigration] rules support the customer
journey. The expected outcome would warn the customer/traveller – may be even add a
reminder to his/her profile/to-do data (or calendar) .
Define test cases: structure + data = specification
For each of the test stories, "in-scope" for the AirPortal, test cases should be defined. Here again we
see the advantage of the traceability of airline bookings: "input" and "expectations" are easy to
define. Please note the advantage of availability & pricing data to be stored in the AirPortal
database; like real-time flight information, these are subject to change; which makes the testing
more complex (expectations are fixed, outcomes can vary).
Elaborated test cases:

D3
1) Adult books from Barcelona (Spain) to Amsterdam (Netherlands), airline code = HV
a) departure 20/09/2014 at 14:00:00Z
b) departure 21/09/2014 at 14:00:00Z
2) Adult books from Amsterdam (Netherlands) to Barcelona (Spain), airline code = HV
a) departure 21/09/2014 at 05:55:00
b) departure 21/09/2014 at 12:25:00Z
c) departure 21/09/2014 at 18:50:00Z
The design of the AirPortal calls for feature allowing a "third party" airline to be booked; hence we
create a KLM booking. "Guidelines" force us to produce a test story
3) Adult books from Amsterdam (Netherlands) to London (Britain), airline code = KL
a. departure 20/09/2014 at 14:00:00Z
4) * booking can’t be made because seats are not available *
Adult books from Amsterdam (Netherlands) to London (Britain), airline code = KL
b. departure 20/09/2014 at 14:00:00Z
With many test cases defined; a “data footprint” starts to assemble. Test cases include many
different combinations of data, both valid and invalid data, and for each case, the expectations are
defined. The follow-on activity of ‘developing’ (in Harmony’s case configuring) becomes achievable;
because all the test cases (the data footprint) acts as a tangible specification.
Agility & the importance of test stories (as specifications, as test cases)
A major advantage of Agile methods is that it isn’t necessary to be “complete", i.e. there’s no need
to have all functionality defined in prior to starting development. In AirPortal terms, a good example
is an including a new function, requirement, (see previous section) … what to do if there are not
enough seats available. This requires the creation of a new test story and defining test cases. What
has to be done/defined is …
1) … the decision [the system] should take, in concise terms, when there are no seats, by:
a) the story text , which is the description / explanation
b) the test case data [records/rows]–
i) when does this situation happen …
ii) what are the conditions for the system to submit, if any, an alternative
2) Sufficient test cases need to be defined
a) Thus allowing the designer/configurator/“developer” to decide how to implement2
.
Creating (developing) the actual AirPortal [system]
Next is a brief description of the development of the actual system, we keep this limited – as the
focus of this article is describing the D3
method, and not the actual Harmony implementation.
The presence of test cases makes it possible to implement data entry [user interface], define rules
and expressions, such as calculations. After all, our motto is "data drives development".
2
can the decisions be solved with a simple rule, or is it necessary (desirable) to implement a decision table

D3
To set up the data structure, most notably the data model, we invoked the assistance of a Harmony
expert. "Enterprise" IT systems requires Harmony's Relationship Kernel – which is a very powerful
feature to define data relationships. It does this by describing data (see Figure 2), instead of
[graphically] modelling data. Harmony RK offers special queries, defined as expressions in the rules
section, replacing traditional SQL queries.
Figure 2 Create the data model - define relationships through descriptive text
After defining the data structures and populating this with data, it’s possible to use the Testimony
specifications and turn this into a [Harmony] implementation. Since Testimony fields/attributes are
[spreadsheet] column names and Harmony fields/attributes are dialog items, we ensure that the
[Testimony] fields correspond with [Harmony] dialog items.
In the configuration expressions need to be created – such as the total number of passengers and
ticket prices.
Note that the reference files (such as REF_Airline, REF_Airport) are created within the application;
this avoids being dependent upon external data sources and/or web services.
Testing: automated, regression.
With (parts of) the system ready, and with test cases defined , it’s possible to start testing.
Testimony inputs are mapped to Harmony dialogs, Testimony "expectations" compare Harmony’s
“decision support” [outcomes]. The results ("outcomes") are listed as "good/successful" or
"bad/failed". All testing is done automated.
An important advantage is that Testimony supports regression testing; after each deployment (sprint
resulting in a new version of the AirPortal) a test run is started. New test cases which are the
specifications for the that sprint, the new version, are marked for testing, and with all previously
defined test cases, are to be part of the test run. Testimony then runs all tests, thus ensuring that
older, previously defined tests pass, as they had before, as well as testing the new functionality.

D3
How to deal with Production Acceptance Testing (PAT)
Traditional IT projects require tasks to test performance and scalability. Not so in the case of using
Harmony, where we witness the benefits of "architected" solutions. The probability of performance
degradation and / or scalability issues to occur is negligible; especially in the "AirPortal case". In case
of extremely complex systems (with 10.000's of business rules implemented) the above issues could
arise on the basis of large number of users in combination with an improper “sized” deployment /
production environment. Only in these cases, it is desirable that specific tests need to be run to
determine identified risks.
Concluding
True successes of Agile methods materialize significantly if the total development life cycle evolves
to a higher level; for which new tooling is required. The transformation from traditional to Agile
involves more than just the Agile testing of the input DFT or TDD.
New, ready to deploy, tools must facilitate the [agile] automation of system development. Harmony
and Testimony deliver on this, offering ready-to-use SaaS platforms, supporting the IT development
process without the need to create additional IT support features (such as creating web services for
testing). The philosophy is to use less IT experts (and those who are involved can actually have lower
level of expertise) and still accomplish the same in less time; without risking quality degradation. The
quality aspects are both functional and non-functional; the is solved by Harmony because of its
“built-to-break” architecture.
The challenge: http://www.dutchopenhackathon.com/
The Testimony page – the Liquid Sequence page
Click here Harmony and [Shiphol/Transavia] API’s in action.

D3
Notes
The samples below are from other projects; no relevance to Airport/travel industry
Sample cases – how input & expectations are used as a specification
Input Expectations
[test story] Fraud detection, a person, uniquely identified by National Insurance # (NI) , requests a benefit
3
Person=Tim, NI=JG-12-13-16A, Benefit=study grant Occurrence=1, Request Information= OK
Harold, SC-49-55-64-S, study grant Occurrence=1, Request = OK
Charles, SC-49-55-64-S, study grant Occurrence=2, Request = Pay attention
Jimmy, SC-49-55-64-S, study grant Occurrence=3, Request = Pay attention
Jimmy, SC-49-55-64-S, study grant Occurrence=4, Request = Pay close attention
Add more test cases (occurrences 5,6, 7,8,9)
Albert, SC-49-55-64-S, study grant Occurrence=10, Request = definitely fraudulent
NI= SC-49-55-64-S, Request= definitely fraudulent Fraud manager must decide which action to take
Yvonne, AB-54-55-21X, housing allowance Occurrence=1, Request = OK
NI= NI=JG-12-13-16A, Request=OK Need Family name, address, bank account number for pay-out
[test story] Need Family name, address, bank account number in case his request [check] = OK
Tim Howard,34 Elgin Av London, GB54BARC20992012345678 The 1
st
payment will be made to your account in 30 days
Tim Howard,34 Elgin Av London, NL39RABO0300065264 Invalid bank account; must be national account
[test story] Fraud manager must action
Fraud mngr=Ken, action= visit, by= inspector, within=24hrs Inspector, visit, who=Albert, NI= SC-49-55-64-S
4
Using the above a specification can be created:
1. We have a person [the requestor] – identified by name an NI …
2. … who requests a benefit
The system must
3. Count the number of requests (occurrences) for each NI/benefit combination
4. For a given number of occurrences the system should provide an indication
Sample Use Case
UCA2 - HRM manager adjusts information about an archived case
 HRM manager opens case in the state “Archived”
 HRM manager selects the workflow action “Archive duration change”
 HRM manager chooses the archiving parameters
 The system calculates the “Save-till” date
 The HRM manager confirms the archiving operation
 The system saves the new “Save-till” date and the case remains in the workflow state
“Archived”
3
Benefits are limited to study grant, housing allowance, Social benefit request.
4
We know of course that address details need to be known for a visit. For oversight reasons there are left out.

D3 data driven development in practice - the AirPortal for Schiphol and Transavia Airlines

Recommended

Recommended

More Related Content

What's hot

What's hot (8)

Similar to D3 data driven development in practice - the AirPortal for Schiphol and Transavia Airlines

Similar to D3 data driven development in practice - the AirPortal for Schiphol and Transavia Airlines (20)

More from 112Motion

More from 112Motion (20)

Recently uploaded

Recently uploaded (20)

D3 data driven development in practice - the AirPortal for Schiphol and Transavia Airlines