In today’s data-driven word, the adage “garbage in, garbage out”, has never had more significance. As military leaders start to leverage AI/ML more and more in the decision-making process, the importance of having accurate, relevant, and timely information on which to base those decisions is going to be key to the successful execution of the current and future offset strategies. Providing that information requires a secure, agile, and robust Data Supply Chain Pipeline (DSCP) to curate data during its lifecycle. In this session, we will discuss some of the key elements of a good data management practice including policy, process, organization, and how Amazon can enable the DSCP by building to the right solution, fit-for-purpose and fit-for-use, to the problem across all classifications of data

1. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Stephen Moon
Senior Solutions Architect, Department of Defense Team
Data Supply Chain Pipeline

2. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
What is Supply Change Management?
Wikipedia... “In commerce, supply chain management (SCM), the
management of the flow of goods and services, involves the movement
and storage of raw materials, of work-in-process inventory, and of
finished goods from point of origin to point of consumption.
Interconnected or interlinked networks, channels and node businesses
combine in the provision of products and services required by end
customers in a supply chain. Supply-chain management has been
defined as the "design, planning, execution, control, and monitoring of
supply chain activities with the objective of creating net value, building a
competitive infrastructure, leveraging worldwide logistics, synchronizing
supply with demand and measuring performance globally.”

3. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Customer Use Case
• An organization has tens, hundreds, or thousands of disparate,
heterogeneous systems, applications, or devices across multiple
organizational domains.
• Each organizational domain has multiple systems of record storing
data about the same entities and it’s attributes
• Leaders need to be able to ask single-domain and multi-domain
questions and receive a single, accurate answer
• For example… what is my force readiness if a conflict arises in
<region>?
• This is a multi-domain question

4. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
What is the Importance of Data?
• Data is a critical organizational asset, not an IT function asset
• Data + Context + Relationships = Information
• Information provides strategic, tactical, and operational advantages by
enabling execution of the mission more rapidly with greater agility and
precision than the adversary.

5. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Data Handling
! Without good quality data, the best modeling techniques in the world won’t
deliver accurate results
• Prepare raw data for model training
• Training Sets, Development Sets, Test Sets
• Historical event data required to build a machine learning model should be
stored in the data warehouse
Importance of Data for Machine / Deep Learning

6. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Data Supply Chain Pipeline Mission Statement
Democratize data by creating an accurate and consistent 360° view of
the organization with the purpose of providing actionable strategic,
tactical, and operational insight in order to enable execution excellence.

7. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Data Supply Chain Pipeline Operating Model
Current State Future State
Ross, Jeanne W, et al. Enterprise Architecture As Strategy: Creating a Foundation for Business Execution. Harvard Business Review
Press, 2006.
https://www.amazon.com/dp/B004OC07EE/ref=dp-kindle-redirect?_encoding=UTF8&btkr=1

8. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Volume
Velocity
Variety
Veracity
Value
Kilobytes (KB)
Megabytes (MB)
Gigabytes (GB)
Terabytes (TB)
Petabytes (PB)
Time-based
Event-driven
────────
KB/s
MB/s
GB/s
Structured | Semi-structured | Unstructured
Accuracy
Authenticity
Consistency
Reliability
Decision Making
Strategic
Tactical
Operational
The Five Vs of Data

9. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Collect
&
Process
Organize
&
Catalog
Explore
&
Discover
Analyze
&
Report
Suppliers Consumers
Search
&
Prepare
Data Supply Chain Pipeline (DSCP)

10. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Name Build It and Make Them Come
Statement Customers should by able to access the data where it lives regardless of their own location
Name Minimal Disruption
Statement Minimize/Eliminate disruption to data producing systems, applications, or devices
Name Configure-to-Customize
Statement Leverage configurable components to meet at least 80% of the requirements
Name Decoupling
Statement Pipeline stages are independent of one another
DSCP Architecture & Design Principles

11. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
DSCP
Architecture

12. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
DSCP Design

13. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Minimal Viable Product (MVP)
Initial Operating Capability (IOC)
Collection & Processing

14. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
DSCP Collection
! There is no “tool” for collecting data from disparate, heterogeneous
suppliers
• Focus on cultivating the organization competencies and the processes
for engaging with suppliers
• Build tiger teams who understand the organizational domains
• Develop templates for Memorandums of Understanding (MoU) and
Interface Control Documents (ICD)
• Use AWS Cloud Services to build and enable the right technology
solutions for data collection use cases

15. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
DSCP Collection
The result will be a small set of common Collection &
Processing patterns that can be standardized, automated,
and scaled to hundreds or thousands of suppliers on AWS’
Cloud Services.

16. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Architecture Pattern Design Pattern Methodologies
Extraction, Transformation, & Loading Change Data Capture
• Push (Source initiated)
• Pull (Target initiated)
• Transaction Log (Synchronous /
• Slowly Changing Dimensions (Type 0:6)
Enterprise Application Integration Publish / Subscribe
• Topic-based
• Content-based
DSCP Collection Architecture & Design Patterns

17. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Interest
Influence
Control
Control
Applications/Systems which are owned and/or directly
managed by the collecting organization
Influence
Applications/Systems of which the collecting
organization is an internal or external stakeholder but
does not own or manage the application/system
Interest
Applications/Systems of which the collecting
organization has a concern for the data but does not
have control or influence over the application/system
Why is this important?
Determines how data is going to be collected!
DSCP Collection, Circles of Concern

18. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Hard Rules Soft Rules
Quality rules that are independent of
business value or meaning.
Example 1
A social security number must be
numeric and contain exactly 9 digits.
Example 2
An address must be valid per the USPS.
Policy rules based on context that add
or alter business value or meaning.
Example 2
A mailing address for prescription drugs
cannot be a P.O. box.
Example 1
A social security number and date of
birth must be a unique combination.
DSCP Processing, Business Rules

19. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
DSCP
Technical L0

20. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
DSCP Technical L1

21. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
DSCP Technical L2, Structured

22. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
DSCP Next Steps
1) Engage your AWS Solutions Architect
2) Develop a set of questions
3) Identify the information required to answer the questions
4) Identify suppliers of the relevant data
5) Build the solution platform
6) Collect & Process the data
7) Start answering questions

23. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Please complete the session survey in
the summit mobile app.

24. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Thank You!

25. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
moonstep@amazon.com
Senior Solutions Architect
Department of Defense (DoD)
Worldwide Public Sector (WWPS)
Amazon Web Services (AWS)
Stephen Moon