Query, Extract, Transform, Persist, Repeat. How many thousands of times has this workflow been codified? What if it never needs to be written again? AWE is a method that enables developers to retrieve data via APIs without the need to write API aware code. This method has been shown to reduce development time by at least two orders of magnitude. This presentation describes a method that enables developers to focus on WHAT they want to retrieve from an API, not HOW to retrieve it. Specifically, we have abstracted out the standard workflow for interacting with an API. We utilize the API documentation and a document that describes what data items are required by the application. We further analyze the API documentation to “string together” the calls required to retrieve the data items. Stop the insanity of pedantic codification!!

2. Question:
Can you name all the droids?
There’s a
ReST API for
that!!

3. © Copyright 2017 Dell Inc.3
How do we find the droids?
• Look at the documentation for necessary resources
• Borrow connection code from a previous project
(or Stack Overflow)
• New code to test and maintain
• Getting the answer requires several requests
(typical of APIs)
• Locked into the current version of the API

4. API Workflow Engine
Erik S. Hansen | erik.hansen@dell.com | @ErikAsCode
Mark Malamut
Dell EMC - Data Protection Division
November 1, 2017

5. © Copyright 2017 Dell Inc.5
Motivation
• Need: Gather information from various software products
• Challenge: Many products to support
– Many transport methods (e.g. ReST, SMI-S, SOAP, CLI)
– Many authentication methods (e.g. HTTP Basic, Token, Oauth)
– Many pagination techniques (e.g. links, custom parameters)
• Goals:
– Stop duplicating code to access the various APIs
– Focus on WHAT data we need, not HOW to get the data
– Frictionless upgrades to utilize new API versions and features

6. © Copyright 2017 Dell Inc.6
Traditional API Interaction: Workflow
• The QETAR (pronounced: Cuter) Model
• Query an API function for information
• Extract data from the API function response
• Transform data as necessary
• Act upon the data (if desired) – e.g. Persist
• Repeat until the desired data has been extracted
• Typically this workflow is entangled within the
application logic
Extract
Transform
Query
Act

7. © Copyright 2017 Dell Inc.7
ReST API Query by Example
https://swapi.co/api/species
https://swapi.co/api/species/?page=2
https://swapi.co/api/species/?page=3
…
…
https://swapi.co/api/people/2/ → C-3PO
https://swapi.co/api/people/3/ → R2-D2
https://swapi.co/api/people/8/ → R5-D4
https://swapi.co/api/people/23/ → IG-88
https://swapi.co/api/people/87/ → BB8
• Many nested queries
• Target data across
multiple resources

8. © Copyright 2017 Dell Inc.8
Application
Interface
Responds with
Data
Action
Traditional API Client Experience
API
• API code is siloed within
separate applications
• The API workflow becomes a
part of your application

9. © Copyright 2017 Dell Inc.9
AWE Developer Experience
Interface
Responds with
Data
Extract or
Transform
Data
AWE
API
Action
Application
Request
Sent to
Interface
• Application requests data
• Don’t care how it’s obtained
• All traversal and data transformations
handled by AWE

10. © Copyright 2017 Dell Inc.10
AWE Design
Build Execution
Plan
Application
Makes Request
AWE
Execute Plan
Result Returned

11. © Copyright 2017 Dell Inc.11
Execution
Plan
Builder
API Definition
AWE Configuration
Execution Plan
Data mappings
API communication
APIs to be executed
Data Transformations
Goal actions
API Definitions &
Data Structures
• Plan re-generated when a config change is detected
• Plans reused for all instances of the same API and
AWE Configuration
Execution Plan Construction

12. © Copyright 2017 Dell Inc.12
Execution Plan – Star Wars API
Execution PlanSWAPI Open API Spec
People
Species
/people
/species Planets
Films
Starships

13. © Copyright 2017 Dell Inc.13
Execution
Engine
Execution Plan
Transport
API info,
Credentials
Action
Interface
API
API request
Transport specific
request
API response Transport specific
response
Workflow Engine Execution
Data
Target
Application

14. © Copyright 2017 Dell Inc.14
AWE Execution – Star Wars API
AWE InternalApplication
https://swapi.co/api/species
https://swapi.co/api/species/?page=2
https://swapi.co/api/species/?page=3
…
Filters:
Species.name: “Droid”
Targets: [“People.name”]
Result
C-3PO
R2-D2
R5-D4
IG-88
BB8

15. © Copyright 2017 Dell Inc.15
Case Study: Storage Array Discovery
• EMC Storage Array has a ReST management API
– but no API Doc (WADL)
• WADL and AWE Configuration created to discover multiple levels of
information about the storage array
• No changes to AWE code
• 1 Hour to create both the WADL and AWE Configuration
with no prior experience using this API
• Biggest challenge (common to any discovery):
– Determining the data of interest
– What to do with the data (persist, manipulate, act upon)

16. © Copyright 2017 Dell Inc.16
AWE Benefits
• Developers focus on data, not how to get it
• Abstracts most communication, authentication and paging methods
• Less time and cost to implement new features
• Less code to test
• Zero-downtime updates
Extract
Transform
Query
Act

17. That’s
AWEsome!
These are the
Droids you’re
looking for!