Building a Data Driven Culture and AI Revolution With Gregory Little | Current 2022 Transforming business or mission through AI/ML doesn't start with technology but with culture…and an audit. At least as much is true for the US Department of Defense (DoD), which presents significant modernization challenges because of its mission scope, expansive global footprint, and massive size - with over 2.8 million people, it is the largest employer in the world. Greg Little discusses how establishing the DoD’s annual audit became a surprising accelerator for the department’s data and analytics journey. It revealed the foundational needs for data management to run a $3 trillion in assets enterprise, and its successful implementation required breaking through deeply entrenched cultural and organizational resistance across DoD. In this session, Greg will discuss what it will take to guide the evolution of technology and culture in parallel: leadership, technology that enables rapid scale and a complete & reliable data flow, and a data driven culture.

1. OFFICE OF THE SECRETARY OF DEFENSE
CHIEF DIGITAL AND AI OFFICE
Building a Data Driven Culture and AI Revolution
for the World’s Largest Employer
1

2. Our mission
2
Accelerate DoD adoption of data, analytics,
and AI to generate decision advantage from the
boardroom to the battlefield

3. Our Environment
3
Mission Data
BATTLESPACE
AWARENESS
ELECTRONIC
WARFARE
COMMAND
AND CONTROL
FINANCIAL
MANAGEMENT
MANPOWER
ACQUISITION &
PROCUREMENT
LOGISTICS
READINESS
CYBER
Readiness Data
Business Data
GFM
MEDICAL
PROPERTY

4. Our Path Forward
4
Enterprise Data, Business Analytics, and AI/ML enablement pyramid
AI and Analytics
Outcomes
Data Architecture
Data ecosystem
Analytics and AI infrastructure
Vision
• What are we trying to achieve?
• What value do we hope to capture?
• Which are the main AI and analytics use cases?
• How much value do we expect from each?
• What are their key constraints and requirements?
• Have we thought of AI assurance?
Enablement Questions
• Do we have the right talent and operating model?
• Do we have a thorough analytics and AI process and
capabilities?
• Do we have single sources of truth for key information
needs?
• Have we launched basic data hygiene actions?
• Do we know what data we have, what it means, and
how to access it?
• How should we grow and manage the data ecosystem?
(e.g. enable accessible, timely, trustworthy, secure data)
• Provide enterprise data management, analytics, and AI
capabilities to accelerate decision advantage from the
boardroom to the battlefield
• JADC2
• Business performance optimization (e.g. business
analytics/metrics)
• AI assurance as-a-service
• Talent management (embedded tactical data teams,
embedded performance teams, upskilling, recruiting)
• Advana, JOS, AI/MLOps platform (includes things like
model catalog, feature store, etc.)
• Data labeling as-a-service
Our priorities in these areas
Data Catalog OSD Policy
• Advana
• Data Engineering
• Data Ontology Foundry
• Data Quality
• Data Catalogue
• API Infrastructure

5. 5

6. Our Challenge
6
Culture Technology Debt
And a lot more…

7. 1. Don’t underestimate the power of
compliance
7

8. One good thing that came out of it was Advana – DoD’s Data
Management and Analytics Environment
8
Security
Network &
Infrastructure
Service Layer/ API
MLOps/Self Service
AI/ML SW Factory
Data Governance
Data Ops
UI / Apps
Platform
Service
Portal &
Home Page
Data Catalog
Decision
Support
Tools/Products
Self/Service Data/BI
Tools (COTS)
User Profile
File Store
Search
Access
Control
Ontology/
Semantic
Data
Collaboration
• Service Desk
• Knowledge Base
• Product Management and
Document Management
• Email Notifications
• Atlassian-as-a-Service
Service Abstraction Layer / Engine
AI/ML Gateway
Data Gateway
Access Control
• RBAC/ABAC
• PII/PHI Masking and handling
• Classification Guidance
• Complex Policy Enforcement
Data Quality
• Accuracy
• Completeness
• Timeliness
• Consistency
Data Acquisition
• Manual Upload/SFTP/API
• Streaming/Batch
• JSON/XML
• REST/SOAP
Data Storage
• S3/Blob (Scalable)
• Parquet/Delta
• DW, Delta
• Bronze/Silver/Gold
• Graph database
Data Processing
• Elastic and MPP
• Spark
• CPU/GPU
• Tagging/Curating
Data Orchestration
• Auto Promotion
• Cross-Domain Solution
• Self-Service Promotion
• Automation
Service Bus
• Messaging (Pub/Sub)
• Service Registry / Discovery
• Security
Security
• IDAM
• Encryption
• Firewall/ continuous Pen Test
• 3rd Party Assessments
Container
Orchestration
• Monitoring
• Metadata Repo
• Security
• Container Registry
CI/CD
• Build
• Scan
• Test
• Deploy
Monitoring
• Logging/Alerting
• Anomalous Behavior
• Performance
• Service Availability
Network
• CAP/BCAP/IAP
• Cross-Domain Integration
• Multi-Cloud Integration
Cloud Infrastructure
• AWS (IL 5, IL6, JWICS) + Azure (NIPR/SIIPR/JWICS)
• Scalable and Elastic
• Regions + Availability Zones (support 24x7 global
SLAs)
• IDS, Firewalls, etc.
Data Catalog/Governance
• Metadata Capture
• Search
• Data Lineage
• Federated Catalog
Self/Service AI/ML
Dev Tools (COTS)
Data Modeling
• Ontology
• Reference/Master Data
• Common Data Models
• Data Architecture
Experiment and Test
• Package/Lib Mgmt.
• Code/Low Code Dev & RPA
Automation
• Feature Store
• Train/Evaluate/Validate
• SW Dev and Test
• Self-Service Publish Model
Deployment and Operations
• Source Code Control
• CI/CD Publish/Deploy
• Model Registry
• Model Serving and Monitoring
• Model Inference Service
• Model Performance
Data Gateway
Automation
Models
API Gateway
Existing Enterprise Capabilities
Global Search
Reference Architecture Enterprise Capabilities
NIPRNET SIPRNET

9. 9
2. Go for the bottom-line (outcomes)

10. 10

11. 11
3. It’s about trust

12. 12

13. 13
Explainable AI
Lineage
Transparency on issue
Data Quality Score

14. 14
4. Depersonalize decision making

15. 15

16. 16

17. 17
5. Empower your Analysts

18. Empower your analysts
• Tools are restricting, bosses are suffocating, and big
organizations prefer predictability
• Create an environment that encourages risk
• Execution strategy:
• Hold Analysts accountable for insights, and then set them free
• Critical thinking should not be underrated
18

19. 19
6. Ownership of the Analytics Needs
to Sit with the Business

20. Ownership of Analytics: Business
• Think, imagine, and move at the pace of our changing world
• Ownership close to outcomes, proactive, and analytical needs
• Successfully analytics usually outside of IT
• We have found a hub and spoke analytical model to be
successful
20

21. In the end we will have happy
executives and analysts
21
Before Data
Driven
After Data
Driven