The document outlines the top 10 common pitfalls in distributed system management, emphasizing the importance of data modeling, understanding data flow, and maintaining controlled states. It warns that ignoring system evolution and performance can lead to significant issues, urging proactive complexity management. The key to successful integration lies in leveraging proven technologies and designing interfaces early to avoid complications later.

1. Your systems. Working as one.
Top 10 Ways to Mess Up your
Distributed System
Stan Schneider, PhD

2. Distributed Systems Experience

3. Trend: Scale
• More things producing
and consuming data
• Greater volume of data
• System of systems
integration
System of
systems

4. Problem
By the time you find out if your infrastructure is
good, it’s too late
5/30/2012 4

5. Top 10 Ways to Mess Up Your
Distributed System
5/30/2012 5

6. Gloss over your data model
5/30/2012 6

7. Who uses the FACE Data Model?
FACE Definition (Standards) System Integration
Proposed Updates
Core Data Model
Definitions
Data
Descriptions
Associations
Packaging System
Constraints Integrator Data Model
OTS
Portable
Components
Supplier
Software Development
7

8. Misunderstand your data flow
5/30/2012 8

9. Data Flow Models
Point-to-Point Client/Server Publish/Subscribe Data-Centric
Pub/Sub Messaging
Brokered
ESB
Daemon
Data-Centric
Publish/Subscribe (DCPS)
9

10. Allow uncontrolled state
5/30/2012 10

11. Controlled State
• Data centric
– Single source of truth
– Known structure
– Clear rules for access, changes, updates
• Technologies
– Database
– Data-centric middleware
5/30/2012 11

12. Ignore evolution
5/30/2012 12

13. Evolve Models & Behaviors
• Extensible infrastructure
• Routing
– Connect data models: guarding, transformation
• Service bus
– Match behaviors and patterns between components
5/30/2012 13

14. Make all communications reliable
5/30/2012 14

15. Asset Tracking Example
Transactional design:
• 12,000 tracks
• 11 servers with 88 cores
• Poor reliability and uptime
• 1.5M SLOC
• 2-8 years to develop
• Custom, proprietary
DDS design:
• 250,000 tracks
• 80% of a single core
• Full redundancy
• 50k SLOC
• Proof of concept in under a week
• 100% standards based
5/30/2012 15

16. Save system integration strategy for the end
5/30/2012 16

17. Example: Naval Integration
• Systems of systems
become
exponentially
complex
• Reduce coupling to
reduce complexity
• Define interfaces
early and carefully
5/30/2012 17

18. Think performance or scale isn’t
important, or can be done “later”
5/30/2012 18

19. • Tuning a slow system
doesn’t work
• Building a fast system
takes design
5/30/2012 19

20. Don’t anticipate business drivers
5/30/2012 20

21. Wind Power
• Grid control
– Continuous performance
tuning of turbines
• Business control
– Predictive and preventive
maintenance
– Smart diagnostics
– Production planning 21

22. Assume all implementations
of the standard are the same
5/30/2012 © 2012 RTI • COMPANY CONFIDENTIAL 22

23. 5/30/2012 © 2012 RTI • COMPANY CONFIDENTIAL 23

24. Go it alone
5/30/2012 © 2012 RTI • COMPANY CONFIDENTIAL 24

25. • Hawaii warning signs
5/30/2012 25

26. Believe there are only 10 ways to mess up
5/30/2012 26

27. RTI Overview
• Market Leader
– Over 70% DDS mw market share1
– Largest embedded middleware vendor2
• Standards Leader
– Supported Standards: DDS, JMS, RTPS, SQL,
WSDL/SOAP, HTTP/Rest, C, Java, C++, .net
– OMG Board of Directors
• Maturity Leader
– 15+ years of commercial availability
– Diverse industries: defense, finance, medical,
industrial control, power generation,
communications
– 500+ design wins
– 350,000+ licensed copies
– TRL 9
1Embedded Market Forecasters
2VDC Analyst Report

28. Top 10 Ways to Mess Up
• Gloss over your data model
• Misunderstand your data flow
• Allow uncontrolled state
• Ignore evolution
• Make all communications reliable
• Save system integration strategy for the end
• Think performance or scale isn’t important, or can be done
“later”
• Don’t anticipate business drivers
• Assume all implementations of the standard are the same
• Go it alone
5/30/2012 28

29. Summary
• Problem: By the time you find out if your
infrastructure is good, it’s too late
• Solution: Proactively manage complexity
– Complexity must exist somewhere; choose where
– Data centric infrastructure pulls complexity out of
every application
– Leverage proven technology & experience
5/30/2012 29

30. Your Systems.
Working as OneSM