Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...
Fielding Systems-of-Systems, Riding the agile sw tiger
1. The Fourth National Conference
INCOSE_IL 2007
• Session: Integration – Unsolved Question
• Lecture Name: Fielding Systems-of-Systems
– Ride the Software Tiger
• Presented by Sergey Tozik
1 Integration-unsolved question
Fielding SoS – Ride the SW Tiger INCOSE_IL 2007
2. 2 Integration-unsolved question
Fielding SoS – Ride the SW Tiger INCOSE_IL 2007
3. Outline
• Challenge:
Fielding of SW-intensive SoS
• Method:
Knowledge driven IVVT
• Technique:
Explorative Modeling
3 Integration-unsolved question
Fielding SoS – Ride the SW Tiger INCOSE_IL 2007
4. SoS: Complexity of Operation
Specify Operate Modify
Needs Systems
System Integrate
Capability
Capability
Capability
Capability
System
System
Test
4 Integration-unsolved question
Fielding SoS – Ride the SW Tiger INCOSE_IL 2007
5. SoS: Emergent Dependency
Project A Project B
independent
Capability A Capability B
Change Change No Change No Change Change
System 1 System 2 System 3 System 2 System 3
Reqs Reqs Reqs
System 1 System 2 System 3
Capability A dependent Capability B
5 Integration-unsolved question
Fielding SoS – Ride the SW Tiger INCOSE_IL 2007
6. SW Challenge
• Simple designs, Agile development
very short development
• Interdependence of systems
abundance of surprises
very long fielding times
• Challenge for integrators & testers:
hang for your life and still deliver
6 Integration-unsolved question
Fielding SoS – Ride the SW Tiger INCOSE_IL 2007
7. Hang for Your
life, dear…
It’s AGILE !!!!
7 Integration-unsolved question
Fielding SoS – Ride the SW Tiger INCOSE_IL 2007
8. What Users Want?
1. More Mission Capabilities
2. Easier Operation
3. No Surprises
8 Integration-unsolved question
Fielding SoS – Ride the SW Tiger INCOSE_IL 2007
9. Elements of Solution
• Focus on “Knowledge Creation”
instead of “Test against Spec”
• Hunt for unintended consequences
• Know what works and field it!
9 Integration-unsolved question
Fielding SoS – Ride the SW Tiger INCOSE_IL 2007
10. MaSK Methodology
• Explorative Modeling
• Semi-automatic test generation
• Session-based testing
• Knowledge factory concept
10 Integration-unsolved question
Fielding SoS – Ride the SW Tiger INCOSE_IL 2007
11. Integration Engineer Roles
Information
Information Gatherer Tester
Directions of Inquire Test Ideas
Analyst
Knowledge
Stakeholder
• Information Gatherer searches the sources for
information and clues about past and supposed behavior
• Tester creates and executes tests to capture information
about actual behavior
• Analyst directs the gatherers, creates knowledge and
morphs it to stakeholders’ needs
11 Integration-unsolved question
Fielding SoS – Ride the SW Tiger INCOSE_IL 2007
12. Knowledge Generation Steps
1. Gather Initial Batch of Information
2. Analyze Information
3. Create Knowledge
4. Detect Knowledge Gaps
5. Direct the Hunters for more Info
6. Gather More Information
7. Go to Step 2
Never Stop Asking Questions!
12 Integration-unsolved question
Fielding SoS – Ride the SW Tiger INCOSE_IL 2007
13. Sources of Information
1. Users’ Manuals
2. Requirements
3. Design Documentation
4. Interface Control Documents
5. Test Results
6. Interviews of Developers
7. Interviews of Users
13 Integration-unsolved question
Fielding SoS – Ride the SW Tiger INCOSE_IL 2007
14. The Knowledge Factory Workflow
Gather Info Info Create/Refine Models
Info
Hunting Use Model Technical Model
Loop
1..N 1..N
Question Analyse Models Insight
1..N
Knowledge
1..N Tests
Refinement
Stakeholder Loop
Value Loop
Plan, Execute Tests
Customer
Testing
Publish Loop
Test Results
1..N
Report Analyse Test Results
14 Integration-unsolved question
Fielding SoS – Ride the SW Tiger INCOSE_IL 2007
15. Explorative Modeling
1. Motivation: technique for fast learning
with limited domain expertise
2. Language: Cause-Effect Graphs
3. Use Models captures Business Processes in
Users’ Language
4. Technical Models capture Design
Information in Developers’ Language
15 Integration-unsolved question
Fielding SoS – Ride the SW Tiger INCOSE_IL 2007
16. Have You a SPEC?
Explorative No?!!
Modeling
Tool
I beg You,
Not Again!!!!
16 Integration-unsolved question
Fielding SoS – Ride the SW Tiger INCOSE_IL 2007
17. Basic link in Cause-Effect Graph
Data, Matter, E Data, Matter,
nergy Energy
Trigger
Output Entity
Input Entity
Processing
Event
Pre-Contition
Post-Condition
System Event
Environment causes changes in
State system/environment Environment
State
17 Integration-unsolved question
Fielding SoS – Ride the SW Tiger INCOSE_IL 2007
18. Knowledge Gaps Discovery
1. Not enough information to complete
Use Model
2. Not enough information to understand
processes initiated by System Event
3. Inconsistencies in information sources
4. Developer’s insecurity (Oops factor)
18 Integration-unsolved question
Fielding SoS – Ride the SW Tiger INCOSE_IL 2007
19. Example – Story of Two Cities
• Two neighbor cities
• Each city has Traffic Light Control System (TLCS)
• Every TLCS has manual and automatic modes
• Each TLCS did the job well until some day suddenly
the effectiveness of the AUTO mode have dropped.
• Why?
• We’ll try to generate clues using Exploratory
Modeling technique
19 Integration-unsolved question
Fielding SoS – Ride the SW Tiger INCOSE_IL 2007
20. Use Model
Focus: visible system behavior, user actions
Identify System Events (Identify Trigger)
• Set System to Manual (User)
• Set System to Automatic (User)
• Change Single Traffic Light Mode (User)
• Change Traffic Light Pattern (System, Periodic)
• Respond to Traffic Light Pattern (Natural)
Connect Events with I/O Entities/Conditions
Build Cause-Effect Graph
20 Integration-unsolved question
Fielding SoS – Ride the SW Tiger INCOSE_IL 2007
21. Cause-Effect Graph
Nothing happens without a reason!
Manual Respond to
Traffic Lights
Mutual
Influence
Respond to
Traffic Lights
Feedback City A
Traffic Pattern City B
Loop Traffic Pattern
Auto
Feedback Auto
Loop Feedback
City A
Loop
Traffic Light Pattern System B
System A
Generate Traffic Cycle Clock Cycle Clock
Light Pattern - City A
City B
Generate Traffic Traffic Light Pattern
Light Pattern - City B
Periodic
Change Activation
Traffic Light Change
Traffic Light
Disable when AUTO
Display Traffic
Disable when AUTO Pattern
Display Traffic
Pattern
System A System B
Auto/Man Auto/Man Set to MANUAL
Set to MANUAL
Operator City B
Operator City A
Set to AUTO Transient Set to AUTO
Generator
21 Integration-unsolved question
Fielding SoS – Ride the SW Tiger INCOSE_IL 2007
22. Directions for Inquiry
1. Analyze mutual influence and concurrency
of automatic control feedback loops
2. Analyze mutual influence and concurrency
of manual control feedback loops
3. Analyze mutual influence of automatic and
manual feedback loops
4. Analyze concurrency of AUTO/MANUAL
transients
22 Integration-unsolved question
Fielding SoS – Ride the SW Tiger INCOSE_IL 2007
23. Direct Test Case Derivation
1. Identify EVENT
2. Prepare INPUT ENTITYs for the EVENT
3. Set INPUT CONDITIONs for the EVENT
4. Initiate EVENT
5. Verify OUTPUT ENTITYs
6. Verify OUTPUT CONDITIONs
23 Integration-unsolved question
Fielding SoS – Ride the SW Tiger INCOSE_IL 2007
24. 5-Step Test Cycle
1. Design Initial Test Event
2. Execute Initial Test Event, Gather Data
3. Analyze Data, Identify Knowledge Gaps
4. Execute Investigative Test, Gather Data
5. Analyze Data, Draw Conclusions, Report
24 Integration-unsolved question
Fielding SoS – Ride the SW Tiger INCOSE_IL 2007
25. Technical Models
• Motivation: enrich understanding
of System Events
• Analysis of cause-effect chains
triggered by System Events
• Utilize developers’ expertise
• Discover missing conditions and entities
• Technique: anything useful!
25 Integration-unsolved question
Fielding SoS – Ride the SW Tiger INCOSE_IL 2007
26. What’s Next?
• Enrich Technical Analysis
• Discover appropriate CAD tools
• Add Simulation Capabilities
• Add Knowledge Management Tools
• Evaluate utility of Systems Theory
26 Integration-unsolved question
Fielding SoS – Ride the SW Tiger INCOSE_IL 2007
27. Summary
• SW + SoS = the real challenge
• Methodology: Knowledge Factory
• Techniques: Explorative Modeling
• There’s still a lot to learn
to ride the tiger and survive!
27 Integration-unsolved question
Fielding SoS – Ride the SW Tiger INCOSE_IL 2007