My PhD Thesis

1. Mission Simulation Lab
HICEE
Mission Simulation Lab
HICEE
Mission Simulation Lab
HICEE
Mission Simulation Lab
HICEE
Tangible Collaboration applied in
Space Systems Concurrent Engineering
Concept Studies
Christopher Shneider Cerqueira Ana Maria Ambrosio Claudio Kirner

2. Mission Simulation Lab
HICEE
Mission Simulation Lab
HICEE
Roadmap
10:35 2

3. Mission Simulation Lab
HICEE
Mission Simulation Lab
HICEE
Mission Simulation Lab
HICEE
Mission Simulation Lab
HICEE
Context and Motivation
TUI – Tangible User Interfaces
10:35 3

4. Mission Simulation Lab
HICEE
Mission Simulation Lab
HICEE
10:35 4
Movie: Hidden Figures (2016)
Portraits NASA’s Mercury Program (~50s)
https://www.hq.nasa.go
v/office/pao/History/tas
kgrp.html

5. Mission Simulation Lab
HICEE
Mission Simulation Lab
HICEE
Space Engineering
System Engineering
Concurrent Engineering
Space Systems
Concept Studies
(Phase 0)
Context
10:35 5
OPPORTUNITY
Reintroduce physical
collaborations to
manipulate models.
Industry
4.0
Space 4.0
DST / MBSE
GUI
Modelling and
Simulation
TUI
Augmented
Reality
Internet
of Things
active-connected artefacts

6. Mission Simulation Lab
HICEE
Mission Simulation Lab
HICEE
Objective and Questions
10:35 6
TUI
Augmented
Reality
Internet
of Things
Objective:
This thesis objective is to propose (and
show the feasibility of the use of) TUI
artefacts as the way engineers collaborate
models during Space Systems Concurrent
Engineering Concept Studies Sessions,
reintroducing physical cognitive artefacts
to represent the study data and
collaborative interactions.
Main Question: Can Tangible User Interfaces be
integrated to handle models in CE Sessions?
Question 1 – How does the team of a CE Session
manipulate models?
Question 2 – How adapted are the current tools
to collaborate during a CE Session?
Question 3 - How does the collaboration among
the members occur during a CE Session?

7. Mission Simulation Lab
HICEE
Mission Simulation Lab
HICEE
Mission Simulation Lab
HICEE
Mission Simulation Lab
HICEE
CE Sessions
CE Sessions – Space Systems Concurrent Engineering Concept Studies
10:35 7

8. Mission Simulation Lab
HICEE
Mission Simulation Lab
HICEE
CE Sessions
10:35 8
multidisciplinary groups to
accomplish the multimodality
coverage and convergence of
Complex, Coupled and Multidisciplinary
Products
IDM
Facility
Software
Process
TEAM
Concurrent
Engineering
• Document-centric tool-set (DCT)
documents the information directly in
document artefacts,
• Model-centric tool-set (MCT) documents
the information that represents the
system in model artefacts
Modelling Approach
Model-Based &
Model-Centric
Model-Based &
Document-Centric
Types of Facilities
physical facilities
where the team of
specialists meets
and collaborates
to conceive the
Space Systems.
• decompose the mission statement
• establish a preliminary technical
requirements specification;
• identify possible mission concepts,
• evaluate a preliminary programmatic
and risk assessment.
Space Systems Concept
Studies
SMAD
ECSS-M-ST-10C

9. Mission Simulation Lab
HICEE
Mission Simulation Lab
HICEE
Information Exchange (through Computational Systems)
10:35 9
Single Sheet
(first attempts / now via Cloud Worksheet)
Multiple Sheet via Data Base
(Proposed Approach)
Multiple Sheet
(Current Approach)
Model-Centric Tool-set
(Future)
IDM – Integrated Design Model (“Data Set”)
DCT
MCT

10. Mission Simulation Lab
HICEE
Mission Simulation Lab
HICEE
Mission Simulation Lab
HICEE
Mission Simulation Lab
HICEE
TUI
Tangible User Interfaces
10:35 10

11. Mission Simulation Lab
HICEE
Mission Simulation Lab
HICEE
TUI
10:35 11
Real VirtualR + V V + R
R VRAR AV
R
IR
VR
IVR
AR
HR
AV
IAV
IR IVRHR IAV
Before
90's
After
2000's
Static
Systems
Deterministic
Systems
Intelligent and
Integrated Systems
R=Real World, VR=VirtualReality, AR=Augmented Reality,
AV=Augmented Virtuality, CR=Cross Reality, HR=Hyper-Reality,
IoT=Internet of Things, I=Intelligence
TechnologicalEvolution
time
VirtualInterface
Mixed Realities
CR
(IoT)
Real Interface
Tangible
User
Interfaces
Digital
Physical
Digital Information
(Model)
touch/movement
1st loop – result of immediate
artefact manipulation
2nd loop – through
computationalmodel
sense
change sense
change
actuate
physically
actuatevirtually
Intangiblerepresentation
(video/audio feedback)
TangibleRepresentation = control
& actuatorparts
& actuated video
3rd loop – by
actuation into
the artefact
Artefact
Interaction
Models
TUI
Spatial
Augmented
Reality
Internet of
Things
User Interface
Styles
Handle
Artefacts
Collaborative
Environment
It is physical!
Ergonomics
(human
factors)
Peripheral
Interaction
“tangible” term derives from the Latin words “tangibilis” and “tangere”, meaning “to touch”

12. Mission Simulation Lab
HICEE
Mission Simulation Lab
HICEE
Mission Simulation Lab
HICEE
Mission Simulation Lab
HICEE
Reviews / Analyses / Proposals
10:35 12

13. Mission Simulation Lab
HICEE
Mission Simulation Lab
HICEE
Reviews / Analyses / Proposals
10:35 13
CE
Collaborative
Interactions
CE
Facility
Collaborative
Behaviours
Collaboration
Model Types
CE
Sessions
Handle
Artefacts
TUI
Elements
Taxonomy

14. Mission Simulation Lab
HICEE
Mission Simulation Lab
HICEE
Formality in CE Facilities
10:35 14
Collaborative Virtual Environment Heuristics
Germani et al. (2010)
[1] Teamwork
[2] Communication
[3] Human Involvement
[4] Cognitive Reaction

15. Mission Simulation Lab
HICEE
Mission Simulation Lab
HICEE
Model Types used in CE Sessions
10:35 15
[1] modelling formalism
[2] specialist use
[3] computational interpretation
[4] collaboration
[5] storage method
Team Xc Facility
Goddard ICMD Facility DLR Facility
JAXA Facility
*Foundry Furnace
Free
Models
Loose
Models
Context
Specific
Context
Independent

16. Mission Simulation Lab
HICEE
Mission Simulation Lab
HICEE
Collaborative Interactions
10:35 16
CE
Collaborative
Interactions
Workspace
• Style
• Big Picture
• Detail Level
• Exploration
• Design Selection
• Input of information
• Conduct of sharing
• Remote
SECESA
Information
Exchange
DCT & MCT
CE
Facility
Entities
Parameter
• Visual Appearance
• Navigation
System Element
• Manipulation
• Interfaces
• Structure
• Behaviour
• Manipulate Parameter
Model
• Move/Retrieve
• Manipulate
• Visual Appearance
• Navigation
• Inspection
• Attach / Attach final
• Handle System Elements
ECSS-TM-10-25A
Model
Discipline Worksheet
System Element
Sheets Abstract Content
Parameter
Cells

17. Mission Simulation Lab
HICEE
Mission Simulation Lab
HICEE
TUI Taxonomy
10:35 17
Metaphor (none, verb, none-verb, full), style (passive, active), purpose (instrument, object,
instrument-object), instance (spatial, constructive, relational, constructive-relational, associative)
Construction (volumes, hinges, plugs, handles,
straps, wheels), materials (plastic, wood, paper,
metal, foam, styrofoam, fabric), shape (static,
manually incremented, controllable material)
Sensor Data (orientation, positioning,
depth, weight, temperature, gestures),
Sensor Type (magnetic, resistive,
capacitive, inertial, visual, global positioning,
contact, time of flight)
Actuator Action (movement,
temperature change, sound, light, spin/torque,
wind), Actuator Type (motor,
heater/cooler, vibrator, speaker, LED, laser,
pump, LCDs)
Data Gathering (visible light, infrared,
depth, communication), Feedback-loop
(monitor, projector, glasses, laser, magnetic,
mechanical), Reaction Presence
(direct, indirect)
Data Source (artefact, facility, could,
hybrid), Processing Place (artefact,
facility, cloud, hybrid)
Tangible Spatial Awareness
(none, mutual perception, environment
awereness),
Embodiement (full, nearby, environmental,
distant), Tangible Assigned Functionality
(permanente, programmable, transcient),
Detectability of Interaction (simple skill,
composite skills), Feedback (continuous, discrete,
filtered), Simultaneous use (single, multiple)

18. Mission Simulation Lab
HICEE
Mission Simulation Lab
HICEE
Mission Simulation Lab
HICEE
Mission Simulation Lab
HICEE
Proof of Concept Design
Tangible Collaboration applied into Space Systems Concurrent Engineering Concept
Studies
10:35 18

19. Mission Simulation Lab
HICEE
Mission Simulation Lab
HICEE
Tangible Collaboration applied in CE Sessions
10:35 19
Handle
Artefacts
Use Scenario
Human
To
Data
Human
To
Human
Human
To
Facility
Interaction
Vocabulary
TUI
Design
Entities
Entities in
OPM
CE
Collaborative
Interactions

20. Mission Simulation Lab
HICEE
Mission Simulation Lab
HICEE
Entites Representation in OPM
10:35 20
Entities
Model
System Element
Parameter
System Element’s and Parameter’s Stereotypes
view
View 2 : Alfa View
SD : SD
View 1: Template View
view
(a) OPM Domain
Mission
System Element
Template Alfa : Mission
Segment
Alfa : Mission
CubeSat Alfa : Segment ITA GS : Segment
Parameter
zoom
CubeSat Alfa : Segment
U Size : Parameter
1 U 2 U
ITA GS : Segment
Receiving Style : Parameter
SDR
zoom
SD 2.1 : CubeSat Alfa in-zoomed SD 2.2 : ITA GS in-zoomed
Christopher handles presentation.
OPM exhibits OPM Examples and OPM Ecosystem.
OPM based MBCE exhibits Architecture and Our Work.
presentation consumes OPM, CE, and MBSE.
presentation yields OPM based MBCE.
(CH1) OPD – Object-Process Diagram
(CH2) OPL – Object-Process Language
(CH3) Simulation
Object Process Methodology (OPM)

21. Mission Simulation Lab
HICEE
Mission Simulation Lab
HICEE
Entites Representation in OPM
10:35 21
(b) Graph View
(c) Gray indicating stereotype
links
Entities
Model
System Element
Parameter
System Element’s and Parameter’s Stereotypes
view
View 2 : Alfa View
SD : SD
View 1: Template View
view
(a) OPM Domain
Mission
System Element
Template Alfa : Mission
Segment
Alfa : Mission
CubeSat Alfa : Segment ITA GS : Segment
Parameter
zoom
CubeSat Alfa : Segment
U Size : Parameter
1 U 2 U
ITA GS : Segment
Receiving Style : Parameter
SDR
zoom
SD 2.1 : CubeSat Alfa in-zoomed SD 2.2 : ITA GS in-zoomed

22. Mission Simulation Lab
HICEE
Mission Simulation Lab
HICEE
TUI DESIGN
10:35 22
Use Scenario
Handle
Artefacts
general, hybrid, centralized and indirect
Tangible & Intangible
Representations (aura)
Central Interaction
Controller
The Intangible Representation (aura) is not a
direct metaphor to the manipulated entities
Tangible without
visual meaning
ARTEFACT
(Tangible)
AURA
(Intangible)
Data
(CE Entities)
TUI
Design

23. Mission Simulation Lab
HICEE
Mission Simulation Lab
HICEE
10:35 23
Specialist Workspace Collaborative Workspace
Totem Share
provide
Knowledge
Controls
Big Picture
Show content
open
couple
couple
Walk through
Models
Walk through
Model s Internal
System Elements
Walk through
Model s
Parameters
Walk through
Parameter s
Data
Walk through
Coupled
ModelsCreate a model
branch into
workspace
AURA
(Intangible)

24. Mission Simulation Lab
HICEE
Mission Simulation Lab
HICEE
Auras
10:35 24
Artefact
AuraArtefact
Entity
Aura

25. Mission Simulation Lab
HICEE
Mission Simulation Lab
HICEE
H2H / H2D / H2F
10:35 25
Camera and depth
sensor equipment
Projector
Fixing Support
Projection Surface
with the Artefacts
• Perceptions: Touch,
Position, Gesture,
Movement
• Interfaces:
• Feedback loops
• Work Environment: Table
Formats, Ergonomics,
Reactive Interaction Places

26. Mission Simulation Lab
HICEE
Mission Simulation Lab
HICEE
10:35 26
Human
To
Facility

27. Mission Simulation Lab
HICEE
Mission Simulation Lab
HICEE
Interaction Vocabulary
10:35 27
• Indirect metaphor
• Noun – represents a thing
• Model, Sys Element,
Parameter and its modes /
states
• Verb – represents an
action
• Move, attach, show inside...

28. Mission Simulation Lab
HICEE
Mission Simulation Lab
HICEE
BigPicture Aura
10:35 28

29. Mission Simulation Lab
HICEE
Mission Simulation Lab
HICEE
Other Auras
10:35 29
ControlsKnowledgeShareTotem

30. Mission Simulation Lab
HICEE
Mission Simulation Lab
HICEE
Video
10:35 30

31. Mission Simulation Lab
HICEE
Mission Simulation Lab
HICEE
Mission Simulation Lab
HICEE
Mission Simulation Lab
HICEE
Evaluation
10:35 31

32. Mission Simulation Lab
HICEE
Mission Simulation Lab
HICEE
Evaluation
10:35 32
Question 2 – How adapted are the
current tools to collaborate
during a CE Session?
Question 3 - How does the
collaboration among the members
occur during a CE Session?
Question 1 – How does the team
of a CE Session manipulate
models?
Main Question:
Can Tangible User
Interfaces be
integrated to handle
models in CE
Sessions?
Raised
Questions
Collaboration
Comparison
Analytical Architecture Analysis
Descriptive Informed Argument

33. Mission Simulation Lab
HICEE
Mission Simulation Lab
HICEE
Collaboration Comparison
10:35 33
Entities
Parameter
• Visual Appearance
• Navigation
System Element
• Manipulation
• Interfaces
• Structure
• Behaviour
• Manipulate Parameter
Model
• Move/Retrieve
• Manipulate
• Visual Appearance
• Navigation
• Inspection
• Attach / Attach final
• Handle System Elements
ECSS-TM-10-25A
Workspace
• Style
• Big Picture
• Detail Level
• Exploration
• Design Selection
• Input of information
• Conduct of sharing
• Remote
SECESA
...

34. Mission Simulation Lab
HICEE
Mission Simulation Lab
HICEE
Questions:
10:35 34
Team Xc Facility
Goddard ICMD Facility DLR Facility
JAXA Facility
*Foundry Furnace
Free
Models
Loose
Models
Context
Specific
Context
Independent
Question 2 – How adapted are the
current tools to collaborate during a CE
Session?
IDM
Facility
Software
Process
TEAM
Concurrent
Engineering
Document-Centric Concurrent
Engineering
Model-Centric Concurrent
Engineering
Facility
MBSE
Framework
TEAM
Concurrent
Engineering
CE using DCT CE using MCT
Question 3 - How does the
collaboration among the members
occur during a CE Session?
Question 1 – How does the team of a
CE Session manipulate models?

35. Mission Simulation Lab
HICEE
Mission Simulation Lab
HICEEMain Question: Can Tangible User Interfaces be
integrated to handle models in CE Sessions?
10:35 35

36. Mission Simulation Lab
HICEE
Mission Simulation Lab
HICEE
Mission Simulation Lab
HICEE
Mission Simulation Lab
HICEE
Conclusions
10:35 36

37. Mission Simulation Lab
HICEE
Mission Simulation Lab
HICEE
Conclusions
• TUIs are a
“Multidisciplinary
Development”
10:35 37
it is possible, requires a team development group

38. Mission Simulation Lab
HICEE
Mission Simulation Lab
HICEE
Development “issues”
2014 2015 2016 2017
Christopher
Specialist
(OPM)
oF JAVA
JavaScript
(Angular / GoJS)
MDESatSim
BD
Sirius (EMF)
Relational
SECESA
SECESA
Graph
Excel
MySQL
MariaDB
PostgreSQL
Collab
Projector
CNPq
Artefact
Unity
Controller
Unreal
oF
Arduino
Esp
Artefact
Tracking
Excel
ARuco
IR
Through Kinect
ARToolKitPlus
Electronics
Mechanics
CubeSat
Shape
Generic 3D printed
Hand
Tracking
REST Comm
Aura
Designs
Comm
Neo4J
Through
openCV Through
HW
Disciplines

39. Mission Simulation Lab
HICEE
Mission Simulation Lab
HICEE
Conclusions
• TUIs allows the use of computational
models as free models.
• The collaboration interaction returns
to be person-to-person without a
physical barrier due
desktops/laptops over the table.
• The collabration with artefacts focus
on the people over the data.
10:35 39

40. Mission Simulation Lab
HICEE
Mission Simulation Lab
HICEE
Publications & Other Activities
10:35 40
Structuring a Cross-Reality
Environment to support a Concurrent
Engineering Process for Space Mission
Concept (SECESA 2014)
Structuring Spatial Cross Reality
Applications using openFrameworks
and Arduino
(SVR 2014)
Using Virtual, Augmented and Cross Reality
in INPE’s Satellite Simulators (SVR 2015)
Structuring on-demand Software User
Interfaces based in Spatial Augmented
Reality to Control Hardware (Arduino) (SVR
2015)
A Model Based Concurrent Engineering
Framework using ISO-19450 Standard
(SECESA 2016)
Initial Risk Management in Educational and
Technological Space Missions (WETE2017)
Data Collection Subsystem Modelling and
Simulation using Simulink
(WETE2017)
Alfa Project Mission Definition
(WETE2017)
Describing Model Behaviour using OPM to
an Operational Satellite Simulator
(WETE2017)
Using ISO-19450 to Describe and Simulate a
Small Sat Operational Scenario (WETE2017)
Two Independent Processes of
Verification Applied to a Satellite
Simulator (WETE2016)
A Framework for Automated Model
Validation Applied to Pico-satellite
Electrical Power Subsystem (WETE2016)
Towards an Automated Hybrid Test and
Simulation Framework to Functional
Verification of Nano-satellites’ Electrical
Power Supply Subsystem (AIAA 2016)
Creating and maintaining a workshop for
graduate course in space engineering and
technology and its usefulness to the
training of future researchers (ITST2015)
Eickhoff
Course CAPES Report
Workshops
Workshops
Panel
SMAD course
IEMA
Teaching internship
ITA – Rocket Design
WETE
CI
CTEE
Others MDE
Course

41. Mission Simulation Lab
HICEE
Mission Simulation Lab
HICEE
Contributions
• CE Sessions
• Information exchange approaches : DCT & MCT
• Collaborations: Workspace & Entities
• CE Model types: Free, Loose, Context Specific &
Context Independent
• Facility directioning: formal x informal
• TUI Develop
• Replicable/Extendable artefact: based in IoT
• Taxonomy of elements
• Application in Space Engineering
• Interaction Tangible Vocabulary to CE Session
10:35 41
Team Xc Facility

42. Mission Simulation Lab
HICEE
Mission Simulation Lab
HICEE
Contributions
• MBSE in CE Sessions
• Use OPM as baseline to Model Centric Tooling
• Graph mapping
• Data collection
• Model Transformation
• Relevant to Industry 4.0 / Space 4.0 trend:
reintroduce (active-connected) physical
artefacts in engineering – now enabled to
manipulate models
10:35 42
Basket of techniques
. . .
OPM
MBSE
System
Engineering
Model Based
Development

43. Mission Simulation Lab
HICEE
Mission Simulation Lab
HICEE
Mission Simulation Lab
HICEE
Mission Simulation Lab
HICEE
Future Works
10:35 43

44. Mission Simulation Lab
HICEE
Mission Simulation Lab
HICEE
Future Motto: Creative Work Interconnection (INCOSE)
updates
Space 4.0
Big Data and
Analytics
Modelling and
Simulation
Vertical and
Horizontal
System
Integration
Internet
Of Things
Security
Cloud
Additive Manufacting
Industry
4.0
integrates
Augmented
Reality
Collaborative Robots
10:35 44
TUI
Authoring
Tool
Modelling
and
Simulation
Multiple
Layers
Artificial
Intelligence
MBSE &
Systems
Integration
Hyper Reality
Concurrent
Engineering
Collaboration

45. Mission Simulation Lab
HICEE
Mission Simulation Lab
HICEE
“On the News”
10:35 45
Microsoft Surface Dial (late 2016) Nintendo Labo (2018)

46. Mission Simulation Lab
HICEE
Mission Simulation Lab
HICEE
“With the passage of time, the psychology of people stays
the same, but the tools and objects in the world
change. Cultures change. Technologies change. The
principles of design still hold, but the way they get applied
needs to be modified to account for new activities, new
technologies, new methods of communication and
interaction.”
Don Norman
“The DESIGN of EVERYDAY THINGS, Revised and Expanded 2013’s Edition”
10:35 46

47. Mission Simulation Lab
HICEE
Mission Simulation Lab
HICEE
10:35 47