IEEE Aerospace 2019, Big Sky, MT, USA

1. Visualization of Software Architectures in Virtual
Reality and Augmented Reality
Andreas Schreiber1, Lisa Nafeie1,2, Artur Baranowski1,2,
Peter Seipel1, Martin Misiak2
1 Intelligent and Distributed Systems
German Aerospace Center (DLR)
Cologne, Germany
2 TH Köln – University of Applied
Sciences, Cologne, Germany
> IEEE Aerospace 2019 > A. Schreiber, L. Nafeie, A. Baranowski, P. Seipel, M. Misiak • Visualization of Software Architectures in Virtual Reality and Augmented Reality > 03.03.2019DLR.de • Chart 1

2. Intelligent and Distributed Systems – Research Topics
> IEEE Aerospace 2019 > A. Schreiber, L. Nafeie, A. Baranowski, P. Seipel, M. Misiak • Visualization of Software Architectures in Virtual Reality and Augmented Reality > 03.03.2019DLR.de • Chart 2
Machine
Learning
BlockchainsWorkflows &
Provenance
Distributed
Systems
Software
Analytics
</>
Software
Engineering

3. Large Software Systems for Simulation, Data Analytics, System Design, ...
> IEEE Aerospace 2019 > A. Schreiber, L. Nafeie, A. Baranowski, P. Seipel, M. Misiak • Visualization of Software Architectures in Virtual Reality and Augmented Reality > 03.03.2019DLR.de • Chart 3

4. OSGi-based Applications
Large applications at DLR are
based on OSGi (Eclipse RCP)
• RCE: Distributed Design
and Simulation
• VirSat: Satellite Design
Developed by Teams
• >5 members
• Usually high labor turnover
> IEEE Aerospace 2019 > A. Schreiber, L. Nafeie, A. Baranowski, P. Seipel, M. Misiak • Visualization of Software Architectures in Virtual Reality and Augmented Reality > 03.03.2019DLR.de • Chart 4
Our Use Cases
• Getting an impression of the
dimensions of the application
• Introducing a new member of the
development team
• Starting to work on a new module
• Checking for abnormalities in
the architecture

5. OSGi
Module system and service platform for Java
• Bundles
• Normal components (JARs) with extra
manifest headers
• Contain packages, which contain
classes
• Services
• Java objects with Java interfaces
• Connect bundles dynamically
• Dependencies
• Import and Export of code between
bundles
> IEEE Aerospace 2019 > A. Schreiber, L. Nafeie, A. Baranowski, P. Seipel, M. Misiak • Visualization of Software Architectures in Virtual Reality and Augmented Reality > 03.03.2019DLR.de • Chart 5
Source: D. Seider, A. Schreiber, T. Marquardt and M. Brüggemann, "Visualizing
Modules and Dependencies of OSGi-Based Applications," 2016 IEEE Working
Conference on Software Visualization (VISSOFT), Raleigh, NC, 2016, pp. 96-100.

6. Repository Mining – Scanning GIT Repositories with jQAssistant
> IEEE Aerospace 2019 > A. Schreiber, L. Nafeie, A. Baranowski, P. Seipel, M. Misiak • Visualization of Software Architectures in Virtual Reality and Augmented Reality > 03.03.2019DLR.de • Chart 6
Data Mining
Repository
git
OSGi Application
Graph Database
Neo4j
.MF
Java XML
Visualization
VR IslandViz
Unity3D / C#
Devices
Oculus Rift /
HTC Vive
Scanning and
Analysis
JQAssistant
L. Nafeie and A. Schreiber, “Visualization of Software Components and Dependency
Graphs in Virtual Reality”. 24th ACM Symposium on Virtual Reality Software and
Technology (VRST 2018), Tokyo, Japan, pp. 133:1-133:2

7. Data Model – Meta Graph in Neo4j
> IEEE Aerospace 2019 > A. Schreiber, L. Nafeie, A. Baranowski, P. Seipel, M. Misiak • Visualization of Software Architectures in Virtual Reality and Augmented Reality > 03.03.2019DLR.de • Chart 7

8. Analyzing and Querying – CYPHER Graph Query Language (Neo4j)
CYPHER „Query“ with one „Clause“
MATCH
(a)-[r:RELATIONSHIP_TYPE]->(b)
RETURN a, r, b
Nodes in our data model
• Packages: PackageFragment
• Classes: CompilationUnit
Relationships in our data model
• Dependencies: EXPORTS, IMPORTS
• Service implementations and
dependencies:
HAS_SERVICE_COMPONENT,
HAS_IMPLEMENTATION_CLASS, …
> IEEE Aerospace 2019 > A. Schreiber, L. Nafeie, A. Baranowski, P. Seipel, M. Misiak • Visualization of Software Architectures in Virtual Reality and Augmented Reality > 03.03.2019DLR.de • Chart 8
MATCH
(pf:PackageFragment)-[c:CONTAINS]->
(cu:CompilationUnit),
()-[e:EXPORTS]->(),
()-[i:IMPORTS]->(),
()-[hs:HAS_SERVICE_COMPONENT]->(),
()-[hi:HAS_IMPLEMENTATION_CLASS]->(),
()-[ps:PROVIDES_SERVICE]->(),
()-[hfh:HAS_FRAGMENT_HOST]->(),
()-[hba:HAS_BUNDLE_ACTIVATOR]->(),
()-[rb:REQUIRES_BUNDLE]->()
RETURN pf,c,cu,e,i,hs,hi,ps,hfh,hba,rb

9. Software Visualization
> IEEE Aerospace 2019 > A. Schreiber, L. Nafeie, A. Baranowski, P. Seipel, M. Misiak • Visualization of Software Architectures in Virtual Reality and Augmented Reality > 03.03.2019DLR.de • Chart 9
Source: L. Merino, M. Ghafari and O. Nierstrasz, "Towards Actionable Visualisation
in Software Development," 2016 IEEE Working Conference on Software Visualization
(VISSOFT), Raleigh, NC, 2016, pp. 61-70.

10. Island Metaphor
> IEEE Aerospace 2019 > A. Schreiber, L. Nafeie, A. Baranowski, P. Seipel, M. Misiak • Visualization of Software Architectures in Virtual Reality and Augmented Reality > 03.03.2019DLR.de • Chart 10
Classes
Multi-storey buildings with a new
storey for every n lines of codes
Packages
Continuous regions
Bundles
Islands with multiple regions;
each island with distinct shape

11. Dependencies
> IEEE Aerospace 2019 > A. Schreiber, L. Nafeie, A. Baranowski, P. Seipel, M. Misiak • Visualization of Software Architectures in Virtual Reality and Augmented Reality > 03.03.2019DLR.de • Chart 11
Ports
Incoming and outgoing package
dependencies
Arrows
Strength and direction of a package
dependency
Services
OSGi service interfaces and service
components

12. > IEEE Aerospace 2019 > A. Schreiber, L. Nafeie, A. Baranowski, P. Seipel, M. Misiak • Visualization of Software Architectures in Virtual Reality and Augmented Reality > 03.03.2019DLR.de • Chart 12

13. Visualization in Virtual Reality
> IEEE Aerospace 2019 > A. Schreiber, L. Nafeie, A. Baranowski, P. Seipel, M. Misiak • Visualization of Software Architectures in Virtual Reality and Augmented Reality > 03.03.2019DLR.de • Chart 13
Source: M. Misiak, D. Seider, S. Zur, A. Fuhrmann, and A. Schreiber, ”Immersive
Exploration of OSGi-based Software Systems in Virtual Reality," 25th IEEE
Conference on Virtual Reality and 3D User Interfaces (IEEE VR 2018), Reutlingen,
Germany, 2018.

14. > IEEE Aerospace 2019 > A. Schreiber, L. Nafeie, A. Baranowski, P. Seipel, M. Misiak • Visualization of Software Architectures in Virtual Reality and Augmented Reality > 03.03.2019DLR.de • Chart 14
Virtual Table

15. Services and Service Dependencies
> IEEE Aerospace 2019 > A. Schreiber, L. Nafeie, A. Baranowski, P. Seipel, M. Misiak • Visualization of Software Architectures in Virtual Reality and Augmented Reality > 03.03.2019DLR.de • Chart 15

16. Navigation
> IEEE Aerospace 2019 > A. Schreiber, L. Nafeie, A. Baranowski, P. Seipel, M. Misiak • Visualization of Software Architectures in Virtual Reality and Augmented Reality > 03.03.2019DLR.de • Chart 16

17. Virtual Tablet
> IEEE Aerospace 2019 > A. Schreiber, L. Nafeie, A. Baranowski, P. Seipel, M. Misiak • Visualization of Software Architectures in Virtual Reality and Augmented Reality > 03.03.2019DLR.de • Chart 17

18. Visualization in Augmented Reality
> IEEE Aerospace 2019 > A. Schreiber, L. Nafeie, A. Baranowski, P. Seipel, M. Misiak • Visualization of Software Architectures in Virtual Reality and Augmented Reality > 03.03.2019DLR.de • Chart 18

19. Information for Selected Bundles – Wire Frame and Information Panel
> IEEE Aerospace 2019 > A. Schreiber, L. Nafeie, A. Baranowski, P. Seipel, M. Misiak • Visualization of Software Architectures in Virtual Reality and Augmented Reality > 03.03.2019DLR.de • Chart 19

20. > IEEE Aerospace 2019 > A. Schreiber, L. Nafeie, A. Baranowski, P. Seipel, M. Misiak • Visualization of Software Architectures in Virtual Reality and Augmented Reality > 03.03.2019DLR.de • Chart 20
Interaction in AR – „Air Tap“ Gesture and Voice Control

21. Exploration of Software Systems – Example Tasks for RCE
> IEEE Aerospace 2019 > A. Schreiber, L. Nafeie, A. Baranowski, P. Seipel, M. Misiak • Visualization of Software Architectures in Virtual Reality and Augmented Reality > 03.03.2019DLR.de • Chart 21
Selecting
Bundles
Showing
Bundle
dependencies
Exploring
Service
dependencies
Showing
Services

22. Current and Future Work
General work for all visualizations
• User studies
(with eye tracking and questionnaires)
• Natural language interaction
(digital assistants and chatbots)
• Support for multiple users
Specific visualizations and visual concepts
• Software history (evolution)
• Understanding of rationale, intent, and
implication
> IEEE Aerospace 2019 > A. Schreiber, L. Nafeie, A. Baranowski, P. Seipel, M. Misiak • Visualization of Software Architectures in Virtual Reality and Augmented Reality > 03.03.2019DLR.de • Chart 22

23. > IEEE Aerospace 2019 > A. Schreiber, L. Nafeie, A. Baranowski, P. Seipel, M. Misiak • Visualization of Software Architectures in Virtual Reality and Augmented Reality > 03.03.2019DLR.de • Chart 23
Thank You!
Questions?
Andreas Schreiber (@onyame)
Head of Department
Andreas.Schreiber@dlr.de
Lisa Nafeie (@LisaNafeie)
Student Assistant Software Visualization
Lisa.Nafeie@dlr.de
DLR Intelligent and Distributed Systems
http://www.DLR.de/sc/ivs