This is a talk on the iLab remote laboratory system given at REV 2014 (International Conference on Remote Engineering and Virtual Instrumentation) at Porto, Portugal. The talk was presented in the Best Paper section of the conference (but didn't win :( ). Check out the paper through IEEE Explore.
Real-time Animation of Equipment in a Remote Laboratory
1. The University of Queensland REV 2014, Porto, Portugal
Real-time Animation of
Equipment in a Remote
Laboratory
Mark Schulz
Feng Chen
Len Payne
The University of Queensland
1
2. The University of Queensland REV 2014, Porto, Portugal
Real-time Animation of
Equipment in a Remote
Laboratory
Mark Schulz
Feng Chen
Len Payne
The University of Queensland
2
This is me. I had the idea to introduce real-time data. I got the idea while I was playing around with a messaging system to transport sensor data. I also
believe that the data should be widely and freely available, as someone may be able to make a use for it that is completely new or novel.
3. The University of Queensland REV 2014, Porto, Portugal
Real-time Animation of
Equipment in a Remote
Laboratory
Mark Schulz
Feng Chen
Len Payne
The University of Queensland
3
This is the student I set the task of making my idea real. He was a Java programmer who had no idea about real-time messaging, graphics, animation, or
remote laboratories before the start of the project. Over two semesters he got this working.
4. The University of Queensland REV 2014, Porto, Portugal
Real-time Animation of
Equipment in a Remote
Laboratory
Mark Schulz
Feng Chen
Len Payne
The University of Queensland
4
This is the developer of the iLab software. He rewrote the iLab software in Java, and introduced the idea of a separate Experiment Engine, which led to
the development of a “farm” of rigs which can be shared seamlessly between users. he helped Feng Chen interface the real-time hooks - but not a lot of
help.
5. The University of Queensland REV 2014, Porto, Portugal
UQ Radiation Experiment Rig
5
1. This is part of a live video feed of the equipment.
2. If you were a student, would you have an understanding of how the equipment works just from this view? How can I be sure that a student using this
rig has the same mental model of this rig as me? Will student engage if no understanding?
6. The University of Queensland REV 2014, Porto, Portugal
UQ User
Interface
6
To illustrate the point: here is the first interface to the experiment rig. Here we are selecting the distances at which to make the measurements, the
number of measurements to make and the time for each measurement.
Has this helped that understanding of the rig?
7. The University of Queensland REV 2014, Porto, Portugal
User view of
Status
User must click ‘Check’
button to discover time to
completion.
7
Here we check whether the experiment is completed.
CLICK
The user must continually hit the ‘Check’ button to get an update from the rig.
8. The University of Queensland REV 2014, Porto, Portugal
Results Page
8
Here we see the results obtained AFTER the experiment has been completed.
9. The University of Queensland REV 2014, Porto, Portugal9
Here’s the question: Do you really understand how these results were produced?
!
A better approach was taken by the folks at Northwestern University.
10. The University of Queensland REV 2014, Porto, Portugal
Northwestern
University
User Interface
10
iLabCentral.org
Here is the first page of the experiment - note that students have already done a lot of preparation work before getting to this part of the experiment —
check out the website as it was designed with the help of high school teachers, instructional designers, and web designers.
!
The top left corner is a Flash animation that students can manipulate to gain a clearer understanding of how the equipment works.
11. The University of Queensland REV 2014, Porto, Portugal
Animation in Action
11
How would we be able to drive this animation: NOT from the sliders and buttons, but from REAL DATA?
There requires a lot of work to test that students do, in fact, build a better model using real data vs a simulation plus the non-intuitive interface, Euan
would agree here
12. The University of Queensland REV 2014, Porto, Portugal
The Next Steps?
GIVEN:
Using Batch Mode iLab Shared Architecture
12
Display the animation (HTML5, CSS,
Javascript)
REQUIRED TO:
Collect data from rig
Transmit data to the animation(s)
We are using the iLab Shared Architecture for our remote labs - albiet a modified version of this. Had to be able to display in the browser with no installed
adding so that software can run anywhere. (== 2D but no 3D yet)
13. The University of Queensland REV 2014, Porto, Portugal
Client
Service Broker
Lab
Server
Standard
Batch-Mode
iLab Shared
Architecture
13
This is the standard architecture for batch mode operation.
We don’t want to change this basic architecture, but enhance it without breaking all our other experiments.
14. The University of Queensland REV 2014, Porto, Portugal
Client
Service Broker
Lab
Server
Experiment
Engine
Enhanced iLab
Batch
Architecture
14
This is a previously report extension: separating out the ‘Experiment Engine’ from the Lab Server. We run our Service Broker and Lab Server software on
centrally maintained hardware (good and bad points to this). BUT, it does allow us to do …
15. The University of Queensland REV 2014, Porto, Portugal
Client
Service Broker
Lab
Server
Experiment
Engine
Experiment
Engine
Experiment
Engine
iLab
Experiment
“Farm”
15
Users see one interface to up to five rigs. Users get allocated a rig on a round robin basis. Easy to drop one out due to failure or maintenance.
MUST ensure that experiment is not compromised by swapping rigs between runs!!
16. The University of Queensland REV 2014, Porto, Portugal
Client
Service Broker
Lab
Server
Equipment
Engine
Message
Server
Display
“Collect data
from the rig”
Mosquitto message server
utilising MQTT:
optimised for M2M
communication
!
http://mosquitto.org/
http://mqtt.org/
16
Here’s what we did to add real-time delivery of data. This is a publish/suscribe messaging system
17. The University of Queensland REV 2014, Porto, Portugal17
Topic:
ilab/radiation-1/Radioactivity/Simulation/Raw/Status
PUBLISH
Experiment Engine
Open Display
Software
Client Display
SUBSCRIBE
SUBSCRIBE
....
Publish-Subscribe Model
‘Experiment Engine’ PUBLISHes a message to a topic on the message server. Message server sends message to each SUBSCRIBEr connected to the
system. Message server is able to handle large numbers of topics, subscribers, and messages per second.
18. The University of Queensland REV 2014, Porto, Portugal
“Transmit
data to the
animation(s)”
18
Client
Service Broker
Lab
Server
Equipment
Engine
Message
Server
Display
DisplayDisplay Display
Extensions to this are to add real-time data into a number of external software subsystems: a data base to enable replay of real-time messages to drive a
simulation to allow tinkering with the system by the user. Also collection of statistical data about usage, etc. — learning analytics
19. The University of Queensland REV 2014, Porto, Portugal
“Display the
Animation”
Uses Adobe Edge (HTML5, CSS, Javascript) for animation (no Flash)
19
This is where it gets interesting — we have no movie because the student kept part of the system running on his laptop and not on our servers. Didn’t find
this out until recently when we tried to run up his demo again. However, we are heading into a total rewrite of his code, so no damage for the future.
!
Points to note: status covers ‘selecting source’, ‘moving GM tube’, … as for Northwestern interface, but driven from REAL DATA.
20. The University of Queensland REV 2014, Porto, Portugal
Some Lessons Learnt
2D animations can be reasonable for real-time animation.
20
Don’t let students use their own laptop for the final demonstration
of a server based product!
Engineers are more data focussed, students still trying to
understand what they are doing.
User Interface Design (UI UX) is CRITICAL!
Don’t let students choose their favourite programming language
Language: The animation involves a Java plugin in this case because they didn’t feel comfortable with Javascript.
UI and UX are ESSENTIAL - we need to use experts not engineers who build the rigs! More than what a graphical designer can provide, actually need an
animator!
Our experience with educational designers indicates these people need to be more involved in our work.
21. The University of Queensland REV 2014, Porto, Portugal
Extension
Work
• Put more hooks into the
operational data.
• Add storage for replay
of experiment.
• Add more visualisations
of the data, e.g., queue
data for progress bar.
21
Client
Service Broker
Lab
Server
Equipment
Engine
Message
Server
Display
DisplayDisplay
Database
Externalise more of the internal operation of every part of the software.
Place in queue and progress bar for job are next things to get going. Helps link to video to know what you are looking at.
Use federate servers to control what data is seen where, and to control loads on message servers.
22. The University of Queensland REV 2014, Porto, Portugal
RT Mapping of Service Broker
Access to UQ Radiation Lab
Extend to user access: where in the world is the
current user?
22
Here is an example of one real-time extension when we expose location data. Se don’t get data in the lab server of where the user is, so we just mapped
the service broker. Need to extend the protocol to give GPS coordinates of users (not expose the identity).
23. The University of Queensland REV 2014, Porto, Portugal
Thank You
Questions Comments?
23
https://github.com/nicochen/AnimaitonRadioactivityiLab
24. The University of Queensland REV 2014, Porto, Portugal
New iLab Sneak Preview
New Service Broker: now a service
Support for alternative authentication schemes
(Facebook, Google, Blackboard via LTI)
User-generated interfaces to Lab servers
RESTful client interface to “legacy” iLab
LabServers
RESTful LabServer interface for new experiments.
24
New service broker written in Node.js!!
25. The University of Queensland REV 2014, Porto, Portugal
New UQ iLab
on Photon
Entanglement
Test platform for our extended
architecture.
!
Due for initial deployment in July
2014
!
Equipment is already in use in a
honours physics course, just not
available remotely.
25
ADVERTISEMENT
Editor's Notes
This is me. I had the idea to introduce real-time data. I got the idea while I was playing around with a messaging system to transport sensor data. I also believe that the data should be widely and freely available, as someone may be able to make a use for it that is completely new or novel.
This is the student I set the task of making my idea real. He was a Java programmer who had no idea about real-time messaging, graphics, animation, or remote laboratories before the start of the project. Over two semesters he got this working.
This is the developer of the iLab software. He rewrote the iLab software in Java, and introduced the idea of a separate Experiment Engine, which led to the development of a “farm” of rigs which can be shared seamlessly between users. he helped Feng Chen interface the real-time hooks - but not a lot of help.
This is part of a live video feed of the equipment.
If you were a student, would you have an understanding of how the equipment works just from this view? How can I be sure that a student using this rig has the same mental model of this rig as me? Will student engage if no understanding?
To illustrate the point: here is the first interface to the experiment rig. Here we are selecting the distances at which to make the measurements, the number of measurements to make and the time for each measurement.
Has this helped that understanding of the rig?
Here we check whether the experiment is completed.
<CLICK>
The user must continually hit the ‘Check’ button to get an update from the rig.
Here we see the results obtained AFTER the experiment has been completed.
Here’s the question: Do you really understand how these results were produced?
A better approach was taken by the folks at Northwestern University.
Here is the first page of the experiment - note that students have already done a lot of preparation work before getting to this part of the experiment — check out the website as it was designed with the help of high school teachers, instructional designers, and web designers.
The top left corner is a Flash animation that students can manipulate to gain a clearer understanding of how the equipment works.
How would we be able to drive this animation: NOT from the sliders and buttons, but from REAL DATA?
<<There requires a lot of work to test that students do, in fact, build a better model using real data vs a simulation plus the non-intuitive interface, Euan would agree here>>
We are using the iLab Shared Architecture for our remote labs - albiet a modified version of this. Had to be able to display in the browser with no installed adding so that software can run anywhere. (==> 2D but no 3D yet)
This is the standard architecture for batch mode operation.
We don’t want to change this basic architecture, but enhance it without breaking all our other experiments.
This is a previously report extension: separating out the ‘Experiment Engine’ from the Lab Server. We run our Service Broker and Lab Server software on centrally maintained hardware (good and bad points to this). BUT, it does allow us to do …
Users see one interface to up to five rigs. Users get allocated a rig on a round robin basis. Easy to drop one out due to failure or maintenance.
MUST ensure that experiment is not compromised by swapping rigs between runs!!
Here’s what we did to add real-time delivery of data. This is a publish/suscribe messaging system
‘Experiment Engine’ PUBLISHes a message to a topic on the message server. Message server sends message to each SUBSCRIBEr connected to the system. Message server is able to handle large numbers of topics, subscribers, and messages per second.
Extensions to this are to add real-time data into a number of external software subsystems: a data base to enable replay of real-time messages to drive a simulation to allow tinkering with the system by the user. Also collection of statistical data about usage, etc. —> learning analytics
This is where it gets interesting — we have no movie because the student kept part of the system running on his laptop and not on our servers. Didn’t find this out until recently when we tried to run up his demo again. However, we are heading into a total rewrite of his code, so no damage for the future.
Points to note: status covers ‘selecting source’, ‘moving GM tube’, … as for Northwestern interface, but driven from REAL DATA.
Language: The animation involves a Java plugin in this case because they didn’t feel comfortable with Javascript.
UI and UX are ESSENTIAL - we need to use experts not engineers who build the rigs! More than what a graphical designer can provide, actually need an animator!
Our experience with educational designers indicates these people need to be more involved in our work.
Externalise more of the internal operation of every part of the software.
Place in queue and progress bar for job are next things to get going. Helps link to video to know what you are looking at.
Use federate servers to control what data is seen where, and to control loads on message servers.
Here is an example of one real-time extension when we expose location data. Se don’t get data in the lab server of where the user is, so we just mapped the service broker. Need to extend the protocol to give GPS coordinates of users (not expose the identity).