Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Human computer learning interaction in a virtual laboratory by Vasilis Zafeiropoulos (HOU)

Artificial Intelligence webinar week, day 1: Human computer learning interaction in a virtual laboratory by Vasilis Zafeiropoulos (HOU)

  • Login to see the comments

  • Be the first to like this

Human computer learning interaction in a virtual laboratory by Vasilis Zafeiropoulos (HOU)

  1. 1. Human-Computer Learning Interaction in a Virtual Laboratory Vasilis Zafeiropoulos PhD Candidate HELLENIC OPEN UNIVERSITY School of Science and Technology
  2. 2. Learning Practices Pyramid2
  3. 3. Οn-site laboratory education - no plain sailing  Large number of trainees  Short time for training  Lab equipment not really completely available (sensitive, wear and tear, consumables)  Risks of accidents (health and maintenance) 3
  4. 4. About Onlabs  Simulation of a real Biology Lab  Main characteristics:  3D  Realism  Interactivity  Genre: Modern Adventure Games  Development under:  Hive3D by Eyelead Software (2012-2015)  Unity (2016-now) 4
  5. 5. Onlabs Screenshot5
  6. 6. Purposes Decongestion of real labs Familiarizing oneself with equipment - safely Protection of lab equipment from damages Capability of infinite trial & error testing (more) Pleasant training process (more) Effective learning Virtual lab education is to complement on- site lab education, not substitute it! 6
  7. 7. Experiments simulated in Onlabs (1)  Α. Use of Optical Microscope 1. Configuration of the microscope (e.g. adjusting light, iris, lenses) 2. Creation of a test specimen (a slide with a piece of paper and water on it) 3. Microscoping with all objectives lenses 7
  8. 8. Experiments simulated in Onlabs (2)  Electrophoresis 1. Use of the electric scale for scaling the various powders 2. Use of electronic pipette for drawing and pouring precise quantities of fluids 3. Use of magnetic stirrer for fully dissolving and stirring the components of a dilution 4. Use of automatic pipette for extracting DNA from PRC tubes 5. Use of microwave oven for dissolving agarose powder in electrophoresis buffer 6. Use electrophoresis device for separating nucleic acid samples by size 7. Use UV viewer for visualizing nucleic acid bands 8
  9. 9. Operation Modes  Experimentation Mode The human user makes free use of all the lab equipment  Instruction Mode The computer guides the human user to complete an experiment  Evaluation Mode The computer evaluates the performance of a human user on conducting an experiment  Computer Training Mode A human expert (expert ≠ user) teaches the computer with Machine Learning to:  rate properly (Rater Training Sub-Mode), i.e. provide an accurate score for the user’s performance)  play properly (Bot Training Sub-Mode), i.e. conduct an experiment
  10. 10. Instruction Mode
  11. 11. Evaluation Mode
  12. 12. Scoring Mechanism (Evaluation Mode)  Consists of two parts:  Success Rate [0-100%] (how “close to” or “far from” the user is from the experiment’s final state)  For the i-th action performed, a score xi ∈ [0,1] is assigned, e.g.: ― if the microscope is connected to the socket (1st action), score x1 = 1; otherwise x1 = 0. ― if the microscope switch is turned on (2nd action), score x2 = 1; otherwise x2 = 0. ― if the microscope light intensity knob is set to 18 (3rd action), score x3 = 1; for any other case x3 lies within (0,1)  Each action has different significance, so we intuitively define a particular weight wi for each one of them  Success rate is the weighted average of the various scores xi: 𝑠𝑢𝑐𝑐𝑒𝑠𝑠 𝑟𝑎𝑡𝑒 ← σ 𝑖=1 𝑛 𝑤 𝑖∙𝑥 𝑖 σ 𝑖=1 𝑛 𝑤 𝑖 ∙ 100  From weights wi, we create a weight vector 𝑤 = 𝑤1, 𝑤2, … , 𝑤 𝑛 𝑇 (useful for Machine Learning later…)  Penalty points (they are received whenever the user performs actions in wrong order)  The Aggregate Score [0-100%] is calculated combining the Success Rate and the Penalty Points
  13. 13. Computer Training Mode (1)  Rater Training Sub-Mode The student plays various sessions The human expert evaluates each session The computer scoring mechanism is adjusted according to human expert’s feedback Machine learning techniques used:  Genetic Algorithm  Artificial Neural Network
  14. 14. Rater Training Sub-Mode
  15. 15. Computer Training Mode (2)  Bot Training Sub-Mode The computer plays a session by itself The human expert evaluates each session The computer learns how to play correctly Machine Learning technique used:  Reinforcement Learning (Under development…)
  16. 16. Bot Training Sub-Mode
  17. 17. Genetic Algorithm (Rater Training Mode)  A GA simulates biological evolution  Our GA is interactive (a human supervisor contributes to the learning process)  In our GA, weight vectors are the chromosomes  Our first generation consists of 30 randomly produced weight vectors.  Weight vectors, like chromosomes, compete against each other with respect to each one’s fitness (expressed by a fitness function)  For each play session, the score produced by a weighted vector and the score provided by the human expert are compared by our fitness function  According to weight vectors’ fitness:  A fixed percentage of them are selected (directly copied) to the new generation  A fixed percentage of them are chosen for crossover (reproduction) with each other and their offspring are put into the new generation  A fixed percentage of the chromosomes in the produced generation are mutated  The GA stops after a termination condition is satisfied; our GA’s termination condition is 50 generations  The fittest weight vector of the final generation is the training result with our GA
  18. 18. Artificial Neural Network (Rater Training Mode)  An ANN simulates neural networks in the brain  Our ANN consists of 3 layers of neurons  To each neuron of an ANN come weights from the neurons of the previous layer  Our ANN has different weights and provides a different scoring mechanism from the ones in Evaluation Mode!  Our ANN’s initial weights wj→i are randomly produced  For each play session, our ANN produces a single value (score) as output through its rightmost neuron  For each play session, the human expert provides their own score, too  The error between the ANN’s score and the human expert’s score is back-propagated through the ANN and its weights are re-configured  An ANN is re-trained several times (epochs); our ANN is being trained for 1000 epochs  The training result is the ANN’s final weights
  19. 19. Training and Testing  Train on set A, test on set B  Train on all sets, test on all sets (biased)  Train on Experts E1, E2, E3, test on E4 (cross-validation among experts)  Train on Expert E1, test on E1 (cross-validation within the same expert)  Train on Classifications C1, C2, test on C3 (cross-validation among performance classifications*)  Train on Groups G1, G2, G3, G4, test on G5 (cross-validation among various groups of training sets**)  Calculation of Mean Squared Error (MSE) on each testing set  Comparison of GA’ fittest weight vector to intuitive weight vector of evaluation mode * Classifications (of user’s performance): Low, Medium, High ** Groups of 12 training sets (one classification from each expert in each group)
  20. 20. Genetic Algorithm Training Results
  21. 21. Artificial Neural Network Training Results (1)
  22. 22. Artificial Neural Network Training Results (2)  Mean Squared Error to Epochs graph (train on all, test on all):
  23. 23. Testing & Evaluation by Students  Focus group: biology-oriented students with minimum or zero previous knowledge in science topics  Written conceptual pre-tests and post-tests  Questionnaires to express opinion of satisfaction, motivation, engagement, etc.  Practice examinations in the wet lab  The Virtual Lab Group (educated with Onlabs) and the Control Group (educated without Onlabs) conducted a 22 steps microscopy experiment  Results: ▪ a. I completed this step easily ▪ b. I completed this step on difficulty ▪ c. I couldn’t complete this step by myself – I asked for help
  24. 24. Future Work  Simulation of the rest of lab instruments and biology experiments  Simulation of instruments and experiments at:  Chemistry lab  Physics lab  Any other place with instruments and procedures  Insertion of more than one bots in the virtual lab  Let them interact, co-operate and compete with the human user and each other  Evaluate learning outcomes in terms of:  speed  accuracy  use of resources etc. 24
  25. 25. The grand goal  Reduce the cost of laboratory education  Elevate lab education at-a-distance to at least the same level of learning effectiveness with on-site lab education 25
  26. 26. Development Team  Vasilis Zafeiropoulos  Dimitris Kalles  Argyro Sgourou  Achilles Kameas  Evgenia Paxinou  Kostas Mitropoulos Thank you!