Successfully reported this slideshow.
RESEARCH COLLABORATION@ FOUR EYES LABSelan R. dos Santos – selan@dimap.ufrn.brImagina Lab / UFRN – imagina.dimap.ufrn.brIN...
Outline• My Background - Why am I here?• Previous research - What I’ve done  • Visualization  • Games  • Virtual reality• ...
My Background• PhD in Computing, 2004, at School of  Computing, Leeds, UK, advisor: Professor Ken Brodlie• PhD focus: Visu...
My Background• I’m also affiliated to the INCT-MACC: National Institutes of Science and Technology – Medicine Assisted by ...
Imagina Lab• Recently created research lab, with faculty members working on  • Artificial Intelligence, Computer Games, Sc...
Imagina Lab • Several desktops and workstations; phantom omni • New equipment arriving soonSensable Omni Phantom NVis nVis...
Imagina Lab• Little time for research: total (6/2) courses/year, for  (under/grad) students; administrative roles; advise ...
Previous Research – PhD (1/2)• Multidimensional/Multivariate visualization• Motivation:  • High dimensionality is a challe...
Previous Research – PhD (2/3)• Solution  • A uniform approach to the filtering of both multidimensional and    multivariat...
Previous Research – PhD (3/3)            Example        Step 1• 4-dimensional chained Rosenbrock function• Implementation ...
Previous Research – PhD (3/3) Step 2: select dimensions   Step 3: change window
Previous Research – PhD (3/3) Step 2: select dimensions   Step 3: change window
Previous Research – PhD (3/3)
Previous Research – PhD (3/3)
Previous Research – COSSENA (1/4)• Camera Orientation to Support Search Navigation• Motivation/Problem  • Social virtual e...
Previous Research – COSSENA (2/4)• Solution  • Assist wayfinding with a game-inspired navigational tool based on    the sm...
Previous Research – COSSENA (3/4)2. Let user choose a destination (he may define priorities)3. Exocentric view of an avata...
Previous Research – COSSENA (4/4)• Experiment  • Desktop VR  • 29 participants, between-subjects design  • Task: find 7 ta...
Previous Research - GOLD• Goal-Oriented Level Generation for Action/Adventure  Games – GOLD• Motivation  • Current game’s ...
Previous Research - GOLD• Overview
Previous Research - GOLD• Overview
Previous Research - GOLD• Overview
Previous Research - GOLD• Solution 1. Goal planning   • A sequence of goals, a list of <obstacles, solutions> + probabilit...
Previous Research - GOLD• Output plan: [a, b, c, d, e, f, g, h, i, j]
Previous Research - GOLD2. Layout planning  • Grid of cells in which we position obstacles and solutions
Previous Research - GOLD• Obstacles are assigned to doors of rooms
Previous Research - GOLD3. Geometry generation  • Generate the scene base on the layout and chunks of cell
Previous Research - GOLD3. Geometry generation  • Generate the scene base on the layout and chunks of cell
Previous Research - GOLD• Contribution  • The related works do only stages 1 and 2 OR stage 3• Ongoing work• Problem: how ...
Previous Research - Handcopter• Problem  • Post-stroke rehabilitation of patients with hand impairment is mainly    done b...
Previous Research - Handcopter• Results  • Positive: patients enjoyed the game  • Positive: they wanted to take the game h...
Research proposal #1- Effects ofpanoramic view on navigation• Problem  • Brain-computer interfaces and eye-tracking demand...
Effects of panoramic view on navigation• Motivation  • Brain-computer interfaces and eye-tracking demands more time to    ...
Effects of panoramic view on navigation
Effects of panoramic view on navigation
Effects of panoramic view on navigation• Problem  • When the user looks up or down the distortion is applied both    horiz...
Effects of panoramic view on navigation
Effects of panoramic view on navigation
Effects of panoramic view on navigation• Research questions  • How panoramas affect object recognition for selection tasks...
Research proposal #2 – AugmentedReality Scenes with Coherent Illumination• Motivation  • Shadows, realistic lighting and s...
AR Scenes with Coherent Illumination• Related work  • Pessoa et al. [2010] proposed a technique to generate a    photoreal...
AR Scenes with Coherent Illumination• Objective  • Real time automatic generation of illumination coherent AR scenes    us...
AR Scenes with Coherent Illumination• Proposal  • Focus initially on directional light determination (for basic rendering ...
AR Scenes with Coherent Illumination• Proposal  • For the cases where only two normals are available, we are plannig    to...
Research proposal #3 – EnhancingScatterplot visualization with Haptics• Motivation  • Vis techniques usually rely on visua...
Enhancing Scatterplot Vis with Haptics• Proposal  • Use haptic properties in visualization to alleviate the information   ...
Enhancing Scatterplot Vis with Haptics• Method  • We have organized our study in two experiments  • Stage I    • Observe p...
Enhancing Scatterplot Vis with Haptics• Partial results  • We tested the relative perception of different intensities of e...
Enhancing Scatterplot Vis with Haptics• Currently, we’re designing the experiments for next stage
Research proposal• Still have a couple of other problems (long presentation!)• Please contact me for details• Short papers...
Other topics of interest• Audiometry project (2 years)• L+N (3 years project)
Upcoming SlideShare
Loading in …5
×

Presentation Selan dos Santos 4Eyes Lab

242 views

Published on

short presentation given on 2-Nov-2012 at the Four Eyes Lab, UCSB.

Published in: Education
  • Be the first to comment

  • Be the first to like this

Presentation Selan dos Santos 4Eyes Lab

  1. 1. RESEARCH COLLABORATION@ FOUR EYES LABSelan R. dos Santos – selan@dimap.ufrn.brImagina Lab / UFRN – imagina.dimap.ufrn.brINCT-MACC/CNPq – macc.lncc.br
  2. 2. Outline• My Background - Why am I here?• Previous research - What I’ve done • Visualization • Games • Virtual reality• Research proposals - What I’m planning to do here
  3. 3. My Background• PhD in Computing, 2004, at School of Computing, Leeds, UK, advisor: Professor Ken Brodlie• PhD focus: Visualization of complex data• Since 2006 – assistant professor at DIMAp/UFRN, Natal• Mid-sized Brazilian university: one of a few hundred public universities funded by the Federal or State governments• DIMAp: 35 faculty members• Imagina Lab: 6 faculty (imaging, graphics and AI)
  4. 4. My Background• I’m also affiliated to the INCT-MACC: National Institutes of Science and Technology – Medicine Assisted by Scientific Computing,http://macc.lncc.br • Modeling and simulation of the human cardiovascular system, the simulation of surgical procedures and medical procedures for craniofacial traumatism • Advanced processing of medical images, visualization and 3D reconstruction of patterns with medical relevance, and its applications in the modeling and computational simulation of physiological systems and image based diagnosis. • Collaborative virtual environments for virtual and augmented reality, as well as for telemanipulation in the medicine for training and formation of human resources, and for surgical planning. • Public health monitoring and multimedia traffic in medical videoconferences. • High performance distributed computing for medical applications in the areas mentioned above.• Financial support from the Science without Borders – The Brazilian scientific mobility program “aimed at helping students and research to undertake research in the best and most relevant universities around the world.”
  5. 5. Imagina Lab• Recently created research lab, with faculty members working on • Artificial Intelligence, Computer Games, Scientific and Data Visualization, VR, Digital Image Processing, Image Reconstruction from Projections, Modeling of Curves and Surfaces, Computational Geometry, Digital Topology• Currently, I’m advising: 3 PhD, and 5 M.Sc• PhD/MSc are both course- and research-based
  6. 6. Imagina Lab • Several desktops and workstations; phantom omni • New equipment arriving soonSensable Omni Phantom NVis nVisor ST50 OptiTrack NaturalPoint
  7. 7. Imagina Lab• Little time for research: total (6/2) courses/year, for (under/grad) students; administrative roles; advise students• But, after 5 year we are encourage to pursue a sabbatical leave – time to boost research!• My objectives are: • Establish collaboration ties (government funded grants) • Open a channel for student exchange program: sandwich PhD & undergrad students year abroad (6-12 months) • Learn from Four Eyes experience
  8. 8. Previous Research – PhD (1/2)• Multidimensional/Multivariate visualization• Motivation: • High dimensionality is a challenge for data visualization • Multidimensional relates to multiple independent dimensions: ex. To understand the behavior of a objective function in the n- dimensional space around the optimum • Multivariate relates to large tables and the need to comprehend the relationship among attributes• Problem: • To reduce dimensionality to make it compatible with the display dimensionality
  9. 9. Previous Research – PhD (2/3)• Solution • A uniform approach to the filtering of both multidimensional and multivariate data, to allow data extraction subject to constraints on their position or value within an n-dimensional window; the filtered data is presented as several subspaces based on choices of dimensions to display• Implementation • A pair of operations on data: (1) definition of a window of interest – in the multiD this is a windows defining the domain of interest, whereas in the multiV it is a region defining a range of interest of the variables; and (2) reduction of dimensions or variables.
  10. 10. Previous Research – PhD (3/3) Example Step 1• 4-dimensional chained Rosenbrock function• Implementation in IRIS Explorer
  11. 11. Previous Research – PhD (3/3) Step 2: select dimensions Step 3: change window
  12. 12. Previous Research – PhD (3/3) Step 2: select dimensions Step 3: change window
  13. 13. Previous Research – PhD (3/3)
  14. 14. Previous Research – PhD (3/3)
  15. 15. Previous Research – COSSENA (1/4)• Camera Orientation to Support Search Navigation• Motivation/Problem • Social virtual environments, such as Second Life or MMPORG, rely on large-scale complex to present a virtual world • People from various background want to be able to navigate these VEs but without assistance they get lost or become disoriented • Navigate these worlds to find interesting places or wayfinding requires an accurate mental model of the environment – this takes time and repeated navigation
  16. 16. Previous Research – COSSENA (2/4)• Solution • Assist wayfinding with a game-inspired navigational tool based on the smooth control of the camera’s point of view to indicate a route to a target location; i.e. to support search and exploratory navigation tasks without a previous mental map of the VE1. Determine a roadmap of the environment – a graph
  17. 17. Previous Research – COSSENA (3/4)2. Let user choose a destination (he may define priorities)3. Exocentric view of an avatar; automatic determination of the camera’s point of view: a. Azimuth & elevation to find a clear view of the avatar and the farthermost visible part of a collision-free path to target b. In case of occlusion, move camera a position hit by a ray coming from the avatar’s head towards the original camera position
  18. 18. Previous Research – COSSENA (4/4)• Experiment • Desktop VR • 29 participants, between-subjects design • Task: find 7 targets, in predetermined order • 3 independent variables: cossena, compass, printed map • 2 dependent variables: time & divergence between optimal trail and trail left by participants• Results • <Show in paper> • Navigation assisted by cossena was better then control groups • Users with no experience in navigating a desktop VE had similar performance to experienced users
  19. 19. Previous Research - GOLD• Goal-Oriented Level Generation for Action/Adventure Games – GOLD• Motivation • Current game’s complexity leads to increasingly high costs • Middleware and engines may reduce development time; but there is no middleware for art • Procedural Content Generation (PCG) may alleviate this problem to a certain degree: reduces developing cost and time; replay value, and reduces loading time• Problem • Focusing on PCG for Platform/adventure games • Gameplay based on interaction with environment, puzzles, and actions
  20. 20. Previous Research - GOLD• Overview
  21. 21. Previous Research - GOLD• Overview
  22. 22. Previous Research - GOLD• Overview
  23. 23. Previous Research - GOLD• Solution 1. Goal planning • A sequence of goals, a list of <obstacles, solutions> + probabilities (game-dependent) • Goals: must be visited by player (new item or ability, boss) • Obstacles: prevent player to get through (wall, door) • Solutions: allow player to overcome obstacles (ability, key) • Ultimately, solutions, obstacles can be regarded as goals
  24. 24. Previous Research - GOLD• Output plan: [a, b, c, d, e, f, g, h, i, j]
  25. 25. Previous Research - GOLD2. Layout planning • Grid of cells in which we position obstacles and solutions
  26. 26. Previous Research - GOLD• Obstacles are assigned to doors of rooms
  27. 27. Previous Research - GOLD3. Geometry generation • Generate the scene base on the layout and chunks of cell
  28. 28. Previous Research - GOLD3. Geometry generation • Generate the scene base on the layout and chunks of cell
  29. 29. Previous Research - GOLD• Contribution • The related works do only stages 1 and 2 OR stage 3• Ongoing work• Problem: how do we evaluate this? • Related work has done subjective evaluation only • Publish a game and collect user input? • Show the results to experienced level designers? • Open to suggestions…
  30. 30. Previous Research - Handcopter• Problem • Post-stroke rehabilitation of patients with hand impairment is mainly done by physical therapy sessions • They often are long, tiresome, and tedious • Stimulate patients and keep them committed to the treatment • We had to detect small amplitude movements of the fingers; this was a requirement defined by the physiotherapist• Solution • Serious game controlled by a low-cost vision-based input device: a regular camera, a 20L bottle of water, Allegro+OpenCV • Input had to capture flexion & extension of fingers • Avoid calibration to skin tone • < show video >
  31. 31. Previous Research - Handcopter• Results • Positive: patients enjoyed the game • Positive: they wanted to take the game home with them • Interesting: they wanted to play over and over to improve score and “beat” the score of the other patient (and vice-versa) • Negative: patients complained about the glove • Remove glove and introduce a skin tone calibration • System ready to a large-scale test in the university hospital • Open question: How to measure (quantify) the amount of movements done in one game session?
  32. 32. Research proposal #1- Effects ofpanoramic view on navigation• Problem • Brain-computer interfaces and eye-tracking demands more time to execute a single command than devices like mouse and keyboard
  33. 33. Effects of panoramic view on navigation• Motivation • Brain-computer interfaces and eye-tracking demands more time to execute a single command than devices like mouse and keyboard • Interaction time required to navigate 3D scenes may take longer because user needs to change viewpoint • We want to improve navigation time by testing panoramic projections (360 degrees) – avoid change of viewing orientation • Understand how panoramic view affects selection and spatial knowledge of the 3D environment
  34. 34. Effects of panoramic view on navigation
  35. 35. Effects of panoramic view on navigation
  36. 36. Effects of panoramic view on navigation• Problem • When the user looks up or down the distortion is applied both horizontally and vertically, making navigation difficult• Solution Coupled-clipping Decoupled-clipping
  37. 37. Effects of panoramic view on navigation
  38. 38. Effects of panoramic view on navigation
  39. 39. Effects of panoramic view on navigation• Research questions • How panoramas affect object recognition for selection tasks • How panoramas influence spatial awareness after navigation • Comparison of navigation experiences (search and exploration) • Understand the influence of scenario design (branching, long hallways, etc.) on navigation • Compare type of panoramas: cylindrical, spherical, coupled- clipping, decoupled-clipping, perspective
  40. 40. Research proposal #2 – AugmentedReality Scenes with Coherent Illumination• Motivation • Shadows, realistic lighting and shading are important aspects that may improve the degree of realism in a computer generated scene • Sometimes we need to have augmented scene blend with the real world in a visually coherent manner• Problem • How to capture real world information to produce lighting coherence in augmented scenes?
  41. 41. AR Scenes with Coherent Illumination• Related work • Pessoa et al. [2010] proposed a technique to generate a photorealistic AR scene using BRDFs • Requires a previous non automated training phase • Yeoh and Zhou [2009] technique was able to render scenes with coherent illumination and shadows • Requires a complex marker with AAA battery, a white A4 sheet, a fiducial marker and a series of squares printed in different gray tones • Lee and Jung [2011] used a rectangular texture with a mirrored sphere on top of it • The mirrored sphere may disrupt user interaction • Aittala [2010] used a rectangular texture and a Ping-Pong ball • Illumination is tracked according to sphere position, which may be different from the marker location, thus creating visual inconsistency
  42. 42. AR Scenes with Coherent Illumination• Objective • Real time automatic generation of illumination coherent AR scenes using a simple marker• Problem • How to capture real world information to produce lighting coherence in augmented scenes? • We want to avoid a training phase, previous preparation of the scene, and dealing with complex markers
  43. 43. AR Scenes with Coherent Illumination• Proposal • Focus initially on directional light determination (for basic rendering techniques and shadowing) • Given a real scene and a textured cube (marker), we want to define a vector d as a directional light • d is a linear combination of the visible cube’s normals (ni) • wi may be determined by comparing the brightness of each visible texture with its corresponding reference texture used for tracking – dissimilarity measurement
  44. 44. AR Scenes with Coherent Illumination• Proposal • For the cases where only two normals are available, we are plannig to segment the cube’s shadow and determine a shadow vector s • For the cases where only one normal is visible, we may rely on frame coherence to estimate d• Open questions • Good method to compare dissimilarity among tracked and reference textures • Since we know the marker geometry, is it possible to use shadow segmentation to illumination parameter retrieval?
  45. 45. Research proposal #3 – EnhancingScatterplot visualization with Haptics• Motivation • Vis techniques usually rely on visual channel to foster insight • In multivariate data we have more than 3 variables or attributes • In scatterplots these variables are mapped to visual properties of a marker: position (3), size (1), color (1), shape (1), etc. • But, visualization may overload the visual channel! • Haptic visualization rely on special devices to simulate a great variety of effects, such as touch, vibration, pressure, weight, etc.• Problem • How can we harness haptics to improve multivariate vis? • Can we find an efficient “haptic mapping” similar to the traditional visual properties mapping? • Vision is far more powerful than the other senses
  46. 46. Enhancing Scatterplot Vis with Haptics• Proposal • Use haptic properties in visualization to alleviate the information overload of the visual channel • We want to use haptics in two different ways: • Replacing visual properties • Reinforcing visual properties• Objective • Understand the interplay between visual and haptic properties • Propose a set of design guidelines for haptic visualization
  47. 47. Enhancing Scatterplot Vis with Haptics• Method • We have organized our study in two experiments • Stage I • Observe participants interacting with several haptics properties • Select the ones with best data/perception rate • Stage II • Combine or replace visual and haptic properties in scatterplot and observe efficiency of visualization • Materials: Phantom Omni, Chai 3D, custom made vis software
  48. 48. Enhancing Scatterplot Vis with Haptics• Partial results • We tested the relative perception of different intensities of effect: viscosity, magnetism, resistance (to perforation), vibration, and stiffness • Best effects: vibration (A), resistance, and viscosity • Limit of perception: 20% difference in intensity
  49. 49. Enhancing Scatterplot Vis with Haptics• Currently, we’re designing the experiments for next stage
  50. 50. Research proposal• Still have a couple of other problems (long presentation!)• Please contact me for details• Short papers describing each one of these projects Thank you!
  51. 51. Other topics of interest• Audiometry project (2 years)• L+N (3 years project)

×