More Related Content
Viewers also liked
Similar to Ross Tredinnick - Rebecca J. Holz Research Data Management Talk 4/16/2013
Similar to Ross Tredinnick - Rebecca J. Holz Research Data Management Talk 4/16/2013 (20)
Recently uploaded
Recently uploaded (20)
Ross Tredinnick - Rebecca J. Holz Research Data Management Talk 4/16/2013
- 1. Technology and Data of the Living Environments Laboratory Ross Tredinnick Systems Programmer 2012-2013 Rebecca J. Holz Series in Research Data Management April 16, 2013
- 2. Outline What is the LEL? My Background / Role What is the CAVE? Our Data Digital Curation Example Projects Conclusions / Questions? 2
- 3. Faculty Collaborators Kevin Ponto Patricia Flatley Robert Radwin Catherine Arnott-Smith Kevin Eliceiri Michael Gleicher Meg Mitchell Assistant Professor, Brennan Discovery Discovery Fellow, Associate Discovery Fellow and Professor of Assistant Design Studies Theme Fellow, Professor of Professor-Library and Director, Laboratory for Computer Sciences Professor, Art Leader, Professor of Biomedical Engineering Information Studies Optical and Computational Staff Nursing and Industrial and Industrial and Systems Engineering Instrumentation Undergraduates Engineering Kendra Kreutz Ross Tredinnick Dana Paz Megan Kinneberg Associate Lab Director System Programmer Joe Kohlmann Vito Freese Andrew Morland Research Intern Graduate Students Post Docs Aaron Barthlomew F. Daniel Nicolalde Karen Chen Edmond Ramly Carrie Roy Dan Nathan-Roberts AHRQ Postdoctoral Fellow
- 4. Goals Create Innovative Home Care Technology 4
- 5. Goals Advance the Art and Science of Design Design Virtual Reality Virtual Reality Prototype Test Implement 5
- 6. Goals Advance the Science of Virtual Reality 6
- 7. Collaboration Research Interest / Projects with UW: Art Art History Biology Biomedical Engineering Chemistry Classics Surgery Industry Partnerships with: Industrial and Systems Engineering Manufacturing Library and Information Sciences Construction / BIM Mechanical Engineering Biotech Physical Therapy . . . And more Design Through Immersive Visualization 7
- 8. Background 8
- 9. My Role “Systems Programmer” • Aid researchers by designing and implementing 3D scenarios to help in answering their research questions • Write software to help visualize new types of research data across different disciplines • Adapt existing VR frameworks to collaborate with other universities • Help publish new findings • Help with Lab Tours • Handle industry work (visualization of data / models) • Maintain lab hardware and software 9
- 10. What is a CAVE? Cave Automatic Virtual Environment A “Virtual Reality Theater” Invented 1992 10
- 11. LEL CAVE 11
- 12. LEL CAVE Intersense IS900 Head Tracking 3D Active Stereo Glasses 12 HD 3D Projectors Wands (3D Input Device) Dual View capability 8 PCs Software! 12
- 13. Software 13
- 14. Data Pipelines Example Conversion: Source Data = SketchUp file (.skp) -> Conversion Tool: Usage of SkpToOSG exporter Scenario Data = model in OpenSceneGraph 14
- 15. Scenario Data Engine 3D Models Scripts / Code Textures Animations 15
- 16. Scenario Pipeline Game Industry LEL 16
- 17. 3D Models Photographs + Basic Modeling 3D Scanned using Depth Camera Online Digital Curation Systems!!!! http://sketchup.google.com/3dwarehouse/ http://www.3dvia.com/search/ http://archive3d.net/ 17
- 18. Audio Online Sound http://www.soundboard.com/ Curation Sites Online “Text-To-Speech” http://www.oddcast.com/home/demos/tts/tts_example.php 18
- 19. Scenario Pipeline Scripts 19
- 20. Research Data Research Data • Type / Amount = dependent on project • 2D Traditional (txt, csv, excel) • 3D Movement • Video (FRAPS, CAVE Web Cam) • Data stored on server, in future: * Research Data is generated by the Scenario * * 20
- 21. Our Digital Curation Spring 2012 - Project started with Prof Dorothea Salo’s Digital Curation Class (LIS 855) • Katherine Hill, Lisa Abler, Kristin Briney, Ashley Guy, Molly Khan, Sarah Leeman and Brittany Sheveland • Initial design and implementation via Drupal • Functions – search, organization, preservation, editing via VCS integration • Suggested Git VCS system • Licensing and Reuse 21
- 22. LEL Project and Object Metadata Map Developer Create GIT repository Developer Project Manager 1. Create object in Drupal 2..Complete required object metadata fields: 1. Create project in Drupal 2. Create Scenario/Project Taxonomy Object Name Object Category* File Name 3. Complete required project metadata fields: • Example: Computer chair • Controlled Vocabulary • Example: • Example: Furniture chair_v07_2007_02_21.jpeg Project Title Scenario/Project* GIT Repository URI • Example: Cave Automatic • Controlled Vocabulary • http://yourserver.com/~you/ Scenario/Project* GIT Repository URI Date Last Modified Virtual Environment (CAVE) • Example: Kitchen proj.git • Controlled Vocabulary • http://yourserver.com/~you/ • YYYY-MM-DD • Example: Kitchen proj.git • 2012-03-24 4. Create known non-required project metadata fields. 3. Create known non-required object metadata fields. Project Start Date, Project Licensing & Reuse Approval Date, Funding Stipulation* Object Description Type of Object Object Creator • Controlled Vocabulary • Example: Black computer • Free text tags • Last name, first name, or Approval Date • Free to Reuse Without chair, swivels • Example: Chair, stool full organization name with • YYYY-MM-DD Permission abbreviation in parentheses • 2013-12-07 • Example: Google Images or Object Contributor File Type Smith, Joe • Last name, first name, or • .xxx Company Affiliation* Grant/Funder Link to Project URL full organization name with • Example .doc • Controlled Vocabulary • Use entity name with • URL abbreviation in parentheses • ABC Architecture abbreviation in • http://...edu • Example: Google Images or Creation Software* Compatible Software* parentheses Smith, Joe • Controlled Vocabulary • Controlled Vocabulary • National Institute of Health • Example: Google SketchUp • Example: Virtual Lab (NIH) 5. Save Licensing & Reuse* Notes • Controlled Vocabulary • Example: Object had x issue • Example: Free to reuse during testing without restrictions Using the Metadata Map Developer 4. Save • Follow the map to Add Object in GIT complete all metadata fields in Drupal. *Controlled Vocabulary/Taxonomy • For definitions of metadata fields, see the Data To create, edit, or delete a term: 1.Click “Structure” in the Admin menu Final Stage: All project and Dictionary. 2.Select “Taxonomy” 3.Select “list terms” located on the right of the particular object metadata should be • Refer to the Training Manual for further taxonomic term to be changed 4.Select “+ Add term” to create a new term completed at this time. instructions. 5.Select “edit” next to a term to rename or delete it 6.Click “Save” to finish 04/2012
- 23. Our Digital Curation Summer 2012 - Joe Kohlmann continued work • Git VCS Integration custom module • Downloadable zip-files of entries Summer 2012 – Joe, Prof Smith, Simon Smith • Improved Taxonomy + entry of data • User Definable Tags + Categories from: • Art & Architecture Thesaurus • http://www.getty.edu/research/tools/vocabularies/aat/ • Library of Congress Subject Headings • http://authorities.loc.gov/ 23
- 24. Digital Curation 24
- 25. Our Digital Curation Spring 2013 – Project continues with Prof Dorothea Salo’s Digital Curation Class • Learning about other 3D digital curation sites and improving our site’s taxonomy Spring 2013 – Ross • SketchUp direct to Curation export • Screenshots of models Future: More users, make public facing? 25
- 26. Digital Curation 26
- 27. Scenario Pipeline 27
- 28. Why Needed? 28
- 29. Other 3D Data Volumes (“Image Stacks”) : Medical Applications Scientific Visualization Point Clouds: Color & Distance information Millions of points acquired via laser scanners Any format a researcher may be working with! 29
- 30. Example Projects Say It To See It How and when will the web truly come to 3D? What will it look like? Or will 3D come to the web? Or both? Word Cake Humanities Collaboration Visualizing sequences of text 30
- 31. Conclusions LEL = A collaborative research lab! • 2 Categories of Data • Scenario creation data (lots of variety) • Research data from scenarios • Digital Curation helps manage all of this. • The future of VR and its applications is once again on the rise! 31
- 32. Conclusions Researchers Open House - 3rd Monday of each month: 11 am to 12 pm • May 13th 10:30 – 12 pm Public tours: 1st Saturday of each month • Noon-1 pm Science Saturday at WID Let us know if you have data / collaboration / research ideas! rdtredinnick@wisc.edu 32
- 33. Questions? wid.wisc.edu/research/lel www.youtube.com/user/LivEnvLab 33
Editor's Notes
- Outcomes:An understanding of the LEL,VR technology, 3D data we work with, 3D scenarios, 3D digital curationUpcoming exciting VR technology
- One of the themes of the Wisconsin Institute for Discovery (Epigenetics, Systems Biology, Bionates, Optimization, GLS, C4, CCT)Directed by Patti BrennanFounded in the Spring of 2010, CAVE Spring 2011Consists of Undergrads, Graduate Students, Academic Staff, Faculty, Post-Docs, CollaboratorsEnhance the understanding of: human behavior, design of devices and environments, interactions, processes, and perception - via immersive visualizationSolve challenges of an interdisciplinary nature, requiring collaboration of individuals across the research spectrum
- The tools in our lab afford us the ability to simulate nearly any type of house hold environment, with this abilityThe lab is founded on the goals of and belief that Behavior - by studying and understanding behaviors, researchers can discover clues to improving healthDesign – in addition, having access to a wide variety of environments plus a greater understanding of human behavior within those environments will lead to innovative health-care designIn addition to this goal, another goal of our lab is to (transition):
- We not only have the ability to simulate any home environment, but also simulate and visualize many other types of environmentsThe left picture shows a user interacting with a 3D sculpting program we developed called SculptUp that enables free form painting of solid geometry in 3D space. It is a very intuitive tool that allows creation of complex models that would otherwise require expertise in traditional desktop based 3D design applications to create. Models can be output to formats that allow them to be 3D printed.Our lab can visualize environments that have a wide range of scale, from being able to take tiny molecular structures and blow them up in size to become surrounded by them to better understand their formTo Bringing the entire galaxy down to a scene that fits inside of our display environment to become the coolest ever educational simulation for kids to understand the solar systemRapid prototypingBeing able to skip the physical prototype stage – current example of a design firm (DC) viewing a design in the CAVE and questioning whether their client will need to even build a physical prototype. Stat about Medicine from IEEE VR Keynote – Satava – physical training cadavers cost ~$850,000 a piece and can only be used onceWhy not use a virtual simulation instead that can be re-used? We need to look to this direction as a whole
- “a realistic simulation of an environment, including three-dimensional graphics, by a computer system using interactive software and hardware.”“a computer-generated environment that, to the person experiencing it, closely resembles reality”Kevin’s definition - “Sensory Substitution” – all sensesNot just visual simulation, but attempts at simulating sound, touch (haptics), and even smell?VRST, 3DUI & VR conferences the last 2 yearsVR is becoming a new medium and we are staying on the forefront of this change
- Great variety of VR applications and dataEndless possibilitiesMortenson Example, Medical Light intersecting with the medical room designQuote from CAVE 2 article : Andreas Linninger, professor of bioengineering and lead researcher of a project that measures and models blood flow in the brains of patients with stroke."We had been looking at computer models of a particular patient's brain for several months," said Linninger, "but within five minutes of putting the model into the CAVE2, the chief endovascologist said we had connected certain arteries in a way that was inconsistent with anatomy." With that revelation, their model could be corrected.http://www.nsf.gov/news/news_summ.jsp?WT.mc_ev=click&WT.mc_id=USNSF_51&cntn_id=126209Immersive visualization allows scientists to look at data in ways they haven’t been able to before in order to help them answer research questions.Traditional desktop applications show data on small screens on a flat surface, we as humans natively see in 3D
- Wisconsin CS, Computer Graphics InterestMinnesota Masters CS – first experiences with VR 2004-2006 Digital Design ConsortiumWorked with VR hardware (HMD) and tracking systems with applications for architecture and design Destineer – Tools Engineer 2006-2009Human Head – Technology Programmer 2009-2012I helped artists and designers to create 3D environments for gamesDesigned and implemented 3D tools to optimize their workflows and decrease iteration timeFeb 2012 LEL!
- Not the best title, but hard to figure out something else, if anyone has any ideas that encompass these things, let me knowWill define “VR framework” shortly
- Before I talk about our CAVE let’s discuss what a CAVE is and some brief historyHow many people have heard of a CAVE (before ours)?Original = 4 sidesOne of two most popular forms of display for VR (other being a HMD)Rear projectionProjectors Cover 6 Walls around you with synchronized images of an environmentTracking provides realistic perspective-correct rendering (scene updates based on your head-movement)Hardware combined with software to create an interactive, immersive experienceOften more than one PC needed to run the system – a small clusterSince then – more sides, higher resolution better tracking, better computers in general!
- 4 PCs for one viewer, 7 for twoOnly dual view capable C6 in existence Mention sliding doorMirrors above and below that reflect projector images seamlessly onto the ceiling and floor
- Discuss how everything fits together to make the CAVE runVR = becoming a software problem (because hardware is lowering in cost and getting much better)
- Custom software is needed to render scenes (VR software is not equal to desktop software)CAVE Hardware = devices just described in previous slideVR Frameworks are pieces of software that act as the link between the CAVE hardware and rendering engines. The frameworks make it easier to setup seamless projections of scenes within the engines. as well as setting up head tracking, and usage of 3D input devices within an interactive simulation.Engines are the pieces of software that actually makes use of the data to make an interactive simulation (drawing of the scene, playing of the sounds, animations, etc.)No unified framework / engine because people have different requirements due to differences in research / and/or hardware setup. Universities often write their own frameworks.VR Frameworks can adapt more than one rendering engine. Each engine may be tailored to render certain types of data better than others, for example there are specific engines for rendering complex chemical and biological structures.Each rendering engine may require more than one type of data to effectively simulate and render a scene. A “data pipeline”.As you can see this quickly becomes a tree of possibilities… and a lot of different data involved.Known Frameworks: CalVR, VRJuggler, MiddleVR, Syzygy, CAVELib, Equalizer, OmegaLib, FreeVR, VruiKnown Used Rendering Engines: OSG, Ogre, Unity, UDK, VMDThe variety of frameworks and engines can prevent collaboration – in the future it would be good for the industry to work to adapt same frameworks and engines.The next section will take a closer look at one of our common scenario data pipelines (highlighted orange)
- Desktop 3D design file formats can’t always just automatically load within VR softwareConversion tools are needed to format data for viewing within the CAVENot all source files may need to go through a conversion process in order to be ready for the engine, it depends on the capabilities of the software and the engine.Each pipeline may have a different number of source data file typesShorter pipelines = a future goal – make software that can handle source data formats directly or aim for fewer conversion steps if this is not possible.What types of data do we work with in general and possibly need to work out a conversion for? (Transition)
- What is a 3D model?Can be either text or binaryFiles contain geometry, material, information Geometry within these files is represented by 3D points, often with connection information to form trianglesSoftware/hardware can then render these triangles to make them visible on screenTextures: painting an image file onto a 3D model by way of “texture mapping”Animations make models move…Code glues together models, textures, sounds, animations to make a 3D interactive simulation / scenarioHow do we know what to gather in terms of assets? Depends upon the experiment or project. Now that we know what data we work with, how do we as a lab put it all together to make an interactive scenario? (Transition to scenario pipeline slide)
- Producers / Publishers / Leads define content needs to the “tools programmer”Tools programmer works with artists and designers to program and create custom 3D design applications that the artists and designers use to make content. As well, the tools programmer writes helper scripts and plugins that interface with commercial 3D design applications they use to help convert source data to scenario data.VCS: management of changes to documents, computer programs, large web sites, and other collections of informationInquire about who has heard of VCS / uses it currently…Game industry: professional artists and level designers produce a lot of the content themselvesLet’s see how this general pipeline is both similar and different to how our pipeline works at the LEL(Press)The LEL has no professional artists on hand.(Press)Nor do we have professional 3D level designers(Press)We’re not making a full game, rather smaller scenarios for research and visualization purposes(Press)Professors and researchers provide the content requirements instead of producers and leads(Press)We don’t use Perforce as a VCS due to costs and also the design of our digital curation system (which I’ll focus on shortly)(Press)Due to the smaller number of people involved with the lab (particularly in comparison numbers-wise to a typical game studio) and smaller projects in general, I don’t fully write 3D design tools from scratch that I often did in the industry, instead it makes more sense to adapt existing available 3D design tools (SketchUp) and write plugins and helper scripts that perform the transforming of data that we need to run a scene in the CAVE.(Press)We may not have professional artists and designers, but we have students! And from the ones we have available are handier than any old artist or designer anyways. They all are great to work with and very driven and I’ve been very impressed by their ability to help our lab. (Press)Professors / Researchers content requirements driven by their Hypothesis and questions, which they discuss with students (sometimes me directly) then I receive content requirements from students and the professors and work to prepare an interactive scenario based on these requirements.So in a sense, students are partially taking the role of designers (Megan example w/ OR and other scenes)(Press)So without professional designers and artists where do we obtain our data? (Transition)
- Photographs and basic modeling works only very well on flat surfaces and can be time consuming.Kinect-based scanning works faster but is still a little ways off in terms of ease of use and quality of outcome – sometimes color isn’t easily retainedThe grand source: Emphasize importance of online 3D model repositories (curation!)Models are also often textured, so we don’t have to worry about needing to manually texture much.
- Where do we get audio? – (no professional sound recording studio or much sound recording knowledge)Again more online digital curation systems.So let’s go back and fill in our scenario pipeline with the new information we know (transition)
- An exporter handles the creation of the levels / scripts which glue together the assets we get from the curation sites.We also emphasize that the research data is generated from the newly created CAVE scenario.What type of research data is generated from the CAVE scenario?Next I will hit on the answer to this, followed by introducing our own digital curation system that we use to keep track of all of our scenario data
- Example: virtual exertions data amount vs. imagination / perceptual calibLabkey: LabKey Server is open source software that helps scientists organize, analyze, and share biomedical research data. R plugins, Relational Database support.Transition – because scenarios create our research data, they are pretty important, so from day 1 back in 2010, Patti has always emphasized the need for a curation system for our lab. Next I’ll discuss how this has taken place (Transition)
- Taxonomy = controlled vocabularyCategories = standardizedTags = user definedProvided suggested naming conventions, work flow
- Setup so that a project manager and the developer can work together to add content to the curation site on a project-to-project basis.Got an initial working drupal-based framework up and running for us as their final project
- Joe’s addition of the Git module makes the site not only a management and searching site but also a system for version control-based editing of models and scenarios.
- Top = searchLeft hand side – can quickly look at objects by category or tagMain Table:Left Column: TitleMiddle Column = Categories on top, Tags on bottomContributor column = attribution to who made / added the dataSecond From Right column = edit drupal entry (when logged in)Rightmost column = git clone and zip download
- After a user clicks on the entries title from the main page
- Briefly back to our pipeline:(Press)Our curation system sits on a git-backend that lets us curate our scenario data.(Press) – the final pipelineIn the Future: Students using 3D Design Tools directly more and interfacing with the curation system and outside curation sites
- Lower costIncrease in higher quality 3D devicesWill lead to an increase in use and application – which will lead to a need for more and more 3D dataThe more of this data we can share, the better the applications, collaborations and progress.Mention I’ve tried Google Glass & Rift
- Volumes nowPoint clouds maybe in the near future