Good afternoon. I’m Derick Campbell and I’m here to take you on a journey through the past, present, and future of Scientific Data Visualization – spanning over 30 thousand years.Our team in MSR creates free tools for computer and research scientists. During this presentation I’ll show you a few of the ones we have in development now.This week I’m celebrating 14 years with Microsoft. 14 years sounds like a lot – and huge visualization advances have occurred in this time. In the context of science however, this is just a small fraction.Our journey covers a much larger timescale.
Tony introduced the Fourth Paradigm yesterday which describes the journey that science has taken over the last several thousand years.During this time, scientists have embraced new ideas and new ways of working – and new ways of visualizing information.Computers introduced the ability to simulate the real world around us, perform calculations previously unattainable, and display rich dynamic images.Networks and distributed devices provide the opportunity to measure vast quantities of data – around the world, in our oceans, beyond the depths of space, and even – within – where our eyes and ears can’t take us.Scientific Visualization has been on a similar journey – where the next most important paradigm is just now happening right as we speak… let’s dive in and take a look. To begin,let’s start at the beginning…
Visualization is as old as art itself, when drawings were used to convey information about tribal history. Handing down information about life in the past for generations to come. In their wildest dreams, I bet the artists had no expectations that their work would persist and be discovered 32 thousand years later. Just incredible.What I find annoying about this art on the left, is that I have thousands of years of evolution on my side – and I cannot paint a horse as well as the artist here did.In northern Spain, landscape drawings have been found scratched into stone tablets.Does this drawing also document life in the village? Perhaps it was used to plan a hunting expedition?This map is more within my capabilities – it looks like the kinds of maps I draw. I like it. The painting – not so much.From http://en.wikipedia.org/wiki/Chauvet_Cave and http://www.discoveryon.info/2009/08/worlds-oldest-map.html
The sciences of math and medicine have been captured visually for thousands of years.Controversy surrounds the clay tablet on the left. Different researchers have suggested it may represent Pythagorean triples, trigonometry, or geometric terms. A particularly interesting and in-depth book on the tablet provides compelling research to suggest it contains math exercises used for teaching.The National Library of Medicine stores the papyrus shown on the right.This particular asthma remedy is prepared as a mixture of herbs, and heated on a brick for inhalingTablet from http://en.wikipedia.org/wiki/Plimpton_322Papyrus from http://en.wikipedia.org/wiki/Ebers_papyrus
There are many old maps of various parts of the world. This is the CantinoPlanisphere, or Cantino World Map.I chose this map to show you – as it has a very interesting history.It was named after Alberto Cantino – a spy. He successfully smuggled this map from Portugal to Italy in 1502, acting as an agent for the Duke of Ferrara.This map is noteworthy for portraying a part of the Brazilian coast, discovered by the Portuguese explorer Pedro Alvares Cabral.You can also see the islands of the Caribbean, the Florida coastline, as well as Africa, Europe, and Asia.From http://en.wikipedia.org/wiki/Cantino_planisphere
Leonardo da Vinci was many things, including scientist, mathematician, engineer, and inventor.He is considered to be one of the greatest painters of all time, and perhaps the most diversely talented person to have ever lived.Leonardo took an observational approach to science – describing phenomenon in utmost detail rather than through experiments or theoretical explanation.His over 200 pages of anatomical drawings – two of which are shown here – were left to his heir to publish.This was an overwhelming task that would eventually take over a hundred years to only do in part.From http://en.wikipedia.org/wiki/Leonardo_da_Vinci
Early theoretical science, like the work on magnetism shown here, used hand-drawn figures to explain theories.Other works written during this time take a similar approach. Sir Issac Newton’s “Mathematical Principles of Natural Philosophy” published in 1687 also uses many hand-drawn pictures to demonstrate his laws of motion and universal gravitation.I’d like to call out a theme we’re seeing here – the scientists and researchers of history have often had to be artists too – if they really wanted to make an impact.From http://en.wikipedia.org/wiki/De_Magnete, http://www.gutenberg.org/files/33810/33810-h/33810-h.htm
Charles Minard was a pioneer in the use of graphics in engineering and science.He is famous for this work pictured, that tells the story of Napolean’sdisasterous Russian campaign of 1812. It shows several variables in a single two dimensional image.Edward Tufte, a professor of statistics at Yale University, suggested that “it may well be the best statistical graph ever drawn”.From http://en.wikipedia.org/wiki/File:Minard.pngThe map's French caption reads: Figurative Map of the successive losses in men of the French Army in the Russian campaign 1812-1813.Drawn up by M. Minard, Inspector General of Bridges and Roads in retirement. Paris, November 20, 1869.The numbers of men present are represented by the widths of the colored zones at a rate of one millimeter for every ten-thousand men; they are further written across the zones. The red [now brown] designates the men who enter into Russia, the black those who leave it. —— The information which has served to draw up the map has been extracted from the works of M. M. Thiers, of Segur, of Fezensac, of Chambray, and the unpublished diary of Jacob, pharmacist of the army since October 28th. In order to better judge with the eye the diminution of the army, I have assumed that the troops of prince Jerome and of Marshal Davoush who had been detached at Minsk and Moghilev and have rejoined around Orcha and Vitebsk, had always marched with the army.The scale is shown on the center-right, in "lieues communes de France" (common French league) which is 4,444m (2.75 miles).The lower portion of the graph is to be read from right to left. It shows the temperature on the army's return from Russia, in degrees below freezing on the Réaumur scale. (Multiply Réaumur temperatures by 1¼ to get Celsius, e.g. −30°R = −37.5 °C) At Smolensk, the temperature was −21° Réaumur on November 14th.
These are the first graphic images ever generated by machine.Ben Laposky was a mathematician and artist from Iowa. He created these Oscillons – his term - in 1950 – showing how graphic images can be generated electronically and then recorded on high-speed film.From http://stage.itp.nyu.edu/history/timeline/Oscillons.html, http://design.osu.edu/carlson/history/images/pages/oscillon4_jpg.htm, and http://www.digitalartmuseum.org/laposky/index.htm
One of the earliest computerized color visualizations was produced in 1969 – only 42 years ago – by Dr. Louis Frank from the University of Iowa. He plotted the energy spectra of spacecraft plasma by showing energy against time.The color in this visualization represents the number of plasma particles per second measured at a specific point in time. Using color to represent at least one dimension of data is a frequently used technique in scientific visualization today.From http://design.osu.edu/carlson/history/lesson18.html
The 1980s were an important decade for scientific visualization.In 1981, Ron Baeker from the University of Toronto created the famous “Sorting out Sorting” computer video – which demonstrates how various sorting algorithms work and how their performance compares.Pixar was selling high end computer workstations for visualization. (Not well apparently, they were going out of business before they turned the company into an animation studio.)And in 1986, the NSF sponsored a panel on Graphics and Image Processing – which resulted in several recommendations. ClickThey released this special issue of Visualization in Scientific Computing and recommended that the NSF establish a new initiative, workshops, and federal funding around visualization hardware and software.Sorting out Sorting from http://www.youtube.com/watch?v=F3oKjPT5KhgPixar Volume Visualization from http://www.youtube.com/watch?v=ckE5U9FsgsE
This brings us to the 1990s – where we start to see computer visualization being used across many scientific domains.The Hubble Space Telescope is launched, revolutionizing astronomy through the most detailed visible-light images of the universe’s most distant objects. It is important to note the cross domain impact of sharing these images. Hubble observations also lead to breakthroughs in astrophysics – such as accurately determining the rate of expansion of the universe.The Global Positioning System becomes fully operational - enabling GPS devices to provide navigation and mapping to anyone.The Human Genome project begins, resulting in an accurate and complete human genome sequence being made available to scientists and researchers in 2003.The world-wide web and HTML are created – and networks across the planet start bringing people, businesses, and information together in ways we are only now beginning to appreciate.Graphical User Interfaces in home and workplace computers become commonplace. Everyone with a computer now has the ability to create stunning visualizations.From:http://en.wikipedia.org/wiki/1990s_in_science_and_technologyhttp://www.visualcomplexity.com/vc/project.cfm?id=77http://en.wikipedia.org/wiki/Human_Genome_Projecthttp://en.wikipedia.org/wiki/Hubble_Space_Telescope
Clearly, the 2000s provide a rich set of visualizations to admire. I’ve only selected a few – otherwise we’d be here all day.This image of the fibre pathways in a monkey brain really captured my attention – not only because it is beautiful, but because the diffusion spectrum imaging used to create it has introduced new ways of looking at how the brain functions. This picture only shows 10% of the 1.5 million fibres in this brainFrom http://brain.oxfordjournals.org/content/130/3/630.full
The original Mandelbrot set is a distinctive and easily recognizable two-dimensional fractal shape – as seen on the lower left of this slide.It is one of the most popular mathematical visualizations.This 3D Mandelbulb on the right, created by Daniel White, has similar properties to the Mandelbrot – including an elaborate boundary that reveals progressively ever-finer recursive detail at increasing magnifications.On the top left you can see how the fractal image changes with an extreme zoom.From http://www.skytopia.com/project/fractal/mandelbulb.html
Now we hit the 2010s – the first decade of this century. Visualizations just keep getting better and more interesting.Paul Butler is an intern with Facebook.He created this really fascinating image using the R programming languageWhat I find so interesting about this visualization is that none of what you can see is a representation of geo-political borders or coastlines. Everything seen here represents a human relationship.What I find so disappointing about this picture is that you can barely make out my native country – Canada. Canada is bigger than the US – but not here!Almost 70% of Canadians are online – and more than 50% of them use Facebook – what gives?Then I remembered that 3/4s of the Canadian population live within 90 miles of the US border. It’s way too cold in northern Canada.I also find it interesting that “custom programmed” visualizations have in a sense replaced the hand drawn visualizations of history. But this is also a shame – that scientists oftenneed to be artists and programmers to visualize their data and translate it into knowledge.From http://www.facebook.com/notes/facebook-engineering/visualizing-friendships/469716398919
If you study social networks or have a need to draw network graphs, Microsoft Research sponsors the development of an Excel companion tool called NodeXL.NodeXL is an open source release – available as a free download on Codeplex.If your data contains vertices, edges, and even images – NodeXL can quickly draw a network graph in a variety of shapes or customized layouts. It can use Twitter and Facebook data – or any network data you may need to analyze.This particular demonstration of NodeXL shows how twitter feeds can be used to identify influential social media users – such as the influential women attending BlogHer 2010. ClickAnd all of these visualizations were also created using NodeXL.From http://www.flickr.com/photos/marc_smith/4882883368/in/set-72157626300420584 and http://www.connectedaction.net/X - # of followersY - # of tweetsNodeXL.codeplex.com
Rick showed these images in his presentation briefly yesterday. I’d like to tell you the story behind it.TheTerapixel project was an attempt by our team in MSR to create the largest and clearest image of the night sky. Several people didn’t think we could do it.We started with the Digitized Sky Survey – thousands of images taken of the northern and southern skies over a period of 50 years.A64-node high-performance cluster was used to deal with dozens of images each: addressing coordinate mapping, varying levels of brightness, noise, and color saturation.Particularly problematic was addressing the Vignetting in these images – a darkening of the edges and corners of each plate. Vignetting is what makes the old stitched image that you see on the left look like a golf ball rather than a seamless image.Stitching, smoothing, and global image optimization were then applied to create a single spherical image greater than 1 terapixel in size.Incidentally, it would take half a million HDTVs – filling up an football field - to view this image at its full resolution.You can view the results of this work from your own computer in Worldwide Telescope.If you have high-end data processing needs like what we had to do with TeraPixel, be sure to visit Felipe and Sean in booth #13 tomorrow at Demofest – to discuss Cluster and Cloud computingFrom http://research.microsoft.com/en-us/projects/terapixel/
Who you deliver knowledge to can provide important insight into how it might be visualized for maximum effect.To help patients better understand their injuries – what better way to show them what’s happening than to project the image right on their body.By enhancing scans of internal limbs with annotations through the use of a digital camera,pico-projector, and IR sensor –MSR researchers are using creative visualization placement to explore how to improve doctor patient discussions on patient health, medical procedures, and the positive impact of physical therapy.This is a great example of how useful visualizations can truly improve communications, understanding, and impact.From http://research.microsoft.com/apps/video/default.aspx?id=147896http://www.aboutprojectors.com/news/2011/05/17/microsoft-research-unveils-anatonme/
MSR Cambridge, in collaboration with Johns Hopkins Medical Institute, Oxford, and several other universities have been working on a variety of medical image visualization solutions.This particular video shows how medical expertise and modern machine learning as Rick described yesterday can be combined with human medical scans… Enabling automatic detection and segmentation of healthy anatomy and the identification of anomalies or medical issues.The practical applications of these advances include computer-aided health diagnosis, personalized medicine, and new interfaces for surgical intervention.I’m particularly excited about this technology’s potential reduce or eliminate invasive and uncomfortable procedures for health diagnosis. I don’t want to be poked and prodded and have devices inserted inside me just to see if I’m ok. Life is tough enough.And this brings us to the current day. What might the future hold?To help think about the future, I’ve found it helpful to look at the various categories of visualization.From http://research.microsoft.com/en-us/projects/medicalimageanalysis/default.aspx
This fairly exhaustive listing of visualization methods – which itself is an interesting visualization – was created by the good folks at Visual Literacy.If you visit the site, you’ll notice this is an interactive visual – as you hover over each type you’ll see an example of the method with a short description.It’s a great resource. I learned quite a bit by going through it myself. If you are trying to identify which visualization to use in a specific situation, it can be a very useful tool.For the purposes of our discussion today however, I find all of these choices a bit overwhelming. I wonder if there might be a simpler model we could use?What if we think about the purpose of a visualization…From http://www.visual-literacy.org/periodic_table/periodic_table.html
I found a couple of very interesting, simpler models – which I’ve taken some liberties with to create what you see here.This model places visualizations into two very broad categories – based on the purpose and impact of the visual.On the left we have Analyzing – or the use of visualizations for individual research and analysis.This turns data and information into my knowledge or my team’s knowledge.On the right we have what I’ve called storytelling – which includes publishing and sharing of visuals to help people teach and learn. Or even to influence public policy.This type of visualization turns data and information into our knowledge – or ideally humanities knowledge.Of course some types of visuals fit into both categories, and the categories are closely related – analyzing can lead to storytelling, and storytelling can lead to analyzing.
Now I’d like to take you through three new technologies currently in development within MSR for visualization.They are primarily in the storytelling category – though I think you’ll see some applicability for individual research as well.Key to our philosophy – you shouldn’t have to be an artist or a computer programmer to create beautiful experiences for others to learn about your research.
Rich Interactive Narratives – or RIN – is an exciting new experience authoring technologycoming out of MSR India. RIN can be used for creating rich, interactive visualizations.They play like a video, and can include narration, deep zoom images, photosynths, and even Silverlight controlled experiences.They are richer than videos however – at any point the user can pause the experience and explore on their own. I’m going to let you explore the TechFest narrative shown on this slide on your own – let’s take a look at a narrative that includes some interesting scientific data visualization.
Visit RIN online at http://digitalnarratives.net/Demo:Try the Telling Stories with Data narrativeLet the narrative progress until a single bird is on the screenPause the narrative, and explore the endangered species in other ways. Clear the current filters, filter by country, etc.Continue the narrative while noting that RIN brings the Pivot control right back to where it was before.After demo:When I showed this presentation to my family before coming to Cartegena – yes, my family really knows how to have a good time – my daughter was truly inspired by this particular narrative. Right after, she went to the site to browse this narrative herself. Let’s summarize what’s what happened here – a guy in India was interested in an endangered bird, and he influenced a teenager in the US to learn more about endangered animals. Would a published PDF have had the same impact?Yesterday you saw Oscar’s demo on Zentity, which you can use to automatically visualize your data in Pivot. Be sure to visit Oscar in Booth #8 tomorrow at DemoFest.If you’d like to learn more about RIN, where you’ll be able to build tours combining Pivot, maps, and other forms of rich media, visit Sridhar in Booth #12 at DemoFest tomorrow.
Chronozoom is the exciting deep-zoom prototype Tony showed us yesterday. It introduced a new way to visually explore Big History – a field that studies history across long time frames through the eyes of multiple disciplines.Our team today is working on the next generation of ChronoZoom – which will allow hundreds of authors to add various subjects, rich media, and tours to an interactive timeline that begins 13.7 billion years ago with the Big Bang, all the way to current events.Our engineering team has been in close collaboration with Big Historians to ensure we get the experience just right. In fact, at our last engineering review – two of the worlds top Big Historians, David Christian and Walter Alvarez joined us in Redmond. I’m sure our team didn’t feel any pressure at all. It was at this review that Roland Saekow, one of Walter’s students and the person who connected Big History to deep-zoom technologies – gave each of us a small Delorean car, like the one in the movie Back to the Future.He said, “This is for you – because what we’re really building here is a time machine.”From: http://eps.berkeley.edu/~saekow/chronozoom/
Visit ChronoZoom online at http://chronozoom.cloudapp.net/firstgeneration.aspxDemo:Press the right arrow once, deep zoom to New Years Eve before 2001Continue pressing the right arrow, pausing while the image zooms to the next stop in the tourDiscuss how this system will look with scientific and historic results across many disciplines throughout timeRemind views that this current demo is a static deep-zoom image – our future solution will be dynamically generated based on user content provided by historians, scientists, and online data-sources
I need your help! Our team is currently building this next generation Chronozoom service, and we’ll be bringing it online with interactive experiences and tours over the next several months.As you can imagine, Chronozoom gets more and more exciting and useful the more history we include. I’d like us to ensure that Latin American history is richly represented.If you are an expert on the types of subjects shown here for your region, or you know who is – we want to talk with you.Come and meet with Lee Dirks in Booth #7 tomorrow at DemoFest – introduce yourself so we can involve you early.You know, I’m really excited about our ability to interest the next generation in history through Chronozoom. I can imagine my ten year old son Parker browsing all of history, learning about different cultures, important events, the history of humanity and the cosmos.Of course, the chances that Parker will actually use Chronozoom to learn the history his teacher wants him to learn is pretty low. But at least he’ll be learning history.
You’re already familiar with Worldwide Telescope | Earth from Rick’s presentation yesterday. You can browse the many beautiful objects in the sky and the planets in our solar system right from your PC.What you may not know is how easy it is to use Worldwide Telescope | Earth to create custom tours – including your own text, images, and sound.Just last week, our team shipped a public Beta release of an Excel Add-in for Worldwide Telescope | Earth – that allows you to bring your earth and sky-based data right into a rich 3D environment for exploration. Now anyone in this room can create and share the kinds of demos Rick gave yesterday with these free tools.Let’s take a look at the types of tours that can be created when you combine your data with a powerful visualization tool. DemoFrom http://www.worldwidetelescope.org/excelplugin.aspx
Demo uses WWT | Earth and the Excel add-in from From http://www.worldwidetelescope.org/excelplugin.aspx.During demo:Play a tour.Stop tour and explore.Show time series data.Show how to visualize half a million rows from Excel in three dimensions in WWT | Earth.Discuss making a tour yourself.After demo:If you’d like to learn more about creating earth and sky based visualizations yourself, please visit Dean and Kris in Booth #4. They also have a very cool demo that uses Kinect to operate Worldwide Telescope | Earth.
After taking a close look at these tools and others coming outto support scientific data visualization…It seems to me that this simplified model is lacking something important – something crucial for the world of science, as Phil was describing yesterday.As technology makes it easier for people to teach and learn – and we can share and explore each other’s datasets and conclusions in highly visual ways…Our visual solutions are no longer constrained to showing one research outcome at a time.This is our new visualization paradigm. ClickWe’re entering a period of technology-enabled collaboration unlike what we previously thought of as collaboration. Of course scientists collaborate – but this isn’t just about the ability to find and see each other’s work.Technology is enabling us to truly layer diverse datasets and information together in ways that will lead to new conclusions and new knowledge. We can provide the information and story we want to tell, in a way that the viewer can stop and explore in their own way – unconstrained by our perspective. Unbounded exploration is available to us if we use new technologies in the right ways.Issac Newton said it best…
“If I have seen a little further it is by standing on the shoulders of Giants.”But Phil used this quote yesterday – thanks Phil – last night I thought up a new quote to help us think about this new paradigm for scientific visualization.It’s all about inspiration. Computer and research scientists want to make an impact.If you want to change the world, please join us – let’s do it together.Let’s give the next generation a real boost with unprecedented sharing of our data, information, and knowledge through collaborative environments like the ones I’ve shown today.
Several of the live demos I showed today you can try for yourself – visit our Research Accelerators page shown here to learn more and try our software.Please take the time to visit our booths at Demofest tomorrow to learn more about the tools we’ve shown you – plus several other exciting new technologies.To stay informed about these new tools for visualization as they progress, follow Microsoft Research on Facebook or Twitter.Thank you.
Visualizing Scientific Data - LATAM Faculty Summit 2011
Before You Begin…<br />This presentation is best viewed as a Slide Show. Animations!<br />My speakers notes are included in the notes– lots of details<br />Videos in the original deck have been replaced with hyperlinked images for size. Click to view online.<br />Many of the graphics are hyperlinked for your browsing convenience.<br />If you like this deck, or have any ideas for improvement, please let me know.<br />DerickC<br />DerickAtMS<br />See this presentation (and others) online: http://docs.com/@derickc<br />email@example.com<br />
Visualizing Scientific Data<br />Derick Campbell<br />Director of Engineering, Microsoft Research Connections<br />
The Fourth Paradigm<br />A thousand years ago – Experimental Science<br />Description of natural phenomena<br />Last few hundred years – Theoretical Science<br />Newton’s Laws, Maxwell’s Equations…<br />Last few decades – Computational Science<br />Simulation of complex phenomena<br />Today – Data-Intensive Science<br />Data sets from many sources…<br />Data captured by instruments<br />Data generated by simulations<br />Data generated by sensor networks<br />http://fourthparadigm.org<br />http://blogs.nature.com/fourthparadigm<br />
Chauvet Cave art from southern France (32,000 years ago)<br />Contains the earliest known paintings.<br />Stone tablet from northern Spain(14,000 years ago)<br />Contains the earliest known representation of a landscape.<br />
Babylonian Clay Tablet - Plimpton 322 (1800 BC)<br />Early example of Babylonian mathematics.<br />EbersPapyrus (1500 BC)<br />Asthma prescription on Egyptian medical papyrus.<br />
Cantino World Map (1502)<br />Earliest known map showing Portuguese discoveries of the east and west.<br />
Leonardo da Vinci (1452-1519)<br />Leonardo’s drawings in science and engineering are as impressive as his artistic work.<br />
“On the Magnet and Magnetic Bodies, and on That Great Magnet the Earth” by William Gilbert (1600)<br />Describes Earth’s magnetic field. Begins the modern science of geomagnetism.<br />
French Army losses in the 1812 Russian campaign, by Charles Minard (1869)<br />Infographic shows losses of men, their movements, and temperature.<br />
Oscillons, by Ben Laposky (1950)<br />The first graphic images generated by an electronic (analog) machine.<br />
Energy Spectra of Spacecraft Plasma, Dr. Louis Frank (1969)<br />One of the earliest color visualizations done by computer.<br />
Sorting out Sorting, Ron Baeker (1981)<br />Visual demonstration of sorting algorithms.<br />Click to watch video<br />Click to watch video<br />Volume Visualization, Pixar (1987)<br />Before making commercial movies, Pixar sold high-end computers.<br />
Individual research and analysis<br />Turns data and information into my knowledge<br />Analyzing<br />Storytelling<br />Publishing and sharing<br />Teaching and learning<br />Turns data and information into our knowledge<br />
Microsoft Research visualizations in the near future…<br />
ChronoZoom: History in its broadest possible context<br />Challenge: The exploration of Big History, with smooth transition from billions of years down to individual nanoseconds.<br />This is what Walter Alvarez, Professor of Earth and Planetary Science at University of Berkeley set out to do. And he did it, with the help of Microsoft Research and the Live Labs team. <br />A new service in development that allows researchers to browse, overlay, and explore interdisciplinary data sources<br />
Interactive Opportunities with ChronoZoom<br />Exploring Life<br />Scientific Works<br />Cultural Heritage<br />National History<br />Classification and Evolution<br />Significant Events<br />
Worldwide Telescope | Earth<br />A seamless visual environment<br />Sky and earth-based visualizations<br />Create and share tours of your data<br />Introducing an Excel Add-in for geo-spatial data visualizations<br />
Demo of WWT | Earth for scientific visualization<br />
Analyzing<br />Storytelling<br />Open Collaboration<br />The next paradigm in scientific visualization<br />Cooperation through the sharing of data and information generates new knowledge.<br />
“If I have seen a little further it is by standing on the shoulders of Giants.“ <br />– Issac Newton, 1676<br />“How many people would you like to inspire?“<br />