How do computers see in 3D?
•Download as PPTX, PDF•
0 likes•198 views
Presented at Julia Robinson Math Festival Seattle, March 22, 2014
Recommended
More Related Content
Similar to How do computers see in 3D?
Similar to How do computers see in 3D? (20)
How do computers see in 3D?
- 1. Kathleen Tuite Julia Robinson Math Festival
- 3. Math + Photography + Computer Science
- 7. How do humans judge depth? ?
- 8. How do humans judge depth? ? 5ft45° 90°
- 9. How do humans judge depth? 5ft 5ft45° 90°
- 11. Cameras are like eyes
- 13. Stereograph of Yosemite (1901)
- 19. What if we had many eyes?
- 20. lots of photos ~ lots of eyes
- 21. Maybe we can make a collage?
- 22. We can make a 3D model!
- 23. How to make a 3D model from photos
- 24. Step 1: Find recognizable pieces (features) of photo • Corners of windows • Distinct texture • Building details
- 26. Step 2: Find the same features in multiple photos Typically there are 100s of matches between photos
- 28. Step 3: Triangulate to figure out how far away things are
- 29. Step 3: Triangulate to figure out how far away things are
- 31. Step 3 in a bit more detail • Don’t know what the 3D model should look like • Don’t know where the photos were taken from • Do know what matches between different photos Hey computer, can you figure this out for me? No problem!
- 33. Can we reconstruct all of UW campus in 3D?
- 35. Campus map Info about a 3D model Teams
- 37. Go outside and take pictures of buildings Add pictures to flags Computer vision algorithms Capture flags for your team and conquer buildings Look for flags on the map
- 41. Feedback for an accepted photo 1058 new points! Players only earn points for taking useful photos
- 42. Seed Models Players can start their own seed models from small photo collections
- 44. UW Empire Cornell Empire
- 45. Competition Details Two rounds Six weeks total 45 players Over 109,000 photos!
- 46. Coverage of UW Campus
- 50. What kind of math went into this? • Geometry and trigonometry – How a camera captures a 3D scene in a 2D photos – How to extract 3D shape from lots of different 2D photos – Lots of matrices and linear algebra behind the scenes (the computer does the hard part)
- 51. Programming and computers can help us do more math • Thousands of photos • Millions of 3D points • Set up the problem and have the computer do the hard work
- 52. Depth cameras
- 53. Depth cameras
- 54. Project Tango: a cell phone with a depth camera
- 55. Take photos and try Photosynth!
- 57. Project Euler (sounds like “Oiler”)
- 58. Thanks! • Cool links – http://photosynth.net/ – https://projecteuler.net/ – http://www.poppy3d.com/ Kathleen Tuite ktuite@cs.washington.edu www.superfiretruck.com
Editor's Notes
- The first thing I’m going to show you…
- LengthLength and widthLength and width and depth
- Here’s a 2d videogame… mario. Every object has a Width, height… and they move, but they’re FLAT like pieces of paper. always at the same depth on the screen
- How many of you have seen minecraft? This minecraft world is made out of things with volume, like lego blocks… things that we can stack
- How do you judge depth? Supposetheres a cat in front of us and we want to figure out the distnace to the catUse geometry to figure this out. Make a triangle…
- We use geometry to figure out how far away stuff is
- Use trigonometry to get this length… kknow the cat is five feet away from the original position.
- Don’t have to move around… have two eyesMakes a triangle between our two eyes and some point in the world
- When you model this with geometry… triangles
- When this camera takes a picture, it looks like this… two pictures from different viewpoints. Slightly differennt…
- Toy I grew up with
- this poppy device is like the modern version of this, excpet you put in your eyephone
- Doesn’t work because the building is in a 3D world, it’s not flat, and the pictures don’t line up
- Things that are easy for a computer to detect… really good at corners and where the color changes a lot
- We use geometry to figure out how far away stuff is
- We use geometry to figure out how far away stuff is
- Super technical, but I want to share it with you guys anyway.Gives you shape by moving
- Geometry program on the computer figures it out
- For us, when we got people to play photocity, the latter happened. And what’s surprising about our game is that while less than 50 people played, and only a small fraction of those made a significant contribution, we still collected over 100,000 photos and put together one of the largest, most comprehensive set of photos of a single location.
- This is the website for PhotoCity. It’s got a big map of the campus, and you can click on anything on the map to get more info, which shows up on the side. At the bottom, there’s a bar that says how many flags each team has.
- Here’s a more detailed picture of the map with castles and flags representing 3D models in the world that players can grow by taking photos.
- Now let me walk you through a cycle of play.The player starts out by identifying a target building on the map…And then they go OUTSIDE and take photosAnd upload them to the flag that they’re targetingWe run our computer vision to find out how much new geometry was created and whether the player was able to capture any flags or conquer any buildings!
- Flags are big virtual columns at GPS locations in the world. We’ve colored all the points in this particular flag column orange to direct the players’ attention to expanding this model to the left
- When the model grows, the flag gets filled in, and eventually captured by a player.
- And then Our system automatically places a new flag as the model grows
- Players extend buildings and capture flags by taking and uploading photos. Their photos are automatically accepted or rejected by the system, and when they’re accepted, they actually add 3D points to the 3D model. This shows where the new points came from – they’re the glowing orange points – so players can learn what works about their pictures when they work.
- One thing players can do instead of expanding existing buildings, is go out and start their own from scratch. You need anywhere from 20 to 200 photos to start a seed.5 continents around the world
- A bunch of different seed models by players
- So, now I’m going to tell you about the big competition we had last spring between university of washington and cornell.
- Two round that each lasted 3 weeks. We got about 45 players.
- This shows the coverage of photos over UW campus – each blue dot is where a photo was taken
- And here’s a flythrough of one part of the university of washington campus – this is red square at UW and this model was made from a 3,000-photo subset of all the photos.
- Here’s the last video I’m going to show you of the results produced by these players and this game. This particular model was made from 7000 photos and contains almost 20 million 3D points. As computer vision algorithms get even better, we’ll be able to use the original 50,000 photos of uw and 50,000 photos of Cornell and run these through new algorithms and techniques to get even better-looking models.
- Cornell arts quad. 5000 images (1/10th of the total images at cornell)As computer vision techniques get better, we’ll be able to rerun these photos and generate clearer models
- We’re not doing repetitive math problems by hand
- How many of you have an xbox with a kinect? Lets your xbox see in 3D
- Right now, kinect is probably the only depth camera you have… you’re going to grow up in a world where these cameras are everywhere… in your cell phone, in your parents’ car
- Free program from microsoft that lets you build models from photos
- Whenyoureinteractign with it, press c… shows you the geometry of the situation
- ive always loved math… math degree in college… ive found that being good at math and programming and using those skills together lets me work on really interesting problems that also involve my hobbies like photography.
- Here are some links for you to follow up with at home… photosynth in particular because it’ll do the 3d reconstruction stuff I was talking about for you with your own photos