Introduction to my Research

515 views
393 views

Published on

Personal Homepage
http://www.hci.iis.u-tokyo.ac.jp/~kylo/

1 Comment
1 Like
Statistics
Notes
No Downloads
Views
Total views
515
On SlideShare
0
From Embeds
0
Number of Embeds
3
Actions
Shares
0
Downloads
6
Comments
1
Likes
1
Embeds 0
No embeds

No notes for slide
  • 2013 Luckily be here with Sato Lab.
  • I started learning Japanese at the first year in Univ.NTU, National Taiwan University33,000 students (UT40,000)1.6km2 (~= UT)
  • After ended my first year in university, I figured out the life here is not what I want.So I planed a travel, to see people on the opposite of the earth,How they act, how they
  • Since there is a required bachelor thesis project at our department,I choose the lab lead by Prof. Lin, which focus on image processing and inspection technique.Start to play with OpenGL, OpenCV, and the GUI software Boaland C++
  • 2013 Luckily be here with Sato Lab.2012 ICPR and ACCV works, sponsored by Panasonic Scholarship2012 Join Academia Sinica, literally "Central Research Academy”.~50 CS P.I. (principal investigator) in two institution. Has paper on competitive conf. such as ICCV, CVPR, MM, every year.2011 Navy, duty and vomit on the ship. Political warfare2010 Internship , HsinChu, Taiwan intern in Intelligent Video Processing Research Group • Automatic 2D-to-3D Conversion of video for 3D TV• Survey the standards of stereoscopic video• Depth map generation, depth fusion, saliency detection 2010 graduate2010 Paper Tuna fish species classification using support vector ma-chine”. International Symposium on Machinery and Mechatronics for Agricultural and Biosys-tems Engineering (ISMAB) 2009 UTech Machine Vision Prize, 3rd place • Human Face Sex Recognition (3 out of 24 teams from Taiwan and China)• Part of winning team with prize NTD 200,000.2009Robotics Competition American Society of Agricultural and Biological Engineers Annual International Meeting• Build a multi-robot system equipped with Infra-Red sensors and Ultrasonic sensor• Each robot cooperate with Server through Zigbee wireless communication• Lead a team of 4 member for 6 months preparation (rank 4 out of 8 teams from U.S.)2008 Participating in the Lab, began to learn OpenGL and OpenCV2007 Language School at Univ. Pennsylvania, United State2006 National Taiwan University (rank 80, 2012 QS)
  • 2013 Luckily be here with Sato Lab.2012 ICPR and ACCV works, sponsored by Panasonic Scholarship2012 Join Academia Sinica, literally "Central Research Academy”.~50 CS P.I. (principal investigator) in two institution. Has paper on competitive conf. such as ICCV, CVPR, MM, every year.2011 Navy, duty and vomit on the ship. Political warfare2010 Internship , HsinChu, Taiwan intern in Intelligent Video Processing Research Group • Automatic 2D-to-3D Conversion of video for 3D TV• Survey the standards of stereoscopic video• Depth map generation, depth fusion, saliency detection 2010 graduate2010 Paper Tuna fish species classification using support vector ma-chine”. International Symposium on Machinery and Mechatronics for Agricultural and Biosys-tems Engineering (ISMAB) 2009 UTech Machine Vision Prize, 3rd place • Human Face Sex Recognition (3 out of 24 teams from Taiwan and China)• Part of winning team with prize NTD 200,000.2009Robotics Competition American Society of Agricultural and Biological Engineers Annual International Meeting• Build a multi-robot system equipped with Infra-Red sensors and Ultrasonic sensor• Each robot cooperate with Server through Zigbee wireless communication• Lead a team of 4 member for 6 months preparation (rank 4 out of 8 teams from U.S.)2008 Participating in the Lab, began to learn OpenGL and OpenCV2007 Language School at Univ. Pennsylvania, United State2006 National Taiwan University (rank 80, 2012 QS)
  • Introduction to my Research

    1. 1. Research ReviewKuo-Yen Lo羅国彦ロ コウエン2013.4.18Sato Laboratory, University of Tokyo
    2. 2. Short Curriculum Vitae• Personal Homepagehttp://www.hci.iis.u-tokyo.ac.jp/~kylo/• Period of past yearsUniversity (2006 – 2010)Internship (2010 – 2011)Research Assistant (2012 – 2013)• Two main research topic:1. 2D-to-3D conversion2. Photo Aesthetics.• Wrap up
    3. 3. Short Curriculum Vitae• Language: English (TOEIC935), JLPT(N1), Chinese(Native)• Programming: C/C++, Matlab, Java(android)Technique: SIFT/SURF/HOG, K-means, GMM, kNN, SVM, PCA/LDA/ITML, bad-of-visual word, bilateral filter.• Have traveled to: USA, Korea.Want to travel to: China, Thailand, Spain• Why Japan-- historical and cultural connection-- camera companies and electronics maker here
    4. 4. Visual CuesLow-levelMathematicsMachineLearningPsychologyComputer Vision
    5. 5. Overview2006NTU2007UPenn2008OpenCV2009RoboticsContestGenderRecognitionContest2012ResearchAssistantICPR2012ACCV_w20122010ISMABNTUGraduateInternship@ TV corp.2011Navy
    6. 6. National Taiwan University2006NTU2007UPenn2008OpenCVPeople in Vision• Yi-Ping Hung(MM12, CVPR11, CHI11, UIST11)• Yung-Yu Chuang(CVPR12*3)• C.J. Lin (Libsvm)• H.T. Lin(ICML12, NIPS12, CVPR11KDD12 Champion)• Winston H. Hsu(MM12*6)33,0001928 B.C.World rank 80
    7. 7. Summer School in UPenn2006NTU2007UPenn2008OpenCVSummer Language ProgramUniversity of Pennsylvania
    8. 8. Join the Lab2006NTU2007UPenn2008OpenCVBiophotonics andBioimaging LaboratoryProf. Ta-Te LinOpenGLOpenCVBorlandC++ Builder
    9. 9. Robotics Contest @ ASABE 20092009RoboticsContestGenderRecognitionContest• Problem:Detecting and Positioning the circular obstacle• Technique:Graphical simulation(OpenGL), Sensor• Material:Boe-Bot Toolkit, IR sensor, Ultrasonicsensor, Zigbee wireless communication
    10. 10. Robotics Contest @ ASABE 20092009RoboticsContestGenderRecognitionContestPlease Visit the following linkfor viewing the video:http://youtu.be/8EjON8Y2OJ0
    11. 11. Gender Recognition Contest2009RoboticsContestGenderRecognitionContest• Problem:Recognize the gender with single face image• Technique:Viola-Jones Face detector (OpenCV)Feature-based alignmentFalse-alarm checkRotate the image 35 degreeto detect all possible tilt face
    12. 12. Gender Recognition Contest2009RoboticsContestGenderRecognitionContest• Problem:Recognize the gender with single face image• Technique:Viola-Jones Face detector (OpenCV)Feature-based alignmentFalse-alarm checkUse Eye detectorto wrap the face tountilt view.
    13. 13. Gender Recognition Contest2009RoboticsContestGenderRecognitionContest• Problem:Recognize the gender with single face image• Technique:Viola-Jones Face detector (OpenCV)Feature-based alignmentFalse-alarm checkUtilize Skin color model, Eyeand Mouth detector to filterthe false-positive result fromthe V-J face detector.
    14. 14. Gender Recognition Contest2009RoboticsContestGenderRecognitionContest• Problem:Recognize the gender with single face image• Technique:Viola-Jones Face detector (OpenCV)Feature-based alignmentFalse-alarm check• Performance1.2 second per 480*320 image• Result~85% face detection accuracy~75% gender recognition accuracyWin 3rd place among 20 teams (Taiwan andChina). Bonus 60,0000yen.
    15. 15. Fish Recognition2010ISMABNTUGraduateInternship@ TV corp.• Problem:Tuna species recognition for fisheryconservation and management• Task:Detection and ClassificationBigeyeYellowfinAlbacore3 spices are considered2011Navy
    16. 16. Fish Recognition• Problem:Tuna species recognition for fisheryconservation and management• Task:Detection and ClassificationFish Image are capturedin certain lighting conditionwith measurement plate.Body part is smooth,makes it reflect light well.72%2010ISMABNTUGraduateInternship@ TV corp.2011Navy
    17. 17. Fish Recognition• Problem:Tuna species recognition for fisheryconservation and management• Task:Detection and ClassificationB Y AB 89 10 5Y 9 86 3A 10 9 8184%34% 72% 52% 58%Confusion Matrix(Head)(Abdomen)(Tail fin) (Tail)Discriminate part!2010ISMABNTUGraduateInternship@ TV corp.2011Navy
    18. 18. Yeh, Graduation!2010ISMABNTUGraduateInternship@ TV corp.2011Navy
    19. 19. 2D-to-3D conversion2010ISMABNTUGraduateInternship@ TV corp.2011Navy• Problem:Generating 3D videofrom 2D content.• Inspiration:3D information isrecovered by depthcuesCaptured View + Depth
    20. 20. 2D-to-3D conversion2010ISMABNTUGraduateInternship@ TV corp.2011NavyReality Comfort• Accurate depth map• Correct depth order• Real-time processing• Clear boundary• Temporal smoothness• Visual impression
    21. 21. How people perceive depth?2010ISMABNTUGraduateInternship@ TV corp.2011Navy1. Low-level cue 2. Scene Recognition
    22. 22. 2D-to-3D conversion2010ISMABNTUGraduateInternship@ TV corp.2011NavyVideo frame + motion estimation[ICCE 2009]Approaches1. Depth map by motion2. Depth map by saliency3. Depth map by priorinformation fusion
    23. 23. 2D-to-3D conversion2010ISMABNTUGraduateInternship@ TV corp.2011Navy Video frame + Saliency map [SDA 2010]Approaches1. Depth map by motion2. Depth map by saliency3. Depth map by priorinformation fusion
    24. 24. Introduction to Bilateral Filter2010ISMABNTUGraduateInternship@ TV corp.2011NavyBilateral Filter [Tomasi, ICCV98]:f(x) h(x)“Bi” lateral = Spatial term + Range term
    25. 25. Introduction to Bilateral Filter2010ISMABNTUGraduateInternship@ TV corp.2011NavyBilateral Filter [Tomasi, ICCV98]:f(x) h(x)“Bi” lateral = Spatial term + Range term
    26. 26. Introduction to Bilateral Filter2010ISMABNTUGraduateInternship@ TV corp.2011NavyBilateral Filter [Tomasi, ICCV98]:f(x) h(x)“Bi” lateral = Spatial term + Range term
    27. 27. Application of Bilateral Filter2010ISMABNTUGraduateInternship@ TV corp.2011Navy“Bi” lateral = Spatial term + Range termSmooth Target Edge-Preserving ResultAnd this one?
    28. 28. 2D-to-3D conversion2010ISMABNTUGraduateInternship@ TV corp.2011NavyPrior fusion [Siggraph 2009]1. Decide Geometric perspective2. Integrate Image and Depth map by Bilateral filter
    29. 29. One-year in Navy2010ISMABNTUGraduateInternship@ TV corp.2011Navy
    30. 30. Academia Sinica2012ResearchAssistantICPR2012ACCV2012Institute of Information Science(Central Research Academy)People in Vision• Chu-Song Chen (CVPR12, CVPR11*2)• Mark H. Liao (MM12*2, MM11*2)• Y.-C. Frank Wang (ECCV12, CVPR12)• Yen-Yu Lin (CVPR13, MM12, TPAMI11)Prof. Chen
    31. 31. PHOTO AESTHETICS CLASSIFICATIONPredicting the visual appealing quality of photos
    32. 32. good?jcar@DPChallenge
    33. 33. good?Mnet @ DPChallenge
    34. 34. Which one is better?
    35. 35. Voted by online photo communityAverage: 5.088 votesAverage: 7.292 votes
    36. 36. Reason?
    37. 37. Reason?BoundaryAlternating repetition(Texture)Contrast Levels of scaleRoughnessStrong centersPositive spaceLocal symmetriesThe VoidNot-separatenessGood shapeGradientsEchoesSimplicity and Inner CalmDeep interlock and ambiguityColorCompositionHarmoniumRichness
    38. 38. ApplicationImage Search & Management Photo evaluation systemEmbedded Camera system Media analysis
    39. 39. Photo Aesthetics2012ResearchAssistantICPR2012ACCV2012• Problem:Recognition the appealing quality ofphoto by computational approaches.• Technique:Image analysis, Pattern recognition,Crowdsourcing, Psychology, Photography• Application
    40. 40. ICPR 20122012ResearchAssistantICPR2012ACCV2012As a Pattern Recognition Problem…Comparison of feature1. Edge distribution, Color histogram,Hue, Saturation.. [Ke, CVPR06]2. SIFT + BOV [Marchesotti , ICCV11]3. Composition layout (Edge + HSV),Color palette, contrast.. [Proposed]ResultItem Speed on PC AccuracyCVPR06 0.2s 81%ICCV11 4s 85%Proposed 0.16s 84%[ Photo aesthetics assessment with efficiency ]
    41. 41. Extraction of Color InformationExtract NDominant colors(we set N=5)K-Nearest Neighbor(K=20)List of PalettesDictionaryHQ Palettes DictionaryLQ Palettes DictionaryPalettes of Photo
    42. 42. Retrieved by FrequencyRetrieved by Kmeans(Cluster Center)Proposed(Weighted Kmeans)Finding the Dominant Colors
    43. 43. Video Demo2012ResearchAssistantICPR2012ACCV2012[ Intelligent Photographing Interface with On-Device Aesthetic Quality Assessment ]Please Visit the following linkfor viewing the video:http://youtu.be/o8mKuTfO6ao
    44. 44. Discussion 1Device : On-line assistive camera system• Contextual Information (Viewing angle)camera < human• Feedbackfrom analysis to advice• Human behaviorWhat do people take?How do people take?• ComputationServer-based v.s. Device• Market and Needs
    45. 45. Discussion 2Algorithm: photo aesthetic value assessment• Definition of photo aestheticsExpert v.s. Volkswagen• Labeling processIndividual bias and variance.Absolute or Relative evaluationEffect of Labeling order• Quantify photo aestheticModeling, the Personalization
    46. 46. Thanks for your attention!10 ratings5.00/7 average5 ratings4.90/7 average

    ×