SIGGRAPH 2014 Course on Computational Cameras and Displays (part 4)Matthew O'Toole
Recent advances in both computational photography and displays have given rise to a new generation of computational devices. Computational cameras and displays provide a visual experience that goes beyond the capabilities of traditional systems by adding computational power to optics, lights, and sensors. These devices are breaking new ground in the consumer market, including lightfield cameras that redefine our understanding of pictures (Lytro), displays for visualizing 3D/4D content without special eyewear (Nintendo 3DS), motion-sensing devices that use light coded in space or time to detect motion and position (Kinect, Leap Motion), and a movement toward ubiquitous computing with wearable cameras and displays (Google Glass).
This short (1.5 hour) course serves as an introduction to the key ideas and an overview of the latest work in computational cameras, displays, and light transport.
Optical Computing for Fast Light Transport AnalysisMatthew O'Toole
Optical Computing for Fast Light Transport Analysis
Matthew O'Toole and Kiriakos N. Kutulakos. ACM SIGGRAPH Asia, 2010.
We present a general framework for analyzing the transport matrix of a real-world scene at full resolution, without capturing many photos. The key idea is to use projectors and cameras to directly acquire eigenvectors and the Krylov subspace of the unknown transport matrix. To do this, we implement Krylov subspace methods partially in optics, by treating the scene as a black box subroutine that enables optical computation of arbitrary matrix-vector products. We describe two methods—optical Arnoldi to acquire a low-rank approximation of the transport matrix for relighting; and optical GMRES to invert light transport. Our experiments suggest that good-quality relighting and transport inversion are possible from a few dozen low-dynamic range photos, even for scenes with complex shadows, caustics, and other challenging lighting effects.
SIGGRAPH 2014 Course on Computational Cameras and Displays (part 3)Matthew O'Toole
Recent advances in both computational photography and displays have given rise to a new generation of computational devices. Computational cameras and displays provide a visual experience that goes beyond the capabilities of traditional systems by adding computational power to optics, lights, and sensors. These devices are breaking new ground in the consumer market, including lightfield cameras that redefine our understanding of pictures (Lytro), displays for visualizing 3D/4D content without special eyewear (Nintendo 3DS), motion-sensing devices that use light coded in space or time to detect motion and position (Kinect, Leap Motion), and a movement toward ubiquitous computing with wearable cameras and displays (Google Glass).
This short (1.5 hour) course serves as an introduction to the key ideas and an overview of the latest work in computational cameras, displays, and light transport.
3D Shape and Indirect Appearance by Structured Light TransportMatthew O'Toole
3D Shape and Indirect Appearance by Structured Light Transport
Matthew O'Toole, John Mather, and Kiriakos N. Kutulakos. CVPR, 2014.
Abstract:
We consider the problem of deliberately manipulating the direct and indirect light flowing through a time-varying, fully-general scene in order to simplify its visual analysis. Our approach rests on a crucial link between stereo geometry and light transport: while direct light always obeys the epipolar geometry of a projector-camera pair, indirect light overwhelmingly does not. We show that it is possible to turn this observation into an imaging method that analyzes light transport in real time in the optical domain, prior to acquisition. This yields three key abilities that we demonstrate in an experimental camera prototype: (1) producing a live indirect-only video stream for any scene, regardless of geometric or photometric complexity; (2) capturing images that make existing structured-light shape recovery algorithms robust to indirect transport; and (3) turning them into one-shot methods for dynamic 3D shape capture.
Primal-Dual Coding to Probe Light Transport
Matthew O'Toole, Ramesh Raskar, and Kiriakos N. Kutulakos. ACM SIGGRAPH, 2012.
Abstract:
We present primal-dual coding, a photography technique that enables direct fine-grain control over which light paths contribute to a photo. We achieve this by projecting a sequence of patterns onto the scene while the sensor is exposed to light. At the same time, a second sequence of patterns, derived from the first and applied in lockstep, modulates the light received at individual sensor pixels. We show that photography in this regime is equivalent to a matrix probing operation in which the elements of the scene's transport matrix are individually re-scaled and then mapped to the photo. This makes it possible to directly acquire photos in which specific light transport paths have been blocked, attenuated or enhanced. We show captured photos for several scenes with challenging light transport effects, including specular inter-reflections, caustics, diffuse inter-reflections and volumetric scattering. A key feature of primal-dual coding is that it operates almost exclusively in the optical domain: our results consist of directly-acquired, unprocessed RAW photos or differences between them.
SIGGRAPH 2014 Course on Computational Cameras and Displays (part 2)Matthew O'Toole
Recent advances in both computational photography and displays have given rise to a new generation of computational devices. Computational cameras and displays provide a visual experience that goes beyond the capabilities of traditional systems by adding computational power to optics, lights, and sensors. These devices are breaking new ground in the consumer market, including lightfield cameras that redefine our understanding of pictures (Lytro), displays for visualizing 3D/4D content without special eyewear (Nintendo 3DS), motion-sensing devices that use light coded in space or time to detect motion and position (Kinect, Leap Motion), and a movement toward ubiquitous computing with wearable cameras and displays (Google Glass).
This short (1.5 hour) course serves as an introduction to the key ideas and an overview of the latest work in computational cameras, displays, and light transport.
SIGGRAPH 2014 Course on Computational Cameras and Displays (part 4)Matthew O'Toole
Recent advances in both computational photography and displays have given rise to a new generation of computational devices. Computational cameras and displays provide a visual experience that goes beyond the capabilities of traditional systems by adding computational power to optics, lights, and sensors. These devices are breaking new ground in the consumer market, including lightfield cameras that redefine our understanding of pictures (Lytro), displays for visualizing 3D/4D content without special eyewear (Nintendo 3DS), motion-sensing devices that use light coded in space or time to detect motion and position (Kinect, Leap Motion), and a movement toward ubiquitous computing with wearable cameras and displays (Google Glass).
This short (1.5 hour) course serves as an introduction to the key ideas and an overview of the latest work in computational cameras, displays, and light transport.
Optical Computing for Fast Light Transport AnalysisMatthew O'Toole
Optical Computing for Fast Light Transport Analysis
Matthew O'Toole and Kiriakos N. Kutulakos. ACM SIGGRAPH Asia, 2010.
We present a general framework for analyzing the transport matrix of a real-world scene at full resolution, without capturing many photos. The key idea is to use projectors and cameras to directly acquire eigenvectors and the Krylov subspace of the unknown transport matrix. To do this, we implement Krylov subspace methods partially in optics, by treating the scene as a black box subroutine that enables optical computation of arbitrary matrix-vector products. We describe two methods—optical Arnoldi to acquire a low-rank approximation of the transport matrix for relighting; and optical GMRES to invert light transport. Our experiments suggest that good-quality relighting and transport inversion are possible from a few dozen low-dynamic range photos, even for scenes with complex shadows, caustics, and other challenging lighting effects.
SIGGRAPH 2014 Course on Computational Cameras and Displays (part 3)Matthew O'Toole
Recent advances in both computational photography and displays have given rise to a new generation of computational devices. Computational cameras and displays provide a visual experience that goes beyond the capabilities of traditional systems by adding computational power to optics, lights, and sensors. These devices are breaking new ground in the consumer market, including lightfield cameras that redefine our understanding of pictures (Lytro), displays for visualizing 3D/4D content without special eyewear (Nintendo 3DS), motion-sensing devices that use light coded in space or time to detect motion and position (Kinect, Leap Motion), and a movement toward ubiquitous computing with wearable cameras and displays (Google Glass).
This short (1.5 hour) course serves as an introduction to the key ideas and an overview of the latest work in computational cameras, displays, and light transport.
3D Shape and Indirect Appearance by Structured Light TransportMatthew O'Toole
3D Shape and Indirect Appearance by Structured Light Transport
Matthew O'Toole, John Mather, and Kiriakos N. Kutulakos. CVPR, 2014.
Abstract:
We consider the problem of deliberately manipulating the direct and indirect light flowing through a time-varying, fully-general scene in order to simplify its visual analysis. Our approach rests on a crucial link between stereo geometry and light transport: while direct light always obeys the epipolar geometry of a projector-camera pair, indirect light overwhelmingly does not. We show that it is possible to turn this observation into an imaging method that analyzes light transport in real time in the optical domain, prior to acquisition. This yields three key abilities that we demonstrate in an experimental camera prototype: (1) producing a live indirect-only video stream for any scene, regardless of geometric or photometric complexity; (2) capturing images that make existing structured-light shape recovery algorithms robust to indirect transport; and (3) turning them into one-shot methods for dynamic 3D shape capture.
Primal-Dual Coding to Probe Light Transport
Matthew O'Toole, Ramesh Raskar, and Kiriakos N. Kutulakos. ACM SIGGRAPH, 2012.
Abstract:
We present primal-dual coding, a photography technique that enables direct fine-grain control over which light paths contribute to a photo. We achieve this by projecting a sequence of patterns onto the scene while the sensor is exposed to light. At the same time, a second sequence of patterns, derived from the first and applied in lockstep, modulates the light received at individual sensor pixels. We show that photography in this regime is equivalent to a matrix probing operation in which the elements of the scene's transport matrix are individually re-scaled and then mapped to the photo. This makes it possible to directly acquire photos in which specific light transport paths have been blocked, attenuated or enhanced. We show captured photos for several scenes with challenging light transport effects, including specular inter-reflections, caustics, diffuse inter-reflections and volumetric scattering. A key feature of primal-dual coding is that it operates almost exclusively in the optical domain: our results consist of directly-acquired, unprocessed RAW photos or differences between them.
SIGGRAPH 2014 Course on Computational Cameras and Displays (part 2)Matthew O'Toole
Recent advances in both computational photography and displays have given rise to a new generation of computational devices. Computational cameras and displays provide a visual experience that goes beyond the capabilities of traditional systems by adding computational power to optics, lights, and sensors. These devices are breaking new ground in the consumer market, including lightfield cameras that redefine our understanding of pictures (Lytro), displays for visualizing 3D/4D content without special eyewear (Nintendo 3DS), motion-sensing devices that use light coded in space or time to detect motion and position (Kinect, Leap Motion), and a movement toward ubiquitous computing with wearable cameras and displays (Google Glass).
This short (1.5 hour) course serves as an introduction to the key ideas and an overview of the latest work in computational cameras, displays, and light transport.
Imaging objects obscured by occluders is a significant challenge for many applications. A camera that could “see around corners” could help improve navigation and mapping capabilities of autonomous vehicles or make search and rescue missions more effective. Time-resolved single-photon imaging systems have recently been demonstrated to record optical information of a scene that can lead to an estimation of the shape and reflectance of objects hidden from the line of sight of a camera. However, existing non-line-of-sight (NLOS) reconstruction algorithms have been constrained in the types of light transport effects they model for the hidden scene parts. We introduce a factored NLOS light transport representation that accounts for partial occlusions and surface normals. Based on this model, we develop a factorization approach for inverse time-resolved light transport and demonstrate high-fidelity NLOS reconstructions for challenging scenes both in simulation and with an experimental NLOS imaging system.
Magnetic tracking is one of miscellaneous motion capture methods, and maybe the oldest. However, its working principle is rarely introduced in detail perhaps due to its early adaptation resides in military and medical industry. Due to my interest in VR & animation MoCap, I’ve spent some time digging into the very depth of it and would like to share some non-confidential knowledge of it with you.
In this slide, a short history of magnetic tracking will be visited, followed by its working principle and algorithm simulation. Hope you enjoy it.
If you wanna discuss something in depth with me, please don’t hesitate to contact me via: dibao.wang@gmail.com
This slide was presented in a talk invited by mCube Inc. last month in Taipei, Taiwan. Here to share with those who may be interested. And also welcome to invite me for sharing related topics.
SIGGRAPH 2014 Course on Computational Cameras and Displays (part 1)Matthew O'Toole
Recent advances in both computational photography and displays have given rise to a new generation of computational devices. Computational cameras and displays provide a visual experience that goes beyond the capabilities of traditional systems by adding computational power to optics, lights, and sensors. These devices are breaking new ground in the consumer market, including lightfield cameras that redefine our understanding of pictures (Lytro), displays for visualizing 3D/4D content without special eyewear (Nintendo 3DS), motion-sensing devices that use light coded in space or time to detect motion and position (Kinect, Leap Motion), and a movement toward ubiquitous computing with wearable cameras and displays (Google Glass).
This short (1.5 hour) course serves as an introduction to the key ideas and an overview of the latest work in computational cameras, displays, and light transport.
This talk provides additional details around the hybrid real-time rendering pipeline we developed at SEED for Project PICA PICA.
At Digital Dragons 2018, we presented how leveraging Microsoft's DirectX Raytracing enables intuitive implementations of advanced lighting effects, including soft shadows, reflections, refractions, and global illumination. We also dove into the unique challenges posed by each of those domains, discussed the tradeoffs, and evaluated where raytracing fits in the spectrum of solutions.
Wave-Based Non-Line-of-Sight Imaging Using Fast f–k Migration | SIGGRAPH 2019David Lindell
We introduce a wave-based image formation model for the problem of non-line-of-sight (NLOS) imaging. Inspired by inverse methods used in seismology, we adapt a frequency-domain method, f-k migration, for solving the inverse NLOS problem.
This is my previous work (a decade ago) regarding modeling, simulation and design of single-axis CMOS MEMS Gyroscope. Hope it helps those who are still working in this field.
PR-278: RAFT: Recurrent All-Pairs Field Transforms for Optical FlowHyeongmin Lee
이번 논문은 ECCV2020에서 Best Paper를 받은 논문으로, 기존 방법들과는 다르게 반복적인 Update를 통해 Optical Flow를 예측하여 꽤나 높은 성능을 기록한 논문입니다.
paper link: https://arxiv.org/pdf/2003.12039.pdf
video link: https://youtu.be/OnZIDatotZ4
In this talk, we present results from the real-time raytracing research done at SEED, a cross-disciplinary team working on cutting-edge, future graphics technologies and creative experiences at Electronic Arts. We explain in detail several techniques from “PICA PICA”, a real-time raytracing experiment featuring a mini-game for self-learning AI agents in a procedurally-assembled world. The approaches presented here are intended to inspire developers and provide a glimpse of a future where real-time raytracing powers the creative experiences of tomorrow.
이번에 다룰 논문은 "Learning by Analogy: Reliable Supervision From Transformations for Unsupervised Optical Flow Estimation"이라는 논문입니다. 얼마 전에 발표드렸던 FlowNet 논문처럼 이 논문도 Deep Learning을 통해 Optical Flow를 학습하는 방법입니다. 다른 점이 하나 있다면, Unsupervised 방식으로 학습이 진행된다는 점입니다. Supervised 방식 만큼이나 Unsupervised 방식으로 Optical Flow를 학습하는 연구 역시 이미 많이 진행이 되어 왔는데요, 오늘 소개 드릴 논문에서는 Data Augmentation을 통한 Consistency를 활용하여 성능을 높이는 방식을 채용한 경우를 소개드리고자 합니다.
영상 링크: 이번에 다룰 논문은 "Learning by Analogy: Reliable Supervision From Transformations for Unsupervised Optical Flow Estimation"이라는 논문입니다. 얼마 전에 발표드렸던 FlowNet 논문처럼 이 논문도 Deep Learning을 통해 Optical Flow를 학습하는 방법입니다. 다른 점이 하나 있다면, Unsupervised 방식으로 학습이 진행된다는 점입니다. Supervised 방식 만큼이나 Unsupervised 방식으로 Optical Flow를 학습하는 연구 역시 이미 많이 진행이 되어 왔는데요, 오늘 소개 드릴 논문에서는 Data Augmentation을 통한 Consistency를 활용하여 성능을 높이는 방식을 채용한 경우를 소개드리고자 합니다.
Visual odometry & slam utilizing indoor structured environmentsNAVER Engineering
Visual odometry (VO) and simultaneous localization and mapping (SLAM) are fundamental building blocks for various applications from autonomous vehicles to virtual and augmented reality (VR/AR).
To improve the accuracy and robustness of the VO & SLAM approaches, we exploit multiple lines and orthogonal planar features, such as walls, floors, and ceilings, common in man-made indoor environments.
We demonstrate the effectiveness of the proposed VO & SLAM algorithms through an extensive evaluation on a variety of RGB-D datasets and compare with other state-of-the-art methods.
Imaging objects obscured by occluders is a significant challenge for many applications. A camera that could “see around corners” could help improve navigation and mapping capabilities of autonomous vehicles or make search and rescue missions more effective. Time-resolved single-photon imaging systems have recently been demonstrated to record optical information of a scene that can lead to an estimation of the shape and reflectance of objects hidden from the line of sight of a camera. However, existing non-line-of-sight (NLOS) reconstruction algorithms have been constrained in the types of light transport effects they model for the hidden scene parts. We introduce a factored NLOS light transport representation that accounts for partial occlusions and surface normals. Based on this model, we develop a factorization approach for inverse time-resolved light transport and demonstrate high-fidelity NLOS reconstructions for challenging scenes both in simulation and with an experimental NLOS imaging system.
Magnetic tracking is one of miscellaneous motion capture methods, and maybe the oldest. However, its working principle is rarely introduced in detail perhaps due to its early adaptation resides in military and medical industry. Due to my interest in VR & animation MoCap, I’ve spent some time digging into the very depth of it and would like to share some non-confidential knowledge of it with you.
In this slide, a short history of magnetic tracking will be visited, followed by its working principle and algorithm simulation. Hope you enjoy it.
If you wanna discuss something in depth with me, please don’t hesitate to contact me via: dibao.wang@gmail.com
This slide was presented in a talk invited by mCube Inc. last month in Taipei, Taiwan. Here to share with those who may be interested. And also welcome to invite me for sharing related topics.
SIGGRAPH 2014 Course on Computational Cameras and Displays (part 1)Matthew O'Toole
Recent advances in both computational photography and displays have given rise to a new generation of computational devices. Computational cameras and displays provide a visual experience that goes beyond the capabilities of traditional systems by adding computational power to optics, lights, and sensors. These devices are breaking new ground in the consumer market, including lightfield cameras that redefine our understanding of pictures (Lytro), displays for visualizing 3D/4D content without special eyewear (Nintendo 3DS), motion-sensing devices that use light coded in space or time to detect motion and position (Kinect, Leap Motion), and a movement toward ubiquitous computing with wearable cameras and displays (Google Glass).
This short (1.5 hour) course serves as an introduction to the key ideas and an overview of the latest work in computational cameras, displays, and light transport.
This talk provides additional details around the hybrid real-time rendering pipeline we developed at SEED for Project PICA PICA.
At Digital Dragons 2018, we presented how leveraging Microsoft's DirectX Raytracing enables intuitive implementations of advanced lighting effects, including soft shadows, reflections, refractions, and global illumination. We also dove into the unique challenges posed by each of those domains, discussed the tradeoffs, and evaluated where raytracing fits in the spectrum of solutions.
Wave-Based Non-Line-of-Sight Imaging Using Fast f–k Migration | SIGGRAPH 2019David Lindell
We introduce a wave-based image formation model for the problem of non-line-of-sight (NLOS) imaging. Inspired by inverse methods used in seismology, we adapt a frequency-domain method, f-k migration, for solving the inverse NLOS problem.
This is my previous work (a decade ago) regarding modeling, simulation and design of single-axis CMOS MEMS Gyroscope. Hope it helps those who are still working in this field.
PR-278: RAFT: Recurrent All-Pairs Field Transforms for Optical FlowHyeongmin Lee
이번 논문은 ECCV2020에서 Best Paper를 받은 논문으로, 기존 방법들과는 다르게 반복적인 Update를 통해 Optical Flow를 예측하여 꽤나 높은 성능을 기록한 논문입니다.
paper link: https://arxiv.org/pdf/2003.12039.pdf
video link: https://youtu.be/OnZIDatotZ4
In this talk, we present results from the real-time raytracing research done at SEED, a cross-disciplinary team working on cutting-edge, future graphics technologies and creative experiences at Electronic Arts. We explain in detail several techniques from “PICA PICA”, a real-time raytracing experiment featuring a mini-game for self-learning AI agents in a procedurally-assembled world. The approaches presented here are intended to inspire developers and provide a glimpse of a future where real-time raytracing powers the creative experiences of tomorrow.
이번에 다룰 논문은 "Learning by Analogy: Reliable Supervision From Transformations for Unsupervised Optical Flow Estimation"이라는 논문입니다. 얼마 전에 발표드렸던 FlowNet 논문처럼 이 논문도 Deep Learning을 통해 Optical Flow를 학습하는 방법입니다. 다른 점이 하나 있다면, Unsupervised 방식으로 학습이 진행된다는 점입니다. Supervised 방식 만큼이나 Unsupervised 방식으로 Optical Flow를 학습하는 연구 역시 이미 많이 진행이 되어 왔는데요, 오늘 소개 드릴 논문에서는 Data Augmentation을 통한 Consistency를 활용하여 성능을 높이는 방식을 채용한 경우를 소개드리고자 합니다.
영상 링크: 이번에 다룰 논문은 "Learning by Analogy: Reliable Supervision From Transformations for Unsupervised Optical Flow Estimation"이라는 논문입니다. 얼마 전에 발표드렸던 FlowNet 논문처럼 이 논문도 Deep Learning을 통해 Optical Flow를 학습하는 방법입니다. 다른 점이 하나 있다면, Unsupervised 방식으로 학습이 진행된다는 점입니다. Supervised 방식 만큼이나 Unsupervised 방식으로 Optical Flow를 학습하는 연구 역시 이미 많이 진행이 되어 왔는데요, 오늘 소개 드릴 논문에서는 Data Augmentation을 통한 Consistency를 활용하여 성능을 높이는 방식을 채용한 경우를 소개드리고자 합니다.
Visual odometry & slam utilizing indoor structured environmentsNAVER Engineering
Visual odometry (VO) and simultaneous localization and mapping (SLAM) are fundamental building blocks for various applications from autonomous vehicles to virtual and augmented reality (VR/AR).
To improve the accuracy and robustness of the VO & SLAM approaches, we exploit multiple lines and orthogonal planar features, such as walls, floors, and ceilings, common in man-made indoor environments.
We demonstrate the effectiveness of the proposed VO & SLAM algorithms through an extensive evaluation on a variety of RGB-D datasets and compare with other state-of-the-art methods.
Transformer Architectures in Vision
[2018 ICML] Image Transformer
[2019 CVPR] Video Action Transformer Network
[2020 ECCV] End-to-End Object Detection with Transformers
[2021 ICLR] An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
Determination of System Geometrical Parameters and Consistency between Scans ...David Scaduto
Digital breast tomosynthesis (DBT) requires precise knowledge of acquisition geometry for accurate image reconstruction. Further, image subtraction techniques employed in dual-energy contrast-enhanced tomosynthesis require that scans be performed under nearly identical geometrical conditions. A geometrical calibration algorithm is developed to investigate system geometry and geometrical consistency of image acquisition between consecutive digital breast tomosynthesis scans, according to requirements for dual-energy contrast-enhanced tomosynthesis. Investigation of geometrical accuracy and consistency on a prototype DBT unit reveals accurate angular measurement, but potentially clinically significant differences in acquisition angles between scans. Further, a slight gantry wobble is observed, suggesting the need for incorporation of gantry wobble into image reconstruction, or improvements to system hardware.
Efficient architecture to condensate visual information driven by attention ...Sara Granados Cabeza
This are the slides from my PhD dissertation. I developed a new representation map for visual information (such as disparity, optical flow, etc.) that I've called "semidense" representation. This novel representation reduces the memory and bandwidth needs for embedded platforms and real-time systems.
COSC 426 Lecture 5 on Mathematical Principles Behind AR Registration. Given by Adrian Clark from the HIT Lab NZ at the University of Canterbury, August 8, 2012
SkyStitch: a Cooperative Multi-UAV-based Real-time Video Surveillance System ...Kitsukawa Yuki
パターン・映像情報処理特論において論文を紹介した時の発表資料です。
Xiangyun Meng, Wei Wang, and Ben Leong. 2015. SkyStitch: A Cooperative Multi-UAV-based Real-time Video Surveillance System with Stitching. In Proceedings of the 23rd ACM international conference on Multimedia (MM '15). ACM, New York, NY, USA, 261-270. DOI=http://dx.doi.org/10.1145/2733373.2806225
Defect detection in circlips using image processing in ni lab viewSayali Bodhankar
A circlip (circular-clip) is a semi-flexible metal ring used as a fastener in advanced industrial machinery. Faults or defects in the circlip can cause the entire machinery to fall off. Hence, sorting of circlips as "good" or "faulty" is imperative.
Similar to Calibration Issues in FRC: Camera, Projector, Kinematics based Hybrid Approach (URAI 2012) (20)
A very private, introductory notes on reinforcement learning.
Focusing to understand DQN by Google DeepMind.
Mainly based on the nice article
"DEMYSTIFYING DEEP REINFORCEMENT LEARNING"by Tambet Matiisen on
http://neuro.cs.ut.ee/demystifying-deep-reinforcement-learning/
Spatial AR: Toward Augmentation of Ambient and Effective Interaction ChannelJoo-Haeng Lee
기존의 모바일 증강현실 기술은 실세계와 가상세계를 연결하는 새로운 상호작용 채널을 제공하고자 하지만 장치 의존성 면에서 상호작용을 제약하는 태생적인 한계를 보이고 있다. 프로젝터를 사용하여 정보, 인터페이스, 콘텐츠를 실세계 사물과 그 주변에 직접 투사하여 증강하고자 하는 공간증강현실은 이러한 기존의 모바일 증강현실이 갖는 문제점을 해결하고자 한다. 특히, 오감과 물리법칙에 기반한 실세계 상호작용을 가능한 그대로 유지하면서 새로운 인터액션 채널을 제공하여 실질적인 증강의 효과를 주고자 한다. 이러한 증강 상호작용은 실사물을 손으로 직접 만지며 작업하는 예술가와 노약자들에게도 도움을 줄 수 있어야 한다. 본 슬라이드에서는 위의 언급을 시각적으로 표현하고 있다.
New geometric interpretation and analytic solution for quadrilateral reconstr...Joo-Haeng Lee
Poster presentation for ICPR 2014 paper.
Title: New geometric interpretation and analytic solution for quadrilateral reconstruction
Author: Joo-Haeng Lee (ETRI)
New geometric interpretation and analytic solution for quadrilateral reconstr...Joo-Haeng Lee
Accepted as poster presentation for ICPR 2014, Stockholm, Sweden on August 24~28, 2014.
[Revised Version]
Title: New geometric interpretation and analytic solution for quadrilateral reconstruction
Author: Joo-Haeng Lee
Affiliation: Human-Robot Interaction Research Team, ETRI, KOREA
Abstract:
A new geometric framework, called generalized coupled line camera (GCLC), is proposed to derive an analytic solution to reconstruct an unknown scene quadrilateral and the relevant projective structure from a single or multiple image quadrilaterals. We extend the previous approach developed for rectangle to handle arbitrary scene quadrilaterals. First, we generalize a single line camera by removing the centering constraint that the principal axis should bisect a scene line. Then, we couple a pair of generalized line cameras to model a frustum with a quadrilateral base. Finally, we show that the scene quadrilateral and the center of projection can be analytically reconstructed from a single view when prior knowledge on the quadrilateral is available. A completely unknown quadrilateral can be reconstructed from four views through non-linear optimization. We also describe a improved method to handle an off-centered case by geometrically inferring a centered proxy quadrilateral, which accelerates a reconstruction process without relying on homography. The proposed method is easy to implement since each step is expressed as a simple analytic equation. We present the experimental results on real and synthetic examples.
[Submitted Version]
Title: Generalized Coupled Line Cameras and Application in Quadrilateral Reconstruction
Abstract:
Coupled line camera (CLC) provides a geometric framework to derive an analytic solution to reconstruct an unknown scene rectangle and the relevant projective structure from a single image quadrilateral. We extend this approach as generalized coupled line camera (GCLC) to handle a scene quadrilateral. First, we generalize a single line camera by removing the centering constraint that the principal axis should bisect a scene line. Then, we couple a pair of generalized line cameras to model a frustum with a quadrilateral base. Finally, we show that the scene quadrilateral and the center of projection can be analytically reconstructed from a single view when prior knowledge on the quadrilateral is available. ...
화홍문 사진 모음 및 편액에 대한 CLC 기반의 사각형 복원 (An Application of CLC on a framed picture ...Joo-Haeng Lee
수원 화성 화홍문 편액에 선분카메라쌍 (CLC) 기반의 사각형 복원 기법을 적용해 보았습니다. 사진들은 인터넷에서 수집했습니다.
Applied CLC-based rectangle reconstruction on a framed picture of Hwahongmun, Hwaseong Fortress, KOREA.
Inverse Perspective Projection of Convex QuadrilateralsJoo-Haeng Lee
Presenter: Joo-Haeng Lee
Affiliation: ETRI
Venue: ACDDE 2011 (Asian Conference on Design and Digital Engineering)
- VR and Multimedia Workshop
- Session VRM-2 Paper ID 116
Comment:
- The talk was given at ACDDE 2011.
- The full paper was published in ETRI Journal: Joo-Haeng Lee "an analytic solution to projector pose estimation problem, " 34(6), 2012.
- Paper link: http://etrij.etri.re.kr/Cyber/BrowseAbstract.jsp?vol=34&num=6&pg=978
- The dual problem for the camera was solved and presented in ICPR 2012: Joo-Haeng Lee, "Camera calibration from a single image based on coupled line cameras and rectangle constraint."
Camera Calibration from a Single Image based on Coupled Line Cameras and Rect...Joo-Haeng Lee
ICPR 2012 Paper Abstract
Title: Camera Calibration from a Single Image Based on Coupled Line Cameras and Rectangle Constraint
Author: Lee, Joo-Haeng (ETRI)
Scheduled for presentation during the Regular Session "Poster Shotgun (04): CV" (TuPSAT2), Tuesday, November 13, 2012, 08:30−09:00, Multi-Purpose Hall
21st International Conference on Pattern Recognition, November 11-15, 2012, Tsukuba International Congress Center, Tsukuba, Japan
This information is tentative and subject to change. Compiled on February 13, 2013
Note on Coupled Line Cameras for Rectangle Reconstruction (ACDDE 2012)Joo-Haeng Lee
The presentation file for the talk in ACDDE 2012.
http://www.acdde2012.org/
It deals with the research result published in ICPR 2012 with the title as "Camera Calibration from a Single Image based on Coupled Line Cameras and Rectangle Constraint"
https://iapr.papercept.net/conferences/scripts/abstract.pl?ConfID=7&Number=70
Ribs and Fans of Bezier Curves and Surfaces with ApplicationsJoo-Haeng Lee
Explains newly found geometric features of Bezier curves and surfaces called "rib and fan.
- Author: Joo-Haeng Lee
- Affiliation: ETRI
- Date: 2007-12-07
Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...UiPathCommunity
💥 Speed, accuracy, and scaling – discover the superpowers of GenAI in action with UiPath Document Understanding and Communications Mining™:
See how to accelerate model training and optimize model performance with active learning
Learn about the latest enhancements to out-of-the-box document processing – with little to no training required
Get an exclusive demo of the new family of UiPath LLMs – GenAI models specialized for processing different types of documents and messages
This is a hands-on session specifically designed for automation developers and AI enthusiasts seeking to enhance their knowledge in leveraging the latest intelligent document processing capabilities offered by UiPath.
Speakers:
👨🏫 Andras Palfi, Senior Product Manager, UiPath
👩🏫 Lenka Dulovicova, Product Program Manager, UiPath
Securing your Kubernetes cluster_ a step-by-step guide to success !KatiaHIMEUR1
Today, after several years of existence, an extremely active community and an ultra-dynamic ecosystem, Kubernetes has established itself as the de facto standard in container orchestration. Thanks to a wide range of managed services, it has never been so easy to set up a ready-to-use Kubernetes cluster.
However, this ease of use means that the subject of security in Kubernetes is often left for later, or even neglected. This exposes companies to significant risks.
In this talk, I'll show you step-by-step how to secure your Kubernetes cluster for greater peace of mind and reliability.
State of ICS and IoT Cyber Threat Landscape Report 2024 previewPrayukth K V
The IoT and OT threat landscape report has been prepared by the Threat Research Team at Sectrio using data from Sectrio, cyber threat intelligence farming facilities spread across over 85 cities around the world. In addition, Sectrio also runs AI-based advanced threat and payload engagement facilities that serve as sinks to attract and engage sophisticated threat actors, and newer malware including new variants and latent threats that are at an earlier stage of development.
The latest edition of the OT/ICS and IoT security Threat Landscape Report 2024 also covers:
State of global ICS asset and network exposure
Sectoral targets and attacks as well as the cost of ransom
Global APT activity, AI usage, actor and tactic profiles, and implications
Rise in volumes of AI-powered cyberattacks
Major cyber events in 2024
Malware and malicious payload trends
Cyberattack types and targets
Vulnerability exploit attempts on CVEs
Attacks on counties – USA
Expansion of bot farms – how, where, and why
In-depth analysis of the cyber threat landscape across North America, South America, Europe, APAC, and the Middle East
Why are attacks on smart factories rising?
Cyber risk predictions
Axis of attacks – Europe
Systemic attacks in the Middle East
Download the full report from here:
https://sectrio.com/resources/ot-threat-landscape-reports/sectrio-releases-ot-ics-and-iot-security-threat-landscape-report-2024/
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...DanBrown980551
Do you want to learn how to model and simulate an electrical network from scratch in under an hour?
Then welcome to this PowSyBl workshop, hosted by Rte, the French Transmission System Operator (TSO)!
During the webinar, you will discover the PowSyBl ecosystem as well as handle and study an electrical network through an interactive Python notebook.
PowSyBl is an open source project hosted by LF Energy, which offers a comprehensive set of features for electrical grid modelling and simulation. Among other advanced features, PowSyBl provides:
- A fully editable and extendable library for grid component modelling;
- Visualization tools to display your network;
- Grid simulation tools, such as power flows, security analyses (with or without remedial actions) and sensitivity analyses;
The framework is mostly written in Java, with a Python binding so that Python developers can access PowSyBl functionalities as well.
What you will learn during the webinar:
- For beginners: discover PowSyBl's functionalities through a quick general presentation and the notebook, without needing any expert coding skills;
- For advanced developers: master the skills to efficiently apply PowSyBl functionalities to your real-world scenarios.
Smart TV Buyer Insights Survey 2024 by 91mobiles.pdf91mobiles
91mobiles recently conducted a Smart TV Buyer Insights Survey in which we asked over 3,000 respondents about the TV they own, aspects they look at on a new TV, and their TV buying preferences.
Generating a custom Ruby SDK for your web service or Rails API using Smithyg2nightmarescribd
Have you ever wanted a Ruby client API to communicate with your web service? Smithy is a protocol-agnostic language for defining services and SDKs. Smithy Ruby is an implementation of Smithy that generates a Ruby SDK using a Smithy model. In this talk, we will explore Smithy and Smithy Ruby to learn how to generate custom feature-rich SDKs that can communicate with any web service, such as a Rails JSON API.
Essentials of Automations: Optimizing FME Workflows with ParametersSafe Software
Are you looking to streamline your workflows and boost your projects’ efficiency? Do you find yourself searching for ways to add flexibility and control over your FME workflows? If so, you’re in the right place.
Join us for an insightful dive into the world of FME parameters, a critical element in optimizing workflow efficiency. This webinar marks the beginning of our three-part “Essentials of Automation” series. This first webinar is designed to equip you with the knowledge and skills to utilize parameters effectively: enhancing the flexibility, maintainability, and user control of your FME projects.
Here’s what you’ll gain:
- Essentials of FME Parameters: Understand the pivotal role of parameters, including Reader/Writer, Transformer, User, and FME Flow categories. Discover how they are the key to unlocking automation and optimization within your workflows.
- Practical Applications in FME Form: Delve into key user parameter types including choice, connections, and file URLs. Allow users to control how a workflow runs, making your workflows more reusable. Learn to import values and deliver the best user experience for your workflows while enhancing accuracy.
- Optimization Strategies in FME Flow: Explore the creation and strategic deployment of parameters in FME Flow, including the use of deployment and geometry parameters, to maximize workflow efficiency.
- Pro Tips for Success: Gain insights on parameterizing connections and leveraging new features like Conditional Visibility for clarity and simplicity.
We’ll wrap up with a glimpse into future webinars, followed by a Q&A session to address your specific questions surrounding this topic.
Don’t miss this opportunity to elevate your FME expertise and drive your projects to new heights of efficiency.
JMeter webinar - integration with InfluxDB and GrafanaRTTS
Watch this recorded webinar about real-time monitoring of application performance. See how to integrate Apache JMeter, the open-source leader in performance testing, with InfluxDB, the open-source time-series database, and Grafana, the open-source analytics and visualization application.
In this webinar, we will review the benefits of leveraging InfluxDB and Grafana when executing load tests and demonstrate how these tools are used to visualize performance metrics.
Length: 30 minutes
Session Overview
-------------------------------------------
During this webinar, we will cover the following topics while demonstrating the integrations of JMeter, InfluxDB and Grafana:
- What out-of-the-box solutions are available for real-time monitoring JMeter tests?
- What are the benefits of integrating InfluxDB and Grafana into the load testing stack?
- Which features are provided by Grafana?
- Demonstration of InfluxDB and Grafana using a practice web application
To view the webinar recording, go to:
https://www.rttsweb.com/jmeter-integration-webinar
Epistemic Interaction - tuning interfaces to provide information for AI supportAlan Dix
Paper presented at SYNERGY workshop at AVI 2024, Genoa, Italy. 3rd June 2024
https://alandix.com/academic/papers/synergy2024-epistemic/
As machine learning integrates deeper into human-computer interactions, the concept of epistemic interaction emerges, aiming to refine these interactions to enhance system adaptability. This approach encourages minor, intentional adjustments in user behaviour to enrich the data available for system learning. This paper introduces epistemic interaction within the context of human-system communication, illustrating how deliberate interaction design can improve system understanding and adaptation. Through concrete examples, we demonstrate the potential of epistemic interaction to significantly advance human-computer interaction by leveraging intuitive human communication strategies to inform system design and functionality, offering a novel pathway for enriching user-system engagements.
GraphRAG is All You need? LLM & Knowledge GraphGuy Korland
Guy Korland, CEO and Co-founder of FalkorDB, will review two articles on the integration of language models with knowledge graphs.
1. Unifying Large Language Models and Knowledge Graphs: A Roadmap.
https://arxiv.org/abs/2306.08302
2. Microsoft Research's GraphRAG paper and a review paper on various uses of knowledge graphs:
https://www.microsoft.com/en-us/research/blog/graphrag-unlocking-llm-discovery-on-narrative-private-data/
Builder.ai Founder Sachin Dev Duggal's Strategic Approach to Create an Innova...Ramesh Iyer
In today's fast-changing business world, Companies that adapt and embrace new ideas often need help to keep up with the competition. However, fostering a culture of innovation takes much work. It takes vision, leadership and willingness to take risks in the right proportion. Sachin Dev Duggal, co-founder of Builder.ai, has perfected the art of this balance, creating a company culture where creativity and growth are nurtured at each stage.
Builder.ai Founder Sachin Dev Duggal's Strategic Approach to Create an Innova...
Calibration Issues in FRC: Camera, Projector, Kinematics based Hybrid Approach (URAI 2012)
1. URAI 2012, Session TD4: Robot & Future Info Device
Calibration Issues in FRC:
Camera, Projector, Kinematics based
Hybrid Approach
Joo-Haeng Lee ETRI
Kosuke Maegawa Ritsumeikan University
Jong-Seung Park Ritsumeikan University
Joo-Ho Lee Ritsumeikan University
2. Agenda
• Introduction: FRC
• Example Application: Robotic Spatial AR (RSAR)
• Calibration Issues: Camera, Projector, Kinematics
• Summary
• Q&A
2 Joo-Haeng Lee (joohaeng at etri.re.kr)
3. ETRI FRC 2010
3 Joo-Haeng Lee (joohaeng at etri.re.kr)
4. ETRI FRC 2011
4 Joo-Haeng Lee (joohaeng at etri.re.kr)
5. ETRI FRC 2012
5 Joo-Haeng Lee (joohaeng at etri.re.kr)
6. ETRI FRC 2012
• Major components for RSAR
• RSAR = Robotic Spatial Augmented Reality
7. ETRI FRC 2012
• Major components for RSAR
• RSAR = Robotic Spatial Augmented Reality
Robotis Logitech
Optoma
Dynamixel HD Pro Webcam
PK-320
MX-28 C920
15. Motivation
• Calibration really matters in RSAR!
• camera to capture the geometry of the world
• projector to display on the real-world surface
• kinematics to control and sense the motion
17. Calibration: Camera
• Camera Model
• Qc = Mc Xwc G
• Qc: image in the camera
• Mc: camera internal
• Xwc: camera external
• G: geometry in the world
19. Calibration: Camera
• Chang’s method in OpenCV
• internal and external parameters + lens distortion
• Issues
• geometric constraints should be considered for
precise calibration of other components such as
kinematics
21. Calibration: Projector
• Projection Model
• Qp = Mp Xwp Gp
• Gp = Xwp-1 Mp-1 Qp
• Qp: image to be projected
• Mp: projector internal
• Xwp: projector external
• Gp: projected area in the world
22. Calibration: Projector
• Chang’s method in OpenCV
• If a projector is not moving or well aligned, we can
apply Chang’s method as in the camera case
23. Calibration: Projector
• Chang’s method in OpenCV
• If a projector is not moving or well aligned, we can
apply Chang’s method as in the camera case
• Issues
• However, for a moving projector, we need to handle
lens shift, which cannot be solved using Chang’s.
24. Calibration: Projector
• Tsai’s method with custom implementation
• Can handle lens shift: no need to specify the image
size
25. Calibration: Projector
• Tsai’s method with custom implementation
• Can handle lens shift: no need to specify the image
size
• Constrained concave programming based on
Lagrangian multiplier method: Qp = P G
26. Calibration: Projector
• Tsai’s method with custom implementation
• Can handle lens shift: no need to specify the image
size
• Constrained concave programming based on
Lagrangian multiplier method: Qp = P G
• RQ decomposition: P = Mp Xwp
27. Calibration: Projector
• Tsai’s method with custom implementation
Fig. An optical center of a projector (in green) that is approximated from camera frustums
from data set #2. Each frustum is aligned in the common coordinate frame of a camera.
The optical center of a camera (in white) is the origin of the frame. Optical centers of a
projector computed using the previous method (assuming no-lens shift) are in gray.
A pair of red and blue points is the
closest points between two frustum
Extended rectangles (in orange) from the partial rectangles (in blue).(in gray) of a
Fig. edges. The average is the Six optical centers
approximate optical center of a projector assuming the no lens-shift.
projector (in green). Computed using the fixed internal
30. Calibration: Projector-Camera
• Projector-camera system
• Calibrated camera: Mc and Xwc
• Calibrated projector: Mp and Xwp
• Transformation between projector and camera
• Xcp = Xwp Xcw = Xwp Xwc -1
31. Calibration: Projector-Camera
• Projector-camera system
• Calibrated camera: Mc and Xwc
• Calibrated projector: Mp and Xwp
• Transformation between projector and camera
• Xcp = Xwp Xcw = Xwp Xwc -1
• Transformation from the world to the projector
32. Calibration: Projector-Camera
• Projector-camera system
• Calibrated camera: Mc and Xwc
• Calibrated projector: Mp and Xwp
• Transformation between projector and camera
• Xcp = Xwp Xcw = Xwp Xwc -1
• Transformation from the world to the projector
• Xwp(t) = Xcp Xwc(t)
33. Calibration: Kinematics
• Precise calibration of kinematics is required for
the quality of RSAR application in FRC
• (ex) inverse kinematics