Theory of Augmented Reality

1. Augmented
Reality
Gia Muhammad
State Islamic University Syarif Hidayatullah

2. Apa itu Augmented
Reality?
• Virtual Reality
o Games
o Simulasi
o Dll
• AR
o Augmented – Penambahan
o Reality – Realitas (Lingkungan Nyata)
o AR – Realitas yang ditambahkan
• Terminologi AR (Azuma, 1997)
o Kombinasi Realitas dan Virtual
o Interaktif dalam Waktu Nyata
o Berjalan dengan model 3D
GiaMuhammad | Praktikum Advance Technology Multimedia
Department of Computer Science, Faculty of Science and Technology - State Islamic University (UIN) Syarif Hidayatullah

3. Pemakaian AR di dunia
industri
• Advertising/Branding/Digital Marketing
• Simulasi
o Medical
o Militer
• Edukasi
GiaMuhammad | Praktikum Advance Technology Multimedia
Department of Computer Science, Faculty of Science and Technology - State Islamic University (UIN) Syarif Hidayatullah

4. Perangkat penunjang AR
GiaMuhammad | Praktikum Advance Technology Multimedia
Department of Computer Science, Faculty of Science and Technology - State Islamic University (UIN) Syarif Hidayatullah

5. Media untuk melihat AR
• Monitor atau Smart Phone (Handheld Type)
• Goggles (Head Mounted Device)
• Contact Lens
GiaMuhammad | Praktikum Advance Technology Multimedia
Department of Computer Science, Faculty of Science and Technology - State Islamic University (UIN) Syarif Hidayatullah

6. Klasifikasi Tracking di AR
• Marker-Based Tracking
o Marker Tradisional/Konvensional (Citra Hitam Putih dengan Border, ARToolKit)
o 2D Barcode
• ARToolKitPlus
• ARTag & WebTag
• QR Code
o Hybrid Marker-Based – Konvensional & 2D Barcode
• QR Code Marker
• Markerless Tracking/Feature-Based Tracking
o Gambar Berwarna
o GPS
o Biometrik
• Wajah
• Tubuh
• Hand Gesture
• Dll
• Hybrid Tracking – Marker-based dan Feature-Based
o Barcode-Assisted (Park, et al, 2011)
o Sensor
o Model Based
GiaMuhammad | Praktikum Advance Technology Multimedia
Department of Computer Science, Faculty of Science and Technology - State Islamic University (UIN) Syarif Hidayatullah

7. Marker-Based Tracking
• Marker Tradisional (Citra Hitam Putih dengan Border,
ARToolKit) [1]
• 2D Barcode
o ARToolKitPlus [2]
o ARTag & WebTag [3]
o QR Code [4]
• Hybrid Marker-Based
o QR Code Marker [5]
1 2 3 4 5
GiaMuhammad | Praktikum Advance Technology Multimedia
Department of Computer Science, Faculty of Science and Technology - State Islamic University (UIN) Syarif Hidayatullah

8. GiaMuhammad | Praktikum Advance Technology Multimedia
Department of Computer Science, Faculty of Science and Technology - State Islamic University (UIN) Syarif Hidayatullah
Markerless/Feature-Based
Tracking
• Gambar Berwarna
• GPS
• Biometrik
o Wajah
o Tubuh
o Hand Gesture
o Dll

9. Bidang Ilmu yang Terkait
• Pengolahan Citra Digital
• Computer Vision
o Model Kamera & Kalibrasi Kamera
o Perspective Transformation
o Segmentasi Citra
o Template Matching
o SURF/SIFT
o Optical Flow
o Stereo Vision
• Grafika Komputer
GiaMuhammad | Praktikum Advance Technology Multimedia
Department of Computer Science, Faculty of Science and Technology - State Islamic University (UIN) Syarif Hidayatullah

10. AR Framework/Tools
• ARToolKit
• ARToolKitPlus
• ARTag
• NyARToolKit
• FlarToolKit
• PaperVision 3D
• StudierStube
• Vuforia
• DFusion
• metaIO
• bazAR
• Dan Masih banyak lagi
GiaMuhammad | Praktikum Advance Technology Multimedia
Department of Computer Science, Faculty of Science and Technology - State Islamic University (UIN) Syarif Hidayatullah

11. Framework untuk
Penelitian AR
• Sering dipakai oleh peneliti-peneliti di dunia
o OpenCV, EmguCV, SimpleCV
• Image Processing
• Computer Vision
• Machine Learning
o OpenNI
o OpenFramework
• Framework 3D Modelling
o OpenGL
o Ogre3D
o OpenSpace3D
o Unity3D
• Framework Tambahan jika diperlukan
o OpenCL
GiaMuhammad | Praktikum Advance Technology Multimedia
Department of Computer Science, Faculty of Science and Technology - State Islamic University (UIN) Syarif Hidayatullah

12. Sistem AR sederhana
GiaMuhammad | Praktikum Advance Technology Multimedia
Department of Computer Science, Faculty of Science and Technology - State Islamic University (UIN) Syarif Hidayatullah

13. Komputasi pada AR
Marker/Feature-Based*
1. Kalibrasi Kamera
2. Deteksi Marker
o Low Level Image Processing
3. Proyeksi Perspektif
4. Identifikasi
o Template Matching
o PCA (Principal Component Analysis)
o Decoding 2D Barcode dengan Algoritma Reed-Solomon
5. Pose Estimation
o RPP (Robust Planar Pose)
o ARToolKit Pose Estimation
6. Rendering 3D
GiaMuhammad | Praktikum Advance Technology Multimedia
Department of Computer Science, Faculty of Science and Technology - State Islamic University (UIN) Syarif Hidayatullah

14. Deteksi Marker
• Algoritma Deteksi Marker oleh Hirokazu Kato
(ARToolKit)
GiaMuhammad | Praktikum Advance Technology Multimedia
Department of Computer Science, Faculty of Science and Technology - State Islamic University (UIN) Syarif Hidayatullah

15. Deteksi Marker
• Low Level Image Processing
o Akuisisi
o Grayscaling
o Thresholding
o Connected Components
o Contouring
o Deteksi Sudut
• Deteksi Marker dengan OpenCV
o Grayscaling
o Thresholding
o Contouring
o Ekstraksi Sudut
GiaMuhammad | Praktikum Advance Technology Multimedia
Department of Computer Science, Faculty of Science and Technology - State Islamic University (UIN) Syarif Hidayatullah

16. Deteksi Marker
• Software Tools
o Microsoft Visual C++ 2008/2010
o OpenCV 2.1
o ARToolKit 2.7.1
• Hardware Tools
o Web Camera 1.3 mp
GiaMuhammad | Praktikum Advance Technology Multimedia
Department of Computer Science, Faculty of Science and Technology - State Islamic University (UIN) Syarif Hidayatullah

17. Reference
• Siltanen S. “Theory and applications of marker-based
augmented reality”, 2012
• Bradski S, Kaehler A. “Learning OpenCV”, 2008
• Agusta. G.M. “QR Code Tracking with Merging on
Conventional Marker based Backpropagation Neural
Network”
GiaMuhammad | Praktikum Advance Technology Multimedia
Department of Computer Science, Faculty of Science and Technology - State Islamic University (UIN) Syarif Hidayatullah