SlideShare a Scribd company logo
Unity3D Audio Grabber
design
Unity3D is tested on version 4.6
Jungsoo Nam
namjungsoo@gmail.com
2015/03/17
FMOD Ex
• Libunity.so uses FMOD Ex statically(with .c sources or .a
archive) linked.
• NOT uses libfmodex.so
• NOT will be applied by function hooking with Elf32_Ehdr like
eglSwapBuffers()’s method.
• C++ based library, but C wrapper classes.
• C functions can be hooked.
IDA analysis – libunity.so
• ONLY Imports tab’s functions can be hooked. FMOD’s are in
Exports tab. (Function pointers are NOT in imports entry
table.)
FMOD hooking
• FMOD C API hooking is essential.(libkamcordaudio.so does like
this)
• Getting/sharing FMOD system object is important for interoperating
AudioGrabber with Unity3D FMOD audio.
• So we have to hook FMOD_System_Create(FMOD_SYSTEM** system)
• How to hook FMOD?
• Find another entry table.(exports table)
• In VideoGrabber, we found imports table.(libunity.so imports libGLES*.so function)
• Next we have to find exports table.(libunity.so exports some functions to client)
• Or use “dlsym()” again.
• orig_FMOD_System_Create = dlsym(RTLD_DEFAULT,
"FMOD_System_Create");
• LOGD("FMOD_System_Create: 0x%x", orig_FMOD_System_Create);
Final AudioGrabber design
• AudioGrabber will be like below:
• Link libfmodex.so.
• Hook some FMOD functions and get FMOD system object.
• Use our FMOD functions in libfmodex.so, manipulate FMOD audio
and get audio buffer data.
• Put audio buffer data into MediaCodec.
• Mix and encode video and audio to .MP4
Thank you

More Related Content

Similar to Unity3D Audio Grabber design

Plug-ins & Third-Party SDKs in UE4
Plug-ins & Third-Party SDKs in UE4Plug-ins & Third-Party SDKs in UE4
Plug-ins & Third-Party SDKs in UE4
Gerke Max Preussner
 
Pharo foreign function interface (FFI) by example by Esteban Lorenzano
Pharo foreign function interface (FFI) by example by Esteban LorenzanoPharo foreign function interface (FFI) by example by Esteban Lorenzano
Pharo foreign function interface (FFI) by example by Esteban Lorenzano
FAST
 
Reproducibility and automation of machine learning process
Reproducibility and automation of machine learning processReproducibility and automation of machine learning process
Reproducibility and automation of machine learning process
Denis Dus
 
Autotools pratical training
Autotools pratical trainingAutotools pratical training
Autotools pratical training
Thierry Gayet
 
Pharo 64bits
Pharo 64bitsPharo 64bits
Pharo 64bits
Pharo
 
Golang
GolangGolang
Golang
Felipe Mamud
 
3 Git
3 Git3 Git
Go for SysAdmins - LISA 2015
Go for SysAdmins - LISA 2015Go for SysAdmins - LISA 2015
Go for SysAdmins - LISA 2015
Chris McEniry
 
Golang execution modes
Golang execution modesGolang execution modes
Golang execution modes
Ting-Li Chou
 
Git slides
Git slidesGit slides
Git slides
55020
 
Git for Excel
Git for ExcelGit for Excel
Git for Excel
xlwings
 
Package management and creation in Gentoo Linux
Package management and creation in Gentoo LinuxPackage management and creation in Gentoo Linux
Package management and creation in Gentoo Linux
Donnie Berkholz
 
Go & multi platform GUI Trials and Errors
Go & multi platform GUI Trials and ErrorsGo & multi platform GUI Trials and Errors
Go & multi platform GUI Trials and Errors
Yoshiki Shibukawa
 
Introduction to MonoTouch and Monodroid/Mono for Android
Introduction to MonoTouch and Monodroid/Mono for AndroidIntroduction to MonoTouch and Monodroid/Mono for Android
Introduction to MonoTouch and Monodroid/Mono for Android
Chris Hardy
 
10 clues showing that you are doing OSGi in the wrong manner - Jerome Moliere
10 clues showing that you are doing OSGi in the wrong manner - Jerome Moliere10 clues showing that you are doing OSGi in the wrong manner - Jerome Moliere
10 clues showing that you are doing OSGi in the wrong manner - Jerome Moliere
mfrancis
 
Building a Cross-Platform Mobile SDK in Rust.pdf
Building a Cross-Platform Mobile SDK in Rust.pdfBuilding a Cross-Platform Mobile SDK in Rust.pdf
Building a Cross-Platform Mobile SDK in Rust.pdf
IanWagner13
 
44CON London 2015 - Reverse engineering and exploiting font rasterizers: the ...
44CON London 2015 - Reverse engineering and exploiting font rasterizers: the ...44CON London 2015 - Reverse engineering and exploiting font rasterizers: the ...
44CON London 2015 - Reverse engineering and exploiting font rasterizers: the ...
44CON
 
Static PIE, How and Why - Metasploit's new POSIX payload: Mettle
Static PIE, How and Why - Metasploit's new POSIX payload: MettleStatic PIE, How and Why - Metasploit's new POSIX payload: Mettle
Static PIE, How and Why - Metasploit's new POSIX payload: Mettle
Brent Cook
 
Unit i
Unit iUnit i
Unit i
vijay gupta
 
IS - section 1 - modifiedFinal information system.pptx
IS - section 1 - modifiedFinal information system.pptxIS - section 1 - modifiedFinal information system.pptx
IS - section 1 - modifiedFinal information system.pptx
NaglaaAbdelhady
 

Similar to Unity3D Audio Grabber design (20)

Plug-ins & Third-Party SDKs in UE4
Plug-ins & Third-Party SDKs in UE4Plug-ins & Third-Party SDKs in UE4
Plug-ins & Third-Party SDKs in UE4
 
Pharo foreign function interface (FFI) by example by Esteban Lorenzano
Pharo foreign function interface (FFI) by example by Esteban LorenzanoPharo foreign function interface (FFI) by example by Esteban Lorenzano
Pharo foreign function interface (FFI) by example by Esteban Lorenzano
 
Reproducibility and automation of machine learning process
Reproducibility and automation of machine learning processReproducibility and automation of machine learning process
Reproducibility and automation of machine learning process
 
Autotools pratical training
Autotools pratical trainingAutotools pratical training
Autotools pratical training
 
Pharo 64bits
Pharo 64bitsPharo 64bits
Pharo 64bits
 
Golang
GolangGolang
Golang
 
3 Git
3 Git3 Git
3 Git
 
Go for SysAdmins - LISA 2015
Go for SysAdmins - LISA 2015Go for SysAdmins - LISA 2015
Go for SysAdmins - LISA 2015
 
Golang execution modes
Golang execution modesGolang execution modes
Golang execution modes
 
Git slides
Git slidesGit slides
Git slides
 
Git for Excel
Git for ExcelGit for Excel
Git for Excel
 
Package management and creation in Gentoo Linux
Package management and creation in Gentoo LinuxPackage management and creation in Gentoo Linux
Package management and creation in Gentoo Linux
 
Go & multi platform GUI Trials and Errors
Go & multi platform GUI Trials and ErrorsGo & multi platform GUI Trials and Errors
Go & multi platform GUI Trials and Errors
 
Introduction to MonoTouch and Monodroid/Mono for Android
Introduction to MonoTouch and Monodroid/Mono for AndroidIntroduction to MonoTouch and Monodroid/Mono for Android
Introduction to MonoTouch and Monodroid/Mono for Android
 
10 clues showing that you are doing OSGi in the wrong manner - Jerome Moliere
10 clues showing that you are doing OSGi in the wrong manner - Jerome Moliere10 clues showing that you are doing OSGi in the wrong manner - Jerome Moliere
10 clues showing that you are doing OSGi in the wrong manner - Jerome Moliere
 
Building a Cross-Platform Mobile SDK in Rust.pdf
Building a Cross-Platform Mobile SDK in Rust.pdfBuilding a Cross-Platform Mobile SDK in Rust.pdf
Building a Cross-Platform Mobile SDK in Rust.pdf
 
44CON London 2015 - Reverse engineering and exploiting font rasterizers: the ...
44CON London 2015 - Reverse engineering and exploiting font rasterizers: the ...44CON London 2015 - Reverse engineering and exploiting font rasterizers: the ...
44CON London 2015 - Reverse engineering and exploiting font rasterizers: the ...
 
Static PIE, How and Why - Metasploit's new POSIX payload: Mettle
Static PIE, How and Why - Metasploit's new POSIX payload: MettleStatic PIE, How and Why - Metasploit's new POSIX payload: Mettle
Static PIE, How and Why - Metasploit's new POSIX payload: Mettle
 
Unit i
Unit iUnit i
Unit i
 
IS - section 1 - modifiedFinal information system.pptx
IS - section 1 - modifiedFinal information system.pptxIS - section 1 - modifiedFinal information system.pptx
IS - section 1 - modifiedFinal information system.pptx
 

More from Jungsoo Nam

NVIDIA CUDA
NVIDIA CUDANVIDIA CUDA
NVIDIA CUDA
Jungsoo Nam
 
AGDK tutorial step by step
AGDK tutorial step by stepAGDK tutorial step by step
AGDK tutorial step by step
Jungsoo Nam
 
Android Grabber Module Proposal
Android Grabber Module ProposalAndroid Grabber Module Proposal
Android Grabber Module Proposal
Jungsoo Nam
 
Android OpenGL ES Game ImageGrabber Final Report
Android OpenGL ES Game ImageGrabber Final ReportAndroid OpenGL ES Game ImageGrabber Final Report
Android OpenGL ES Game ImageGrabber Final Report
Jungsoo Nam
 
13th kandroid OpenGL and EGL
13th kandroid OpenGL and EGL13th kandroid OpenGL and EGL
13th kandroid OpenGL and EGL
Jungsoo Nam
 
Android RenderScript
Android RenderScriptAndroid RenderScript
Android RenderScript
Jungsoo Nam
 
OpenGL ES EGL Spec&APIs
OpenGL ES EGL Spec&APIsOpenGL ES EGL Spec&APIs
OpenGL ES EGL Spec&APIs
Jungsoo Nam
 
OpenGL Shading Language
OpenGL Shading LanguageOpenGL Shading Language
OpenGL Shading Language
Jungsoo Nam
 

More from Jungsoo Nam (8)

NVIDIA CUDA
NVIDIA CUDANVIDIA CUDA
NVIDIA CUDA
 
AGDK tutorial step by step
AGDK tutorial step by stepAGDK tutorial step by step
AGDK tutorial step by step
 
Android Grabber Module Proposal
Android Grabber Module ProposalAndroid Grabber Module Proposal
Android Grabber Module Proposal
 
Android OpenGL ES Game ImageGrabber Final Report
Android OpenGL ES Game ImageGrabber Final ReportAndroid OpenGL ES Game ImageGrabber Final Report
Android OpenGL ES Game ImageGrabber Final Report
 
13th kandroid OpenGL and EGL
13th kandroid OpenGL and EGL13th kandroid OpenGL and EGL
13th kandroid OpenGL and EGL
 
Android RenderScript
Android RenderScriptAndroid RenderScript
Android RenderScript
 
OpenGL ES EGL Spec&APIs
OpenGL ES EGL Spec&APIsOpenGL ES EGL Spec&APIs
OpenGL ES EGL Spec&APIs
 
OpenGL Shading Language
OpenGL Shading LanguageOpenGL Shading Language
OpenGL Shading Language
 

Recently uploaded

Fueling AI with Great Data with Airbyte Webinar
Fueling AI with Great Data with Airbyte WebinarFueling AI with Great Data with Airbyte Webinar
Fueling AI with Great Data with Airbyte Webinar
Zilliz
 
Deep Dive: AI-Powered Marketing to Get More Leads and Customers with HyperGro...
Deep Dive: AI-Powered Marketing to Get More Leads and Customers with HyperGro...Deep Dive: AI-Powered Marketing to Get More Leads and Customers with HyperGro...
Deep Dive: AI-Powered Marketing to Get More Leads and Customers with HyperGro...
saastr
 
Monitoring and Managing Anomaly Detection on OpenShift.pdf
Monitoring and Managing Anomaly Detection on OpenShift.pdfMonitoring and Managing Anomaly Detection on OpenShift.pdf
Monitoring and Managing Anomaly Detection on OpenShift.pdf
Tosin Akinosho
 
5th LF Energy Power Grid Model Meet-up Slides
5th LF Energy Power Grid Model Meet-up Slides5th LF Energy Power Grid Model Meet-up Slides
5th LF Energy Power Grid Model Meet-up Slides
DanBrown980551
 
"Choosing proper type of scaling", Olena Syrota
"Choosing proper type of scaling", Olena Syrota"Choosing proper type of scaling", Olena Syrota
"Choosing proper type of scaling", Olena Syrota
Fwdays
 
Programming Foundation Models with DSPy - Meetup Slides
Programming Foundation Models with DSPy - Meetup SlidesProgramming Foundation Models with DSPy - Meetup Slides
Programming Foundation Models with DSPy - Meetup Slides
Zilliz
 
“Temporal Event Neural Networks: A More Efficient Alternative to the Transfor...
“Temporal Event Neural Networks: A More Efficient Alternative to the Transfor...“Temporal Event Neural Networks: A More Efficient Alternative to the Transfor...
“Temporal Event Neural Networks: A More Efficient Alternative to the Transfor...
Edge AI and Vision Alliance
 
Northern Engraving | Nameplate Manufacturing Process - 2024
Northern Engraving | Nameplate Manufacturing Process - 2024Northern Engraving | Nameplate Manufacturing Process - 2024
Northern Engraving | Nameplate Manufacturing Process - 2024
Northern Engraving
 
Taking AI to the Next Level in Manufacturing.pdf
Taking AI to the Next Level in Manufacturing.pdfTaking AI to the Next Level in Manufacturing.pdf
Taking AI to the Next Level in Manufacturing.pdf
ssuserfac0301
 
AppSec PNW: Android and iOS Application Security with MobSF
AppSec PNW: Android and iOS Application Security with MobSFAppSec PNW: Android and iOS Application Security with MobSF
AppSec PNW: Android and iOS Application Security with MobSF
Ajin Abraham
 
Nordic Marketo Engage User Group_June 13_ 2024.pptx
Nordic Marketo Engage User Group_June 13_ 2024.pptxNordic Marketo Engage User Group_June 13_ 2024.pptx
Nordic Marketo Engage User Group_June 13_ 2024.pptx
MichaelKnudsen27
 
HCL Notes and Domino License Cost Reduction in the World of DLAU
HCL Notes and Domino License Cost Reduction in the World of DLAUHCL Notes and Domino License Cost Reduction in the World of DLAU
HCL Notes and Domino License Cost Reduction in the World of DLAU
panagenda
 
"Frontline Battles with DDoS: Best practices and Lessons Learned", Igor Ivaniuk
"Frontline Battles with DDoS: Best practices and Lessons Learned",  Igor Ivaniuk"Frontline Battles with DDoS: Best practices and Lessons Learned",  Igor Ivaniuk
"Frontline Battles with DDoS: Best practices and Lessons Learned", Igor Ivaniuk
Fwdays
 
Introduction of Cybersecurity with OSS at Code Europe 2024
Introduction of Cybersecurity with OSS  at Code Europe 2024Introduction of Cybersecurity with OSS  at Code Europe 2024
Introduction of Cybersecurity with OSS at Code Europe 2024
Hiroshi SHIBATA
 
Connector Corner: Seamlessly power UiPath Apps, GenAI with prebuilt connectors
Connector Corner: Seamlessly power UiPath Apps, GenAI with prebuilt connectorsConnector Corner: Seamlessly power UiPath Apps, GenAI with prebuilt connectors
Connector Corner: Seamlessly power UiPath Apps, GenAI with prebuilt connectors
DianaGray10
 
Harnessing the Power of NLP and Knowledge Graphs for Opioid Research
Harnessing the Power of NLP and Knowledge Graphs for Opioid ResearchHarnessing the Power of NLP and Knowledge Graphs for Opioid Research
Harnessing the Power of NLP and Knowledge Graphs for Opioid Research
Neo4j
 
Apps Break Data
Apps Break DataApps Break Data
Apps Break Data
Ivo Velitchkov
 
Generating privacy-protected synthetic data using Secludy and Milvus
Generating privacy-protected synthetic data using Secludy and MilvusGenerating privacy-protected synthetic data using Secludy and Milvus
Generating privacy-protected synthetic data using Secludy and Milvus
Zilliz
 
Y-Combinator seed pitch deck template PP
Y-Combinator seed pitch deck template PPY-Combinator seed pitch deck template PP
Y-Combinator seed pitch deck template PP
c5vrf27qcz
 
Skybuffer SAM4U tool for SAP license adoption
Skybuffer SAM4U tool for SAP license adoptionSkybuffer SAM4U tool for SAP license adoption
Skybuffer SAM4U tool for SAP license adoption
Tatiana Kojar
 

Recently uploaded (20)

Fueling AI with Great Data with Airbyte Webinar
Fueling AI with Great Data with Airbyte WebinarFueling AI with Great Data with Airbyte Webinar
Fueling AI with Great Data with Airbyte Webinar
 
Deep Dive: AI-Powered Marketing to Get More Leads and Customers with HyperGro...
Deep Dive: AI-Powered Marketing to Get More Leads and Customers with HyperGro...Deep Dive: AI-Powered Marketing to Get More Leads and Customers with HyperGro...
Deep Dive: AI-Powered Marketing to Get More Leads and Customers with HyperGro...
 
Monitoring and Managing Anomaly Detection on OpenShift.pdf
Monitoring and Managing Anomaly Detection on OpenShift.pdfMonitoring and Managing Anomaly Detection on OpenShift.pdf
Monitoring and Managing Anomaly Detection on OpenShift.pdf
 
5th LF Energy Power Grid Model Meet-up Slides
5th LF Energy Power Grid Model Meet-up Slides5th LF Energy Power Grid Model Meet-up Slides
5th LF Energy Power Grid Model Meet-up Slides
 
"Choosing proper type of scaling", Olena Syrota
"Choosing proper type of scaling", Olena Syrota"Choosing proper type of scaling", Olena Syrota
"Choosing proper type of scaling", Olena Syrota
 
Programming Foundation Models with DSPy - Meetup Slides
Programming Foundation Models with DSPy - Meetup SlidesProgramming Foundation Models with DSPy - Meetup Slides
Programming Foundation Models with DSPy - Meetup Slides
 
“Temporal Event Neural Networks: A More Efficient Alternative to the Transfor...
“Temporal Event Neural Networks: A More Efficient Alternative to the Transfor...“Temporal Event Neural Networks: A More Efficient Alternative to the Transfor...
“Temporal Event Neural Networks: A More Efficient Alternative to the Transfor...
 
Northern Engraving | Nameplate Manufacturing Process - 2024
Northern Engraving | Nameplate Manufacturing Process - 2024Northern Engraving | Nameplate Manufacturing Process - 2024
Northern Engraving | Nameplate Manufacturing Process - 2024
 
Taking AI to the Next Level in Manufacturing.pdf
Taking AI to the Next Level in Manufacturing.pdfTaking AI to the Next Level in Manufacturing.pdf
Taking AI to the Next Level in Manufacturing.pdf
 
AppSec PNW: Android and iOS Application Security with MobSF
AppSec PNW: Android and iOS Application Security with MobSFAppSec PNW: Android and iOS Application Security with MobSF
AppSec PNW: Android and iOS Application Security with MobSF
 
Nordic Marketo Engage User Group_June 13_ 2024.pptx
Nordic Marketo Engage User Group_June 13_ 2024.pptxNordic Marketo Engage User Group_June 13_ 2024.pptx
Nordic Marketo Engage User Group_June 13_ 2024.pptx
 
HCL Notes and Domino License Cost Reduction in the World of DLAU
HCL Notes and Domino License Cost Reduction in the World of DLAUHCL Notes and Domino License Cost Reduction in the World of DLAU
HCL Notes and Domino License Cost Reduction in the World of DLAU
 
"Frontline Battles with DDoS: Best practices and Lessons Learned", Igor Ivaniuk
"Frontline Battles with DDoS: Best practices and Lessons Learned",  Igor Ivaniuk"Frontline Battles with DDoS: Best practices and Lessons Learned",  Igor Ivaniuk
"Frontline Battles with DDoS: Best practices and Lessons Learned", Igor Ivaniuk
 
Introduction of Cybersecurity with OSS at Code Europe 2024
Introduction of Cybersecurity with OSS  at Code Europe 2024Introduction of Cybersecurity with OSS  at Code Europe 2024
Introduction of Cybersecurity with OSS at Code Europe 2024
 
Connector Corner: Seamlessly power UiPath Apps, GenAI with prebuilt connectors
Connector Corner: Seamlessly power UiPath Apps, GenAI with prebuilt connectorsConnector Corner: Seamlessly power UiPath Apps, GenAI with prebuilt connectors
Connector Corner: Seamlessly power UiPath Apps, GenAI with prebuilt connectors
 
Harnessing the Power of NLP and Knowledge Graphs for Opioid Research
Harnessing the Power of NLP and Knowledge Graphs for Opioid ResearchHarnessing the Power of NLP and Knowledge Graphs for Opioid Research
Harnessing the Power of NLP and Knowledge Graphs for Opioid Research
 
Apps Break Data
Apps Break DataApps Break Data
Apps Break Data
 
Generating privacy-protected synthetic data using Secludy and Milvus
Generating privacy-protected synthetic data using Secludy and MilvusGenerating privacy-protected synthetic data using Secludy and Milvus
Generating privacy-protected synthetic data using Secludy and Milvus
 
Y-Combinator seed pitch deck template PP
Y-Combinator seed pitch deck template PPY-Combinator seed pitch deck template PP
Y-Combinator seed pitch deck template PP
 
Skybuffer SAM4U tool for SAP license adoption
Skybuffer SAM4U tool for SAP license adoptionSkybuffer SAM4U tool for SAP license adoption
Skybuffer SAM4U tool for SAP license adoption
 

Unity3D Audio Grabber design

  • 1. Unity3D Audio Grabber design Unity3D is tested on version 4.6 Jungsoo Nam namjungsoo@gmail.com 2015/03/17
  • 2. FMOD Ex • Libunity.so uses FMOD Ex statically(with .c sources or .a archive) linked. • NOT uses libfmodex.so • NOT will be applied by function hooking with Elf32_Ehdr like eglSwapBuffers()’s method. • C++ based library, but C wrapper classes. • C functions can be hooked.
  • 3. IDA analysis – libunity.so • ONLY Imports tab’s functions can be hooked. FMOD’s are in Exports tab. (Function pointers are NOT in imports entry table.)
  • 4. FMOD hooking • FMOD C API hooking is essential.(libkamcordaudio.so does like this) • Getting/sharing FMOD system object is important for interoperating AudioGrabber with Unity3D FMOD audio. • So we have to hook FMOD_System_Create(FMOD_SYSTEM** system) • How to hook FMOD? • Find another entry table.(exports table) • In VideoGrabber, we found imports table.(libunity.so imports libGLES*.so function) • Next we have to find exports table.(libunity.so exports some functions to client) • Or use “dlsym()” again. • orig_FMOD_System_Create = dlsym(RTLD_DEFAULT, "FMOD_System_Create"); • LOGD("FMOD_System_Create: 0x%x", orig_FMOD_System_Create);
  • 5. Final AudioGrabber design • AudioGrabber will be like below: • Link libfmodex.so. • Hook some FMOD functions and get FMOD system object. • Use our FMOD functions in libfmodex.so, manipulate FMOD audio and get audio buffer data. • Put audio buffer data into MediaCodec. • Mix and encode video and audio to .MP4