SlideShare a Scribd company logo
Pinch & Play
– Vocoder based on Resonance of Object
Special Topics in CDE 6 : Computational interaction design
CDE 20208002
Byounghern, Kim.
2020. 10. 06.
“콕 집어서 말하기”
Motivation
Electronic Kazoo-like Vocoder
Experience
Experience
Pinch,
Play,
Feel,
Collect what you love
With any object
Modeling : System
Device for Pinch Laptop
Play
Modeling : Data Collection
Device for Pinch
Laptop
Analyze and Generate
‘Voice Modulation Method’
Transmitter
Receiver
voice gesture (accent, texture, tone)
Modeling : Data Collection (Ideal Result)
Laptop
Analyze and Generate
‘Voice Modulation Method’
RNN
Invertible System
Modeling : Data Collection (First step)
Laptop
Analyze and Generate
‘Voice Modulation Method’
Mimicking voice
(sampled with a mouth and object)
Modeling :
Transformation
Rendering
(Under 100ms-delay)
Microphone Input
(Continuous Audio Data)
Voice gesture (accent, texture, tone)
Audio Output
(with noise elimination)
Challenges
Fine tuning
(Accent, texture, tone)
Synesthesia
(Visualize)
Milestones
Dec
Fine tuning
Oct
Mimicking
Voice
Generative
model
validation
Nov
Visualization
Implementation
Milestones
Oct
Mimicking
Voice
Generative
model
validation
Voice
Conversion
Thanks
Special Topics in CDE 6 : Computational interaction design
CDE 20208002
Byounghern, Kim.
2020. 10. 06.
Voice Controlled Interface 4 Digital Musical Instruments (Fasciani, 2014)
WaveNet (Google, 2016)
ADAPTIVE WAVENET VOCODER FOR RESIDUAL COMPENSATION IN GAN-BASED VOICE
CONVERSION (Sisman, 2018)

More Related Content

More from ByoungHern Kim

Progress Indicators - Affective Engineering Presentation
Progress Indicators - Affective Engineering PresentationProgress Indicators - Affective Engineering Presentation
Progress Indicators - Affective Engineering Presentation
ByoungHern Kim
 
pre-workshop assignment
pre-workshop assignmentpre-workshop assignment
pre-workshop assignment
ByoungHern Kim
 
(review) Using nonverbal signals to request help during human robot
(review) Using nonverbal signals to request help during human robot(review) Using nonverbal signals to request help during human robot
(review) Using nonverbal signals to request help during human robot
ByoungHern Kim
 
211117 decs seminar
211117 decs seminar211117 decs seminar
211117 decs seminar
ByoungHern Kim
 
Tiny ml study 20201031
Tiny ml study 20201031Tiny ml study 20201031
Tiny ml study 20201031
ByoungHern Kim
 
210604 hci seminar long term use
210604 hci seminar   long term use210604 hci seminar   long term use
210604 hci seminar long term use
ByoungHern Kim
 
210812 decs seminar
210812 decs seminar210812 decs seminar
210812 decs seminar
ByoungHern Kim
 
210901 decs seminar
210901 decs seminar210901 decs seminar
210901 decs seminar
ByoungHern Kim
 
210716 web serial impl
210716 web serial impl210716 web serial impl
210716 web serial impl
ByoungHern Kim
 
Decs seminar 211117
Decs seminar 211117Decs seminar 211117
Decs seminar 211117
ByoungHern Kim
 
Mbed cli install guide
Mbed cli install guideMbed cli install guide
Mbed cli install guide
ByoungHern Kim
 
경상북도교육청 과학영재교육원 4차산업혁명 특강자료
경상북도교육청 과학영재교육원 4차산업혁명 특강자료경상북도교육청 과학영재교육원 4차산업혁명 특강자료
경상북도교육청 과학영재교육원 4차산업혁명 특강자료
ByoungHern Kim
 
Superbin project workshop
Superbin project workshopSuperbin project workshop
Superbin project workshop
ByoungHern Kim
 
proposal using big+thick data
proposal using big+thick dataproposal using big+thick data
proposal using big+thick data
ByoungHern Kim
 
Bdpdp project proposal 20200402
Bdpdp project proposal 20200402Bdpdp project proposal 20200402
Bdpdp project proposal 20200402
ByoungHern Kim
 
Pinocchio ppt 140324
Pinocchio ppt 140324Pinocchio ppt 140324
Pinocchio ppt 140324
ByoungHern Kim
 

More from ByoungHern Kim (16)

Progress Indicators - Affective Engineering Presentation
Progress Indicators - Affective Engineering PresentationProgress Indicators - Affective Engineering Presentation
Progress Indicators - Affective Engineering Presentation
 
pre-workshop assignment
pre-workshop assignmentpre-workshop assignment
pre-workshop assignment
 
(review) Using nonverbal signals to request help during human robot
(review) Using nonverbal signals to request help during human robot(review) Using nonverbal signals to request help during human robot
(review) Using nonverbal signals to request help during human robot
 
211117 decs seminar
211117 decs seminar211117 decs seminar
211117 decs seminar
 
Tiny ml study 20201031
Tiny ml study 20201031Tiny ml study 20201031
Tiny ml study 20201031
 
210604 hci seminar long term use
210604 hci seminar   long term use210604 hci seminar   long term use
210604 hci seminar long term use
 
210812 decs seminar
210812 decs seminar210812 decs seminar
210812 decs seminar
 
210901 decs seminar
210901 decs seminar210901 decs seminar
210901 decs seminar
 
210716 web serial impl
210716 web serial impl210716 web serial impl
210716 web serial impl
 
Decs seminar 211117
Decs seminar 211117Decs seminar 211117
Decs seminar 211117
 
Mbed cli install guide
Mbed cli install guideMbed cli install guide
Mbed cli install guide
 
경상북도교육청 과학영재교육원 4차산업혁명 특강자료
경상북도교육청 과학영재교육원 4차산업혁명 특강자료경상북도교육청 과학영재교육원 4차산업혁명 특강자료
경상북도교육청 과학영재교육원 4차산업혁명 특강자료
 
Superbin project workshop
Superbin project workshopSuperbin project workshop
Superbin project workshop
 
proposal using big+thick data
proposal using big+thick dataproposal using big+thick data
proposal using big+thick data
 
Bdpdp project proposal 20200402
Bdpdp project proposal 20200402Bdpdp project proposal 20200402
Bdpdp project proposal 20200402
 
Pinocchio ppt 140324
Pinocchio ppt 140324Pinocchio ppt 140324
Pinocchio ppt 140324
 

Recently uploaded

Storytelling For The Web: Integrate Storytelling in your Design Process
Storytelling For The Web: Integrate Storytelling in your Design ProcessStorytelling For The Web: Integrate Storytelling in your Design Process
Storytelling For The Web: Integrate Storytelling in your Design Process
Chiara Aliotta
 
Practical eLearning Makeovers for Everyone
Practical eLearning Makeovers for EveryonePractical eLearning Makeovers for Everyone
Practical eLearning Makeovers for Everyone
Bianca Woods
 
Game Concept Presentation for Ukrainian Mythology Based Game With Designs
Game Concept Presentation for Ukrainian Mythology Based Game With DesignsGame Concept Presentation for Ukrainian Mythology Based Game With Designs
Game Concept Presentation for Ukrainian Mythology Based Game With Designs
184804
 
一比一原版(BU毕业证)波士顿大学毕业证如何办理
一比一原版(BU毕业证)波士顿大学毕业证如何办理一比一原版(BU毕业证)波士顿大学毕业证如何办理
一比一原版(BU毕业证)波士顿大学毕业证如何办理
peuce
 
定制美国西雅图城市大学毕业证学历证书原版一模一样
定制美国西雅图城市大学毕业证学历证书原版一模一样定制美国西雅图城市大学毕业证学历证书原版一模一样
定制美国西雅图城市大学毕业证学历证书原版一模一样
qo1as76n
 
Impact of Fonts: in Web and Apps Design
Impact of Fonts:  in Web and Apps DesignImpact of Fonts:  in Web and Apps Design
Impact of Fonts: in Web and Apps Design
contactproperweb2014
 
Timeless Principles of Good Design
Timeless Principles of Good DesignTimeless Principles of Good Design
Timeless Principles of Good Design
Carolina de Bartolo
 
一比一原版(Columbia毕业证)哥伦比亚大学毕业证如何办理
一比一原版(Columbia毕业证)哥伦比亚大学毕业证如何办理一比一原版(Columbia毕业证)哥伦比亚大学毕业证如何办理
一比一原版(Columbia毕业证)哥伦比亚大学毕业证如何办理
asuzyq
 
NHR Engineers Portfolio 2023 2024 NISHANT RATHI
NHR Engineers Portfolio 2023 2024 NISHANT RATHINHR Engineers Portfolio 2023 2024 NISHANT RATHI
NHR Engineers Portfolio 2023 2024 NISHANT RATHI
NishantRathi18
 
UNIT IV-VISUAL STYLE AND MOBILE INTERFACES.pptx
UNIT IV-VISUAL STYLE AND MOBILE INTERFACES.pptxUNIT IV-VISUAL STYLE AND MOBILE INTERFACES.pptx
UNIT IV-VISUAL STYLE AND MOBILE INTERFACES.pptx
GOWSIKRAJA PALANISAMY
 
Maximize Your Content with Beautiful Assets : Content & Asset for Landing Page
Maximize Your Content with Beautiful Assets : Content & Asset for Landing Page Maximize Your Content with Beautiful Assets : Content & Asset for Landing Page
Maximize Your Content with Beautiful Assets : Content & Asset for Landing Page
pmgdscunsri
 
Graphic Design Tools and Software .pptx
Graphic Design Tools and Software   .pptxGraphic Design Tools and Software   .pptx
Graphic Design Tools and Software .pptx
Virtual Real Design
 
Divertidamente SLIDE.pptxufururururuhrurid8dj
Divertidamente SLIDE.pptxufururururuhrurid8djDivertidamente SLIDE.pptxufururururuhrurid8dj
Divertidamente SLIDE.pptxufururururuhrurid8dj
lunaemel03
 
SECURING BUILDING PERMIT CITY OF CALOOCAN.pdf
SECURING BUILDING PERMIT CITY OF CALOOCAN.pdfSECURING BUILDING PERMIT CITY OF CALOOCAN.pdf
SECURING BUILDING PERMIT CITY OF CALOOCAN.pdf
eloprejohn333
 
PDF SubmissionDigital Marketing Institute in Noida
PDF SubmissionDigital Marketing Institute in NoidaPDF SubmissionDigital Marketing Institute in Noida
PDF SubmissionDigital Marketing Institute in Noida
PoojaSaini954651
 
Top Interior Designers in Bangalore.pdf1
Top Interior Designers in Bangalore.pdf1Top Interior Designers in Bangalore.pdf1
Top Interior Designers in Bangalore.pdf1
Decomart Studio
 
Connect Conference 2022: Passive House - Economic and Environmental Solution...
Connect Conference 2022: Passive House -  Economic and Environmental Solution...Connect Conference 2022: Passive House -  Economic and Environmental Solution...
Connect Conference 2022: Passive House - Economic and Environmental Solution...
TE Studio
 
一比一原版(UW毕业证)西雅图华盛顿大学毕业证如何办理
一比一原版(UW毕业证)西雅图华盛顿大学毕业证如何办理一比一原版(UW毕业证)西雅图华盛顿大学毕业证如何办理
一比一原版(UW毕业证)西雅图华盛顿大学毕业证如何办理
kecekev
 
AHMED TALAAT ARCHITECTURE PORTFOLIO .pdf
AHMED TALAAT ARCHITECTURE PORTFOLIO .pdfAHMED TALAAT ARCHITECTURE PORTFOLIO .pdf
AHMED TALAAT ARCHITECTURE PORTFOLIO .pdf
talaatahm
 
ARENA - Young adults in the workplace (Knight Moves).pdf
ARENA - Young adults in the workplace (Knight Moves).pdfARENA - Young adults in the workplace (Knight Moves).pdf
ARENA - Young adults in the workplace (Knight Moves).pdf
Knight Moves
 

Recently uploaded (20)

Storytelling For The Web: Integrate Storytelling in your Design Process
Storytelling For The Web: Integrate Storytelling in your Design ProcessStorytelling For The Web: Integrate Storytelling in your Design Process
Storytelling For The Web: Integrate Storytelling in your Design Process
 
Practical eLearning Makeovers for Everyone
Practical eLearning Makeovers for EveryonePractical eLearning Makeovers for Everyone
Practical eLearning Makeovers for Everyone
 
Game Concept Presentation for Ukrainian Mythology Based Game With Designs
Game Concept Presentation for Ukrainian Mythology Based Game With DesignsGame Concept Presentation for Ukrainian Mythology Based Game With Designs
Game Concept Presentation for Ukrainian Mythology Based Game With Designs
 
一比一原版(BU毕业证)波士顿大学毕业证如何办理
一比一原版(BU毕业证)波士顿大学毕业证如何办理一比一原版(BU毕业证)波士顿大学毕业证如何办理
一比一原版(BU毕业证)波士顿大学毕业证如何办理
 
定制美国西雅图城市大学毕业证学历证书原版一模一样
定制美国西雅图城市大学毕业证学历证书原版一模一样定制美国西雅图城市大学毕业证学历证书原版一模一样
定制美国西雅图城市大学毕业证学历证书原版一模一样
 
Impact of Fonts: in Web and Apps Design
Impact of Fonts:  in Web and Apps DesignImpact of Fonts:  in Web and Apps Design
Impact of Fonts: in Web and Apps Design
 
Timeless Principles of Good Design
Timeless Principles of Good DesignTimeless Principles of Good Design
Timeless Principles of Good Design
 
一比一原版(Columbia毕业证)哥伦比亚大学毕业证如何办理
一比一原版(Columbia毕业证)哥伦比亚大学毕业证如何办理一比一原版(Columbia毕业证)哥伦比亚大学毕业证如何办理
一比一原版(Columbia毕业证)哥伦比亚大学毕业证如何办理
 
NHR Engineers Portfolio 2023 2024 NISHANT RATHI
NHR Engineers Portfolio 2023 2024 NISHANT RATHINHR Engineers Portfolio 2023 2024 NISHANT RATHI
NHR Engineers Portfolio 2023 2024 NISHANT RATHI
 
UNIT IV-VISUAL STYLE AND MOBILE INTERFACES.pptx
UNIT IV-VISUAL STYLE AND MOBILE INTERFACES.pptxUNIT IV-VISUAL STYLE AND MOBILE INTERFACES.pptx
UNIT IV-VISUAL STYLE AND MOBILE INTERFACES.pptx
 
Maximize Your Content with Beautiful Assets : Content & Asset for Landing Page
Maximize Your Content with Beautiful Assets : Content & Asset for Landing Page Maximize Your Content with Beautiful Assets : Content & Asset for Landing Page
Maximize Your Content with Beautiful Assets : Content & Asset for Landing Page
 
Graphic Design Tools and Software .pptx
Graphic Design Tools and Software   .pptxGraphic Design Tools and Software   .pptx
Graphic Design Tools and Software .pptx
 
Divertidamente SLIDE.pptxufururururuhrurid8dj
Divertidamente SLIDE.pptxufururururuhrurid8djDivertidamente SLIDE.pptxufururururuhrurid8dj
Divertidamente SLIDE.pptxufururururuhrurid8dj
 
SECURING BUILDING PERMIT CITY OF CALOOCAN.pdf
SECURING BUILDING PERMIT CITY OF CALOOCAN.pdfSECURING BUILDING PERMIT CITY OF CALOOCAN.pdf
SECURING BUILDING PERMIT CITY OF CALOOCAN.pdf
 
PDF SubmissionDigital Marketing Institute in Noida
PDF SubmissionDigital Marketing Institute in NoidaPDF SubmissionDigital Marketing Institute in Noida
PDF SubmissionDigital Marketing Institute in Noida
 
Top Interior Designers in Bangalore.pdf1
Top Interior Designers in Bangalore.pdf1Top Interior Designers in Bangalore.pdf1
Top Interior Designers in Bangalore.pdf1
 
Connect Conference 2022: Passive House - Economic and Environmental Solution...
Connect Conference 2022: Passive House -  Economic and Environmental Solution...Connect Conference 2022: Passive House -  Economic and Environmental Solution...
Connect Conference 2022: Passive House - Economic and Environmental Solution...
 
一比一原版(UW毕业证)西雅图华盛顿大学毕业证如何办理
一比一原版(UW毕业证)西雅图华盛顿大学毕业证如何办理一比一原版(UW毕业证)西雅图华盛顿大学毕业证如何办理
一比一原版(UW毕业证)西雅图华盛顿大学毕业证如何办理
 
AHMED TALAAT ARCHITECTURE PORTFOLIO .pdf
AHMED TALAAT ARCHITECTURE PORTFOLIO .pdfAHMED TALAAT ARCHITECTURE PORTFOLIO .pdf
AHMED TALAAT ARCHITECTURE PORTFOLIO .pdf
 
ARENA - Young adults in the workplace (Knight Moves).pdf
ARENA - Young adults in the workplace (Knight Moves).pdfARENA - Young adults in the workplace (Knight Moves).pdf
ARENA - Young adults in the workplace (Knight Moves).pdf
 

PnP instrument ( Computational Interaction Design - project proposal )