SlideShare a Scribd company logo
1 of 23
Download to read offline
Super-resolution in sound field recording and
reproduction based on sparse representation
Shoichi Koyama1,2, Naoki Murata1, and Hiroshi Saruwatari1
1The University of Tokyo
2Paris Diderot University / Institute Langevin
November 29, 2016
Sound field reproduction for audio system
Microphone array Loudspeaker array
 Large listening area can be achieved
 Listeners can perceive source distance
 Real-time recording and reproduction can be achieved
without recording engineers
Recording area Target area
November 29, 2016
Sound field reproduction for audio system
Microphone array Loudspeaker array
Telecommunication system
NW
Home Theatre
Live broadcasting
Applications
Recording area Target area
November 29, 2016
Sound field reproduction for audio system
Microphone array Loudspeaker array
Improve reproduction accuracy when # of array elements is small
 # of microphones > # of loudspeakers
– Higher reproduction accuracy within local region of target area
 # of microphones < # of loudspeakers
– Higher reproduction accuracy of sources in local region of recording area
[Koyama+ IEEE JSTSP2015], [Koyama+ ICASSP 2014, 2015]
[Ahrens+ AES Conv. 2010], [Ueno+ ICASSP 2017 (submitted)]
Recording area Target area
Sound Field Recording and Reproduction
November 29, 2016
Recording area Target area
Obtain driving signals of secondary sources (= loudspeakers)
arranged on to reconstruct desired sound field inside
 Inherently, sound pressure and its gradient on is required to obtain
, but sound pressure is usually only known
 Signal conversion for sound field recording and reproduction with ordinary
acoustic sensors and transducers is necessary
Primary
sources
November 29, 2016
Conventional: WFR filtering method
Recording area Target area
Secondary source planeReceiving plane
Primary
sources
Signal
conversion
[Koyama+ IEEE TASLP 2013]
Received
signals
Driving signals
Plane wave Plane wave
Each plane wave determines entire sound field
Signal conversion can be achieved in spatial frequency domain
November 29, 2016
Conventional: WFR filtering method
Target area
Received
signals
Driving signals
Plane wave Plane wave
Each plane wave determines entire sound field
Spatial aliasing artifacts due to plane wave decomposition
Significant error at high freq even when microphone < loudspeaker
Recording area
Signal
conversion
Secondary source planeReceiving plane
Primary
sources
[Koyama+ IEEE TASLP 2013]
Sound field representation for super-resolution
 Plane wave decomposition suffers from spatial aliasing artifacts because
many basis functions are used
 Observed signals should be represented by a few basis functions for accurate
interpolation of sound field
 Appropriate basis function may be close to pressure distribution originating
from sound sources
 To obtain driving signals of loudspeakers, basis functions must be
fundamental solutions of Helmholtz equation (e.g. Green functions)
November 29, 2016
Basis functionReceived
signals
Sound field decomposition into fundamental solutions
of Helmholtz equation is necessary
Sound field decomposition
Generative model of sound field
 Inhomogeneous and homogeneous Helmholtz eq. Distribution of
source components
November 29, 2016
[Koyama+ ICASSP 2014]
Sound field is divided into two regions
Generative model of sound field
 Inhomogeneous and homogeneous Helmholtz eq.
November 29, 2016
[Koyama+ ICASSP 2014]
Green’s function
Inhomogeneous + homogeneous terms
Plane wave
November 29, 2016
Generative model of sound field
 Observe sound pressure distribution on plane
 Conversion into driving signals
Synthesize monopole sources [Spors+ AES Conv. 2008]
Ambient componentsDirect source components
Applying WFR filtering method [Koyama+ IEEE TASLP 2013]
Decomposition into two components can lead to
higher reproduction accuracy above spatial Nyquist freq
November 29, 2016
Sparse sound field representation
・・・・・・・・
Microphone array
Source components
Grid points Sparsity-based signal decomposition
Discretization
Ambient components
Dictionary matrix of Green’s functions
Observed signal Distribution of source components
A few elements of has non-zero values
under the assumption of spatially sparse source distribution
Sparse signal decomposition
 Sparse signal representation in vector form
 Signal decomposition based on sparsity of
November 29, 2016
Minimize -norm of
Group sparsity based on physical properties
November 29, 2016
Group sparse signal models for robust decomposition
• Multiple time frames
• Temporal frequencies
• Multipole components
Decomposition algorithm extending FOCUSS
[Koyama+ ICASSP 2015]
 Sparse signal representation in vector form Structure of sparsity induced
by physical properties
Block diagram of signal conversion
 Decomposition stage
– Group sparse decomposition of
 Reconstruction stage
– and are respectively converted into driving signals
– is obtained as sum of two components
November 29, 2016
Simulation Experiment
 Proposed method (Proposed), WFR filtering method (WFR), and Sound
Pressure Control method (SPC) were compared
 32 microphones (6 cm intervals)and 48 loudspeakers (4 cm intervals)
 : Rectangular region of 2.4x2.4 m, Grid points: (10cm, 20cm) intervals
 Source directivity: unidirectional
 Source signal: single frequency sinewave
Recording area Target area
November 29, 2016
Simulation Experiment
 Signal-to-distortion ratio of reproduction (SDRR)
Recording area Target area
November 29, 2016
Original pressure distribution
Reproduced pressure distribution
November 29, 2016
Frequency vs. SDR
SDRRs above spatial Nyquist frequency were improved
 Source location: (-0.32, -0.84, 0.0) m
Reproduced sound pressure distribution (1.0 kHz)PressureError
November 29, 2016
Proposed WFR SPC
18.1 dB 18.0 dB 19.4 dB
 Source location: (-0.32, -0.84, 0.0) m
SDRR:
Reproduced sound pressure distribution (4.0 kHz)PressureError
November 29, 2016
19.7 dB 6.8 dB 7.8 dB
Proposed WFR SPC
 Source location: (-0.32, -0.84, 0.0) m
SDRR:
Frequency response of reproduced sound field
November 29, 2016
 Frequency response at (0.0, 1.0, 0.0) m
Reproduced frequency response was improved
Conclusion
 Super-resolution sound field recording and
reproduction based on sparse representation
– Conventional plane wave decomposition is suffered from
spatial aliasing artifacts
– Sound field representation using source and plane wave
components
– Sound field decomposition based on spatial sparsity of
source components
– Group sparsity based on physical properties of sound field
– Experimental results indicated that reproduction accuracy
above spatial Nyquist frequency can be improved
November 29, 2016
Thank you for your attention!
Related publications
• S. Koyama and H. Saruwatari, “Sound field decomposition in reverberant environment
using sparse and low-rank signal models,” Proc. IEEE ICASSP, 2016.
• N. Murata, S. Koyama, et al. “Sparse sound field decomposition with multichannel
extension of complex NMF,” Proc. IEEE ICASSP, 2016.
• S. Koyama, et al. “Sparse sound field decomposition using group sparse Bayesian
learning,” Proc. APSIPA ASC, 2015.
• N. Murata, S. Koyama, et al. “Sparse sound field decomposition with parametric
dictionary learning for super-resolution recording and reproduction,” Proc. IEEE
CAMSAP, 2015.
• S. Koyama, et al. “Source-location-informed sound field recording and reproduction
with spherical arrays,” Proc. IEEE WASPAA, 2015.
• S. Koyama, et al. “Source-location-informed sound field recording and reproduction,”
IEEE J. Sel. Topics Signal Process., vol. 9, no. 5, pp. 881-894, 2015.
• S. Koyama, et al. “Structured sparse signal models and decomposition algorithm for
super-resolution in sound field recording and reproduction,” Proc. IEEE ICASSP, 2015.
• S. Koyama, et al. “Sparse sound field representation in recording and reproduction for
reducing spatial aliasing artifacts,” Proc. IEEE ICASSP, 2014.
November 29, 2016

More Related Content

What's hot

Adaptive Median Filters
Adaptive Median FiltersAdaptive Median Filters
Adaptive Median FiltersAmnaakhaan
 
Empire state building presentation
Empire state building presentationEmpire state building presentation
Empire state building presentationGabriela Soto Bazán
 
Strange Buildings
Strange BuildingsStrange Buildings
Strange BuildingsTony DeLiso
 
Acoustics for a music studio
Acoustics for a music studioAcoustics for a music studio
Acoustics for a music studioayan baraniya
 
Earthquakes effects on reinforced concrete buildings
Earthquakes effects on reinforced concrete buildingsEarthquakes effects on reinforced concrete buildings
Earthquakes effects on reinforced concrete buildingsAnoop Shrestha
 
Structure borne
Structure borneStructure borne
Structure borneYNam Ho
 
Why Inclusive Design Is For Us All - Accessibility and Life Safety vs. Inclus...
Why Inclusive Design Is For Us All - Accessibility and Life Safety vs. Inclus...Why Inclusive Design Is For Us All - Accessibility and Life Safety vs. Inclus...
Why Inclusive Design Is For Us All - Accessibility and Life Safety vs. Inclus...Galala University
 
Basic theory of sound
Basic theory of soundBasic theory of sound
Basic theory of soundSeid Adem
 
Acoustical materials
Acoustical materialsAcoustical materials
Acoustical materialsyogitham
 
Kindergarten design focusing on creating healthy and comfortable
Kindergarten design focusing on creating healthy and comfortableKindergarten design focusing on creating healthy and comfortable
Kindergarten design focusing on creating healthy and comfortableSiham EL-Saadi
 

What's hot (20)

Adaptive Median Filters
Adaptive Median FiltersAdaptive Median Filters
Adaptive Median Filters
 
Acoustics
AcousticsAcoustics
Acoustics
 
Empire state building presentation
Empire state building presentationEmpire state building presentation
Empire state building presentation
 
Acostic Barriers
Acostic Barriers Acostic Barriers
Acostic Barriers
 
Strange Buildings
Strange BuildingsStrange Buildings
Strange Buildings
 
Acoustics for a music studio
Acoustics for a music studioAcoustics for a music studio
Acoustics for a music studio
 
Sound fundamentals
Sound fundamentalsSound fundamentals
Sound fundamentals
 
Earthquakes effects on reinforced concrete buildings
Earthquakes effects on reinforced concrete buildingsEarthquakes effects on reinforced concrete buildings
Earthquakes effects on reinforced concrete buildings
 
Acoustics buildings
Acoustics buildingsAcoustics buildings
Acoustics buildings
 
Structure borne
Structure borneStructure borne
Structure borne
 
Taliesin west
Taliesin westTaliesin west
Taliesin west
 
_CONCERT HALL ACOUSTICS.pdf
_CONCERT HALL ACOUSTICS.pdf_CONCERT HALL ACOUSTICS.pdf
_CONCERT HALL ACOUSTICS.pdf
 
Why Inclusive Design Is For Us All - Accessibility and Life Safety vs. Inclus...
Why Inclusive Design Is For Us All - Accessibility and Life Safety vs. Inclus...Why Inclusive Design Is For Us All - Accessibility and Life Safety vs. Inclus...
Why Inclusive Design Is For Us All - Accessibility and Life Safety vs. Inclus...
 
IS 13827
IS 13827IS 13827
IS 13827
 
Basic theory of sound
Basic theory of soundBasic theory of sound
Basic theory of sound
 
Architectural acoustics
Architectural acousticsArchitectural acoustics
Architectural acoustics
 
Urban Design Study
Urban Design StudyUrban Design Study
Urban Design Study
 
Acoustical materials
Acoustical materialsAcoustical materials
Acoustical materials
 
Acaustic
AcausticAcaustic
Acaustic
 
Kindergarten design focusing on creating healthy and comfortable
Kindergarten design focusing on creating healthy and comfortableKindergarten design focusing on creating healthy and comfortable
Kindergarten design focusing on creating healthy and comfortable
 

Viewers also liked

HMMに基づく日本人英語音声合成における中学生徒の英語音声を用いた評価
HMMに基づく日本人英語音声合成における中学生徒の英語音声を用いた評価HMMに基づく日本人英語音声合成における中学生徒の英語音声を用いた評価
HMMに基づく日本人英語音声合成における中学生徒の英語音声を用いた評価Shinnosuke Takamichi
 
Moment matching networkを用いた音声パラメータのランダム生成の検討
Moment matching networkを用いた音声パラメータのランダム生成の検討Moment matching networkを用いた音声パラメータのランダム生成の検討
Moment matching networkを用いた音声パラメータのランダム生成の検討Shinnosuke Takamichi
 
数値解析と物理学
数値解析と物理学数値解析と物理学
数値解析と物理学すずしめ
 
独立性に基づくブラインド音源分離の発展と独立低ランク行列分析 History of independence-based blind source sep...
独立性に基づくブラインド音源分離の発展と独立低ランク行列分析 History of independence-based blind source sep...独立性に基づくブラインド音源分離の発展と独立低ランク行列分析 History of independence-based blind source sep...
独立性に基づくブラインド音源分離の発展と独立低ランク行列分析 History of independence-based blind source sep...Daichi Kitamura
 

Viewers also liked (15)

Hybrid NMF APSIPA2014 invited
Hybrid NMF APSIPA2014 invitedHybrid NMF APSIPA2014 invited
Hybrid NMF APSIPA2014 invited
 
Slp201702
Slp201702Slp201702
Slp201702
 
ILRMA 20170227 danwakai
ILRMA 20170227 danwakaiILRMA 20170227 danwakai
ILRMA 20170227 danwakai
 
Ea2015 7for ss
Ea2015 7for ssEa2015 7for ss
Ea2015 7for ss
 
HMMに基づく日本人英語音声合成における中学生徒の英語音声を用いた評価
HMMに基づく日本人英語音声合成における中学生徒の英語音声を用いた評価HMMに基づく日本人英語音声合成における中学生徒の英語音声を用いた評価
HMMに基づく日本人英語音声合成における中学生徒の英語音声を用いた評価
 
Asj2017 3 bileveloptnmf
Asj2017 3 bileveloptnmfAsj2017 3 bileveloptnmf
Asj2017 3 bileveloptnmf
 
Ica2016 312 saruwatari
Ica2016 312 saruwatariIca2016 312 saruwatari
Ica2016 312 saruwatari
 
Moment matching networkを用いた音声パラメータのランダム生成の検討
Moment matching networkを用いた音声パラメータのランダム生成の検討Moment matching networkを用いた音声パラメータのランダム生成の検討
Moment matching networkを用いた音声パラメータのランダム生成の検討
 
Asj2017 3invited
Asj2017 3invitedAsj2017 3invited
Asj2017 3invited
 
Dsp2015for ss
Dsp2015for ssDsp2015for ss
Dsp2015for ss
 
Apsipa2016for ss
Apsipa2016for ssApsipa2016for ss
Apsipa2016for ss
 
Koyama AES Conference SFC 2016
Koyama AES Conference SFC 2016Koyama AES Conference SFC 2016
Koyama AES Conference SFC 2016
 
Discriminative SNMF EA201603
Discriminative SNMF EA201603Discriminative SNMF EA201603
Discriminative SNMF EA201603
 
数値解析と物理学
数値解析と物理学数値解析と物理学
数値解析と物理学
 
独立性に基づくブラインド音源分離の発展と独立低ランク行列分析 History of independence-based blind source sep...
独立性に基づくブラインド音源分離の発展と独立低ランク行列分析 History of independence-based blind source sep...独立性に基づくブラインド音源分離の発展と独立低ランク行列分析 History of independence-based blind source sep...
独立性に基づくブラインド音源分離の発展と独立低ランク行列分析 History of independence-based blind source sep...
 

Similar to Koyama ASA ASJ joint meeting 2016

SoundSense
SoundSenseSoundSense
SoundSensebutest
 
Audio Source Separation Based on Low-Rank Structure and Statistical Independence
Audio Source Separation Based on Low-Rank Structure and Statistical IndependenceAudio Source Separation Based on Low-Rank Structure and Statistical Independence
Audio Source Separation Based on Low-Rank Structure and Statistical IndependenceDaichi Kitamura
 
HUFFMAN CODING ALGORITHM BASED ADAPTIVE NOISE CANCELLATION
HUFFMAN CODING ALGORITHM BASED ADAPTIVE NOISE CANCELLATIONHUFFMAN CODING ALGORITHM BASED ADAPTIVE NOISE CANCELLATION
HUFFMAN CODING ALGORITHM BASED ADAPTIVE NOISE CANCELLATIONIRJET Journal
 
Defense - Sound space rendering based on the virtual Sound space rendering ba...
Defense - Sound space rendering based on the virtual Sound space rendering ba...Defense - Sound space rendering based on the virtual Sound space rendering ba...
Defense - Sound space rendering based on the virtual Sound space rendering ba...JunjieShi3
 
Future Proof Surround Sound Mixing using Ambisonics
Future Proof Surround Sound Mixing using AmbisonicsFuture Proof Surround Sound Mixing using Ambisonics
Future Proof Surround Sound Mixing using AmbisonicsBruce Wiggins
 
Plane wave decomposition and beamforming for directional spatial sound locali...
Plane wave decomposition and beamforming for directional spatial sound locali...Plane wave decomposition and beamforming for directional spatial sound locali...
Plane wave decomposition and beamforming for directional spatial sound locali...Muhammad Imran
 
Sound Source Localization
Sound Source LocalizationSound Source Localization
Sound Source LocalizationMuhammad Imran
 
Adaptive noise estimation algorithm for speech enhancement
Adaptive noise estimation algorithm for speech enhancementAdaptive noise estimation algorithm for speech enhancement
Adaptive noise estimation algorithm for speech enhancementHarshal Ladhe
 
Audio spot light
Audio spot lightAudio spot light
Audio spot lightMansi Gupta
 
Blind audio source separation based on time-frequency structure models
Blind audio source separation based on time-frequency structure modelsBlind audio source separation based on time-frequency structure models
Blind audio source separation based on time-frequency structure modelsKitamura Laboratory
 
Experimental analysis of optimal window length for independent low-rank matri...
Experimental analysis of optimal window length for independent low-rank matri...Experimental analysis of optimal window length for independent low-rank matri...
Experimental analysis of optimal window length for independent low-rank matri...Daichi Kitamura
 
Transferring Singing Expressions from One Voice to Another for a Given Song
Transferring Singing Expressions from One Voice to Another for a Given SongTransferring Singing Expressions from One Voice to Another for a Given Song
Transferring Singing Expressions from One Voice to Another for a Given SongNAVER Engineering
 
A Combined Sub-Band And Reconstructed Phase Space Approach To Phoneme Classif...
A Combined Sub-Band And Reconstructed Phase Space Approach To Phoneme Classif...A Combined Sub-Band And Reconstructed Phase Space Approach To Phoneme Classif...
A Combined Sub-Band And Reconstructed Phase Space Approach To Phoneme Classif...April Smith
 
Robust Sound Field Reproduction against Listener’s Movement Utilizing Image ...
Robust Sound Field Reproduction against  Listener’s Movement Utilizing Image ...Robust Sound Field Reproduction against  Listener’s Movement Utilizing Image ...
Robust Sound Field Reproduction against Listener’s Movement Utilizing Image ...奈良先端大 情報科学研究科
 
Audiospotlighting
AudiospotlightingAudiospotlighting
AudiospotlightingBiji Raju
 
A Computational Framework for Sound Segregation in Music Signals using Marsyas
A Computational Framework for Sound Segregation in Music Signals using MarsyasA Computational Framework for Sound Segregation in Music Signals using Marsyas
A Computational Framework for Sound Segregation in Music Signals using MarsyasLuís Gustavo Martins
 

Similar to Koyama ASA ASJ joint meeting 2016 (20)

SoundSense
SoundSenseSoundSense
SoundSense
 
Audio Source Separation Based on Low-Rank Structure and Statistical Independence
Audio Source Separation Based on Low-Rank Structure and Statistical IndependenceAudio Source Separation Based on Low-Rank Structure and Statistical Independence
Audio Source Separation Based on Low-Rank Structure and Statistical Independence
 
HUFFMAN CODING ALGORITHM BASED ADAPTIVE NOISE CANCELLATION
HUFFMAN CODING ALGORITHM BASED ADAPTIVE NOISE CANCELLATIONHUFFMAN CODING ALGORITHM BASED ADAPTIVE NOISE CANCELLATION
HUFFMAN CODING ALGORITHM BASED ADAPTIVE NOISE CANCELLATION
 
Audio spotlighting
Audio spotlightingAudio spotlighting
Audio spotlighting
 
Defense - Sound space rendering based on the virtual Sound space rendering ba...
Defense - Sound space rendering based on the virtual Sound space rendering ba...Defense - Sound space rendering based on the virtual Sound space rendering ba...
Defense - Sound space rendering based on the virtual Sound space rendering ba...
 
Future Proof Surround Sound Mixing using Ambisonics
Future Proof Surround Sound Mixing using AmbisonicsFuture Proof Surround Sound Mixing using Ambisonics
Future Proof Surround Sound Mixing using Ambisonics
 
T26123129
T26123129T26123129
T26123129
 
B45010811
B45010811B45010811
B45010811
 
Plane wave decomposition and beamforming for directional spatial sound locali...
Plane wave decomposition and beamforming for directional spatial sound locali...Plane wave decomposition and beamforming for directional spatial sound locali...
Plane wave decomposition and beamforming for directional spatial sound locali...
 
Sound Source Localization
Sound Source LocalizationSound Source Localization
Sound Source Localization
 
Adaptive noise estimation algorithm for speech enhancement
Adaptive noise estimation algorithm for speech enhancementAdaptive noise estimation algorithm for speech enhancement
Adaptive noise estimation algorithm for speech enhancement
 
Audio spot light
Audio spot lightAudio spot light
Audio spot light
 
Blind audio source separation based on time-frequency structure models
Blind audio source separation based on time-frequency structure modelsBlind audio source separation based on time-frequency structure models
Blind audio source separation based on time-frequency structure models
 
F010334548
F010334548F010334548
F010334548
 
Experimental analysis of optimal window length for independent low-rank matri...
Experimental analysis of optimal window length for independent low-rank matri...Experimental analysis of optimal window length for independent low-rank matri...
Experimental analysis of optimal window length for independent low-rank matri...
 
Transferring Singing Expressions from One Voice to Another for a Given Song
Transferring Singing Expressions from One Voice to Another for a Given SongTransferring Singing Expressions from One Voice to Another for a Given Song
Transferring Singing Expressions from One Voice to Another for a Given Song
 
A Combined Sub-Band And Reconstructed Phase Space Approach To Phoneme Classif...
A Combined Sub-Band And Reconstructed Phase Space Approach To Phoneme Classif...A Combined Sub-Band And Reconstructed Phase Space Approach To Phoneme Classif...
A Combined Sub-Band And Reconstructed Phase Space Approach To Phoneme Classif...
 
Robust Sound Field Reproduction against Listener’s Movement Utilizing Image ...
Robust Sound Field Reproduction against  Listener’s Movement Utilizing Image ...Robust Sound Field Reproduction against  Listener’s Movement Utilizing Image ...
Robust Sound Field Reproduction against Listener’s Movement Utilizing Image ...
 
Audiospotlighting
AudiospotlightingAudiospotlighting
Audiospotlighting
 
A Computational Framework for Sound Segregation in Music Signals using Marsyas
A Computational Framework for Sound Segregation in Music Signals using MarsyasA Computational Framework for Sound Segregation in Music Signals using Marsyas
A Computational Framework for Sound Segregation in Music Signals using Marsyas
 

Recently uploaded

Production technology of Brinjal -Solanum melongena
Production technology of Brinjal -Solanum melongenaProduction technology of Brinjal -Solanum melongena
Production technology of Brinjal -Solanum melongenajana861314
 
CDS Fundamentals of digital communication system UNIT 1 AND 2.pdf
CDS Fundamentals of digital communication system UNIT 1 AND 2.pdfCDS Fundamentals of digital communication system UNIT 1 AND 2.pdf
CDS Fundamentals of digital communication system UNIT 1 AND 2.pdfshubhangisonawane6
 
Science (Communication) and Wikipedia - Potentials and Pitfalls
Science (Communication) and Wikipedia - Potentials and PitfallsScience (Communication) and Wikipedia - Potentials and Pitfalls
Science (Communication) and Wikipedia - Potentials and PitfallsDobusch Leonhard
 
6.2 Pests of Sesame_Identification_Binomics_Dr.UPR
6.2 Pests of Sesame_Identification_Binomics_Dr.UPR6.2 Pests of Sesame_Identification_Binomics_Dr.UPR
6.2 Pests of Sesame_Identification_Binomics_Dr.UPRPirithiRaju
 
DNA isolation molecular biology practical.pptx
DNA isolation molecular biology practical.pptxDNA isolation molecular biology practical.pptx
DNA isolation molecular biology practical.pptxGiDMOh
 
Think Science: What Are Eclipses, by Craig Bobchin
Think Science: What Are Eclipses, by Craig BobchinThink Science: What Are Eclipses, by Craig Bobchin
Think Science: What Are Eclipses, by Craig BobchinNathan Cone
 
Speed Breeding in Vegetable Crops- innovative approach for present era of cro...
Speed Breeding in Vegetable Crops- innovative approach for present era of cro...Speed Breeding in Vegetable Crops- innovative approach for present era of cro...
Speed Breeding in Vegetable Crops- innovative approach for present era of cro...jana861314
 
DETECTION OF MUTATION BY CLB METHOD.pptx
DETECTION OF MUTATION BY CLB METHOD.pptxDETECTION OF MUTATION BY CLB METHOD.pptx
DETECTION OF MUTATION BY CLB METHOD.pptx201bo007
 
AICTE activity on Water Conservation spreading awareness
AICTE activity on Water Conservation spreading awarenessAICTE activity on Water Conservation spreading awareness
AICTE activity on Water Conservation spreading awareness1hk20is002
 
Anatomy of Duodenum- detailed ppt presentation for science students
Anatomy of Duodenum- detailed ppt presentation for science studentsAnatomy of Duodenum- detailed ppt presentation for science students
Anatomy of Duodenum- detailed ppt presentation for science studentsgargipatil3210
 
Environmental acoustics- noise criteria.pptx
Environmental acoustics- noise criteria.pptxEnvironmental acoustics- noise criteria.pptx
Environmental acoustics- noise criteria.pptxpriyankatabhane
 
6.1 Pests of Groundnut_Binomics_Identification_Dr.UPR
6.1 Pests of Groundnut_Binomics_Identification_Dr.UPR6.1 Pests of Groundnut_Binomics_Identification_Dr.UPR
6.1 Pests of Groundnut_Binomics_Identification_Dr.UPRPirithiRaju
 
ESSENTIAL FEATURES REQUIRED FOR ESTABLISHING FOUR TYPES OF BIOSAFETY LABORATO...
ESSENTIAL FEATURES REQUIRED FOR ESTABLISHING FOUR TYPES OF BIOSAFETY LABORATO...ESSENTIAL FEATURES REQUIRED FOR ESTABLISHING FOUR TYPES OF BIOSAFETY LABORATO...
ESSENTIAL FEATURES REQUIRED FOR ESTABLISHING FOUR TYPES OF BIOSAFETY LABORATO...Chayanika Das
 
Introduction of Organ-On-A-Chip - Creative Biolabs
Introduction of Organ-On-A-Chip - Creative BiolabsIntroduction of Organ-On-A-Chip - Creative Biolabs
Introduction of Organ-On-A-Chip - Creative BiolabsCreative-Biolabs
 
Environmental Acoustics- Speech interference level, acoustics calibrator.pptx
Environmental Acoustics- Speech interference level, acoustics calibrator.pptxEnvironmental Acoustics- Speech interference level, acoustics calibrator.pptx
Environmental Acoustics- Speech interference level, acoustics calibrator.pptxpriyankatabhane
 
Efficient Fourier Pricing of Multi-Asset Options: Quasi-Monte Carlo & Domain ...
Efficient Fourier Pricing of Multi-Asset Options: Quasi-Monte Carlo & Domain ...Efficient Fourier Pricing of Multi-Asset Options: Quasi-Monte Carlo & Domain ...
Efficient Fourier Pricing of Multi-Asset Options: Quasi-Monte Carlo & Domain ...Chiheb Ben Hammouda
 
Remote patient monitoring :Health care transformation
Remote patient monitoring :Health care transformationRemote patient monitoring :Health care transformation
Remote patient monitoring :Health care transformationfahad Alotaibiu
 
Loudspeaker- direct radiating type and horn type.pptx
Loudspeaker- direct radiating type and horn type.pptxLoudspeaker- direct radiating type and horn type.pptx
Loudspeaker- direct radiating type and horn type.pptxpriyankatabhane
 
Total Legal: A “Joint” Journey into the Chemistry of Cannabinoids
Total Legal: A “Joint” Journey into the Chemistry of CannabinoidsTotal Legal: A “Joint” Journey into the Chemistry of Cannabinoids
Total Legal: A “Joint” Journey into the Chemistry of CannabinoidsMarkus Roggen
 

Recently uploaded (20)

Production technology of Brinjal -Solanum melongena
Production technology of Brinjal -Solanum melongenaProduction technology of Brinjal -Solanum melongena
Production technology of Brinjal -Solanum melongena
 
CDS Fundamentals of digital communication system UNIT 1 AND 2.pdf
CDS Fundamentals of digital communication system UNIT 1 AND 2.pdfCDS Fundamentals of digital communication system UNIT 1 AND 2.pdf
CDS Fundamentals of digital communication system UNIT 1 AND 2.pdf
 
Science (Communication) and Wikipedia - Potentials and Pitfalls
Science (Communication) and Wikipedia - Potentials and PitfallsScience (Communication) and Wikipedia - Potentials and Pitfalls
Science (Communication) and Wikipedia - Potentials and Pitfalls
 
6.2 Pests of Sesame_Identification_Binomics_Dr.UPR
6.2 Pests of Sesame_Identification_Binomics_Dr.UPR6.2 Pests of Sesame_Identification_Binomics_Dr.UPR
6.2 Pests of Sesame_Identification_Binomics_Dr.UPR
 
DNA isolation molecular biology practical.pptx
DNA isolation molecular biology practical.pptxDNA isolation molecular biology practical.pptx
DNA isolation molecular biology practical.pptx
 
Think Science: What Are Eclipses, by Craig Bobchin
Think Science: What Are Eclipses, by Craig BobchinThink Science: What Are Eclipses, by Craig Bobchin
Think Science: What Are Eclipses, by Craig Bobchin
 
Speed Breeding in Vegetable Crops- innovative approach for present era of cro...
Speed Breeding in Vegetable Crops- innovative approach for present era of cro...Speed Breeding in Vegetable Crops- innovative approach for present era of cro...
Speed Breeding in Vegetable Crops- innovative approach for present era of cro...
 
DETECTION OF MUTATION BY CLB METHOD.pptx
DETECTION OF MUTATION BY CLB METHOD.pptxDETECTION OF MUTATION BY CLB METHOD.pptx
DETECTION OF MUTATION BY CLB METHOD.pptx
 
AICTE activity on Water Conservation spreading awareness
AICTE activity on Water Conservation spreading awarenessAICTE activity on Water Conservation spreading awareness
AICTE activity on Water Conservation spreading awareness
 
Anatomy of Duodenum- detailed ppt presentation for science students
Anatomy of Duodenum- detailed ppt presentation for science studentsAnatomy of Duodenum- detailed ppt presentation for science students
Anatomy of Duodenum- detailed ppt presentation for science students
 
Environmental acoustics- noise criteria.pptx
Environmental acoustics- noise criteria.pptxEnvironmental acoustics- noise criteria.pptx
Environmental acoustics- noise criteria.pptx
 
6.1 Pests of Groundnut_Binomics_Identification_Dr.UPR
6.1 Pests of Groundnut_Binomics_Identification_Dr.UPR6.1 Pests of Groundnut_Binomics_Identification_Dr.UPR
6.1 Pests of Groundnut_Binomics_Identification_Dr.UPR
 
ESSENTIAL FEATURES REQUIRED FOR ESTABLISHING FOUR TYPES OF BIOSAFETY LABORATO...
ESSENTIAL FEATURES REQUIRED FOR ESTABLISHING FOUR TYPES OF BIOSAFETY LABORATO...ESSENTIAL FEATURES REQUIRED FOR ESTABLISHING FOUR TYPES OF BIOSAFETY LABORATO...
ESSENTIAL FEATURES REQUIRED FOR ESTABLISHING FOUR TYPES OF BIOSAFETY LABORATO...
 
Introduction of Organ-On-A-Chip - Creative Biolabs
Introduction of Organ-On-A-Chip - Creative BiolabsIntroduction of Organ-On-A-Chip - Creative Biolabs
Introduction of Organ-On-A-Chip - Creative Biolabs
 
Introduction Classification Of Alkaloids
Introduction Classification Of AlkaloidsIntroduction Classification Of Alkaloids
Introduction Classification Of Alkaloids
 
Environmental Acoustics- Speech interference level, acoustics calibrator.pptx
Environmental Acoustics- Speech interference level, acoustics calibrator.pptxEnvironmental Acoustics- Speech interference level, acoustics calibrator.pptx
Environmental Acoustics- Speech interference level, acoustics calibrator.pptx
 
Efficient Fourier Pricing of Multi-Asset Options: Quasi-Monte Carlo & Domain ...
Efficient Fourier Pricing of Multi-Asset Options: Quasi-Monte Carlo & Domain ...Efficient Fourier Pricing of Multi-Asset Options: Quasi-Monte Carlo & Domain ...
Efficient Fourier Pricing of Multi-Asset Options: Quasi-Monte Carlo & Domain ...
 
Remote patient monitoring :Health care transformation
Remote patient monitoring :Health care transformationRemote patient monitoring :Health care transformation
Remote patient monitoring :Health care transformation
 
Loudspeaker- direct radiating type and horn type.pptx
Loudspeaker- direct radiating type and horn type.pptxLoudspeaker- direct radiating type and horn type.pptx
Loudspeaker- direct radiating type and horn type.pptx
 
Total Legal: A “Joint” Journey into the Chemistry of Cannabinoids
Total Legal: A “Joint” Journey into the Chemistry of CannabinoidsTotal Legal: A “Joint” Journey into the Chemistry of Cannabinoids
Total Legal: A “Joint” Journey into the Chemistry of Cannabinoids
 

Koyama ASA ASJ joint meeting 2016

  • 1. Super-resolution in sound field recording and reproduction based on sparse representation Shoichi Koyama1,2, Naoki Murata1, and Hiroshi Saruwatari1 1The University of Tokyo 2Paris Diderot University / Institute Langevin
  • 2. November 29, 2016 Sound field reproduction for audio system Microphone array Loudspeaker array  Large listening area can be achieved  Listeners can perceive source distance  Real-time recording and reproduction can be achieved without recording engineers Recording area Target area
  • 3. November 29, 2016 Sound field reproduction for audio system Microphone array Loudspeaker array Telecommunication system NW Home Theatre Live broadcasting Applications Recording area Target area
  • 4. November 29, 2016 Sound field reproduction for audio system Microphone array Loudspeaker array Improve reproduction accuracy when # of array elements is small  # of microphones > # of loudspeakers – Higher reproduction accuracy within local region of target area  # of microphones < # of loudspeakers – Higher reproduction accuracy of sources in local region of recording area [Koyama+ IEEE JSTSP2015], [Koyama+ ICASSP 2014, 2015] [Ahrens+ AES Conv. 2010], [Ueno+ ICASSP 2017 (submitted)] Recording area Target area
  • 5. Sound Field Recording and Reproduction November 29, 2016 Recording area Target area Obtain driving signals of secondary sources (= loudspeakers) arranged on to reconstruct desired sound field inside  Inherently, sound pressure and its gradient on is required to obtain , but sound pressure is usually only known  Signal conversion for sound field recording and reproduction with ordinary acoustic sensors and transducers is necessary Primary sources
  • 6. November 29, 2016 Conventional: WFR filtering method Recording area Target area Secondary source planeReceiving plane Primary sources Signal conversion [Koyama+ IEEE TASLP 2013] Received signals Driving signals Plane wave Plane wave Each plane wave determines entire sound field Signal conversion can be achieved in spatial frequency domain
  • 7. November 29, 2016 Conventional: WFR filtering method Target area Received signals Driving signals Plane wave Plane wave Each plane wave determines entire sound field Spatial aliasing artifacts due to plane wave decomposition Significant error at high freq even when microphone < loudspeaker Recording area Signal conversion Secondary source planeReceiving plane Primary sources [Koyama+ IEEE TASLP 2013]
  • 8. Sound field representation for super-resolution  Plane wave decomposition suffers from spatial aliasing artifacts because many basis functions are used  Observed signals should be represented by a few basis functions for accurate interpolation of sound field  Appropriate basis function may be close to pressure distribution originating from sound sources  To obtain driving signals of loudspeakers, basis functions must be fundamental solutions of Helmholtz equation (e.g. Green functions) November 29, 2016 Basis functionReceived signals Sound field decomposition into fundamental solutions of Helmholtz equation is necessary Sound field decomposition
  • 9. Generative model of sound field  Inhomogeneous and homogeneous Helmholtz eq. Distribution of source components November 29, 2016 [Koyama+ ICASSP 2014] Sound field is divided into two regions
  • 10. Generative model of sound field  Inhomogeneous and homogeneous Helmholtz eq. November 29, 2016 [Koyama+ ICASSP 2014] Green’s function Inhomogeneous + homogeneous terms Plane wave
  • 11. November 29, 2016 Generative model of sound field  Observe sound pressure distribution on plane  Conversion into driving signals Synthesize monopole sources [Spors+ AES Conv. 2008] Ambient componentsDirect source components Applying WFR filtering method [Koyama+ IEEE TASLP 2013] Decomposition into two components can lead to higher reproduction accuracy above spatial Nyquist freq
  • 12. November 29, 2016 Sparse sound field representation ・・・・・・・・ Microphone array Source components Grid points Sparsity-based signal decomposition Discretization Ambient components Dictionary matrix of Green’s functions Observed signal Distribution of source components A few elements of has non-zero values under the assumption of spatially sparse source distribution
  • 13. Sparse signal decomposition  Sparse signal representation in vector form  Signal decomposition based on sparsity of November 29, 2016 Minimize -norm of
  • 14. Group sparsity based on physical properties November 29, 2016 Group sparse signal models for robust decomposition • Multiple time frames • Temporal frequencies • Multipole components Decomposition algorithm extending FOCUSS [Koyama+ ICASSP 2015]  Sparse signal representation in vector form Structure of sparsity induced by physical properties
  • 15. Block diagram of signal conversion  Decomposition stage – Group sparse decomposition of  Reconstruction stage – and are respectively converted into driving signals – is obtained as sum of two components November 29, 2016
  • 16. Simulation Experiment  Proposed method (Proposed), WFR filtering method (WFR), and Sound Pressure Control method (SPC) were compared  32 microphones (6 cm intervals)and 48 loudspeakers (4 cm intervals)  : Rectangular region of 2.4x2.4 m, Grid points: (10cm, 20cm) intervals  Source directivity: unidirectional  Source signal: single frequency sinewave Recording area Target area November 29, 2016
  • 17. Simulation Experiment  Signal-to-distortion ratio of reproduction (SDRR) Recording area Target area November 29, 2016 Original pressure distribution Reproduced pressure distribution
  • 18. November 29, 2016 Frequency vs. SDR SDRRs above spatial Nyquist frequency were improved  Source location: (-0.32, -0.84, 0.0) m
  • 19. Reproduced sound pressure distribution (1.0 kHz)PressureError November 29, 2016 Proposed WFR SPC 18.1 dB 18.0 dB 19.4 dB  Source location: (-0.32, -0.84, 0.0) m SDRR:
  • 20. Reproduced sound pressure distribution (4.0 kHz)PressureError November 29, 2016 19.7 dB 6.8 dB 7.8 dB Proposed WFR SPC  Source location: (-0.32, -0.84, 0.0) m SDRR:
  • 21. Frequency response of reproduced sound field November 29, 2016  Frequency response at (0.0, 1.0, 0.0) m Reproduced frequency response was improved
  • 22. Conclusion  Super-resolution sound field recording and reproduction based on sparse representation – Conventional plane wave decomposition is suffered from spatial aliasing artifacts – Sound field representation using source and plane wave components – Sound field decomposition based on spatial sparsity of source components – Group sparsity based on physical properties of sound field – Experimental results indicated that reproduction accuracy above spatial Nyquist frequency can be improved November 29, 2016 Thank you for your attention!
  • 23. Related publications • S. Koyama and H. Saruwatari, “Sound field decomposition in reverberant environment using sparse and low-rank signal models,” Proc. IEEE ICASSP, 2016. • N. Murata, S. Koyama, et al. “Sparse sound field decomposition with multichannel extension of complex NMF,” Proc. IEEE ICASSP, 2016. • S. Koyama, et al. “Sparse sound field decomposition using group sparse Bayesian learning,” Proc. APSIPA ASC, 2015. • N. Murata, S. Koyama, et al. “Sparse sound field decomposition with parametric dictionary learning for super-resolution recording and reproduction,” Proc. IEEE CAMSAP, 2015. • S. Koyama, et al. “Source-location-informed sound field recording and reproduction with spherical arrays,” Proc. IEEE WASPAA, 2015. • S. Koyama, et al. “Source-location-informed sound field recording and reproduction,” IEEE J. Sel. Topics Signal Process., vol. 9, no. 5, pp. 881-894, 2015. • S. Koyama, et al. “Structured sparse signal models and decomposition algorithm for super-resolution in sound field recording and reproduction,” Proc. IEEE ICASSP, 2015. • S. Koyama, et al. “Sparse sound field representation in recording and reproduction for reducing spatial aliasing artifacts,” Proc. IEEE ICASSP, 2014. November 29, 2016

Editor's Notes

  1. These are the reproduced sound pressure and time-averaged squared error distributions of the three methods at 1 kHz. The white region indicates the region of high reproduction accuracy. Because 1.0 kHz is below the spatial Nyquist frequency, the sound field was accurately reproduced using these three methods.