The document proposes a computationally efficient method for calculating the Maximum Mean Discrepancy (MMD) between two distributions based on random Fourier features. It shows that by using random features to approximate the kernel, the MMD can be computed in O(BM^2 + B^2) time, where B is the batch size and M is the number of random features, providing a significant speedup over direct kernel computation which takes O(B^3) time. It applies this efficient MMD to the task of voice conversion and demonstrates it achieves better voice quality and naturalness compared to other baselines, as measured by objective metrics and a mean opinion score user study.

2. ‣
‣

3. ‣
‣
O(N3
)
GMMN CMMD

5. {yi} {˜yi}
ϕ(y) ϕ(y)
RKHS RKHS
μ = Cψ(x)
𝔼[ ⋅ ]
˜μ = ˜Cψ(x)
𝔼[ ⋅ ]
x
ψ(x)
RKHS
P(Y|x)
P( ˜Y|x)
CMMD2
= ∥C − ˜C∥2

6. CMMD2
= ∥C − ˜C∥2
̂CMMD2
= Tr [(KY,Y + K˜Y, ˜Y − 2KY, ˜Y)(H + λI)−1
H(H + λI)−1
]
‣
‣
O(N2
)
O(N3
)

7. ‣
‣
‣
CMMD2
= Tr [(KY,Y + K˜Y, ˜Y − 2KY, ˜Y)(H + λI)−1
H(H + λI)−1
]
O(B3
)

8.
kRBF(x, x′) = (exp( −∥x − x′∥2
/2) kRBF(x, x′) ≈
1
M
M
∑
r=1
cos(x⊤
ωr + br)cos(x′⊤
ωr + br)
RBF (RFF)
-1.0
1.0
0.0
-1.0
1.0
0.0
N=1000 M=100

9. ‣
‣
‣
CMMD2
= Tr [(KY,Y + K˜Y, ˜Y − 2KY, ˜Y)(H + λI)−1
H(H + λI)−1
]
O(BM2
+ B2
)

10. ‣
‣

12. ‣
‣
‣

13. 95% confidence interval
p<0.05 p<0.001
MSE
1 2 3 4 5
LOCAL-RAND
LOCAL-CLST
RFF-RAND
RFF-CLST
VOC
Score
(1: , 5: )
2

14. 0 MGC 1 MGC
F0
[cent] [ms] MOS
LOCAL-RAND 0.023 0.012 15.8 2.46 1.61
LOCAL-CLST 0.053 0.022 18.2 3.50 1.71
RFF-RAND 0.021 0.007 1.5 3.77 1.73
RFF-CLST 0.049 0.027 14.0 5.47 1.94

15. ‣
‣
‣

17. 1
MSE
Score
95% confidence interval
p<0.01
LOCAL-RAND
LOCAL-CLST
RFF-RAND
RFF-CLST
VOC
2 3 4 5
(1: 5: )