The document provides several examples of machine learning techniques, including: i) Examples of applying K-SVD dictionary learning to signals with noise and comparing it to other methods like FFT. ii) Examples of using K-SVD regression to predict future values and comparing its performance to other models like k-NN and local AR models. iii) Further examples of K-SVD for tasks like source separation and its use in other algorithms for problems like system identification and time series prediction.

1. @koh_t

2. today’s menu

3. Machine Learning ?

4. Machine Learning ?

5. Machine Learning ?
, ,
???

6. Machine Learning ?
...

7. Machine Learning ?
... !!

8. Machine Learning ?

9. Machine Learning ?
= { , , }
={ , , }

10. for an example...
Q. ? ( )

11. for an example...
Q. ? ( )
…?
PC: he is ... ... ...
…
id: Kan id: poppo
PC: he is Naoto Kan!

12. for another example...
Q. ?

13. for another example...
Q. ?
…?
PC: ... ...
PC: 1$ = 85 ± 0.25

14. many examples...

15. many examples...
…

16. many examples...
…

21. Quotes Daily Data
240
130
110
IBM
90
230
70
200
220
Apple
100
E
50
210
35
30
Microsoft
25
200
20
13:00
40 15
190
30
Dell
20
10
Hour
2005 2006 2007 2008 2009 2010
Index
( )
0 200 400 600 800 1000
0 200 400 600 800 1000
0 200 400 600 800 1000
0 200 400 600 800 1000
0 200 400 600 800 1000
0 200 400 600 800 1000
0 200 400 600 800 1000
0 200 400 600 800 1000
0 200 400 600 800 1000

23. 1 m
1
n×m
X∈X ⊂R
n

24. 1 m
1
n×m
X∈X ⊂R
n m ?
n

25. many examples...

26. many examples...
” ”

27. many examples...
” ”
OK

30. X A,B
X f(X)

31. X A,B
CX = max p(Ck |X)
k
X f(X)
f (X) = argmin ||Y − f (X)||F
f

32. X A,B
CX = max p(Ck |X)
k
X f(X)
f (X) = argmin ||Y − f (X)||F
f

34. my examples...

35. my examples...
i) 1
ii) 1
iii) -5[V], -5 5[V] , +5[V]

36. my examples...
i) 1
ii) 1
iii) -5[V], -5 5[V] , +5[V]
ii) iii)
OK?

37. my bachelor examples...
Monthly Data Daily Data
240
250
240
230
230
220
220
E
E
210
210
200
200
June 15th 13:00
190
190
Day Hour
…

38. my bachelor examples...
Daily Data
hourly Data
240
225
230
220
220
E
210
E
215
200
13:00
210
190
13:00 13:30 14:00
205
Hour
10 Sec
…

39. my bachelor examples...
…
AR: x = (x(1), . . . , x(t))T ∈ X ⊂ Rt
a = (a(1), . . . , a(t))T ∈ A ⊂ Rt
x(t + 1) = a x + N (0, σ )
T 2
0 t t+1

40. my bachelor examples...
…
AR: x = (x(1), . . . , x(t))T ∈ X ⊂ Rt
ARIMA Example
a = (a(1), . . . , a(t))T ∈ A ⊂ Rt
230
observed value
x(t + 1) = a x + N (0, σ )
predicted value
T
2*SE 2
225
220
215
210
205
0 0 5 10 15 t t+1
20 25 30 35
Time

41. my bachelor examples...
i) ˜
X Xr
ii) AR &
X ar = g(Xr )
x(t + 1) = ar x + N (0, σ )
˜ T 2
|˜(t + 1) − x(t)| → Ck
x ˜
Xr
˜
X

42. my bachelor examples...
i) ˜
X Xr
ii) AR &
X ar = g(Xr )
x(t + 1) = ar x + N (0, σ )
˜ T 2
|˜(t + 1) − x(t)| → Ck
x ˜
Xr
˜
X
…

43. my bachelor examples...
Monthly Data Daily Data
240
250
240
230
230
220
220
E
E
210
210
200
200
June 15th 13:00
190
190
Day Hour

44. my bachelor examples...
…
FFT:
K-SVD:

45. my bachelor examples...
…
!"#$%
&'()* FFT: !"#$%
&'()*
K.TAKEUCHI k-SVD(GH<GI<'JKL) k-SVD(GH<GI<'JKL)
+,-(
K-SVD: K.TAKEUCHI
1
+,-(
2
1 2
1.0
1.0
!0.214
./ ./ MN6OPQRST k U MNOPQRS
!0.216
!0.215
0.5
0.5
01 01 DVWQXY. UV6'<+D
!0.218
0.0
0.0
!0.220
E
E
E
E
2301 2301
!0.220
.4 .4
ex)FFT Z[]^,weblet TMNRWX
!0.5
!0.5
!0.222
!0.225
567 567 !0.224
_`Z[abD weblet Z YZ[]'<
!1.0
!1.0
0 20 40 60 80 100 0 20 40 60 80 100 5 10 15 20 5 10 15 20
89 89
RS
Time Time Time Time
k-NN 3 k-NN
4 3 4 Q_`XDaO
Local AR
k-SVD [cdDVWQe
!0.210
Local AR
!0.205
ex) fgh, ij
1.0
1.0
k-SVD k-SVD
!0.210
fghi
0.5
0.5
!0.215
:;'<+ :;'<+
!0.215
E
E
0.0
0.0
E
E
!0.220
5=>? 5=>?
!0.220
!0.5
!0.5
!0.225
@A @A
!0.225
!1.0
!1.0
BCDEF
5 10 15 20 5 10 15 20
BCDEF 0 20 40
Time
60 80 100 0 20 40
Time
60 80 100
Time Time
FFT
FFT K-SVD
Dictionary

46. my bachelor examples...
K-SVD:
i) D
argmin ||X − 2
DZ||F s.t. ∀i ||zi || ≤ C0
D,Z
ii) ˜
X D
$%
)*
UCHI
k-SVD(GH<GI<'JKL)
1 2
|˜(t − 1) − x(t)| k-SVD(GH<GI<'JKL)
x ˜
!"#$%
&'()*
→ Ck
!0.214
MNOPQRST K.TAKEUCHI
!0.216
×
!0.215
UV6'<+DPQRS
!0.218
+,-( k!SVD regression
!0.220
E
E
!0.220
TMNRWX ./ MNOPQRSTUV
!0.222
!0.225
01 WX
!0.224
YZ[]'<+^SP
200
2301
.4 k YDPQDZ[+
D
5 10 15 20 5 10 15 20
Time Time
3 4 Q_`XDaObcde
567
]^-_`aObPQ
150
89
YcdVef
!0.210
E
!0.205
ex) fgh, ij, kl
k-NN
Local AR
PQU l+1 RgXV
!0.210
!0.215
k-SVD
+
X!_h 1-l Dijk
100
!0.215
:;'<+
E
E
ma+<]^-n`
!0.220
!0.220
5=>?
ObPQUopqrs
!0.225
@A
50
!0.225
VWXSTD l+1 Di
0 5 10 15 20 25 30
BCDEF Time
F
5 10 15 20 5 10 15 20 k=60,T=18
Time Time
k-SVD Utuvnwaxyz

47. my bachelor examples...
K-SVD:
i) D
argmin ||X − 2
DZ||F s.t. ∀i ||zi || ≤ C0
D,Z
ii) ˜
X D
$%
)*
UCHI
k-SVD(GH<GI<'JKL)
1 2
|˜(t − 1) − x(t)| k-SVD(GH<GI<'JKL)
x ˜
!"#$%
&'()*
→ Ck
!0.214
MNOPQRST K.TAKEUCHI
!0.216
×
!0.215
UV6'<+DPQRS
!0.218
+,-( k!SVD regression
!0.220
E
E
!0.220
TMNRWX ./ MNOPQRSTUV
!0.222
!0.225
01 WX
!0.224
YZ[]'<+^SP
200
2301
.4 k YDPQDZ[+
D
5 10 15 20 5 10 15 20
Time Time
3 4 Q_`XDaObcde
567
]^-_`aObPQ
150
89
YcdVef
!0.210
E
!0.205
ex) fgh, ij, kl
k-NN
Local AR
PQU l+1 RgXV
!0.210
!0.215
k-SVD
+
X!_h 1-l Dijk
100
!0.215
:;'<+
E
E
ma+<]^-n`
!0.220
!0.220
5=>?
ObPQUopqrs
!0.225
@A
50
!0.225
VWXSTD l+1 Di
0 5 10 15 20 25 30
BCDEF Time
F
5 10 15 20 5 10 15 20 k=60,T=18
Time Time
k-SVD Utuvnwaxyz

48. my examples...

49. my examples...
K-SVD

50. my examples...
K-SVD

51. my examples...
K-SVD
cf: no free lunch theorem

52. @koh_t