Least Square with L0, L1, and L2 Constraint

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Masayuki Tanaka
Jun. 22, 2016
Least Square with
L0, L1, and L2 Constraint

Cost Functions
𝐸2 𝑥 = 𝑦 − 𝑥 2 + 𝜆2 𝑥 2 𝑥 2 = 𝑥2
𝐸1 𝑥 = 𝑦 − 𝑥 2 + 𝜆1 𝑥 1 𝑥 1 =
𝑥 (𝑥 ≥ 0)
−𝑥 (𝑥 < 0)
𝐸0 𝑥 = 𝑦 − 𝑥 2 + 𝜆0 𝑥 0...

Least Square with L0 constraint
𝐸0 𝑥 = 𝑦 − 𝑥 2 + 𝜆0 𝑥 0 𝑥 0 =
0 (𝑥 = 0)
1 (𝑥 ≠ 0)
𝑥0 =
0 (𝑦2 < 𝜆0)
𝑦 (𝑦2
≥ 𝜆0)
Hard thresh...

Least Square with L1 constraint
𝑥1 = sign 𝑦 𝑦 − 𝜆1/2 +
sign 𝜉 =
−1 (𝜉 < 0)
0 (𝜉 = 0)
1 (𝜉 > 0)
𝜉 + = max 𝜉, 0 =
𝜉 (𝜉 > 0)
...

Least Square with L2 constraint
𝜕𝐸2
𝜕𝑥
= 𝑥 − 𝑦 + 𝜆2 𝑥 = 0
𝑥2 =
𝑦
1 + 𝜆2
𝐸2 𝑥 = 𝑦 − 𝑥 2
+ 𝜆2 𝑥 2 𝑥 2 = 𝑥2
Solution:

Derivation of
Least Square with L0 constraint
𝐸0 𝑥 = 𝑦 − 𝑥 2 + 𝜆0 𝑥 0 𝑥 0 =
0 (𝑥 = 0)
1 (𝑥 ≠ 0)
𝑥 = 0 𝑥 ≠ 0
𝐸0 0 = y2 𝐸0 𝑥...

Formulation as a Proximal Operator
prox 𝐿0,𝜌 𝑦 = arg min
𝑥
𝑥 0 +
𝜌
2
𝑦 − 𝑥 2 𝑥 0 =
0 (𝑥 = 0)
1 (𝑥 ≠ 0)
=
0, 𝑦2
<
2
𝜌
1, 𝑦2...

Formulation as a Proximal Operator
prox 𝐿1,𝜌 𝑦 = arg min
𝑥
𝑥 1 +
𝜌
2
𝑦 − 𝑥 2 𝑥 1 =
𝑥 (𝑥 ≥ 0)
−𝑥 (𝑥 < 0)
= sign 𝑦 𝑦 − 1/𝜌 +...

Share Slideshare

LinkedIn
Facebook
Twitter

Embed

Size (px)

Show related Slideshows at end

WordPress Shortcode

Link

Share
Email

ADMM algorithm in ProxImaL by Masayuki Tanaka 1976 views

View on Slideshare

1 of 8 Ad

View on Slideshare

1 of 8 Ad