This 20-minute talk was presented during the Operator Splitting Workshop

1. THE FORWARD-BACKWARD SPLITTING
ALGORITHM WITHOUT COCOERCIVITY?
Saverio Salzo
Istituto Italiano di Tecnologia
Genova, Italy
Workshop on Operator Splitting Methods in Data Analysis
Samsi, Durham, NC, March, 21-23, 2018

2. is a real Hilbert space with scalar product
is maximal monotone
is -cocoercive, that is
ZEROS OF SUM OF MONOTONE OPERATORS
The problem
H h· , ·i
A: H ! 2H
B : H ! H
find ¯x 2 H s.t. 0 2 A(¯x) + B(¯x)
kB(x) B(y)k2
 hx y, B(x) B(y)i

3. is a real Hilbert space with scalar product
is maximal monotone if is
proper, convex, and lower semicontinuous.
is -cocoercive iff it is -Lipschitz
continuous.
MINIMIZATION OF SUM OF FUNCTIONS
The problem
h· , ·i
find ¯x 2 H s.t. 0 2 @f(¯x) + rg(¯x)
@f : H ! 2H f : H ! ] 1, +1]
rg: H ! H (1/ )
H

4. is a real Hilbert space with scalar product
is maximal monotone if is
nonempty, closed, and convex.
is -cocoercive.
VARIATIONAL INEQUALITIES
The problem
find ¯x 2 C s.t. 8 x 2 C h¯x x, B¯xi  0
A = @iC = NC C ⇢ H
B : H ! H
h· , ·iH

5. THE FORWARD-BACKWARD SPLITTING
(Mercier ’79)
The algorithm
if and , then
xk+1 = J kA(xk kB(xk))
Convergence: if k < 2
xk * ¯x 2 zer(A + B)
A = @f B = rg (f + g)(xk) inf(f + g) = o(1/k)
Cocoercivity of is a fundamental assumption!B

6. minimize
x2Rd
+
DKL(b, Ax) + ⌧kxk1
minimize
x2Rd
1
p
kAx bkp
p + ⌧kxk1 (p > 1)
minimize
w2Rd
nX
i=1
L(yi, hw, xii) +
⌧
p
kwkp
p (1 < p < 2)
When cocoercivity is an issue

7. minimize
x2Rd
1
p
kAx bkp
p + ⌧kxk1 (p > 1)
minimize
↵2Rn
1
q
kX⇤
↵kq
q +
1
⌧
nX
i=1
L⇤
(yi, ⌧↵i) (q > 2)
When cocoercivity is an issue
minimize
x2Rd
+
DKL(b, Ax) + ⌧kxk1

8. TSENG’S SPLITTING
(Tseng ’00)
is maximal monotone
is uniformly continuous on bounded sets
is chosen by the backtracking procedure: let
B : H ! H
k
kB(yk) B(xk)k 
k
kyk xkk
2 ]0, 1[
The algorithm converges if
A: H ! 2H
yk = J kA(xk kB(xk))
xk+1 = yk kB(yk) + kB(xk)
Strategies without cocoercivity

9. , is proper, convex, and lsc
is uniformly continuous on bounded sets.
is determined by backtracking line search procedures
(a priori choices are not possible anymore).
THE FORWARD-BACKWARD SPLITTING
FOR MINIMIZATION PROBLEMS
xk+1 = J kA(xk kB(xk))
B = rg
A = @f f : H ! ] 1, +1]
k
The algorithm converges if
S. Salzo, The variable metric forward-backward splitting algorithm under mild
differentiability assumptions. SIOPT 2017.
Strategies without cocoercivity

10. BACKTRACKING LINE SEARCHES
L1)
L2)
L3)
We considered three possible inequalities: let
krg(xk+1) rg(xk)k 
k
kxk+1 xkk
g(xk+1) g(xk) hxk+1 xk, rg(xk)i 
k
kxk+1 xkk2
(f + g)(xk+1) (f + g)(xk)
 (1 ) f(xk+1) f(xk) + hxk+1 xk, rg(xk)i
2 ]0, 1[
L1) L2) L3)
L1) is not quite appropriate for FB algorithm since it leads
to halve the step-sizes w.r.t. L2) and L3)
=) =)
Strategies without cocoercivity

11. FBA FBFA
They work under exactly the same
assumptions!
MINIMIZATION PROBLEMS
Strategies without cocoercivity

12. THE PROBLEM: DEVISE AN APPROPRIATE
BACKTRACKING LINESEARCH THAT REPLACES
THE COCOERCIVITY
What about the FB algorithm for
monotone operators?
Removing cocoercivity

13. Where does cocoercivity enter?
kxk+1 ¯xk2
 kxk ¯xk2
kxk+1 xkk2
2 khxk ¯x, B(xk) B(¯x)i
2 khxk+1 xk, B(xk) B(¯x)i
= kxk ¯xk2
k(2 k)kB(xk) B(¯x)k2
k(xk+1 xk) k(B(xk) B(¯x))k2
FB ALGORITHM FOR MONOTONE OPERATORS

14. Where does cocoercivity enter?
FB ALGORITHM FOR MINIMIZATION PROBLEMS
kxk+1 xk2
 kxk xk2
kxk+1 xkk2
+ 2 k (f + g)(x) (f + g)(xk)
2 k g(xk+1) g(xk) + hxk+1 xk, rg(xk)i
 kxk xk2
+ (2 1)kxk+1 xkk2
+ 2 k (f + g)(x) (f + g)(xk)
+ 2 k (f + g)(xk) (f + g)(xk+1)

15. Possible strategies
only value of B?
would not work!
the values of function (to be determined), that hopefully
should decrease along the iterations?
k
kB(xk+1) B(xk)k2
 hB(xk+1) B(xk), xk+1 xki
The inequality should involve:

16. ,
Possible strategies
PRIMAL DUAL ALGORITHMS
min
x2H
g(x) + h(Lx)
✓
0
0
◆
2 A
✓
¯x
¯v
◆
+ B
✓
¯x
¯v
◆
A
✓
x
v
◆
=
✓
L⇤
v
@h⇤
(v) Lx
◆
B
✓
x
v
◆
=
✓
rg(x)
0
◆
is convex and continuously differentiable.g: H ! R
(Combettes-Pesquet ’12, Condat ’13, Vu ’13)
Should backtracking involve the duality gap function?

17. Thank you!