The document discusses the forward-backward splitting algorithm for finding the zeros of the sum of maximal monotone operators. It notes that the standard convergence analysis relies on one of the operators being cocoercive. It proposes investigating strategies for the algorithm that do not require cocoercivity, such as using backtracking line searches to determine the step size or involving the duality gap function in the backtracking condition. Removing the cocoercivity assumption would expand the applicability of the forward-backward splitting method.

1. THE FORWARD-BACKWARD SPLITTING
ALGORITHM WITHOUT COCOERCIVITY?
Saverio Salzo
Istituto Italiano di Tecnologia
Genova, Italy
Workshop on Operator Splitting Methods in Data Analysis
Samsi, Durham, NC, March, 21-23, 2018

2. is a real Hilbert space with scalar product
is maximal monotone
is -cocoercive, that is
ZEROS OF SUM OF MONOTONE OPERATORS
The problem
H h· , ·i
A: H ! 2H
B : H ! H
find ¯x 2 H s.t. 0 2 A(¯x) + B(¯x)
kB(x) B(y)k2
 hx y, B(x) B(y)i

3. is a real Hilbert space with scalar product
is maximal monotone if is
proper, convex, and lower semicontinuous.
is -cocoercive iff it is -Lipschitz
continuous.
MINIMIZATION OF SUM OF FUNCTIONS
The problem
h· , ·i
find ¯x 2 H s.t. 0 2 @f(¯x) + rg(¯x)
@f : H ! 2H f : H ! ] 1, +1]
rg: H ! H (1/ )
H

4. is a real Hilbert space with scalar product
is maximal monotone if is
nonempty, closed, and convex.
is -cocoercive.
VARIATIONAL INEQUALITIES
The problem
find ¯x 2 C s.t. 8 x 2 C h¯x x, B¯xi  0
A = @iC = NC C ⇢ H
B : H ! H
h· , ·iH

5. THE FORWARD-BACKWARD SPLITTING
(Mercier ’79)
The algorithm
if and , then
xk+1 = J kA(xk kB(xk))
Convergence: if k < 2
xk * ¯x 2 zer(A + B)
A = @f B = rg (f + g)(xk) inf(f + g) = o(1/k)
Cocoercivity of is a fundamental assumption!B

6. minimize
x2Rd
+
DKL(b, Ax) + ⌧kxk1
minimize
x2Rd
1
p
kAx bkp
p + ⌧kxk1 (p > 1)
minimize
w2Rd
nX
i=1
L(yi, hw, xii) +
⌧
p
kwkp
p (1 < p < 2)
When cocoercivity is an issue

7. minimize
x2Rd
1
p
kAx bkp
p + ⌧kxk1 (p > 1)
minimize
↵2Rn
1
q
kX⇤
↵kq
q +
1
⌧
nX
i=1
L⇤
(yi, ⌧↵i) (q > 2)
When cocoercivity is an issue
minimize
x2Rd
+
DKL(b, Ax) + ⌧kxk1

8. TSENG’S SPLITTING
(Tseng ’00)
is maximal monotone
is uniformly continuous on bounded sets
is chosen by the backtracking procedure: let
B : H ! H
k
kB(yk) B(xk)k 
k
kyk xkk
2 ]0, 1[
The algorithm converges if
A: H ! 2H
yk = J kA(xk kB(xk))
xk+1 = yk kB(yk) + kB(xk)
Strategies without cocoercivity

9. , is proper, convex, and lsc
is uniformly continuous on bounded sets.
is determined by backtracking line search procedures
(a priori choices are not possible anymore).
THE FORWARD-BACKWARD SPLITTING
FOR MINIMIZATION PROBLEMS
xk+1 = J kA(xk kB(xk))
B = rg
A = @f f : H ! ] 1, +1]
k
The algorithm converges if
S. Salzo, The variable metric forward-backward splitting algorithm under mild
differentiability assumptions. SIOPT 2017.
Strategies without cocoercivity

10. BACKTRACKING LINE SEARCHES
L1)
L2)
L3)
We considered three possible inequalities: let
krg(xk+1) rg(xk)k 
k
kxk+1 xkk
g(xk+1) g(xk) hxk+1 xk, rg(xk)i 
k
kxk+1 xkk2
(f + g)(xk+1) (f + g)(xk)
 (1 ) f(xk+1) f(xk) + hxk+1 xk, rg(xk)i
2 ]0, 1[
L1) L2) L3)
L1) is not quite appropriate for FB algorithm since it leads
to halve the step-sizes w.r.t. L2) and L3)
=) =)
Strategies without cocoercivity

11. FBA FBFA
They work under exactly the same
assumptions!
MINIMIZATION PROBLEMS
Strategies without cocoercivity

12. THE PROBLEM: DEVISE AN APPROPRIATE
BACKTRACKING LINESEARCH THAT REPLACES
THE COCOERCIVITY
What about the FB algorithm for
monotone operators?
Removing cocoercivity

13. Where does cocoercivity enter?
kxk+1 ¯xk2
 kxk ¯xk2
kxk+1 xkk2
2 khxk ¯x, B(xk) B(¯x)i
2 khxk+1 xk, B(xk) B(¯x)i
= kxk ¯xk2
k(2 k)kB(xk) B(¯x)k2
k(xk+1 xk) k(B(xk) B(¯x))k2
FB ALGORITHM FOR MONOTONE OPERATORS

14. Where does cocoercivity enter?
FB ALGORITHM FOR MINIMIZATION PROBLEMS
kxk+1 xk2
 kxk xk2
kxk+1 xkk2
+ 2 k (f + g)(x) (f + g)(xk)
2 k g(xk+1) g(xk) + hxk+1 xk, rg(xk)i
 kxk xk2
+ (2 1)kxk+1 xkk2
+ 2 k (f + g)(x) (f + g)(xk)
+ 2 k (f + g)(xk) (f + g)(xk+1)

15. Possible strategies
only value of B?
would not work!
the values of function (to be determined), that hopefully
should decrease along the iterations?
k
kB(xk+1) B(xk)k2
 hB(xk+1) B(xk), xk+1 xki
The inequality should involve:

16. ,
Possible strategies
PRIMAL DUAL ALGORITHMS
min
x2H
g(x) + h(Lx)
✓
0
0
◆
2 A
✓
¯x
¯v
◆
+ B
✓
¯x
¯v
◆
A
✓
x
v
◆
=
✓
L⇤
v
@h⇤
(v) Lx
◆
B
✓
x
v
◆
=
✓
rg(x)
0
◆
is convex and continuously differentiable.g: H ! R
(Combettes-Pesquet ’12, Condat ’13, Vu ’13)
Should backtracking involve the duality gap function?

17. Thank you!