Proximal Gradient

Minimizer = Fixed Point of Update Formula

Is it descent or equation solving? Proximal gradient says: both. Minimizer = fixed point.

𝗣𝗿𝗼𝗯𝗹𝗲𝗺
Minimize f + g
where f is convex, closed and proper, and g is convex and smooth.

𝗢𝗽𝘁𝗶𝗺𝗮𝗹𝗶𝘁𝘆
Fermat's optimality condition is

0 ∈ ∂f(x) + ∇g(x).

It can be equivalently rewritten as

x = prox_{𝛼f}(x - 𝛼∇g(x)).

The right hand side first has a gradient descent step x - 𝛼∇g(x) to handle the smooth part, and then a proximal update for f. This is precisely the update formula for proximal gradient.

Check out the short derivation below for why these two conditions are equivalent.