Acknowledgements
My genuine gratitude goes to our amusing yet enlightening professor, Yuanzhi Li, for this fabulous course and all the lively delivered lectures.
Comments
I spent a considerable amount of time figuring out the following lemmas & problems.
I’ve already typed them out as decently organized formulae so I’ll dump them here for reference and some comments (if I’d come back with any).
Table of Contents
Convexity
(1) defined over a convex set
(2) 2nd order differentiable
(3) hessian condition: second order derivative is semi-positive definite
(4) lower linear bound
L-Smoothness
(1) doesn’t require convexity
(2) constrained on speed of “growth”
(3) hessian condition: second derivative has a numeric positive finite upper bound
(4) upper quadratic bound