Velamuvi AIAcademy
Mathematics for AI

Optimization

How models find their best settings. From gradient descent to Adam, with interactive visual intuition.

  • Advanced
  • About 8 hours
  • 3 modules
  • 10 lessons
  • Free

This course is not released yet. We publish courses one at a time, each one complete. The full syllabus is below.

What you will learn

  • The ProblemObjective Functions, Convex vs Non-Convex, Constraints
  • Gradient MethodsBatch Gradient Descent, Stochastic and Mini-Batch, Momentum and more
  • In PracticeLearning Rate Schedules, Regularisation as Optimisation, Diagnosing Training Problems

Course syllabus

Not released yet · full syllabus below

  1. 1

    The Problem

    3 lessons
    1. 1Objective FunctionsSoon
    2. 2Convex vs Non-ConvexSoon
    3. 3ConstraintsSoon
  2. 2

    Gradient Methods

    4 lessons
    1. 4Batch Gradient DescentSoon
    2. 5Stochastic and Mini-BatchSoon
    3. 6MomentumSoon
    4. 7RMSProp and AdamSoon
  3. 3

    In Practice

    3 lessons
    1. 8Learning Rate SchedulesSoon
    2. 9Regularisation as OptimisationSoon
    3. 10Diagnosing Training ProblemsSoon

Recommended next