Introduction to Optimization Techniques:

• Derivative based optimization- Steepest Descent, Newton method
• Derivative free optimization- Random Search, Down Hill Simplex

Optimization:

Optimization is at the heart of almost all machine learning techniques. Choosing the best element from some set of available alternatives. We are optimizing variables and basing our personal …