Optimization: to achieve a certain preset goal with minimal effort.

Mathematical examples:

• (Unconstrained optimization) Find the min. or max. of
  y = a*x^2 + b*x + c.
• (Unconstrained optimization) Find the max. of
  z = 3*(1-x)^2*exp(-(x^2) - (y+1)^2) - 10*(x/5 - x^3 - y^5)*exp(-x^2-y^2) - 1/3*exp(-(x+1)^2 - y^2).

• To design the most efficient vehicle
• To maximize profit on a give amount of investment.
• To come to the class in time.
• To pass the class with minimum effort.

Method of Optimization:

• Derivative-based (for continuous problems):
  • Steepest descent
  • Conjugate method
  • Newton method
  • Gauss-Newton method
  • Levenberg-Marquardt method
• Derivative-free
  • Genetic algorithms
  • Simulated annealing
  • Random search (for continuous problems)
  • Downhill simplex search (for continuous problems)