A Truly Random Search Method:

Step 1:

Choose a start point as the current point.

Step 2:

Add a random vector to the current point in the parameter space and evaluate the objective function at the new point at .

Step 3:

If , set the current point equal to .

Step 4:

Stop if the maximum number of function evaluations is reached. Otherwise, go back to step 2 to find a new point.

Observations That Leads to Heuristics:

Observation 1:

If search in a direction results in a higher objective function, the opposite direction can often lead to a lower objective function.

Observation 2:

Successive successful searches in a certain direction should bias subsequent searching toward this direction. On the other hand, successive failures in a certain direction should discourage subsequent searching along this direction.

A Random Search Method with Some Heuristics:

Step 1:

Choose a start point as the current point. Set initial bias equal to a zero vector.

Step 2:

Add a bias term and a random vector to the current point in the input space and evaluate the objective function at the new point at .

Step 3:

If , set the current point equal to and the bias equal to ; go to step 6. Otherwise, go to the next step.

Step 4:

If , set the current point equal to and the bias equal to ; go to step 6. Otherwise, go to the next step.

Step 5:

Set the bias equal to and go to step 6.

Step 6:

Stop if the maximum number of function evaluations is reached. Otherwise go back to step 2 to find a new point.

Flowchart for random search

Random Search for Continuous Optimization: An Example:
Find the min. of the peaks function:
z = peaks(x, y)
= 3*(1-x)^2*exp(-(x^2) - (y+1)^2) - 10*(x/5 - x^3 - y^5)*exp(-x^2-y^2) - 1/3*exp(-(x+1)^2 - y^2).

Surface plot of z = peaks(x, y)	Contour plot of z = peaks(x, y)

An example of random search

Advantages of Random Search:

• Derivative-free (derivatives of the peaks function).
• Intuitive
• Easy to incorporate new heuristics
• stochastic and less likely to get trapped in local minima

Weakness of Random Search:

• Slow
• Heuristics given above are only good for continuous optimization.

Reported Random Search Applications:

• Neural networks modeling
• Fuzzy modeling
• Some kines of discrete optimization
• All kinds of continuous optimization

Reference: Neuro-Fuzzy and Soft Computing