Download Document

Local Search
• Goal is to find the local maximum (or minimum)
• Example:
– # seconds to spin wheels at 1.0 to move 2.0 meters.
Gradient Descent
• Minimum is found by following the slope of
the function
“Like climbing Everest in thick fog with amnesia”
# gen 100 points with a bias of 25 and 10 variance as a bit of noise
x, y = genData(100, 25, 10)
starter.py
#fix for higher dimensions.
#here, m = 100, n = 2
m, n = np.shape(x)
numIterations= 100000
alpha = 0.0005
theta = np.ones(n) #answer = [offset, slope]
theta = gradientDescent(x, y, theta, alpha, m, numIterations)
print "Our solution line has a y-intercept of ", theta[0], "and a slope of", theta[1]
print "The data we're trying to fit is as follows"
for i in range (0, 100):
print x[i][1], y[i]
def genData(numPoints, bias, variance):
x = np.zeros(shape=(numPoints, 2))
y = np.zeros(shape=numPoints)
# basically a straight line
for i in range(0, numPoints):
# bias feature
x[i][0] = 1
x[i][1] = i
# our target variable
y[i] = (i + bias) + random.uniform(0, 1) * variance
return x, y
gradientDescent.py
def gradientDescent(x, y, theta, alpha, m, numIterations):
xTrans = x.transpose()
replaceMe = .0001
for i in range(0, numIterations):
hypothesis = np.dot(x, theta)
loss = hypothesis - y
# Want the mean squared error here. Only use
# for debugging to make sure we're progressing
cost = np.sum(replaceMe) / (m)
if i % 9999 == 0:
print i, cost
# avg gradient per example
gradient = np.dot(xTrans, replaceMe) / m
# update
theta = theta * replaceMe
return theta
• http://stackoverflow.com/questions/1778458
7/gradient-descent-using-python-and-numpy
• http://www.bogotobogo.com/python/python
_numpy_batch_gradient_descent_algorithm.p
hp
Simulated annealing search
• Improves on gradient descent
• Idea: escape local maxima by allowing some “bad”
moves but gradually decrease their frequency
http://katrinaeg.com/simulated-annealing.html
def anneal(solution):
old_cost = cost(solution)
T = 1.0
T_min = 0.00001
alpha = 0.9
while T > T_min:
i=1
while i <= 100:
new_solution = neighbor(solution)
new_cost = cost(new_solution)
ap = acceptance_probability(old_cost, new_cost, T)
if ap > random():
solution = new_solution
old_cost = new_cost
i += 1
T = T*alpha
return solution, old_cost
• http://apmonitor.com/me575/index.php/Mai
n/SimulatedAnnealing
Genetic Algorithms
• Inspired by nature, based on reproduction and
selection
• Start with randomly generated states (population)
– A population member is represented by state
vector in the variable space (its DNA)
– A fitness function determines the quality of state
• New states generated from two parent states. Throw
some randomness into the mix as well…
Genetic algorithms
Genetic algorithms
• Normalized fitness function:
– 24/(24+23+20+11) = 31%
– 23/(24+23+20+11) = 29%
– … etc.
Genetic algorithms
Probability of selection is
weighted by the normalized
fitness function.
Genetic algorithms
Probability of selection is
weighted by the normalized
fitness function.
Genetic algorithms
Genetic Algorithms
1. Initialize population ( random states)
2. Calculate fitness function
3. Select pairs for crossover (weighted by
normalized fitness)
4. Apply mutation (random)
5. Evaluate fitness of children
6. From the resulting population of individuals,
probabilistically pick of the best
7. Repeat
Example: N-Queens
•
•
•
•
Why does crossover make sense here?
When wouldn’t it make sense?
What would mutation be?
What would a good fitness function be?
16
Genetic Algorithms
• Compared to gradient-based methods:
– Can perform similar random jumps to simulated
annealing
– Greater coverage - multiple state estimates at a
time
– Less sensitive to local optima
– Can be used for both continuous and discrete
variables
• Fitness function
• Population, lots of parameters
• Types of problems can solve
– http://nn.cs.utexas.edu/pages/research/rocket/
– http://nerogame.org/
• Co-evolution
– http://www.cs.utexas.edu/users/nn/pages/research/n
eatdemo.html
– http://www.cs.utexas.edu/users/nn/pages/research/ro
botmovies/clip7.gif
• Where will GAs not do as well in robotics (or in
general)?