Download Optimal Brain Surgeon

Optimal Brain Surgeon
ECE 539 Final Project
Mark Slosarek
Background
Optimal Brain Surgeon Algorithm (OBS) is
a pruning algorithm
Reduces weights to reduce overall
complexity of network
Benefits
A pruned network has several benefits


Quicker Calculations
Less Storage Space required
A pruned network should not have
significantly more error than an nonpruned network
Steps to Perform OBS
Train the given MLP to minimize meansquare error
Calculate the cost of the equation
Compute the inverse Hessian
Find the smallest Saliency
If Saliency is much smaller than meansquare, delete that weight and repeat for
next weight, other go to next step
Update all weights
Tests Performed and Results
To test the effectiveness of the OSB, I
used the wine data from the homework
The pruned network contained
approximately 60% fewer weights
Results from both networks, however
results of pruning were very inconclusive

The results were not very consistant
Conclusion
A pruned network can save much space
and time
My algorithm is not perfect, and could be
recoded to be more efficient
Results not noticeable in simple networks,
but in real world problems, could be
results could be great