Pruning of a large network by optimal brain damage and surgeon: an example from biological sequence analysis

Abstract

Optimal Brain Damage (OBD) and Optimal Brain Surgeon (OBS) represent two popular pruning procedures; however, pruning large networks trained on voluminous data sets using these methods easily becomes intractable. We present a number of approximations and discuss practical issues in real-world pruning, and use as an example a network trained to predict protein coding regions in DNA sequences. The efficiency of OBS on large networks is compared to OBD, and it turns out that OBD is preferable to OBS, since more weights can be removed using less computational effort.