%A W. Nick Street
%A O. L. Mangasarian
%T Improved Generalization via Tolerant Training
%D July 1995; Revised Decemebr 1996
%R 95-11
%I COMPUTER SCIENCES DEPARTMENT, UNIVERSITY OF WISCONSIN
%C MADISON, WI
%X Theoretical and computational justification is given for
improved generalization when the training set is learned with
less accuracy. The model used for this investigation is a
simple linear one. It is shown that learning a training set with a
tolerance $\tau$
improves generalization, over zero-tolerance
training, for any testing set satisfying a certain closeness condition to the
training set. These results, obtained via a
mathematical programming formulation,
are placed in the context of some well-known machine learning results.
Computational confirmation of improved generalization is given for
linear systems (including nine of the twelve real-world data sets
tested), as well as for nonlinear systems such as neural
networks for which no theoretical results are available at present.
In particular, the tolerant training method improves generalization on
noisy, sparse, and over-parameterized problems.