Kernel second-order discriminants versus support vector machines

Support vector machines (SVMs) are the most well known non-linear classifiers based on the Mercer kernel trick. They generally leads to very sparse solutions that ensure good generalization performance. Recently Mika et al. have proposed a new nonlinear technique based on the kernel trick and the Fisher criterion: the nonlinear kernel Fisher discriminant (KFD). Experiments show that KFD is competitive to the SVM classifiers. Nevertheless, it can be shown that there exists distributions such that even though the two classes are linearly separable, the Fisher linear discriminant has an error probability close to 1. In this paper, we propose an alternative strategy based on Mercer kernels that consists in picking the optimum nonlinear receiver in the sense of the best second-order criterion. We also present a strategy for controlling the complexity of the resulting classifier. Finally we compare this new method with SVM and KFD.