ABSTRACT
The multilayer perceptron–back-propagation (MLP–BP) neural network can be considered as a type of stochastic approximation. On the other hand, radial basis function (RBF) neural networks are considered as curve-fitting problems in high-dimensional space. Unlike in MLP-type networks, which can have several layers of neurons, RBF-type networks have only three layers: an input layer, a hidden layer, and an output layer. The main distinguishing characteristics of RBF-type neural networks are that the hidden layer is of high dimensionality having a non-linear activation function, and the output layer is always linear. The mathematical justification for the rationale of using a non-linear transformation followed by a linear transformation has been given by Cover (1965) in his theorem on separability of patterns, which states that a complete pattern classification problem cast in a high-dimensional space non-linearly is more likely to be linearly separable than in a low-dimensional space. Therefore, the dimension of the hidden layer is made high, and it can be said that the higher the dimension of the hidden layer, the more accurate the approximation.
