Title:
Novel-Neuronal-Activation
Download
Description: Feedforward neural network structures have extensively been considered in the
literature. In a significant volume of research and development studies hyperbolic tangent
type of a neuronal nonlinearity has been utilized. This paper dwells on the widely used neuronal
activation functions as well as two new ones composed of sines and cosines, and a sinc
function characterizing the firing of a neuron. The viewpoint here is to consider the hidden
layer(s) as transforming blocks composed of nonlinear basis functions, which may assume
different forms. This paper considers 8 different activation functions which are differentiable
and utilizes Levenberg-Marquardt algorithm for parameter tuning purposes. The studies
carried out have a guiding quality based on empirical results on several training data sets.
To Search:
File list (Check if you may need any files):
Novel Neuronal Activation.pdf
