TITLE:
A Note on Link Functions in Generalized Linear Models and Activation Functions in Deep Learning: An Alternative Activation Function
AUTHORS:
Kazumitsu Nawata
KEYWORDS:
Generalized Linear Model, Deep Learning, Activation Function, ReLU, Tobit Model, Truncated Mean
JOURNAL NAME:
Open Journal of Statistics,
Vol.16 No.5,
October
29,
2026
ABSTRACT: The generalized linear model (GLM) is widely used in econometrics and statistics. In a GLM, a location parameter of the dependent variable—typically the expected value or mean—is linked to a linear combination of independent variables through a link function. The choice of link function is determined by the distribution of the error term. In contrast, activation functions are used in the convolutional layers of deep learning (DL) models. Common activation functions include the indicator (step) function, sigmoid, rectified linear unit (ReLU), and leaky ReLU. Although the universal approximation theorem shows that DL models using these activation functions can approximate a wide class of functions, major challenges remain, including overfitting, local minima, and substantial computational requirements. It is therefore essential to select an appropriate activation function. Link functions in GLMs and activation functions in DL are closely related and often coincide. In this note, we examine the similarities between these functions and propose an activation function derived from the truncated mean. We focus on the theoretical and methodological aspects rather than practical applications. The relationship between link functions and activation functions remains conjectural and warrants further research.