Article citationsMore>>
Qiu, J.T., Wang, J., Yao, S., Guo, K.Y., Li, B.X., Zhou, E.J., Yu, J.C., Tang, T.Q., et al. (2016) Going Deeper with Embedded FPGA Platform for Convolutional Neural Network. 2016 ACM/SIGDA International Symposium on Field-Programmable Gate Arrays, Monterey, February 2016, 26-35.
https://doi.org/10.1145/2847263.2847265
has been cited by the following article:
-
TITLE:
Neural Networks on an FPGA and Hardware-Friendly Activation Functions
AUTHORS:
Jiong Si, Sarah L. Harris, Evangelos Yfantis
KEYWORDS:
Deep Learning, D-ReLU, Dynamic ReLU, FPGA, Hardware Acceleration, Activation Function
JOURNAL NAME:
Journal of Computer and Communications,
Vol.8 No.12,
December
31,
2020
ABSTRACT: This paper describes our implementation of several neural networks built on a field programmable gate array (FPGA) and used to recognize a handwritten digit dataset—the Modified National Institute of Standards and Technology (MNIST) database. We also propose a novel hardware-friendly activation function called the dynamic Rectifid Linear Unit (ReLU)—D-ReLU function that achieves higher performance than traditional activation functions at no cost to accuracy. We built a 2-layer online training multilayer perceptron (MLP) neural network on an FPGA with varying data width. Reducing the data width from 8 to 4 bits only reduces prediction accuracy by 11%, but the FPGA area decreases by 41%. Compared to networks that use the sigmoid functions, our proposed D-ReLU function uses 24% - 41% less area with no loss to prediction accuracy. Further reducing the data width of the 3-layer networks from 8 to 4 bits, the prediction accuracies only decrease by 3% - 5%, with area being reduced by 9% - 28%. Moreover, FPGA solutions have 29 times faster execution time, even despite running at a 60× lower clock rate. Thus, FPGA implementations of neural networks offer a high-performance, low power alternative to traditional software methods, and our novel D-ReLU activation function offers additional improvements to performance and power saving.