A mathematical primer to classical deep learning

Authors

  • Enow Takang Achuo Albert Department of Plant Biology, Faculty of Science, University of Yaoundé I, P.O. Box 812, Yaoundé, Center Region, Cameroon
  • Ngalle Hermine Bille Department of Plant Biology, Faculty of Science, University of Yaoundé I, P.O. Box 812, Yaoundé, Center Region, Cameroon
  • Ngonkeu Mangaptche Eddy Leonard Department of Plant Biology, Faculty of Science, University of Yaoundé I, P.O. Box 812, Yaoundé, Center Region, Cameroon

DOI:

https://doi.org/10.21839/jaar.2024.v9.9169

Keywords:

Deep Learning, Matrix Calculus, Activation Functions, Gradient Descent, Backpropagation

Abstract

This manuscript synthesizes the statistical foundations of classical deep learning by integrating insights from eight seminal works. It covers matrix calculus, neuron layers, weight and bias indexing, cost functions, differentiation of neuron operations, activation functions, bias functions, gradient descent, and backpropagation algorithms. The synthesis aims to provide a comprehensive understanding of the mathematical and statistical principles underpinning deep learning models, facilitating their application and further development in various domains.

Downloads

Download data is not yet available.

References

Alber, M., Bello, I., Zoph, B., Kindermans, P.-J., Ramachandran, P., & Le, Q. (2018). Backprop Evolution. arXiv, arXiv:1808.02822. https://doi.org/10.48550/arXiv.1808.02822

Cabello, J. G. (2022). Mathematical Neural Networks. Axioms, 11(2), 80. https://doi.org/10.3390/axioms11020080

Chen, R. T. Q., Rubanova, Y., Bettencourt, J., & Duvenaud, D. K. (2018). Neural Ordinary Differential Equations. arXiv, arXiv:1806.07366. https://doi.org/10.48550/arXiv.1806.07366

Du, S. S., Jin, C., Lee, J. D., Jordan, M. I., Poczos, B., & Singh, A. (2017). Gradient Descent Can Take Exponential Time to Escape Saddle Points. arXiv, arXiv:1705.10412. https://doi.org/10.48550/arXiv.1705.10412

James, G., Witten, D., Hastie, T., & Tibshirani, R. (2013). An Introduction to Statistical Learning: With Applications in R. New York, US: Springer Science & Business Media. https://doi.org/10.1007/978-1-0716-1418-1

Kılıçarslan, S., Adem, K., & Çelik, M. (2021). An overview of the activation functions used in deep learning algorithms. Journal of New Results in Science, 10(3), 3. https://doi.org/10.54187/jnrs.1011739

Parr, T., & Howard, J. (2018). The Matrix Calculus You Need For Deep Learning. arXiv, arXiv:1802.01528. https://doi.org/10.48550/arXiv.1802.01528

Pick, A. J., & Cole, D. J. (2008). A Mathematical Model of Driver Steering Control Including Neuromuscular Dynamics. Journal of Dynamic Systems, Measurement, and Control, 130(3), 031004. https://doi.org/10.1115/1.2837452

Published

23-09-2024

Issue

Vol 9, 2024

Section

Articles