1. Geoffrey Hinton's 2006 Paper: In 2006, computer scientist Geoffrey Hinton published a seminal paper titled "Deep Learning: A Tutorial on Deep Neural Networks" in Nature magazine. This paper is considered a landmark in the field of artificial neural networks and helped popularize the term "deep learning." Hinton and his colleagues at the University of Toronto are widely regarded as pioneers in the development of deep learning techniques.
2. Yoshua Bengio's Use of the Term: Yoshua Bengio, another prominent researcher in the field, also played a significant role in the popularization of the term "deep learning" in the early 2000s. Bengio and his colleagues at Université de Montréal conducted groundbreaking research on deep learning algorithms, and he frequently used the term "deep learning" in his research publications.
3. Influence of Cognitive Psychology: Some believe that the inspiration for the term "deep learning" may have come from the notion of "deep structure" in cognitive psychology. Deep structure is a term used in linguistics and cognitive psychology to describe the underlying representation or syntax of a language that goes beyond the surface-level features of words and phrases. This concept may have influenced the understanding of deep learning models as capturing underlying patterns and complex relationships in data.
4. Comparison with Traditional Machine Learning: The term "deep learning" was likely coined to differentiate it from traditional machine learning methods. While traditional machine learning algorithms often rely on shallow neural networks or shallow representations of data, deep learning involves the use of deep neural networks with multiple hidden layers. These deep architectures allow for more complex and hierarchical feature extraction, enabling models to learn higher-level representations of data.
5. Historical Context: In the early days of neural network research, shallow neural networks were the norm, and they faced limitations in their representational capabilities and ability to handle complex problems. The emergence of powerful computing resources, such as graphical processing units (GPUs), in the late 2000s made it possible to train deeper neural networks effectively. This historical context contributed to the need for a term that captured the advances and increased complexity of these new approaches, hence the term "deep learning" gained traction.
It is likely a combination of these factors, along with the convergence of research efforts and breakthroughs, that led to the widespread adoption of the term "deep learning" to describe the subfield within machine learning focused on deep neural networks.