# LeNet-5

> type of convolutional neural network

**Wikidata**: [Q94694243](https://www.wikidata.org/wiki/Q94694243)  
**Wikipedia**: [English](https://en.wikipedia.org/wiki/LeNet)  
**Source**: https://4ort.xyz/entity/lenet-5

## Summary
LeNet-5 is a convolutional neural network architecture developed by Yann LeCun and colleagues in 1998. It was one of the first successful applications of convolutional neural networks to handwritten digit recognition.

## Key Facts
- Developed in 1998 by Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner
- Designed for handwritten digit recognition and document analysis
- Consists of 7 layers including 3 convolutional layers and 2 fully connected layers
- Uses average pooling instead of max pooling
- Employs sigmoid activation functions rather than ReLU
- Trained on the MNIST dataset of handwritten digits
- Achieved error rates below 1% on digit recognition tasks
- Published in the paper "Gradient-Based Learning Applied to Document Recognition"
- One of the earliest deep learning models to demonstrate practical success

## FAQs
### Q: What is LeNet-5 used for?
A: LeNet-5 was originally designed for handwritten digit recognition and document analysis tasks. It became the foundation for modern convolutional neural networks used in image classification, object detection, and computer vision applications.

### Q: How many layers does LeNet-5 have?
A: LeNet-5 has 7 layers total: 3 convolutional layers, 2 subsampling (pooling) layers, and 2 fully connected layers. The architecture processes images through these layers to extract features and make predictions.

### Q: What makes LeNet-5 different from modern CNNs?
A: LeNet-5 uses average pooling instead of max pooling, sigmoid activation functions instead of ReLU, and has a much smaller architecture compared to modern deep networks. It also processes smaller 32x32 pixel images rather than the larger inputs used today.

## Why It Matters
LeNet-5 represents a landmark achievement in the history of deep learning as one of the first convolutional neural networks to demonstrate practical success in real-world applications. Before LeNet-5, neural networks were largely theoretical constructs with limited practical utility. This architecture proved that deep learning could solve complex pattern recognition problems with high accuracy, specifically in handwritten digit recognition for bank check processing and postal address reading. The success of LeNet-5 validated the convolutional approach to image processing and laid the groundwork for the deep learning revolution that would follow decades later. Its architecture principles - local receptive fields, shared weights, and spatial subsampling - became fundamental concepts that modern CNNs still rely on today. Without LeNet-5's demonstration of what was possible, the field of computer vision might have developed very differently.

## Notable For
- First successful application of convolutional neural networks to real-world problems
- Achieved error rates below 1% on handwritten digit recognition
- Introduced architectural concepts that became standard in modern CNNs
- Demonstrated the practical viability of backpropagation for deep networks
- Published in one of the most cited papers in computer vision history

## Body
### Architecture Details
LeNet-5 processes 32x32 pixel grayscale images through its 7-layer architecture. The first convolutional layer uses 6 filters of size 5x5, producing 28x28 feature maps. The first subsampling layer applies average pooling with a 2x2 window and stride of 2, reducing dimensions to 14x14. The second convolutional layer uses 16 filters of size 5x5, creating 10x10 feature maps. The second subsampling layer again applies average pooling, producing 5x5 feature maps. These are then flattened and passed through two fully connected layers with 120 and 84 units respectively, before the final output layer with 10 units for digit classification.

### Training and Performance
The network was trained using stochastic gradient descent with momentum on the MNIST dataset containing 60,000 training images and 10,000 test images of handwritten digits 0-9. LeNet-5 achieved test error rates around 0.95%, significantly better than previous methods. The training process took several days on the hardware available in 1998, which would be considered extremely slow by modern standards. The use of average pooling rather than max pooling was a deliberate choice based on the belief that averaging would provide better generalization for the digit recognition task.

### Historical Context
Developed at AT&T Bell Labs in the late 1990s, LeNet-5 emerged during a period when neural networks were largely out of favor in the AI community. The architecture demonstrated that deep learning approaches could outperform traditional computer vision techniques and handcrafted feature extractors. Its success in document recognition applications led to commercial deployment in check reading systems used by banks. The publication of LeNet-5 in 1998 helped preserve knowledge about convolutional networks during the "AI winter," ensuring these techniques would be available when computational power and datasets finally caught up decades later.

## Schema Markup
```json
{
  "@context": "https://schema.org",
  "@type": "Thing",
  "name": "LeNet-5",
  "description": "A convolutional neural network architecture developed in 1998 for handwritten digit recognition",
  "sameAs": [
    "https://www.wikidata.org/wiki/Q1398103",
    "https://en.wikipedia.org/wiki/LeNet"
  ]
}