image recognition

Hello Deep Learning: Convolutional networks - Bert Hubert's writings

2 months ago

Convolutional networks (CNNs) use kernels to scan for features in images, making them robust to pixel position changes.
Max-pooling reduces sensitivity to feature location by downsampling, using non-overlapping windows to keep only the maximum values.
GELU activation function often outperforms ReLU by retaining more information from negative inputs.
The described CNN design for handwritten letter recognition includes three convolutional+max-pooling stages, followed by three fully connected layers and LogSoftMax.
Training convolutional networks is computationally intensive (e.g., can take a day) due to many operations, though parallel processing helps.
Overfitting is a key challenge; production use typically requires progressing through training, validation, real-world data, and reproducibility.
The network's performance on handwritten letters can be subjective (e.g., g vs. q), leading to about 85% accuracy but over 95% when considering second-best guesses.
Backpropagation in CNNs involves sliding kernels over inputs and gradients to update weights and biases.
Actual implementation swaps digit datasets for letter datasets and uses a ConvoAlphabetModel instead of a simpler digit model.