Hello Deep Learning: Reading handwritten digits - Bert Hubert's writings
a day ago
- The network architecture flattens a 28x28 image to 784x1, then applies three linear layers (128, 64, and 10 units) with ReLU activations between the first two layers.
- LogSoftMax is applied to the final 10 outputs to convert them into log probabilities, and cross-entropy loss is computed by comparing the predicted log probability for the correct digit (via a one-hot vector) to zero.
- Training uses stochastic gradient descent (SGD) with batches of 64 images, accumulating gradients over the batch and updating parameters with a learning rate divided by batch size.
- The model achieves 92.23% accuracy on the test set, with a confusion matrix showing common confusions (e.g., between 7 and 9) but no confusion between digits like 7 and 6.
- The network is heavily dependent on pixel positions and does not capture shape relations, limiting its generalization to uncentered or noisy digits.