Hello Deep Learning: Reading handwritten digits - Bert Hubert's writings

a day ago

The network architecture flattens a 28x28 image to 784x1, then applies three linear layers (128, 64, and 10 units) with ReLU activations between the first two layers.
LogSoftMax is applied to the final 10 outputs to convert them into log probabilities, and cross-entropy loss is computed by comparing the predicted log probability for the correct digit (via a one-hot vector) to zero.
Training uses stochastic gradient descent (SGD) with batches of 64 images, accumulating gradients over the batch and updating parameters with a learning rate divided by batch size.
The model achieves 92.23% accuracy on the test set, with a confusion matrix showing common confusions (e.g., between 7 and 9) but no confusion between digits like 7 and 6.
The network is heavily dependent on pixel positions and does not capture shape relations, limiting its generalization to uncentered or noisy digits.

Hasty Briefsbeta