Flax debugging: making a hash of things
2 days ago
- The author was debugging a JAX/Flax NNX training loop where loss remained constant at 10.82, indicating the model was essentially guessing randomly.
- To verify whether model parameters were being updated, the author used hashing of parameter arrays by converting them to numpy bytes and computing a hash to detect any changes.
- The root cause was using @jax.jit instead of @nnx.jit, which prevented the in-place parameter updates required by Flax NNX. Switching to @nnx.jit resolved the issue, allowing the loss to decrease.