JAX backends and devices
14 hours ago
- JAX defaults to loading data directly onto the GPU, leading to out-of-memory errors when the dataset exceeds VRAM capacity.
- The `jax.devices()` function only returns devices for the default backend (GPU if available, otherwise CPU) unless a specific backend is specified.
- To check available backends, JAX requires trying different backend options and catching errors, as there is no built-in listing method.
- JAX uses a default device concept, which can be temporarily changed using the `jax.default_device` context manager to load data onto the CPU instead of the GPU.
- The solution involves wrapping the data-loading code in `with jax.default_device(jax.devices("cpu")[0]):` to avoid GPU memory issues.
- Data loaded on the CPU can be moved to the GPU during training using `jax.device_put`.