Compare commits

...

2 Commits

Author SHA1 Message Date
torzdf
fbbdf5522c Merge branch 'staging'
Some checks failed
ci/build / conda (${{ matrix.os }}, ${{ matrix.backend }}) (cpu, macos-latest) (push) Has been cancelled
ci/build / conda (${{ matrix.os }}, ${{ matrix.backend }}) (cpu, ubuntu-latest) (push) Has been cancelled
ci/build / conda (${{ matrix.os }}, ${{ matrix.backend }}) (cpu, windows-latest) (push) Has been cancelled
ci/build / conda (${{ matrix.os }}, ${{ matrix.backend }}) (directml, windows-latest) (push) Has been cancelled
ci/build / conda (${{ matrix.os }}, ${{ matrix.backend }}) (nvidia, macos-latest) (push) Has been cancelled
ci/build / conda (${{ matrix.os }}, ${{ matrix.backend }}) (nvidia, ubuntu-latest) (push) Has been cancelled
ci/build / conda (${{ matrix.os }}, ${{ matrix.backend }}) (nvidia, windows-latest) (push) Has been cancelled
ci/build / conda (${{ matrix.os }}, ${{ matrix.backend }}) (rocm, ubuntu-latest) (push) Has been cancelled
ci/build / pip (ubuntu-latest, ${{ matrix.backend }}) (cpu, 3.10) (push) Has been cancelled
ci/build / pip (windows-latest, ${{ matrix.backend }}) (cpu, 3.10) (push) Has been cancelled
ci/build / pip (windows-latest, ${{ matrix.backend }}) (directml, 3.10) (push) Has been cancelled
2025-10-23 18:18:56 +01:00
torzdf
b16b755d55 lr-finder: Log error on NaN 2025-10-23 18:18:39 +01:00

View File

@ -132,7 +132,8 @@ class LearningRateFinder:
for idx in pbar:
model_inputs, model_targets = self._feeder.get_batch()
loss: list[float] = self._model.model.train_on_batch(model_inputs, y=model_targets)
if np.isnan(loss[0]):
if any(np.isnan(x) for x in loss):
logger.warning("NaN detected! Exiting early")
break
self._on_batch_end(idx, loss[0])
self._update_description(pbar)