OSSForks/pytorch
1
0
Fork 0
mirror of https://github.com/zebrajr/pytorch.git synced 2025-12-07 12:21:27 +01:00
Code Issues Actions 38 Packages Projects Releases Wiki Activity

Bug in CosineAnnealingWarmRestarts in optim/lr_scheduler.py (#64758)

Browse Source 
Summary:
## {emoji:1f41b} Bug
'CosineAnnealingWarmRestarts'  object has no attribute 'T_cur'.
In the Constructor of the CosineAnnealingWarmRestarts, we're calling the constructor of the Parent class (_LRScheduler) which inturn calls the step method of the CosineAnnealingWarmRestarts.
The called method tries to update the object's attribute  'T_cur' which is not defined yet. So it raises the error.
This only holds, when we give the value for last_epoch argument as 0 or greater than 0 to the 'CosineAnnealingWarmRestarts', while initializing the object.

![Bug_in_CosineAnnealingWarmRestarts](https://user-images.githubusercontent.com/77477328/132552212-70abc8b5-0357-4c35-90a9-832648bac607.png)
## To Reproduce

Steps to reproduce the behavior:

1. Give the value for the last_epoch argument as zero OR
1. Give the value for the last_epoch argument as a Positive integer.

## Expected behavior

I only expected the 'CosineAnnealingWarmRestarts' object to be initialized.

## Environment

PyTorch version: 1.9.0+cpu
Is debug build: False
CUDA used to build PyTorch: None
ROCM used to build PyTorch: N/A
OS: Ubuntu 20.04.2 LTS (x86_64)
GCC version: (Ubuntu 9.3.0-17ubuntu1~20.04) 9.3.0
Clang version: Could not collect
CMake version: version 3.21.2
Libc version: glibc-2.31
Python version: 3.8.10  [GCC 9.4.0] (64-bit runtime)
Python platform: Linux-5.8.0-59-generic-x86_64-with-glibc2.29
Is CUDA available: False
CUDA runtime version: No CUDA

## Additional context
We can able to solve this bug by moving the line 'self.T_cur = self.last_epoch' above the 'super(CosineAnnealingWarmRestarts,self).__init__()' line. Since we've initialized the "self.T_cur" to the object.

Pull Request resolved: https://github.com/pytorch/pytorch/pull/64758

Reviewed By: ezyang

Differential Revision: D31113694

Pulled By: jbschlosser

fbshipit-source-id: 98c0e292291775895dc3566fda011f2d6696f721
This commit is contained in:
Balaji 2021-09-22 16:36:39 -07:00 committed by Facebook GitHub Bot
parent b80bdcc73b
commit 32f0387ee8
1 changed files with 1 additions and 3 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

4
torch/optim/lr_scheduler.py
View File

@ -1236,11 +1236,9 @@ class CosineAnnealingWarmRestarts(_LRScheduler):
self.T_i = T_0
self.T_mult = T_mult
self.eta_min = eta_min
self.T_cur = last_epoch
super(CosineAnnealingWarmRestarts, self).__init__(optimizer, last_epoch, verbose)
self.T_cur = self.last_epoch
def get_lr(self):
if not self._get_lr_called_within_step:
warnings.warn("To get the last learning rate computed by the scheduler, "