OSSForks/tensorflow
1
0
Fork 0
mirror of https://github.com/zebrajr/tensorflow.git synced 2025-12-06 12:20:11 +01:00
Code Issues Actions 5 Packages Projects Releases Wiki Activity

Add warmup to CosineDecay learning rate schedule

Browse Source 
PiperOrigin-RevId: 513348996
This commit is contained in:
Gabriel Rasskin 2023-03-01 14:52:26 -08:00 committed by TensorFlower Gardener
parent ab5b4eaffd
commit b59d2a159f
2 changed files with 5 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

4
RELEASE.md
View File

@ -47,6 +47,10 @@
`SidecarEvaluator` evaluator. The evaluator regularly evaluates the
model and exports it if the user-defined comparison function determines
that it is an improvement.
* Added warmup capabilities to `tf.keras.optimizers.schedules.CosineDecay`
learning rate scheduler. You can now specify an initial and target
learning rate, and our scheduler will perform a linear interpolation
between the two after which it will begin a decay phase.
* `tf.function`:
* ConcreteFunction (`tf.types.experimental.ConcreteFunction`) as generated

2
tensorflow/tools/api/golden/v2/tensorflow.optimizers.schedules.-cosine-decay.pbtxt
View File

@ -5,7 +5,7 @@ tf_class {
is_instance: "<type \'object\'>"
member_method {
name: "__init__"
argspec: "args=[\'self\', \'initial_learning_rate\', \'decay_steps\', \'alpha\', \'name\'], varargs=None, keywords=None, defaults=[\'0.0\', \'None\'], "
argspec: "args=[\'self\', \'initial_learning_rate\', \'decay_steps\', \'alpha\', \'name\', \'warmup_target\', \'warmup_steps\'], varargs=None, keywords=None, defaults=[\'0.0\', \'None\', \'None\', \'0\'], "
}
member_method {
name: "from_config"