pytorch/caffe2
Jiang Liu ee8d5e5249 Implementation of cosine learning rate training policy (#29017)
Summary:
Implementation of the cosine learning rate in: https://arxiv.org/pdf/1608.03983.pdf.

Mostly inspired from:
https://github.com/pytorch/fairseq/blob/master/fairseq/optim/lr_scheduler/cosine_lr_scheduler.py
Pull Request resolved: https://github.com/pytorch/pytorch/pull/29017

Test Plan:
buck test -v 2 caffe2/caffe2/fb/dper/layer_models/tests/split_1:sparse_nn_test  -- test_composite_cosine_lr_policy

learning rate log with max_lr=0.3, initial_period=20, t_mult=0.95, lr_shrink=0.95: P120327179

https://pxl.cl/PrcP

full canary: https://fburl.com/fblearner/mw69ylsd

Differential Revision: D18195868

Pulled By: grantlj

fbshipit-source-id: 67bdb0b8dd31d040d16b29d0da3115907bd141ef
2019-11-05 18:19:41 -08:00
..
contrib Merge Tensor and Variable. (#28620) 2019-11-04 14:59:57 -08:00
core Handling of failing and terminal async cpu ops (#29052) 2019-11-04 12:01:21 -08:00
cuda_rtc
db
distributed
experiments
ideep Build torch.distributed with Gloo backend on macOS (#25260) 2019-09-05 07:09:50 -07:00
image
mobile Remove API-level guard on NeuralNetworks.h (#22429) 2019-07-01 22:09:11 -07:00
mpi
observers
onnx Support Offline Tensors through ONNXIFI layer 2019-10-31 10:33:42 -07:00
operators Add cuda version for operators BatchSparseToDense and BatchDenseToSparse (#29166) 2019-11-05 13:06:23 -08:00
opt Support Offline Tensors through ONNXIFI layer 2019-10-31 10:33:42 -07:00
perfkernels Back out "Revert D17826873: Adding support to offsets based Fused8BitRowwiseEmbeddingLookup" (#27728) 2019-10-10 23:52:43 -07:00
predictor BlackBoxPredictor OSS part N + 1 : strip fb/predictor/Transforms.h dependency (#23350) (#23350) 2019-08-22 17:11:00 -07:00
proto adding OpProfile proto into ProfDAGProtos to support storing operation cost (#26677) 2019-09-23 20:44:15 -07:00
python Disables test_atomic_ops and testInputOrder (#29145) 2019-11-05 16:53:53 -08:00
quantization use fbgemm's 3d group conv fast path (#29085) 2019-11-05 00:58:49 -08:00
queue
serialize String optimizations related to serialization. (#28230) 2019-10-18 07:39:30 -07:00
sgd Implementation of cosine learning rate training policy (#29017) 2019-11-05 18:19:41 -08:00
share
test
transforms
utils Remove LOG(INFO) from math_cpu.cc (#27001) 2019-09-27 16:37:49 -07:00
video OpenCV 4 compatibility fix for caffe2/video (#24143) 2019-08-10 14:50:20 -07:00
__init__.py
.clang-format
c2_aten_srcs.bzl Remove Tensor.h, TensorMethods.h from src/core. (#27086) 2019-10-06 09:37:50 -07:00
CMakeLists.txt C++ parity, nn::CrossMapLRN2d (#29039) 2019-11-05 15:27:08 -08:00
README.md
release-notes.md
requirements.txt Add requests as a legit dependency (#25596) 2019-09-04 17:43:37 -07:00
VERSION_NUMBER

Caffe2

Jenkins Build Status

Caffe2 is a lightweight, modular, and scalable deep learning framework. Building on the original Caffe, Caffe2 is designed with expression, speed, and modularity in mind.

Questions and Feedback

Please use Github issues (https://github.com/pytorch/pytorch/issues) to ask questions, report bugs, and request new features.

Further Resources on Caffe2.ai