OSSForks/pytorch
1
0
Fork 0
mirror of https://github.com/zebrajr/pytorch.git synced 2025-12-07 00:21:07 +01:00
Code Issues Actions 38 Packages Projects Releases Wiki Activity
67,630 Commits 8,304 Branches 150 Tags 1.5 GiB
71cb13869b
Commit Graph

549 Commits

Author SHA1 Message Date
Yan Wang
a76098ac15 fix optimizer when given single parameters (instead of an iterable) 
When I use the named_parametes to modify the lr and weight decay, I will face a bug. Because the value of the named_parameters return is  torch.nn.paramter.Parameter, not a generator of the Parameter.
 2017-06-05 23:47:56 -04:00
Jiaming Liu
630af4d7d8 add learning rate schedulers (#1370) 2017-05-25 16:21:43 -04:00
Adam Paszke
feef54ec34 Don't modify non-volatile grads in zero_grad 2017-05-10 16:43:14 +02:00
Edward Z. Yang
743e4894d2 Prefix values/indices/sparse_mask/nnz with underscore (#1457) 
As discussed in #1441.

I also added some docs giving clear guidance about how to coalescing
in sparse tensors.

Signed-off-by: Edward Z. Yang <ezyang@fb.com>
 2017-05-03 11:14:10 -04:00
Edward Z. Yang
699755e04f Convert contiguous() call in adagrad to out-of-place coalesce. (#1446) 
We missed this one in f2903332c7dce1fbb7d7d9f18dcfba8e853581df!

Signed-off-by: Edward Z. Yang <ezyang@fb.com>
 2017-05-02 16:51:54 -04:00
Adam Paszke
20aa5b066f Convert some of the functions to new format 
Also, fix a lot of issues that appeared after the previous commits.
 2017-05-01 16:44:56 -04:00
Adam Paszke
2ca787fcf4 Refactor attribute names in autograd 2017-05-01 16:44:56 -04:00
Martin Raison
cd3bbc9dfd more operations and optimizations (hspmm, reorder, ...) 2017-04-18 12:46:54 -07:00
Martin Raison
1018b238ac make gradients contiguous in adagrad 2017-04-18 12:46:54 -07:00
Soumith Chintala
85954032d9 fix doc formatting 2017-04-05 22:02:29 -04:00
Nitish Shirish Keskar
1a04b92226 add note regarding SGD momentum 2017-04-05 20:45:41 -04:00
Tudor Berariu
5c79046d39 Use persistent tensor to store exp_inf (part of optimizer's state) (#1152) 2017-03-31 10:30:31 -04:00
Adam Paszke
1e8cb82a2d Break only after the update in L-BFGS 2017-03-22 18:58:42 -04:00
Nitish Shirish Keskar
b9aef6bc03 Fixing default values for LR and Epsilon (#895) 
It seems that the default values for LR and Epsilon (previously, 1E-2 and 1E-38 respectively) were different from the ones recommended by the authors (2E-3 and 1E-8, respectively). Other packages such as Keras (https://github.com/fchollet/keras/blob/master/keras/optimizers.py#L474) and Lasagne (https://github.com/Lasagne/Lasagne/blob/master/lasagne/updates.py#L612) use the suggested values as well.
 2017-03-22 11:34:39 -04:00
Edouard Delasalles
b1c2714ad5 Add momentum and centered options to RMSProp (#810) 
* add momentum and centered options

Add two options :
 - Momentum (like SGD's momentum)
- Centered RMSprop, as in Graves 2013 ( https://arxiv.org/abs/1308.0850 ) : grad is normalized by running estimation of its variance

* somme PEP8

* bug in default

* bug2

* sign mistake

* alloc of momentum & centered only if needed

* add link to docstring

* some pep8 on docstring

* implement __setstate__() for backward compatibilty

* correct grammar mistake

* multiply by lr when adding delta to params

* rename momentum variables

* change __init__ params order
 2017-03-09 10:04:32 +01:00
Martin Raison
f17cfe4293 sparse tensor operations (#735) 2017-03-03 18:37:03 +01:00
Adam Paszke
3277d83648 Add Nesterov Momentum (#887) 2017-03-01 20:49:59 +01:00
Adam Paszke
bd7a5ad6f0 Make Optimizer.load_state_dict use __setstate__ 2017-02-26 20:02:42 +01:00
Durk Kingma
a25c8555eb Fixed paper references 2017-02-21 00:27:18 +01:00
Adam McCarthy
7926324385 Corrected parameter typo in Adam docstring (#697) 2017-02-07 19:00:10 +01:00
Adam Paszke
57373c7c29 Fix docs 2017-01-28 01:16:04 +01:00
Luke Yeager
3ed720079e [pep8] Fix most remaining lint manually 2017-01-28 01:15:51 +01:00
Luke Yeager
e7c1e6a8e3 [pep8] Fix most lint automatically with autopep8 
Here's the command I used to invoke autopep8 (in parallel!):

    git ls-files | grep '\.py$' | xargs -n1 -P`nproc` autopep8 -i

Several rules are ignored in setup.cfg. The goal is to let autopep8
handle everything which it can handle safely, and to disable any rules
which are tricky or controversial to address. We may want to come back
and re-enable some of these rules later, but I'm trying to make this
patch as safe as possible.

Also configures flake8 to match pep8's behavior.

Also configures TravisCI to check the whole project for lint.
 2017-01-28 01:15:51 +01:00
Adam Paszke
ecfcf39f30 Improve optimizer serialization 
Also, add optimizer.load_state_dict
 2017-01-24 17:30:50 -05:00
Sergey Zagoruyko
2748b920ab make adam have the same lr as lua torch (#576) 2017-01-24 16:35:28 -05:00
Soumith Chintala
4c77ad6ee4 step_rate -> lr in adadelta (#569) 2017-01-24 10:05:59 -05:00
Edouard Delasalles
e374dc1696 add step rate to adadelta (#568) 
Scales `delta` before it is applied to the parameters in order to control the learning rate of the optimizer (inspired from climin optim lib for theano).
Also changed the link to the Adadelta paper to point to the right location.
 2017-01-24 08:48:19 -05:00
Adam Paszke
3238786ea1 Improve optimizer error messages 2017-01-22 18:32:51 -05:00
Adam Paszke
ca555abcf9 fix comments 2017-01-22 18:02:40 -05:00
Adam Paszke
f8ae34706e Port L-BFGS from Lua optim 2017-01-22 18:02:40 -05:00
glample
99f4864674 fixed RMSprop initialization (#485) 
* fixed RMSprop initialization
 2017-01-18 17:05:53 -05:00
Adam Paszke
95f0fa8a92 Change .grad attribute of Variables to be a Variable 2017-01-16 12:59:47 -05:00
Adam Paszke
676ffee542 Check params type in optimizers 2017-01-16 12:59:47 -05:00
Adam Paszke
604e13775f Add optim docs 2017-01-16 12:59:47 -05:00
Adam Paszke
75d850cfd2 Fix optim docs 2016-12-30 00:15:06 -05:00
Sam Gross
126a1cc398 Add Sphinx docs 2016-12-28 00:03:39 +01:00
Adam Paszke
506a40ce44 Remove optim submodule attributes from torch.optim package 2016-12-01 23:14:41 +01:00
Sam Gross
162170fd7b Add optional weight decay to optim.SGD (#269) 2016-11-29 20:35:40 -05:00
Adam Paszke
09493603f6 Change optimizer API 2016-11-08 18:12:56 +01:00
Adam Paszke
df59b89fbb Add more optimizers 2016-11-07 22:50:56 +01:00
Adam Paszke
3cbe66ba8c Change requires_grad default to False 2016-10-05 08:46:34 -07:00
Adam Paszke
99de537a2e Remove CUDA sync points from losses and trainer 2016-10-05 08:46:31 -07:00
Adam Paszke
4db6667923 Allow specifying per-parameter optimization parameters 2016-10-04 18:21:50 -07:00
Adam Paszke
58b134b793 Allow exporting optimizer state as a dict 2016-10-04 17:33:49 -07:00
Soumith Chintala
9842be4b15 setting default dampening value to 0 2016-09-13 10:28:33 -07:00
Adam Paszke
ff785e5f17 Make optimizers accept a closure 2016-08-25 09:23:39 -07:00
Adam Paszke
7bcb2a4081 Initial optim version 2016-08-23 19:03:30 -07:00
Adam Paszke
2f342af22f Move optim to legacy 2016-08-01 12:01:46 -04:00
Adam Paszke
554a1d8336 Add optim 2016-07-21 16:42:06 -04:00