Summary:
In some cases you have an optimized network and a normal
one. And you would like to make sure they produce same results. If
math under the hood is the same, you could do this with a very high
precision compare to a traditional numerical gradient check. One of
the application - RNNs. There we can unroll RNN into Caffe2 graph and
make sure result is the same as in the optimized version using
RecurrentNetworkOp.
Another possible application - graph transformations. We can verify
that after that nets produce same gradients (cc akyrola on memonger,
bwasti on other transformation ideas)
Reviewed By: bwasti
Differential Revision: D5200855
fbshipit-source-id: 0196af187f0c2feb33de4778ea08d0d288fe1017
Summary: Add Python support for arbitrary (unidirectional) recurrent networks with MultiRNNCell abstraction. Since the combined step net for all layers is created at one time (in method _apply), this may be optimizable as-is. LSTM() function is extended to accept a list of numbers of units for the dim_out argument, producing a multi-layer LSTM in that case.
Reviewed By: salexspb
Differential Revision: D4965001
fbshipit-source-id: 39c069468d5b40bf803503cf62046a479ca83cbb
Summary:
1. add net gradient check to dper2 model unittest framework
2. add net gradient check to mtml model
3. refactor the code setting defaults to namedtuple.
Reviewed By: kittipatv
Differential Revision: D4897169
fbshipit-source-id: 4f17dd06ee169aa1158f12f5156614d45d7d97c1
Summary: this is useful to test rnn cells
Reviewed By: dzhulgakov
Differential Revision: D4720641
fbshipit-source-id: baa7df43357ed8af72ede64be3e0a642a40472df
