OSSForks/pytorch
1
0
Fork 0
mirror of https://github.com/zebrajr/pytorch.git synced 2025-12-07 12:21:27 +01:00
Code Issues Actions 38 Packages Projects Releases Wiki Activity
bb08f261f1
pytorch/caffe2/python/operator_test
History
James Cross 53ccbd9a6e soft-coverage attention 
Summary:
Implementation of a new variant of attention module, which contains a recurrent decoder state with vectors corresponding to each source-side word and strictly increasing values, thus enabling it to model the degree to which source words have been translated.

The approach is a variant of the approaches described in https://arxiv.org/pdf/1601.04811.pdf. We simply include the sum of all previous attention weights for encoder words as a new recurrent state (coverage_t). A new linear transform on encoder_outputs is used to produce coverage_weights, which has the same dimensionality as encoder_outputs, and implicitly models the fertility of source-side words (and putting this extra information strain on the encoder network).

Thus the encoder output, the decoder state, and the coverage weights have the same dimensionality for a given source word, and attention logits are calculated as v *  tanh(coverage * coverage_weights + encoder_output + decoder_state).

Note: the entire coverage state for each translation instance is of shape (encoder_length, coverage_units), but the states for the RecurrentNetwork operator, used to train the decoder, must be flat in the data dimension. This state is therefore initialized with shape (encoder_length * coverage_units) [not shown in the open-source library] and reshaped appropriately within the apply_soft_coverage_attention() function.

Differential Revision: D5593617

fbshipit-source-id: 7d0522b5eb0b26f22e8429e4461a459f2f16ed46
2017-08-31 21:21:54 -07:00
..
activation_ops_test.py Fix flaky test due to numerical gradient approximation error. 2017-07-26 18:58:19 -07:00
adagrad_test.py Fix GPU SparseAdaGrad with empty tensors 2017-07-25 23:50:54 -07:00
adam_test.py Fix SparseAdagrad for indices.ndim>1 2017-05-30 12:02:18 -07:00
apmeter_test.py Fixing broken Python tests 2017-06-08 13:34:46 -07:00
atomic_ops_test.py Add unittest.main() lines to test scripts under python/operator_test 2016-11-29 15:18:37 -08:00
batch_box_cox_test.py add box cox transform op 2017-04-27 22:06:43 -07:00
blobs_queue_db_test.py Caffe2: support Tensor in BlobsQueueDB 2017-08-11 11:21:14 -07:00
boolean_mask_test.py Added window mode for caffe2 sequence operator 2017-08-16 21:34:29 -07:00
boolean_unmask_test.py Add CUDA implementation of BooleanUnmask and fixed some bugs in the test 2017-08-01 16:51:40 -07:00
channel_shuffle_test.py Opensourcing channel shuffle 2017-08-25 16:46:31 -07:00
checkpoint_test.py snapshot -> checkpoint 2016-12-15 12:01:30 -08:00
clip_op_test.py Vectorize CPU ClipOp implementation (and add test) 2017-06-26 11:33:13 -07:00
concat_split_op_test.py Add tests for ConcatOp and SplitOp 2017-08-18 14:02:42 -07:00
conv_test.py Adding 1d-2d-3d Schemas for Conv and Pool 2017-08-17 09:45:54 -07:00
conv_transpose_test.py Support new arguments in ConvTranspose 2017-08-31 11:17:32 -07:00
copy_ops_test.py deprecate CNNModelHelper in python/operator_test dir 2017-05-19 12:17:17 -07:00
cosine_embedding_criterion_op_test.py Add unittest.main() lines to test scripts under python/operator_test 2016-11-29 15:18:37 -08:00
counter_ops_test.py AtomicCounter to return previous value on Reset. 2017-02-02 14:59:30 -08:00
crf_test.py deprecate CNNModelHelper in python/operator_test dir 2017-05-19 12:17:17 -07:00
cross_entropy_ops_test.py CUDA version of SigmoidCrossEntropyWithLogits 2017-04-14 16:07:33 -07:00
cudnn_recurrent_test.py Dict fixes/improvements and unittest targets for Python 3 in caffe2 core 2017-06-29 17:05:41 -07:00
dataset_ops_test.py Core unit test fixes for Python 3 2017-06-23 13:22:16 -07:00
distance_op_test.py L1Distance Row-wise, instead of cumulative 2017-08-07 14:09:25 -07:00
dropout_op_test.py Fixes for CuDNNDropoutOp 2017-06-15 22:51:12 -07:00
duplicate_operands_test.py Add unittest.main() lines to test scripts under python/operator_test 2016-11-29 15:18:37 -08:00
elementwise_linear_op_test.py ElementwiseLinearOp 2017-04-17 14:18:27 -07:00
elementwise_logical_ops_test.py Fixing some tests 2017-05-30 09:16:48 -07:00
elementwise_op_broadcast_test.py new SumReduceLike op CPU/GPU implementation and doc 2017-04-13 10:28:46 -07:00
elementwise_ops_test.py Vectorize SigmoidOp on CPU 2017-05-17 20:33:36 -07:00
emptysample_ops_test.py Add unittest.main() lines to test scripts under python/operator_test 2016-11-29 15:18:37 -08:00
extend_tensor_op_test.py Add unittest.main() lines to test scripts under python/operator_test 2016-11-29 15:18:37 -08:00
fc_operator_test.py Make FC op work with empty batch in cuda 2017-08-24 18:52:04 -07:00
filler_ops_test.py Caffe2: diagonal fill op 2017-08-16 13:05:11 -07:00
find_op_test.py FindOp 2017-04-24 15:52:25 -07:00
gather_ops_test.py GPU version of BatchGatherOp 2017-08-17 18:31:10 -07:00
gather_ranges_op_test.py Add unittest.main() lines to test scripts under python/operator_test 2016-11-29 15:18:37 -08:00
given_tensor_fill_op_test.py Fixes for ops without a CUDA backend 2017-03-29 14:36:09 -07:00
group_conv_test.py Make all convolution operators allow optional bias term 2016-12-21 15:14:24 -08:00
gru_test.py Revert D5589309: modify _LSTM into _RNN to adapt GRU 2017-08-10 16:42:41 -07:00
hsm_test.py Core unit test fixes for Python 3 2017-06-23 13:22:16 -07:00
im2col_col2im_test.py Implement gradients for Col2Im and Im2Col operators 2017-08-07 15:51:30 -07:00
image_input_op_test.py ImageInputOp_more_data_augmentation 2017-08-19 14:15:58 -07:00
index_hash_ops_test.py IndexHash 2017-07-07 23:06:11 -07:00
index_ops_test.py Core unit test fixes for Python 3 2017-06-23 13:22:16 -07:00
instance_norm_test.py Fixes zip/izip for Python 3 2017-06-07 00:04:26 -07:00
layer_norm_op_test.py Layer norm brew wrapper 2017-08-17 11:17:47 -07:00
leaky_relu_test.py minor leaky relu bug fixes 2017-07-10 16:04:45 -07:00
lengths_tile_op_test.py add gradient for LengthsTileOp 2017-04-25 14:31:15 -07:00
lengths_top_k_ops_test.py implement LengthsTopK operator 2017-08-07 18:19:29 -07:00
loss_ops_test.py Caffe2: consolidate AveragedLoss with SumElementsOp 2017-04-06 10:35:01 -07:00
lpnorm_op_test.py adding operator lp_norm to support calculating l1 norm and l2 norm 2017-08-02 15:09:08 -07:00
map_ops_test.py CreateMapOp 2017-08-09 13:32:19 -07:00
margin_ranking_criterion_op_test.py Add unittest.main() lines to test scripts under python/operator_test 2016-11-29 15:18:37 -08:00
matmul_op_test.py BatchMatMulOp: use cuBLAS batched strided gemm for CUDA 2017-03-28 11:54:09 -07:00
merge_id_lists_op_test.py Operator to Merge ID_LIST features 2017-08-17 01:16:00 -07:00
mkl_conv_op_test.py MKL convolution operator 2017-01-23 09:59:30 -08:00
mkl_packed_fc_op_test.py MKL convolution operator 2017-01-23 09:59:30 -08:00
mkl_speed_test.py deprecate CNNModelHelper in python/operator_test dir 2017-05-19 12:17:17 -07:00
momentum_sgd_test.py Add tests and GPU impls for sparse optimizers 2017-04-13 11:07:40 -07:00
mpi_test.py Setup MPI before test start 2016-12-19 15:59:32 -08:00
normalize_op_test.py Add NormalizeL1Op for doing L1 nomalization along given axis 2017-07-10 10:10:36 -07:00
one_hot_ops_test.py Caffe2: Write CUDA version of OneHot operator 2017-08-08 18:17:39 -07:00
pack_ops_test.py Registering GPU version of PackSegments using GPUFallbackOp 2017-03-24 16:01:53 -07:00
pack_rnn_sequence_op_test.py added PackRNNSequence and UnpackRNNSequence operators 2017-06-30 09:53:31 -07:00
pad_test.py Support cropping with negative pad sizes in PadImage 2017-04-03 23:47:54 -07:00
partition_ops_test.py Add unittest.main() lines to test scripts under python/operator_test 2016-11-29 15:18:37 -08:00
piecewise_linear_transform_test.py Added PiecewiseLinearTransform CUDA Op 2017-07-07 15:20:00 -07:00
pooling_test.py Added fast path for CUDNN global max pooling 2017-08-23 16:33:06 -07:00
pow_op_test.py CUDA version of elementwise power + rename to Pow + gradient 2017-03-07 10:20:40 -08:00
prepend_dim_test.py PrependDimOp 2017-08-24 18:52:05 -07:00
python_op_test.py Add unittest.main() lines to test scripts under python/operator_test 2016-11-29 15:18:37 -08:00
rank_loss_operator_test.py improve pair_wise_loss operator to support multiple sessions 2017-07-28 15:12:47 -07:00
rebatching_queue_test.py caffe2: rebatching queue for MultiTask 2017-05-02 15:22:46 -07:00
record_queue_test.py Add unittest.main() lines to test scripts under python/operator_test 2016-11-29 15:18:37 -08:00
recurrent_network_test.py Revert D5001637: [Caffe2][RNN] Threaded dependency-aware RNNExecutor (frontier/diagonal execution). 2017-08-16 03:21:49 -07:00
reduce_ops_test.py ReduceBack{Sum|Mean}Op CPU & GPU implementation 2017-03-13 16:19:58 -07:00
reduction_ops_test.py Optimize SumSqrElementsOp for CUDA 2017-06-16 17:03:38 -07:00
relu_op_test.py Change back the function signature of relu gradient to only use 2017-04-13 22:08:09 -07:00
reshape_ops_test.py Allow test discovery in caffe2/python/ 2017-03-14 18:16:41 -07:00
resize_op_test.py added gradients for ResizeNearest (CPU + CUDA) and ref 2017-07-07 14:19:42 -07:00
rnn_cell_test.py soft-coverage attention 2017-08-31 21:21:54 -07:00
segment_ops_test.py CUDA SparseLengthsWeightedSum 2017-08-22 15:42:02 -07:00
sequence_ops_test.py add gpu support for caffe2-seq2seq 2017-03-17 05:19:14 -07:00
shape_inference_test.py TensorInference function for LengthsSum and such 2017-08-31 09:32:48 -07:00
sinusoid_position_encoding_op_test.py Optimizations for Caffe2 SinusoidPositionEncodingOp 2017-08-22 00:04:06 -07:00
softmax_ops_test.py stabilize softmax_ops_test 2017-06-22 13:50:32 -07:00
softplus_op_test.py Fix gradient checking for softplus op 2017-06-21 05:35:43 -07:00
sparse_gradient_checker_test.py Fixes for ops without a CUDA backend 2017-03-29 14:36:09 -07:00
sparse_lengths_sum_benchmark.py Add a quick SparseLengthsSum benchmark. 2017-07-24 13:17:47 -07:00
sparse_ops_test.py Fixes range/xrange for Python 3 2017-06-07 00:04:26 -07:00
sparse_to_dense_mask_op_test.py add gradient for SparseToDenseMask operator 2017-08-01 13:05:03 -07:00
spatial_bn_op_test.py Allow inplace for spatial_bn_op 2017-08-17 09:18:55 -07:00
specialized_segment_ops_test.py Code generator for and high-performance emebding look-up kernels, supporting 2017-08-30 16:22:11 -07:00
square_root_divide_op_test.py float support for square root divide 2017-07-27 17:40:40 -07:00
stats_ops_test.py Core unit test fixes for Python 3 2017-06-23 13:22:16 -07:00
string_ops_test.py Remove map() and filter() in favor of comprehensions 2017-05-30 15:32:58 -07:00
text_file_reader_test.py Core unit test fixes for Python 3 2017-06-23 13:22:16 -07:00
tile_op_test.py Fixes in tests to support numpy >= 0.12 2017-06-15 14:17:20 -07:00
top_k_test.py Implement TopKOp for GPU 2017-06-17 08:47:38 -07:00
unique_uniform_fill_op_test.py UniqueUniformFillOp 2017-02-15 16:00:44 -08:00
utility_ops_test.py Adding a range operator similar to np.arange 2017-08-18 14:45:56 -07:00
video_input_op_test.py Skip Python tests which require opencv or lmdb 2017-06-08 13:34:43 -07:00