pytorch/caffe2/python
Kevin Wilfong 88b1f6619e Return list of AccessedFeatures from get_accessed_features (#23983)
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/23983

While testing I realized that model layers can extract different types of features from the same column.  For example, MultifeedFeaturesTransform uses float and ID list features from the "features" column.

get_accessed_features returns a map from column to AccessedFeatures, and AccessedFeatures only has the feature IDs for one feature type.  This is incompatible with have multiple types of features per column, one type ends up overwriting another in the map.

To fix this, I've modified get_accessed_features to return a map from column to a list of AccessedFeatures objects.

Reviewed By: itomatik

Differential Revision: D16693845

fbshipit-source-id: 2099aac8dc3920dd61de6b6ad5cf343c864803bc
2019-08-14 10:50:27 -07:00
..
docs Fix several DeprecationWarning: invalid escape sequence (#15733) 2019-01-05 08:53:35 -08:00
examples Adding models to jenkins benchmark script (#21010) 2019-05-30 15:17:40 -07:00
helpers Add elementwise_affine for LayerNormGradientOp (#19982) 2019-05-03 15:33:46 -07:00
ideep Removed work around for convolution transpose op since the bug has be… (#22184) 2019-06-25 14:34:34 -07:00
layers Return list of AccessedFeatures from get_accessed_features (#23983) 2019-08-14 10:50:27 -07:00
mint re-enable copy of python files, but be careful that the copy is only … (#14982) 2018-12-11 16:54:08 -08:00
mkl implement operators for DNNLOWP (#18656) 2019-04-10 12:04:39 -07:00
modeling Support plot norm of specific embeddings of a LUT in diagnose_options (#19809) 2019-05-18 01:08:45 -07:00
models Adding ShufflenetV2 to caffe2's benchmark suite. (#20180) 2019-05-23 20:40:17 -07:00
onnx weight_names bug fix (#23848) 2019-08-06 10:58:43 -07:00
operator_test add aligned option to RoIAlign 2019-08-07 21:22:33 -07:00
predictor Supporting Manifold DB in Predictor Exporter (#22334) 2019-07-01 16:17:02 -07:00
rnn Unify cuda and hip device types in Caffe2 python front end (#14221) 2018-11-29 14:00:16 -08:00
serialized_test Output sequence probability with CTC beam search, optional multiple output sequences (#21927) 2019-07-02 17:29:13 -07:00
test Enforce import order to make protobuf cpp implementation in python work (#18560) 2019-04-03 13:17:08 -07:00
trt Skip tests if C2/ONNX models cannot be read (#18494) 2019-03-27 11:21:44 -07:00
__init__.py Revert #17191 and #17215 that no longer apply on Windows (#17567) 2019-03-01 10:37:27 -08:00
_import_c_extension.py Enforce import order to make protobuf cpp implementation in python work (#18560) 2019-04-03 13:17:08 -07:00
allcompare_test.py
attention.py [Caffe2] Update elementwise ops to support numpy style boradcast (#8070) 2018-06-05 15:49:16 -07:00
benchmark_generator.py
binarysize.py
brew_test.py Move tanh function to math (#9328) 2018-07-11 13:59:50 -07:00
brew.py Testing for folded conv_bn_relu (#19298) 2019-04-16 19:04:06 -07:00
build.py
cached_reader.py Pass loop_over optional parameter for cached reader properly. (#21929) 2019-06-19 18:15:32 -07:00
caffe_translator_test.py Fix several ResourceWarning: unclosed file (#15746) 2019-01-09 15:36:53 -08:00
caffe_translator.py Fix several ResourceWarning: unclosed file (#15746) 2019-01-09 15:36:53 -08:00
checkpoint_test.py Revert D9566744: [New Checkpoint] Kill the dummy TaskOutput when task.get_step() (#11164) 2018-08-31 22:25:57 -07:00
checkpoint.py Remove setting logger level in caffe2.python.checkpoint (#19803) 2019-05-10 07:00:58 -07:00
CMakeLists.txt Fix CMakeLists.txt for Int8 python bindings (#15047) 2018-12-11 10:48:47 -08:00
cnn.py Unify cuda and hip device types in Caffe2 python front end (#14221) 2018-11-29 14:00:16 -08:00
compatibility.py migrating deprecated calls without abc module for containers (#11515) 2018-09-13 15:09:22 -07:00
context_test.py
context.py Resolve name conflict of ContextManager (#7244) 2018-06-22 00:41:51 -04:00
control_ops_grad_test.py fix auto grad summing for IfOp where intermediate output needs renaming (#14772) 2018-12-09 08:26:46 -08:00
control_ops_grad.py DeviceScope support for CUDA and testing (#15357) 2019-01-30 18:42:12 -08:00
control_ops_util.py
control_test.py
control.py
convert_test.py New serialization format (#12384) 2018-10-16 16:36:58 -07:00
convert.py New serialization format (#12384) 2018-10-16 16:36:58 -07:00
convnet_benchmarks_test.py Skip convnets benchmark in rocm CI (#17331) 2019-02-20 21:12:24 -08:00
convnet_benchmarks.py
core_gradients_test.py add extra info for the auto gen sum ops 2019-03-27 14:56:32 -07:00
core_test.py Extend Net.RunAllOnGPU() to support RecurrentNetwork op (#15713) 2019-02-08 15:48:42 -08:00
core.py Fix the GPU trainer for NoneCalibration and RNN 2019-07-01 22:55:18 -07:00
crf_predict.py Move crf in caffe2 from fb to oss (#12200) 2018-10-01 18:31:41 -07:00
crf_viterbi_test.py Move crf in caffe2 from fb to oss (#12200) 2018-10-01 18:31:41 -07:00
crf.py Productionize CRF layer in PyText (#10362) 2018-08-22 00:25:26 -07:00
data_parallel_model_test.py Unify cuda and hip device types in Caffe2 python front end (#14221) 2018-11-29 14:00:16 -08:00
data_parallel_model.py skip import nccl and gloo_gpu in cpu machine (#22522) 2019-07-10 11:56:56 -07:00
data_workers_test.py
data_workers.py Fixed log message (#10874) 2018-09-05 09:55:52 -07:00
dataio_test.py Pass loop_over optional parameter for cached reader properly. (#21929) 2019-06-19 18:15:32 -07:00
dataio.py Rearrange stopping condition in CompositeReader (#20062) 2019-05-06 15:06:32 -07:00
dataset.py Update from facebook (#7855) 2018-05-29 11:38:02 -07:00
db_file_reader.py Update from Facebook (#8887) 2018-06-26 14:55:48 -07:00
db_test.py
device_checker.py
dlpack.h Upgrade DLPack 2018-11-12 15:59:46 -08:00
dyndep.py
embedding_generation_benchmark.py
experiment_util.py
extension_loader.py always restore dlopen flag in dyndep (#22958) 2019-07-17 10:26:25 -07:00
filler_test.py caffe2 - Expose tensor filler util to Python (#18886) 2019-04-08 11:54:10 -07:00
functional_test.py Add support for specifying device_option in Functional (#9619) 2018-07-24 14:41:59 -07:00
functional.py Caffe2 Functional enforcing inplace output (#10797) 2018-08-23 22:42:47 -07:00
fused_8bit_rowwise_conversion_ops_test.py
gradient_check_test.py Unify gpu_support variable in python tests (#16748) 2019-02-07 00:29:51 -08:00
gradient_checker.py Adding gradient to Boolean Mask operator (#21423) 2019-06-06 20:48:47 -07:00
gru_cell.py
hip_test_util.py Make CUDNN an alias of MIOPEN for HIP ops (#12278) 2018-10-24 17:07:31 -07:00
hsm_util.py
hypothesis_test_util.py Hypothesis tests: add ability to enforce shape inference (#23935) 2019-08-13 05:32:41 -07:00
hypothesis_test.py Fix SliceGradientOp to handle properly empty batches (#23784) 2019-08-06 02:43:32 -07:00
ideep_test_util.py
layer_model_helper.py Fix the bug in regularizer matching (#23485) 2019-08-02 15:54:48 -07:00
layer_model_instantiator.py
layer_parameter_sharing_test.py Add validator for optimizers when parameters are shared 2019-04-17 21:10:38 -07:00
layer_test_util.py
layers_test.py Return list of AccessedFeatures from get_accessed_features (#23983) 2019-08-14 10:50:27 -07:00
lengths_reducer_fused_8bit_rowwise_ops_test.py make the threshold for acurracy more precise (#17194) 2019-02-20 13:14:11 -08:00
lengths_reducer_rowwise_8bit_ops_test.py
lstm_benchmark.py Unify cuda and hip device types in Caffe2 python front end (#14221) 2018-11-29 14:00:16 -08:00
memonger_test.py Unify gpu_support variable in python tests (#16748) 2019-02-07 00:29:51 -08:00
memonger.py Fix typo "properlyh" 2019-08-09 11:43:04 -07:00
mkl_test_util.py
model_device_test.py Unify gpu_support variable in python tests (#16748) 2019-02-07 00:29:51 -08:00
model_helper_test.py keep net type info when generating model complete net (#11032) 2018-09-04 21:10:06 -07:00
model_helper.py Remove the identical if branch (#18019) 2019-03-15 13:14:26 -07:00
modifier_context.py
mpi_python.cc
muji_test.py Unify cuda and hip device types in Caffe2 python front end (#14221) 2018-11-29 14:00:16 -08:00
muji.py Unify cuda and hip device types in Caffe2 python front end (#14221) 2018-11-29 14:00:16 -08:00
net_builder_test.py
net_builder.py
net_drawer.py Allow customization of blob node in net_drawer (#16915) 2019-02-12 15:02:50 -08:00
net_printer_test.py
net_printer.py Fix spelling errors (#21665) 2019-06-13 15:21:55 -07:00
nomnigraph_test.py nomnigraph - support subgraph visualization (#13795) 2018-11-16 08:19:20 -08:00
nomnigraph_transformations_test.py Add transpose network pass (#13437) 2018-11-01 14:27:07 -07:00
nomnigraph_transformations.py Add transpose network pass (#13437) 2018-11-01 14:27:07 -07:00
nomnigraph.py createUniqueDataNode 2018-10-31 11:16:38 -07:00
normalizer_context.py Update from Facebook (#8887) 2018-06-26 14:55:48 -07:00
normalizer_test.py Update from Facebook (#8887) 2018-06-26 14:55:48 -07:00
normalizer.py Enable alternative LayerNorm impl in FisherGan (#12178) 2018-10-11 17:36:11 -07:00
numa_benchmark.py Revert D13205604: Move numa.{h, cc} to c10/util 2018-12-07 10:01:25 -08:00
numa_test.py Move numa.{h, cc} to c10/util (#15024) 2018-12-12 12:21:10 -08:00
observer_test.py
operator_fp_exceptions_test.py Caffe2 - Add flag to fails if float point exceptions is detected in operator runs (#18040) 2019-03-16 12:28:05 -07:00
optimizer_context.py
optimizer_test_util.py Unify cuda and hip device types in Caffe2 python front end (#14221) 2018-11-29 14:00:16 -08:00
optimizer_test.py Unify gpu_support variable in python tests (#16748) 2019-02-07 00:29:51 -08:00
optimizer.py Add validator for optimizers when parameters are shared 2019-04-17 21:10:38 -07:00
parallel_workers_test.py Add event and event_counter columns to caffe2_usage_tracer table (#21739) 2019-06-13 12:06:02 -07:00
parallel_workers.py
parallelize_bmuf_distributed_test.py Unify gpu_support variable in python tests (#16748) 2019-02-07 00:29:51 -08:00
pipeline_test.py
pipeline.py SNNTest with Data Preproc Service (#11707) 2018-09-17 21:25:49 -07:00
predictor_constants.py
pybind_state_dlpack.cc Upgrade DLPack 2018-11-12 15:59:46 -08:00
pybind_state_dlpack.h Remove PythonOp non-CPU path and PytorchOp (#15417) 2019-01-02 16:36:37 -08:00
pybind_state_gpu.cc add simple memory analyzer and log warning if GPU underutilized (#21024) 2019-05-28 19:58:54 -07:00
pybind_state_hip.cc Fix get_gpu_memory_info in non-cuda builds (#21054) 2019-05-28 23:05:15 -07:00
pybind_state_ideep.cc Upgrade mkldnn-bridge for dnnlowp support (#16308) 2019-04-03 12:47:17 -07:00
pybind_state_int8.cc Renaming meta() to dtype() - 2/2 (#13334) 2018-10-30 18:24:30 -07:00
pybind_state_nomni.cc nomnigraph - support subgraph visualization (#13795) 2018-11-16 08:19:20 -08:00
pybind_state_registry.cc Move registry fully to c10 (#12077) 2018-09-27 03:09:54 -07:00
pybind_state_registry.h Move registry fully to c10 (#12077) 2018-09-27 03:09:54 -07:00
pybind_state.cc weight_names bug fix (#23848) 2019-08-06 10:58:43 -07:00
pybind_state.h Replace caffe2::DeviceGuard with c10::cuda::CUDAGuard (#17623) 2019-03-06 10:48:15 -08:00
python_op_test.py Clean up a couple of items in the C2 test scaffolding (WIP) (#7847) 2018-11-07 09:16:13 -08:00
queue_util.py
record_queue.py
recurrent.py
regularizer_context.py Update from Facebook (#8887) 2018-06-26 14:55:48 -07:00
regularizer_test.py Implement "trimmed lasso" regularization and support all available regularization in a single interface (#22966) 2019-07-17 16:12:31 -07:00
regularizer.py Fix the bug in regularizer matching (#23485) 2019-08-02 15:54:48 -07:00
rnn_cell.py Unify cuda and hip device types in Caffe2 python front end (#14221) 2018-11-29 14:00:16 -08:00
schema_test.py Pass LRU hash output evicted_values to SparseLookup (#21389) 2019-07-02 11:27:37 -07:00
schema.py Pass LRU hash output evicted_values to SparseLookup (#21389) 2019-07-02 11:27:37 -07:00
scope_test.py Add EmptyNameScope to allow you jump out from current scope. (#14631) 2018-12-12 01:39:50 -08:00
scope.py Add EmptyNameScope to allow you jump out from current scope. (#14631) 2018-12-12 01:39:50 -08:00
session_test.py
session.py
sparse_to_dense_mask_test.py Increase static tolerance for negative feature ids 2019-05-20 19:09:22 -07:00
sparse_to_dense_test.py
task_test.py caffe2/python/task: added __repr__ methods to all task definitions (#15250) 2018-12-17 16:02:16 -08:00
task.py A trivial typo fix in caffe2.python (#15907) 2019-01-17 04:57:34 -08:00
test_util.py caffe2 - support flaky operator tests for caffe2 build (#18155) 2019-03-25 16:58:34 -07:00
text_file_reader.py Create Node2Vec ModuleKeeper 2019-04-01 10:36:23 -07:00
timeout_guard.py
toy_regression_test.py Enable junk fill for the default CPU allocator (#13377) 2018-11-08 00:02:37 -08:00
transformations_test.py Remove sinkMaxPool transformation (#17694) 2019-03-12 20:10:46 -07:00
transformations.py support pre-convert filter format for mkldnn training mode and change 'OptimizeForIdeep' to 'OptimizeForMkldnn' (#15171) 2019-03-29 19:00:48 -07:00
tt_core_test.py
tt_core.py
utils_test.py Convert Arguments to dictionary (#13436) 2018-11-01 14:27:05 -07:00
utils.py Query caffe2 operator stats for detailed execution info (#20924) 2019-06-13 23:41:04 -07:00
visualize.py
workspace_test.py Pass Variable into Caffe2 ops, by requiring that the Variable doesn't require grad (#22473) 2019-07-08 11:31:10 -07:00
workspace.py Fix get_gpu_memory_info in non-cuda builds (#21054) 2019-05-28 23:05:15 -07:00