pytorch/test
Jane Xu 09ce9b5877 Store test file in S3 as well for every TestSuite (#52869)
Summary:
We want to store the file names that triggers each test suite so that we can use this data for categorizing those test files.

~~After considering several solutions, this one is the most backwards compatible, and the current test cases in test_testing.py for print test stats don't break.~~

The previous plan did not work, as there are multiple Python test jobs that spawn the same suites. Instead, the new S3 format will store test files (e.g., `test_nn` and `distributed/test_distributed_fork`) which will contain the suites they spawn, which will contain the test cases run within the suite. (Currently, there is no top layer of test files.)

Because of this major structural change, a lot of changes have now been made (thank you samestep!) to test_history.py and print_test_stats.py to make this new format backwards compatible.

Old test plan:
Make sure that the data is as expected in S3 after https://github.com/pytorch/pytorch/pull/52873 finishes.

Pull Request resolved: https://github.com/pytorch/pytorch/pull/52869

Test Plan: Added tests to test_testing.py which pass, and CI.

Reviewed By: samestep

Differential Revision: D26672561

Pulled By: janeyx99

fbshipit-source-id: f46b91e16c1d9de5e0cb9bfa648b6448d979257e
2021-03-02 07:36:00 -08:00
..
backward_compatibility Update backwards compatibility check to ignore reverted op (#52841) 2021-02-25 08:28:38 -08:00
benchmark_utils Revert "Revert D25199264: Enable callgrind collection for C++ snippets" (#48720) 2020-12-02 11:10:11 -08:00
bottleneck_test
cpp [NNC] Handle non literal constant bounds in Unroll. (#53029) 2021-03-02 00:35:27 -08:00
cpp_api_parity Add PixelUnshuffle (#49334) 2020-12-22 20:14:55 -08:00
cpp_extensions Move generator state APIs to ATen (#49589) 2021-01-06 18:26:56 -08:00
custom_backend Removes deprecated preprocess method from the backend interface (#52258) 2021-02-17 17:53:36 -08:00
custom_operator [BE] move more unittest.main() to run_tests() (#50923) 2021-01-25 07:23:09 -08:00
distributed Revert D26696938: Update and expose ZeroRedundancyOptimizer docs 2021-03-02 07:14:23 -08:00
distributions Add sample validation for LKJCholesky.log_prob (#52763) 2021-02-25 16:12:29 -08:00
error_messages
expect Trace linear as aten::linear (#51897) 2021-02-19 10:20:42 -08:00
fx Extend subgraph_rewriter logic (#51532) 2021-02-03 18:14:37 -08:00
jit Fix jit.trace mis-handling of InterfaceType (#53052) 2021-03-02 00:40:19 -08:00
jit_hooks [WIP] JIT Static Hooks: cpp tests (#49547) 2021-01-20 09:12:57 -08:00
mobile [pytorch] reintroduce static dispatch (#51957) 2021-02-19 11:41:39 -08:00
namespace_b
onnx [ONNX] Update LayerNorm symbolic to handle autocasting (#52199) (#52350) 2021-02-19 10:57:15 -08:00
optim
package_a [fx/package] make GraphModules packageable (#51976) 2021-02-23 22:43:00 -08:00
quantization [quant][graphmode][fx][fp16] Add fp16 support for silu (#52865) 2021-03-02 02:11:29 -08:00
scripts
test_img
type_hint_tests Fix mypy type hint for AdaptiveAvgPool2,3d, AdaptiveMaxPool2,3d (#49963) 2021-01-06 09:47:15 -08:00
HowToWriteTestsUsingFileCheck.md
linear.py Move conv to mkldnn (#51483) 2021-03-01 21:19:27 -08:00
module_a.py
run_test.py [JIT] Add torch._C._jit submodule (#52910) 2021-02-26 16:05:05 -08:00
simulate_nccl_errors.py
test_autograd.py Reenables skipped tests for all CUDA versions except 11.2 (#52359) 2021-02-19 15:30:55 -08:00
test_binary_ufuncs.py remove pointless test (#52609) 2021-02-22 16:25:04 -08:00
test_bundled_images.py
test_bundled_inputs.py [Pytorch Mobile] Improve Bundled Inputs Error Checking (#52386) 2021-02-24 13:55:45 -08:00
test_complex.py Add serialization logic for complex numbers (#51287) 2021-01-28 17:25:35 -08:00
test_cpp_api_parity.py
test_cpp_extensions_aot.py Test TORCH_LIBRARY in CUDA extension (#47524) 2020-11-16 13:12:22 -08:00
test_cpp_extensions_jit.py Internal gradcheck wrapper in testing._internal that sets certain flags to True (#51133) 2021-01-29 09:13:37 -08:00
test_cuda_primary_ctx.py
test_cuda.py [ROCm] Enable test cases in test_cuda.py for ROCm (#52739) 2021-03-01 12:54:40 -08:00
test_dataloader.py [ROCm] Enable test cases in test/test_dataloader.py for ROCm (#52766) 2021-03-01 13:32:35 -08:00
test_datapipe.py [DataLoader] Move BufferedShuffle from Dataset to DataPipe (#52141) 2021-03-01 12:54:44 -08:00
test_dataset.py [DataLoader] Rename Functional DataSet to DataPipe (#51488) 2021-02-03 07:01:09 -08:00
test_determination.py Add option for cpp_extensions to compile standalone executable (#47862) 2020-12-01 20:03:08 -08:00
test_dispatch.py Improve docs around Math/DefaultBackend & add PythonDispatcher class. (#50854) 2021-01-25 23:10:36 -08:00
test_expecttest.py [BE] replace unittest.main with run_tests (#50451) 2021-01-13 10:33:08 -08:00
test_foreach.py Probable fix for out of place BinaryOpScalar bad values and/or IMAs on 11.2 (ci-all edition) (#52634) 2021-03-01 12:41:24 -08:00
test_function_schema.py
test_functional_autograd_benchmark.py Skip test_functional_autograd_benchmark during code coverage (#50183) 2021-01-07 11:17:21 -08:00
test_futures.py [JIT/Futures] support set_exception api (#50983) 2021-02-04 20:22:19 -08:00
test_fx_experimental.py [fx] Separate globals assignment from code generation (#51974) 2021-02-23 13:48:03 -08:00
test_fx.py [FX][EZ] Hoist custom class .so loading into setUp (#52883) 2021-02-25 18:46:05 -08:00
test_indexing.py index_put_ for complex tensors on CUDA (#51148) 2021-01-27 09:11:37 -08:00
test_jit_cuda_fuser.py [ROCm] disable tests for ROCm 4.0.1 (#51510) 2021-02-02 14:39:08 -08:00
test_jit_disabled.py [BE] replace unittest.main with run_tests (#50451) 2021-01-13 10:33:08 -08:00
test_jit_fuser_legacy.py
test_jit_fuser_te.py add a full pipeline test for a TypeCheck (#52933) 2021-03-01 10:58:08 -08:00
test_jit_fuser.py Clean up some type annotations in caffe2/test (#49943) 2021-01-13 10:01:55 -08:00
test_jit_legacy.py
test_jit_profiling.py Remove redundant code for unsupported Python versions (#49486) 2021-01-06 12:45:46 -08:00
test_jit_py3.py Workaround Python3.9 limitations in test_jit_py3 (#51088) 2021-01-26 12:49:35 -08:00
test_jit_simple.py Remove redundant code for unsupported Python versions (#49486) 2021-01-06 12:45:46 -08:00
test_jit_string.py [JIT] Allow implicit boolean conversion of containers (#51683) 2021-02-05 00:34:35 -08:00
test_jit.py Add support to compare devices (#53045) 2021-03-01 21:04:43 -08:00
test_kernel_launch_checks.py Fix check_kernel_launches.py for macros and provide extended context (#49365) 2020-12-14 22:09:33 -08:00
test_license.py concantenate LICENSE files when building a wheel (#51634) 2021-02-08 08:28:46 -08:00
test_linalg.py Revert D26375734: Implemented torch.linalg.multi_dot 2021-02-25 00:43:57 -08:00
test_logging.py
test_metal.py
test_mkldnn.py Dont error on 0-dim in convolution (#51922) 2021-03-01 21:22:30 -08:00
test_mobile_optimizer.py [Reland] Update XNNPACK (#52691) 2021-02-24 06:40:38 -08:00
test_multiprocessing_spawn.py
test_multiprocessing.py [BE] unified run_process_no_exception code (#49774) 2021-01-04 13:43:09 -08:00
test_namedtensor.py Enhance Tensor.unflatten to support -1 as the inferred size (#51955) 2021-02-18 08:37:41 -08:00
test_namedtuple_return_api.py make forward AD API private (#51693) 2021-02-04 19:02:29 -08:00
test_native_functions.py
test_nn.py Add 64bit indexing support for softmax (#52713) 2021-02-24 21:39:58 -08:00
test_numba_integration.py Drop unused imports from test (#49973) 2021-01-07 12:09:38 -08:00
test_numpy_interop.py Mypy fixes for pytorch master (#52090) 2021-02-17 10:39:51 -08:00
test_op_aliases.py OpInfo: Added clamp and trunc tests with aliases (#51167) 2021-02-10 05:36:18 -08:00
test_openmp.py
test_ops.py Add OpInfo entries for i0 and logical_not (#51956) 2021-02-23 10:12:05 -08:00
test_optim.py Probable fix for out of place BinaryOpScalar bad values and/or IMAs on 11.2 (ci-all edition) (#52634) 2021-03-01 12:41:24 -08:00
test_overrides.py Fix pickling for Tensor subclasses (redo) (#47732) 2021-02-01 07:32:52 -08:00
test_package.py [fx/package] make GraphModules packageable (#51976) 2021-02-23 22:43:00 -08:00
test_profiler.py Add FLOPS support to the new profiler API. (#51734) 2021-02-05 15:03:35 -08:00
test_pruning_op.py Rowwise Prune op (Add the test to OSS run_test), Make the op private. (#46131) 2021-01-29 06:08:18 -08:00
test_public_bindings.py [JIT] Add torch._C._jit submodule (#52910) 2021-02-26 16:05:05 -08:00
test_pytree.py
test_quantization.py NS for FX: add test for a simple sparsenn model (#52092) 2021-02-18 08:17:57 -08:00
test_reductions.py Enable min & max for Float16 & BFloat16 (#51244) 2021-02-18 23:13:51 -08:00
test_serialization.py Enable tests using named temp files on Windows (#49640) 2020-12-29 09:57:35 -08:00
test_set_default_mobile_cpu_allocator.py [Pytorch] Add python binding to use mobile cpu allocator. (#52323) 2021-02-17 08:42:23 -08:00
test_shape_ops.py Removes nonzero method warning (#51618) 2021-02-04 17:43:43 -08:00
test_show_pickle.py [BE] replace unittest.main with run_tests (#50451) 2021-01-13 10:33:08 -08:00
test_sort_and_select.py Stable sort for the CPU take 2. (#51790) 2021-02-19 09:28:57 -08:00
test_sparse.py [BE] _get_torch_cuda_version should return tuple (#52409) 2021-02-18 09:28:38 -08:00
test_spectral_ops.py [ROCm] enable fft tests (#51581) 2021-02-17 13:27:55 -08:00
test_static_runtime.py Clean up some type annotations in caffe2/test (#49943) 2021-01-13 10:01:55 -08:00
test_tensor_creation_ops.py [testing] Port torch.{repeat, tile} tests to use OpInfo machinery (#50199) 2021-01-19 06:02:27 -08:00
test_tensorboard.py
test_tensorexpr_pybind.py [TensorExpr] Reland: "PyBind: bind ExternalCalls." (#53063) 2021-03-01 20:44:10 -08:00
test_tensorexpr.py Add NNC support for aten::hardtanh (a hot operation in mobilenet v2/v3) (#52394) 2021-02-18 22:56:03 -08:00
test_testing.py Store test file in S3 as well for every TestSuite (#52869) 2021-03-02 07:36:00 -08:00
test_throughput_benchmark.py Enable tests using named temp files on Windows (#49640) 2020-12-29 09:57:35 -08:00
test_torch.py [testing] Add Opinfo for torch.frac and minor fixes (#52660) 2021-03-01 04:58:31 -08:00
test_type_hints.py Mypy fixes for pytorch master (#52090) 2021-02-17 10:39:51 -08:00
test_type_info.py
test_type_promotion.py [numpy] torch.erf{c} : promote integer inputs to float (#48472) 2020-11-27 15:08:40 -08:00
test_unary_ufuncs.py [testing] Add Opinfo for torch.frac and minor fixes (#52660) 2021-03-01 04:58:31 -08:00
test_utils.py Store test file in S3 as well for every TestSuite (#52869) 2021-03-02 07:36:00 -08:00
test_view_ops.py Add tensor.view(dtype) (#47951) 2021-01-08 06:55:21 -08:00
test_vmap.py Add private mechanism to toggle vmap fallback warnings (#51218) 2021-01-28 13:05:00 -08:00
test_vulkan.py
test_xnnpack_integration.py