Taylor Robie dda95e6914 More Timer refinement (#46023) ... Summary: This PR just adds more polish to the benchmark utils: 1) `common.py`, `timer.py`, and `valgrind_wrapper/timer_interface.py` are now MyPy strict compliant. (except for three violations due to external deps.) Compare and Fuzzer will be covered in a future PR. 2) `CallgrindStats` now uses `TaskSpec` rather than accepting the individual fields which brings it closer to `Measurement`. 3) Some `__repr__` logic has been moved into `TaskSpec` (which `Measurement` and `CallgrindStats` use in their own `__repr__`s) for a more unified feel and less horrible f-string hacking, and the repr's have been given a cleanup pass. 4) `Tuple[FunctionCount, ...]` has been formalized as the `FunctionCounts` class, which has a much nicer `__repr__` than just the raw tuple, as well as some convenience methods (`__add__`, `__sub__`, `filter`, `transform`) for easier DIY stat exploration. (I find myself using the latter two a lot now.) My personal experience is that manipulating `FunctionCounts` is massively more pleasant than the raw tuples of `FunctionCount`. (Though it's still possible to get at the raw data if you want.) 5) Better support for multi-line `stmt` and `setup`. 6) Compare now also supports rowwise coloring, which is often the more natural layout for A/B testing. 7) Limited support for `globals` in `collect_callgrind`. This should make it easier to benchmark JIT models. (CC ZolotukhinM) 8) More unit tests, including extensive tests for the Callgrind stats manipulation APIs. 9) Mitigate issue with `MKL_THREADING_LAYER` when run in Jupyter. (https://github.com/pytorch/pytorch/issues/37377) Pull Request resolved: https://github.com/pytorch/pytorch/pull/46023 Test Plan: changes should be covered by existing and new unit tests. Reviewed By: navahgar, malfet Differential Revision: D24313911 Pulled By: robieta fbshipit-source-id: 835d4b5cde336fb7ff0adef3c0fd614d64df0f77		2020-10-15 16:32:53 -07:00
..
callgrind_artifacts.json
test_benchmark_utils.py