Summary:
Fixes https://github.com/pytorch/pytorch/issues/66232
This should be the last immediate task. I anticipate test ownership will change overtime but this is the last big thing to close it out
Pull Request resolved: https://github.com/pytorch/pytorch/pull/67859
Reviewed By: soulitzer
Differential Revision: D32210534
Pulled By: janeyx99
fbshipit-source-id: 7fd835d87d9d35d49ec49de1fcfa29b085133e99
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/66006
Previously, this was resulting in a key collision and a crash.
ghstack-source-id: 139342089
Test Plan: Ran webdriver test locally.
Reviewed By: dhruvbird
Differential Revision: D31281092
fbshipit-source-id: f31311726c681d6d7e0504ff8e84c888af9054f0
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/66005
ghstack-source-id: 139342091
Test Plan: Unit test, and used in a notebook.
Reviewed By: dhruvbird
Differential Revision: D31281091
fbshipit-source-id: 1e4d0713b9796a3d182de9e676c3b3c3b1610d6e
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/60702
- Instead of traversing and counting all tensor memory, collect a map
from storage key to storage info while traversing. Add up sizes at
the end to avoid double counting.
- Count tensor memory from constants as well.
Test Plan: Ran webdriver test.
Reviewed By: dhruvbird
Differential Revision: D29380396
Pulled By: dreiss
fbshipit-source-id: 6d0fd66f677fe23c851aa218387aa4dc59502b1e
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/60701
The unit test previously only tested that the dump could complete
successfully. It was not able to verify that any JS worked properly.
Now we can test the JS as long as webdriver is installed.
Tweaked the implementation of Hider a bit to make it easier for tests to
find and open them.
I disabled the tests by default since I don't want to deal with
webdriver in CI. Enable them with the environment variable
RUN_WEBDRIVER=1.
We could make the tests use headless mode, but it's kind of fun to watch
them run.
Add a test to verify that tensor memory computation is working for the
simple model.
Test Plan: Ran the test.
Reviewed By: dhruvbird
Differential Revision: D29380398
Pulled By: dreiss
fbshipit-source-id: f19d0b05d79ad5a8231e85422976f1889e021c89
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/60699
Also add a unit test for main, which brings the test coverage up to
~98%. Also factor out the "needs importlib.resources" check into a
function for easier reuse.
Test Plan: CI
Reviewed By: dhruvbird
Differential Revision: D29380397
Pulled By: dreiss
fbshipit-source-id: bba16da85bf7bfb4370308e38c844694d01b47eb
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/56868
See __init__.py for a summary of the tool.
The following sections are present in this initial version
- Model Size. Show the total model size, as well as a breakdown by
stored files, compressed files, and zip overhead. (I expect this
breakdown to be a bit more useful once data.pkl is compressed.)
- Model Structure. This is basically the output of
`show_pickle(data.pkl)`, but as a hierarchical structure.
Some structures cause this view to crash right now, but it can be
improved incrementally.
- Zip Contents. This is basically the output of `zipinfo -l`.
- Code. This is the TorchScript code. It's integrated with a blame
window at the bottom, so you can click "Blame Code", then click a bit
of code to see where it came from (based on the debug_pkl). This
currently doesn't render properly if debug_pkl is missing or
incomplete.
- Extra files (JSON). JSON dumps of each json file under /extra/, up to
a size limit.
- Extra Pickles. For each .pkl file in the model, we safely unpickle it
with `show_pickle`, then render it with `pprint` and include it here
if the size is not too large. We aren't able to install the pprint
hack that thw show_pickle CLI uses, so we get one-line rendering for
custom objects, which is not very useful. Built-in types look fine,
though. In particular, bytecode.pkl seems to look fine (and we
hard-code that file to ignore the size limit).
I'm checking in the JS dependencies to avoid a network dependency at
runtime. They were retrieved from the following URLS, then passed
through a JS minifier:
https://unpkg.com/htm@3.0.4/dist/htm.module.js?modulehttps://unpkg.com/preact@10.5.13/dist/preact.module.js?module
Test Plan:
Manually ran on a few models I had lying around.
Mostly tested in Chrome, but I also poked around in Firefox.
Reviewed By: dhruvbird
Differential Revision: D28020849
Pulled By: dreiss
fbshipit-source-id: 421c30ed7ca55244e9fda1a03b8aab830466536d
