Commit Graph

5 Commits

Author SHA1 Message Date
Lily Johnson
cbb6ab6d88 [package] ignore dunder import errors (#61148)
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/61148

Changes `__import__` processing to silently skip cases where the `__import__` statement cannot be parsed. Adds failed imports to a list retrievable by `PackageExporter.failed_dunder_import_list()`.

Test Plan: Imported from OSS

Reviewed By: suo

Differential Revision: D29559680

Pulled By: Lilyjjo

fbshipit-source-id: 2513d0b9ef271c85cadc3f5a013fbd8c8de80b46
2021-07-09 15:27:08 -07:00
Lillian Johnson
2c6f5e8a12 [package] PackageExporter __import__ logic to not parse dynamic cases (#57283)
Summary: Pull Request resolved: https://github.com/pytorch/pytorch/pull/57283

Test Plan: Imported from OSS

Reviewed By: suo

Differential Revision: D28095858

Pulled By: Lilyjjo

fbshipit-source-id: c3cec074e6b2c48a09785fa0c02cd576b7ec94d9
2021-04-29 14:21:33 -07:00
Lillian Johnson
c244d1c540 [package] resolve __import__ calls on export (#55153)
Summary: Pull Request resolved: https://github.com/pytorch/pytorch/pull/55153

Test Plan: Imported from OSS

Reviewed By: suo

Differential Revision: D27504536

Pulled By: Lilyjjo

fbshipit-source-id: 5e3e10f213c6e0cf1755d18eb19727515362f91a
2021-04-21 15:43:15 -07:00
Michael Suo
8c2c9450cc [package] autoformat (#53783)
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/53783

Use isort + black on torch/package/

Test Plan: Imported from OSS

Reviewed By: SplitInfinity

Differential Revision: D26969020

Pulled By: suo

fbshipit-source-id: e2c0738e79bf41b6342355eb7025998178c35dc9
2021-03-15 17:18:43 -07:00
Zachary DeVito
cb75addee4 torch.package - a way to package models and code (#45015)
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/45015

torch.package allows you to write packages of code, pickled python data, and
arbitrary binary and text resources into a self-contained package.

torch.package.PackageExporter writes the packages and
torch.package.PackageImporter reads them.

The importers can load this code in a hermetic way, such that code is loaded
from the package rather than the normal python import system. This allows
for the packaging of PyTorch model code and data so that it can be run
on a server or used in the future for transfer learning.

The code contained in packages is copied file-by-file from the original
source when it is created, and the file format is a specially organized
zip file. Future users of the package can unzip the package, and edit the code
in order to perform custom modifications to it.

The importer for packages ensures that code in the module can only be loaded from
within the package, except for modules explicitly listed as external using :method:`extern_module`.
The file `extern_modules` in the zip archive lists all the modules that a package externally depends on.
This prevents "implicit" dependencies where the package runs locally because it is importing
a locally-installed package, but then fails when the package is copied to another machine.

Test Plan: Imported from OSS

Reviewed By: SplitInfinity

Differential Revision: D23824337

Pulled By: zdevito

fbshipit-source-id: 1247c34ba9b656f9db68a83e31f2a0fbe3bea6bd
2020-09-22 21:21:21 -07:00