Tugsbayasgalan Manlaibaatar
1d02106e03
Preserve source_fn or nn_module_stack in the lifted params ( #105017 )
...
Pull Request resolved: https://github.com/pytorch/pytorch/pull/105017
Approved by: https://github.com/angelayi
2023-07-13 06:03:28 +00:00
zhxchen17
100aff9d4f
[export] Deserialize subgraphs. ( #103991 )
...
Fixes #ISSUE_NUMBER
Pull Request resolved: https://github.com/pytorch/pytorch/pull/103991
Approved by: https://github.com/angelayi , https://github.com/avikchaudhuri
2023-06-26 18:17:44 +00:00
Michael Voznesensky
ec24f1e4cc
Simulate treespec flattening/unflattening ( #101896 )
...
Pull Request resolved: https://github.com/pytorch/pytorch/pull/101896
Approved by: https://github.com/jansel , https://github.com/anijain2305
2023-06-23 10:53:15 +00:00
Michael Voznesensky
e5e9d563c2
Lift user defined attributes into inputs for certain cases (user defined types and tensors) ( #103386 )
...
(1) Lazy (converts to dynamo variable on access only)
(2) Uses existing side effect/reconstruct tech
(3) not tensor opinionated
Pull Request resolved: https://github.com/pytorch/pytorch/pull/103386
Approved by: https://github.com/jansel
2023-06-20 23:45:19 +00:00
Mengwei Liu
a52b6f086d
[export][serde] Add validator to compare deserializer opset version with model opset version ( #103691 )
...
This PR adds a validator to compare model opset version and deserializer opset version. This currently raises exception if any of the version doesn't match.
Note: the validator will only print warning if the op namespace in model is missing from the deserializer.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/103691
Approved by: https://github.com/avikchaudhuri , https://github.com/zhxchen17
2023-06-16 01:36:43 +00:00
Angela Yi
90ef8d58cf
[export] Serialize metadata ( #103274 )
...
Pull Request resolved: https://github.com/pytorch/pytorch/pull/103274
Approved by: https://github.com/zhxchen17
2023-06-15 17:34:12 +00:00
Angela Yi
8dc6001057
[export] Serialize symbolic values ( #103273 )
...
* Modified the SymInt schema to also store the hint of the SymInt if it is represented as a symbol so that when we reconstruct the SymInt, the hint will also exist on the node.
* GraphModuleDeserializer.deserialize now also optionally map of symbol names to range.
ReplaceSymSizeOpPass should not be needed after https://github.com/pytorch/pytorch/pull/103107 lands
Pull Request resolved: https://github.com/pytorch/pytorch/pull/103273
Approved by: https://github.com/avikchaudhuri , https://github.com/zhxchen17
2023-06-13 20:29:47 +00:00
Angela Yi
e930c0fc35
[export] Initial deserialization v2 ( #102716 )
...
v2 of https://github.com/pytorch/pytorch/pull/102126 . mentally stacked on top of https://github.com/pytorch/pytorch/pull/102707
Pull Request resolved: https://github.com/pytorch/pytorch/pull/102716
Approved by: https://github.com/avikchaudhuri , https://github.com/zhxchen17
2023-06-07 16:02:35 +00:00
Angela Yi
3a385656b5
[export] Initial serialization v2 ( #102707 )
...
v2 of https://github.com/pytorch/pytorch/pull/102125 because of git issues
corresponding deserialization diff: https://github.com/pytorch/pytorch/pull/102716
Implementing serialization of the exported program to a python dataclass, and then from that dataclass to json. This is split into a couple of sections:
- `serialize(ep: ep.ExportedProgram, opset_version: Dict[str, int]) -> Tuple[bytes, bytes]` -- takes an exported program object, a dictionary mapping opset namespaces to versions, and returns the serialized exported program in bytes, and separately the state dict serialized in bytes
- `GraphModuleSerializer` class that serializes torch.fx.GraphModule
to the schema.GraphModule dataclass
- `ExportedProgramSerializer` class that serializes torch._export.exported_program.ExportedProgram to the schema.ExportedProgram dataclass
Serialization TODOs:
- [x] pytree spec: https://github.com/pytorch/pytorch/pull/102577
- [ ] higher order ops
- [ ] node metadata (specifically nn_module_stack/source_fn)
- [ ] constraints
- [ ] graph module metadata
The tests are not super comprehensive, but that's because I think it'll be better tested + easier to test once deserialization is implemented.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/102707
Approved by: https://github.com/avikchaudhuri , https://github.com/zhxchen17
2023-06-06 05:12:49 +00:00
Angela Yi
c4028de462
[export] ExportedProgram ( #102259 )
...
Pull Request resolved: https://github.com/pytorch/pytorch/pull/102259
Approved by: https://github.com/ydwu4 , https://github.com/avikchaudhuri , https://github.com/tugsbayasgalan , https://github.com/zhxchen17
2023-05-26 23:36:38 +00:00
Zhengxu Chen
351c2ea2fb
[export] Prototype on serialization schema. ( #101899 )
...
Fixes #ISSUE_NUMBER
Pull Request resolved: https://github.com/pytorch/pytorch/pull/101899
Approved by: https://github.com/angelayi
2023-05-21 06:31:53 +00:00
Angela Yi
ba47a2b227
[export] Pickle of ExportGraphModule ( #100924 )
...
try 2 of reland of https://github.com/pytorch/pytorch/pull/100620 bc merge conflict 😭 ...
Pull Request resolved: https://github.com/pytorch/pytorch/pull/100924
Approved by: https://github.com/tugsbayasgalan
2023-05-09 16:58:24 +00:00
PyTorch MergeBot
f42eae4755
Revert "[export] Pickle of ExportGraphModule ( #100620 )"
...
This reverts commit d4975a5fe0https://github.com/pytorch/pytorch/pull/100620 on behalf of https://github.com/clee2000 due to broke export/test_serialize.py::TestSerialize::test_pickle_dynamic across various jobs d4975a5fe0https://github.com/pytorch/pytorch/pull/100620#issuecomment-1536643519 ))
2023-05-05 18:52:48 +00:00
Angela Yi
d4975a5fe0
[export] Pickle of ExportGraphModule ( #100620 )
...
reland of https://github.com/pytorch/pytorch/pull/100423 bc merge conflict...
Pull Request resolved: https://github.com/pytorch/pytorch/pull/100620
Approved by: https://github.com/mergennachin
2023-05-05 18:21:39 +00:00
PyTorch MergeBot
c4fd76e7b4
Revert "[export] Pickle result of export ( #100423 )"
...
This reverts commit 7226dbcbcehttps://github.com/pytorch/pytorch/pull/100423 on behalf of https://github.com/angelayi due to merge conflict ([comment](https://github.com/pytorch/pytorch/pull/100423#issuecomment-1534163373 ))
2023-05-04 06:41:06 +00:00
Angela Yi
7226dbcbce
[export] Pickle result of export ( #100423 )
...
Pickles the metadata["val"] into TensorMetadata struct so that it'll be retrained when we unpickle.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/100423
Approved by: https://github.com/mergennachin
2023-05-04 06:37:16 +00:00
