Commit Graph

71 Commits

Author SHA1 Message Date
Maciej Bargiel
2cdf98a74d Back out "Removing some dependency edges from Blob to other caffe2"
Summary: The controller you requested could not be found. Original commit changeset: 2ea17724e223

Differential Revision:
D10026321
Ninja: stable broken

fbshipit-source-id: faf87cb7cc0f78c2c10d4aa6fceea279cd27acd6
2018-09-25 01:11:14 -07:00
Sebastian Messmer
17a65bf9b6 Removing some dependency edges from Blob to other caffe2 (#11923)
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/11923

This is pre-work to allow moving Blob to ATen/core, which cannot depend on caffe2 anymore.
(1) Removing the Blob -> Tensor dependency allows us to move Blob to ATen/core and use it inside IValue without having to wait for the Tensor merge to be complete.
(2) In the final Blob design, we want it to be a very small class that doesn't have any special treatment for Tensor (or to be more correct, doesn't allow storing Tensor anymore), so this is anyhow the direction we want to go.

This changes call sites that will have to be moved to IValue later, but they cannot be moved to IValue directly, because for that, IValue first needs to be able to store Blob, which in turn first needs this diff and some other changes coming up in future diffs.

Codemods:
$ codemod --extensions h,hpp,c,cpp,cc "([a-zA-Z0-9_]+)\\.IsTensorType\\(" "BlobIsTensorType(\\1, "
$ codemod --extensions h,hpp,c,cpp,cc "([a-zA-Z0-9_]+)->IsTensorType\\(" "BlobIsTensorType(*\\1, "
$ codemod --extensions h,hpp,c,cpp,cc "([a-zA-Z0-9_]+)\\.GetMutableTensor\\(" "BlobGetMutableTensor(\\1, "
$ codemod --extensions h,hpp,c,cpp,cc "([a-zA-Z0-9_]+)->GetMutableTensor\\(" "BlobGetMutableTensor(*\\1, "

It is, however, not only these codemods because regex based refactoring was only able to match a small amount of the call sites. To catch more, I wouldn've needed a AST aware tool like clangr, which I didn't figure out how to use.

Reviewed By: ezyang

Differential Revision: D9979976

fbshipit-source-id: 2ea17724e223b5b73b44f99362727759ca689e61
2018-09-24 22:57:05 -07:00
Christian Puhrsch
a6630e25af Remove many caffe2::TIndex and replace them with int64_t (#11943)
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/11943

See title

Reviewed By: ezyang

Differential Revision: D9992645

fbshipit-source-id: e8f80d6ea762971513e5e8072975ceea53e1f11a
2018-09-22 18:11:04 -07:00
Sebastian Messmer
b2b05b7c20 Move blob serialization to free functions (#11817)
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/11817

Blob::Serialize() and Blob::Deserialize() are now free functions SerializeBlob(), DeserializeBlob() instead.
This takes away access to Blob internals from them and makes future refactorings easier.

Reviewed By: ezyang

Differential Revision: D9882726

fbshipit-source-id: 3251ebd4b53fc12f5e6924a6e4a8db3846ab3729
2018-09-20 23:27:34 -07:00
Alexander Sidorov
23dd5b4a53 Back out "Open-source ThreadSafeActivationCleaningPredictor"
Summary:
Original commit changeset: bfe253ae5fc8

Apparently Ads push process detected some regression which normal
canaries don't show.
https://fb.facebook.com/groups/1274424122598505/permalink/2597819483592289/

Reviewed By: highker, Prowindy

Differential Revision: D9952807

fbshipit-source-id: 1a3ea249c3b1e2618220c61f3d51468824b6ef10
2018-09-19 21:26:51 -07:00
Bram Wasti
cf5a21e4a1 Add back proto opt disable feature that was lost during refactor (#11875)
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/11875

Seems like the refactor to predictor_config dropped some functionality that is now blocking other teams

rFBS2b30208263c14ce7039f27c618a3b232bf11ee33 is the change that was missed

hoping to land this quickly :)

Reviewed By: jonmorton

Differential Revision: D9948324

fbshipit-source-id: 1628f7c51c06319fa7ca5dc9d59799135bb82c5f
2018-09-19 15:33:26 -07:00
Alexander Sidorov
b46f1b8ca7 Open-source ThreadSafeActivationCleaningPredictor (#11779)
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/11779

Pull Request resolved: https://github.com/pytorch/pytorch/pull/11731

This Predictor provides threadsafe interface and also
cleans-up activations after each run. So in multi-model setup
activation space doesn't explode

Reviewed By: highker

Differential Revision: D9842374

fbshipit-source-id: bfe253ae5fc813e73a347c5147ff6b58d50781ea
2018-09-18 21:56:58 -07:00
Sebastian Messmer
ce6906b051 Narrowing Blob (#11167)
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/11167

Narrow the Blob API as preparation for merging Blob/IValue

- get rid of templated IsType and Operator::InputIsType / OutputIsType
- Use 'using' instead of 'typedef' for DestroyCall (just for readability)

Reviewed By: ezyang

Differential Revision: D9623916

fbshipit-source-id: 952f0b0cf5a525094b02e8d2798dd57a56a9e1d8
2018-09-10 12:40:16 -07:00
Shashank Singh
f1bfe6750f Back out "[caffe2] Update blackbox predictor with new constructor" (#11105)
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/11105

Reverts: D9516972

See this discussion for context: https://fburl.com/w45hb1oc

Reviewed By: highker

Differential Revision: D9587931

fbshipit-source-id: 715247929d819dfa88e1d051021e51c5bf0c4835
2018-08-31 00:55:36 -07:00
Yi Cheng
e550eab3e2 Remove MetaNetDef test case in Predictor (#11052)
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/11052

Delete the test case for Predictor with constructing by MetaNetDef since the constructor
actually has been deprecated. The broken PR is for construcing predictor from DB instance.

Reviewed By: highker

Differential Revision: D9566935

fbshipit-source-id: 5511883953a2d3f6eb0a4f1c5518a1bc4b3ffbdc
2018-08-29 17:55:21 -07:00
Yi Cheng
c99a143eea Update blackbox predictor with new constructor (#10920)
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/10920

Update the black box predictor and the related code to use the
constructor with PredictorConfig.

Reviewed By: highker

Differential Revision: D9516972

fbshipit-source-id: fbd7ece934d527e17dc6bcc740b4e67e778afa1d
2018-08-29 13:31:45 -07:00
Orion Reblitz-Richardson
4cb968fb77 Default hidden visibility (#10752)
Summary:
Flipping to hidden visibility one more time. Let's see what fails.

cc mingzhe09088 pjh5 Yangqing
Pull Request resolved: https://github.com/pytorch/pytorch/pull/10752

Reviewed By: ezyang

Differential Revision: D9526343

Pulled By: orionr

fbshipit-source-id: c0e9c29270e95e1b2e21c598095f720c199e1e52
2018-08-28 15:25:43 -07:00
Edward Yang
91797c0672 Replace direct include of caffe2.pb.h with an intermediary header caffe2_pb.h (#10946)
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/10946

```
codemod -d . --extensions cc,cpp,cu,cuh,h caffe2/proto/caffe2.pb.h caffe2/proto/caffe2_pb.h
```

Reviewed By: houseroad

Differential Revision: D9539945

fbshipit-source-id: 497d04720e8e7e61c05ffe1b23733d0cb774de7e
2018-08-28 11:57:08 -07:00
Yi Cheng
ddc37d7487 Update mobile predictor caller's interface
Summary: Update all the caller for the new interface

Reviewed By: highker

Differential Revision: D9323167

fbshipit-source-id: a39335ceb402db0719f5f2314085ba9a81380308
2018-08-24 23:40:05 -07:00
Yi Cheng
8da4167129 Fix performance regression (#10835)
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/10835

The last diff of constructor cause performance regression in cold run.
This one tried to fix this.

Reviewed By: highker

Differential Revision: D9489617

fbshipit-source-id: a77c2e2c903a73e2ad9806b4f9c209cdb751442f
2018-08-23 19:55:23 -07:00
Yi Cheng
4aa5075cae update the constructor to accept the PredictorConfg only to set up the predictor (#9483)
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/9483

The interface is updated to accept the config to construct the predictor.

Reviewed By: highker

Differential Revision: D8872999

fbshipit-source-id: 3ca54d644970823fc33c0ade9a005e12f52e2b24
2018-08-22 19:11:09 -07:00
Yangqing Jia
0a809fc8b1 build changes to make cpu unified build working. (#10504)
Summary:
Properly annotated all apis for cpu front. Checked with cmake using

cmake -DUSE_ATEN=ON -DUSE_CUDA=OFF -DBUILD_ATEN=ON

and resulting libcaffe2.so has about 11k symbols.
Pull Request resolved: https://github.com/pytorch/pytorch/pull/10504

Reviewed By: ezyang

Differential Revision: D9316491

Pulled By: Yangqing

fbshipit-source-id: 215659abf350af7032e9a4b0f28a856babab2454
2018-08-15 17:22:36 -07:00
Bram Wasti
ec807f2a91 Bail out if netdef has disable_nomnigraph argument
Summary: allow models to override nomnigraph opts

Reviewed By: ajtulloch

Differential Revision: D9035729

fbshipit-source-id: 2b30208263c14ce7039f27c618a3b232bf11ee33
2018-07-31 20:54:46 -07:00
Bram Wasti
fcd567ed15 Enable Optimization on mobile by default
Summary: Re-enable opt by default

Reviewed By: Maratyszcza

Differential Revision: D8525434

fbshipit-source-id: a61253907251a44cfc59e0b50fb1906c5eb20558
2018-07-31 20:54:44 -07:00
Jerry Zhang
aebf3b47ae Remove template parameter from Tensor (#9939)
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/9939

Pull Request resolved: https://github.com/facebookresearch/weakly-supervised-action-detection/pull/13

Pull Request resolved: https://github.com/pytorch/translate/pull/166

Pull Request resolved: https://github.com/pytorch/pytorch/pull/9125

Closes https://github.com/pytorch/pytorch/pull/9125

Use inheritance for polymorphism, and remove template parameter
This is to change the templating in call sites, the core implementations will change later

Before Caffe2 Tensor class was compile-time fixed to bind to a particular device/context. With this change, we're making it a runtime property (stored inside the tensor), but preserve the same semantics. For example, one has to specify device type in order to create a Tensor - there are no uninitialized tensors. More specifically the changes are:

1. We added an extra argument *DeviceType* to most of the constructors of the tensor, e.g. (Tensor(DeviceType type)),
2. Semantics of constructor Tensor(const Tensor<SrcContext>& src, ContextForCopy* context); is changed, in this constructor, the second context is passed in to enable us to call the templated Copy function, it could be in a different context as source and target previously, now we'll enforce that the context should have same device type as src, if it is provided.
3. To preserve 'get-or-construct' semantics of Blob, we added specialized getter Blob::GetMutableTensor that verifies both that Blob contains a Tensor and that it's of a correct type
4. Specifically, Tensor type is not default-constructible any more (as we don't have unknown device tensors) and thus some of the code handling STL containers needs to change

Note: Some changes are postponed just to keep this diff a bit smaller. Please see `TODO`s.

Reviewed By: ezyang, houseroad

Differential Revision: D9024330

fbshipit-source-id: e0b8295d2dc6ebe2963383ded5af799ad17164ba
2018-07-27 10:56:39 -07:00
Yi Cheng
dfa0af093d Move predictor into caffe2/caffe2/predictor (#9548)
Summary:
Pull Request resolved: https://github.com/pytorch/pytorch/pull/9548

Pull Request resolved: https://github.com/pytorch/translate/pull/157

One part of refactor predictor. Move all the files into predictor dir.

Reviewed By: highker

Differential Revision: D8845276

fbshipit-source-id: 1e917464b0c8a042f025128a082c784eaa3b7013
2018-07-26 19:03:40 -07:00