Summary:
The android.cmake.toolchain file we use from a submodule is unmaintained and not updated since 2015.
It causes numerous problems in Caffe2 build:
- Caffe2 can't be built for Android ARM64, because gcc toolchain for ARM64 doesn't support NEON-FP16 intrinsics, and the android.cmake.toolchain we use doesn't allow us specify clang-5.0 from NDK r15c
- Caffe2 can't be built with Android NDK r16 (the most recent NDK version)
- Caffe2 can't be built for Android with Ninja generator
This change updates the build script to use $ANDROID/build/cmake/android.cmake.toolchain instead, which is maintained by Android team, and synchronized with Android NDK version.
As this toolchain file doesn't support "armeabi-v7a with NEON FP16" ABI, I had to disable mobile OpenGL backend, which requires NEON-FP16 extension to build. With some work, it can be re-enabled in the future.
Closes https://github.com/caffe2/caffe2/pull/1740
Differential Revision: D6707099
Pulled By: Maratyszcza
fbshipit-source-id: 8488594c4225deed0323c1e54c8d71c804b328df
Summary:
Lots of unwanted stuff here that shouldn't be in this branch. I just need to make a PR so I can test it
Closes https://github.com/caffe2/caffe2/pull/1765
Reviewed By: orionr
Differential Revision: D6752610
Pulled By: pjh5
fbshipit-source-id: cc93290773640a9eb029f350b17f520ac5f2504e
Summary:
Change the directory name for ipython notebook.
Change the executable name fro ipython to jupyter
Pass arguments to the script to the notebook, instead of fixing --ip='*'. In some setup, --ip='*' cause jupyter notebook not displayed.
Closes https://github.com/caffe2/caffe2/pull/1546
Reviewed By: pietern
Differential Revision: D6460324
Pulled By: sf-wind
fbshipit-source-id: f73d7be96525e2ab97f3d0e7fcb4b1557934f873
Summary:
The scripts/build_local.sh script would always build protoc from the
third_party protobuf tree and override the PROTOBUF_PROTOC_EXECUTABLE
CMake variable. This variable is used by the protobuf CMake files, so
it doesn't let us detect whether the protoc was specified by the user
or by the protobuf CMake files (e.g. an existing installation). This
in turn led to a problem where system installed headers would be
picked up while using protoc built from third_party. This only works
if the system installed version matches the version included in the
Caffe2 tree. Therefore, this commit changes the variable to specify a
custom protoc executable to CAFFE2_CUSTOM_PROTOC_EXECUTABLE, and
forces the use of the bundled libprotobuf when it is specified.
The result is that we now EITHER specify a custom protoc (as required
for cross-compilation where protoc must be compiled for the host and
libprotobuf for the target architecture) and use libprotobuf from the
Caffe2 tree, OR use system protobuf.
If system protobuf cannot be found, we fall back to building protoc
and libprotobuf in tree and packaging it as part of the Caffe2 build
artifacts.
Closes https://github.com/caffe2/caffe2/pull/1328
Differential Revision: D6032836
Pulled By: pietern
fbshipit-source-id: b75f8dd88412f02c947dc81ca43f7b2788da51e5
Summary:
The build_local.sh script current is single thread, which is really slow. Use the same mechanism in build_android.sh to parallelize the build.
Closes https://github.com/caffe2/caffe2/pull/1282
Differential Revision: D5992231
Pulled By: sf-wind
fbshipit-source-id: 01ba06b6efcb0f535f974a2dfffbae9ba385d27d
Summary:
After this, windows should be all green.
Closes https://github.com/caffe2/caffe2/pull/1228
Reviewed By: bwasti
Differential Revision: D5888328
Pulled By: Yangqing
fbshipit-source-id: 98fd39a4424237f2910df69c8609455d7af3ca34
Summary:
There does not exist appropriate build script for Tizen software platform.
This commit is to fix#847.
Signed-off-by: Geunsik Lim <geunsik.lim@samsung.com>
Closes https://github.com/caffe2/caffe2/pull/877
Differential Revision: D5571335
Pulled By: Yangqing
fbshipit-source-id: 12759a3c0cb274ef93d7127b8185341e087f2bfa
Summary:
1. switch the protoc building system from msbuild to cmake
2. set default CMAKE_GENERATE to VS2015
3. set default CMAKE_BUILD_TYPE to Release
4. improve error handling
5. add the generated protobuf include path
6. exclude many optional dependencies from build_windows.bat
Closes https://github.com/caffe2/caffe2/pull/1014
Differential Revision: D5559402
Pulled By: Yangqing
fbshipit-source-id: 019e3a6c3c909154027fa932ce1d6549476b23bb
Summary:
use user defined android ndk path instead of hard code.
Closes https://github.com/caffe2/caffe2/pull/506
Differential Revision: D5162646
Pulled By: Yangqing
fbshipit-source-id: 5093888e15607b3bf6682e05eb91aa94c6206b01
Summary:
Added python-pip and python-numpy into build_raspbian.sh script
because they are not installed in ubuntu/debian minimal image.
Closes https://github.com/caffe2/caffe2/pull/609
Differential Revision: D5121550
Pulled By: Yangqing
fbshipit-source-id: 14dd1450275fcc2aa9d2a06f0982f460528a1930
Summary:
build_ios.sh now have `-fembed-bitcode` flags for cmake and passes these flags to build_host_protoc.sh (which now accepts optional argument `--other-flags`). That allows to use output libs (libCaffe2_CPU.a, libCAFFE2_NNPACK.a, libCAFFE2_PTHREADPOOL.a and libprotobuf-lite.a, libprotobuf.a respectively) in Xcode projects with bitcode enabled.
Bitcode is enabled by default in all projects since Xcode7, is crucial for slicing and is mandatory for watchOS targets. Enabling bitcode for target requires bitcode to be enabled for all dependencies also, so Caffe2 built without bitcode forces developers to switch off bitcode for the whole app.
Closes https://github.com/caffe2/caffe2/pull/457
Reviewed By: bwasti
Differential Revision: D4978644
Pulled By: Yangqing
fbshipit-source-id: 5165abb507fb91bc8c38f7348d6836bccf8fcc22
Summary:
When trying to build caffe2 with python provided by homebrew, I find out there are some errors in the building scripts. The "get_python_cmake_flags.py" script is supposed to find out the correct python library and header file locations. However, due to these errors, this script does not function correctly. After building, caffe2 is linked against the default python library provided by Apple which causes a crash when trying to validate whether or not the installation is successful:
```shell
python -c 'from caffe2.python import core' 2>/dev/null && echo "Success" || echo "Failure"
```
The fix is as simple as follows:
- Add "shell" so that command substitution could work under Makefile.
- Add blank spaces between -D options so that they are treated as options not makefile targets.
- Print the "flags" variable without the newline character so that they could be utilized by command substitution correctly.
Closes https://github.com/caffe2/caffe2/pull/391
Differential Revision: D4943212
Pulled By: Yangqing
fbshipit-source-id: 04d3595fa2d89fe57aed5b6a7a91a95114a82a1b
Summary:
Changed _Android_ to _iOS_ in the comments in scripts/build_ios.sh.
Closes https://github.com/caffe2/caffe2/pull/364
Differential Revision: D4930101
Pulled By: Yangqing
fbshipit-source-id: 8f0a6aa1b43fd57c2f71f1c667c61d1f69b1e061
Summary:
When compiling Caffe2 on a Jetson TX2 using JetPack 3.0, the compilation with the Tegra X1 build script runs through perfectly fine. However, when running
from caffe2.python import workspace
the following error shows up:
> ImportError: No module named six
After installing `six` manually using
sudo pip install six
this works fine. I thus added the `six` module to the install script.
I assume this will also be required for the `build_raspbian.sh` script, however as I could test this, I didn't add it (yet).
Closes https://github.com/caffe2/caffe2/pull/293
Differential Revision: D4914121
Pulled By: Yangqing
fbshipit-source-id: 75947e8c295e1f5ad3f480a025fe8518dd91a957
Summary:
Changed the windows python extension name to ".pyd" and did a manual copy from the {Debug,Release} folder to the main folder for easier automatic build.
Closes https://github.com/caffe2/caffe2/pull/222
Differential Revision: D4771065
Pulled By: Yangqing
fbshipit-source-id: 4a89d409fa66f0979cf4ecf502189b2f9cc11504
Summary:
aaronmarkham this solves your Windows build issue. Basically:
(1) VS 2017 does not have CUDA support yet, and we will be waiting on NVidia to do so.
(2) VS 2015 and 2017 need different cmake generator strings.
This PR shows how to determine those and also updates appveyor to do contbuild guard for the following 3 settings:
- VS2015 without cuda
- VS2017 without cuda
- VS2015 with cuda
Closes https://github.com/caffe2/caffe2/pull/210
Differential Revision: D4745007
Pulled By: Yangqing
fbshipit-source-id: 50952552843abd0eb6f4145d9f132daeee3a6794
Summary:
(Note: previous revert was due to a race condition between D4657831 and
D4659953 that I failed to catch.)
After this, we should have contbuild guarding the Windows build both with
and without CUDA.
This includes a series of changes that are needed to make Windows build,
specifically:
(1) Various flags that are needed in the cmake system, specially dealing
with /MD, /MT, cuda, cudnn, whole static linking, etc.
(2) Contbuild scripts based on appveyo.
(3) For Windows build, note that one will need to use "cmake --build" to
build stuff so that the build type is consistent between configuration and
actual build. see scripts\build_windows.bat for details.
(4) In logging.h, ERROR is already defined by Windows. I don't have a good
solution now, and as a result, LOG(ERROR) on windows is going to be
LOG(INFO).
(5) variable length array is not supported by MSVC (and it is not part of
C++ standard). As a result I replaced them with vectors.
(6) sched.h is not available on Windows, so akyrola 's awesome simple
async net might encounter some slowdown due to no affinity setting on
Windows.
(7) MSVC has a bug that does not work very well with template calls inide
a templated function call, which is a known issue that should be fixed in
MSVC 2017. However for now this means changes to conv_op_impl.h and
recurrent_net_op.h. No actual functionalities are changed.
(8) std host function calls are not supported in CUDA8+MSVC, so I changed
lp_pool (and maybe a few others) to use cuda device functions.
(9) The current Scale and Axpy has heavy templating that does not work
well with MSVC. As a result I reverted azzolini 's changes to the Scale
and Axpy interface, moved the fixed-length version to ScaleFixedSize and
AxpyFixedSize.
(10) CUDA + MSVC does not deal with Eigen well, so I guarded all Eigen
parts to only the non-CUDA part.
(11) In conclusion, it is fun but painful to deal with visual c++.
Differential Revision: D4666745
fbshipit-source-id: 3c9035083067bdb19a16d9c345c1ce66b6a86600
Summary:
After this, we should have contbuild guarding the Windows build both with
and without CUDA.
This includes a series of changes that are needed to make Windows build,
specifically:
(1) Various flags that are needed in the cmake system, specially dealing
with /MD, /MT, cuda, cudnn, whole static linking, etc.
(2) Contbuild scripts based on appveyo.
(3) For Windows build, note that one will need to use "cmake --build" to
build stuff so that the build type is consistent between configuration and
actual build. see scripts\build_windows.bat for details.
(4) In logging.h, ERROR is already defined by Windows. I don't have a good
solution now, and as a result, LOG(ERROR) on windows is going to be
LOG(INFO).
(5) variable length array is not supported by MSVC (and it is not part of
C++ standard). As a result I replaced them with vectors.
(6) sched.h is not available on Windows, so akyrola 's awesome simple
async net might encounter some slowdown due to no affinity setting on
Windows.
(7) MSVC has a
Closes https://github.com/caffe2/caffe2/pull/183
Reviewed By: ajtulloch
Differential Revision: D4657831
Pulled By: Yangqing
fbshipit-source-id: 070ded372ed78a7e3e3919fdffa1d337640f146e
Summary:
MSVC 2015 has known bugs about template functions so these changes aim to fix them - no functional differences introduced.
Closes https://github.com/caffe2/caffe2/pull/179
Reviewed By: ajtulloch
Differential Revision: D4635241
Pulled By: Yangqing
fbshipit-source-id: a282a96e1e626e9440c1e3f3cb15b5b1fa710887
Summary:
This script will attempt to determine files that will be useful for building with the correct python version. Currently on macOS with various python installations CMake fails to determine the correct location of python libraries.
Closes https://github.com/caffe2/caffe2/pull/163
Reviewed By: Yangqing
Differential Revision: D4594954
Pulled By: bwasti
fbshipit-source-id: c2b750ee9608a02fad4ce2f2293f5fa54dc7011c
Summary:
This is essentially https://github.com/caffe2/caffe2/pull/146/ but shipit
failed to trigger task determinator.
Reviewed By: bwasti
Differential Revision: D4557698
fbshipit-source-id: b0e6777957e76df4e23671371098c2c6fe83b55c
Summary: If the PATH doesn't include cmake (such as when android studio wipes all the environment variables), this will still work.
Reviewed By: Yangqing
Differential Revision: D4504653
fbshipit-source-id: 56a8854e3daf6ee1f5b1cbeb83ca175a007dad12
Summary:
Turns out that building on raspbian is easy as a cake for caffe2 - cmake is awesome.
Closes https://github.com/caffe2/caffe2/pull/112
Differential Revision: D4480985
Pulled By: Yangqing
fbshipit-source-id: 5dbe5e1e71d8680dea7a5ec8a9ce7fbe6aa5270a
Summary:
Added cmake for android script under scripts, and set up the travis contbuild target.
Closes https://github.com/caffe2/caffe2/pull/109
Reviewed By: bwasti
Differential Revision: D4468767
Pulled By: Yangqing
fbshipit-source-id: 709f3eb6be24727b0a989d0901dbf377871b122a
