Commit Graph

227 Commits

Author SHA1 Message Date
Austin Anderson
c02de0b95f Roll out Win RBE container with correct VC version 14.27
PiperOrigin-RevId: 381315911
Change-Id: I07b1bff4e538bf5b89e21a833881a941e1d0f5d1
2021-06-24 12:53:26 -07:00
Reed Wanderman-Milne
ac58229f34 Add documentation on the --config=dbg option.
Resolves https://github.com/tensorflow/tensorflow/issues/48919.

PiperOrigin-RevId: 380915403
Change-Id: I6b79f15226efcb13d10f05d0fbd0e8cf01c81583
2021-06-22 16:26:17 -07:00
Scott Zhu
ff964e9279 Update windows RBE image for MSVC version 14.27
PiperOrigin-RevId: 380868165
Change-Id: I0b8c7d07a2ca90b08a8a3c04e34f516f2cdbcb38
2021-06-22 12:40:57 -07:00
Austin Anderson
2951554f51 Print compilation subcommands in release builds
This should help us identify exactly how we're building our releases.

PiperOrigin-RevId: 380829893
Change-Id: I279b789351504d6d6d9bafcac40d3c3a6174db5f
2021-06-22 09:58:00 -07:00
Austin Anderson
fba8213df8 Print compilation subcommands in release builds
This should help us identify exactly how we're building our releases.

PiperOrigin-RevId: 380700058
Change-Id: I21cf9eef338dfdda468afc232d9ac2dbd673637f
2021-06-21 17:37:17 -07:00
TensorFlower Gardener
191f48ee41 Merge pull request #50318 from bas-aarts:bas-devel-no-distinct-host-conf
PiperOrigin-RevId: 380570343
Change-Id: I36ed4558b9e41c3923b9ea6f2ccf87e04cf88dc3
2021-06-21 07:17:14 -07:00
Alex Humesky
ece1c46969
Remove noincompatible_prohibit_aapt1 from bazelrc
`--noincompatible_prohibit_aapt1` is a noop and is being removed from bazel in a future version.

See

https://buildkite.com/bazel/bazel-at-head-plus-downstream/builds/2070#20d4703d-abdf-44cc-a6b6-2f2c9a38bc48

aefd107e16
2021-06-16 18:29:49 -04:00
Bas Aarts
0f9af91115 set --distinct_host_configuration=false by default for linux builds 2021-06-16 15:09:09 -07:00
Scott Zhu
5844cd5faa Update the RBE window image.
This should include latest keras-nightly for testing.

PiperOrigin-RevId: 379588572
Change-Id: I2605381dd9aabbff7429d7685a045db5d67e99f9
2021-06-15 15:44:58 -07:00
Rahul Joshi
bf36815372 Enable use of sandboxed strategy only for linux and android builds.
This addresses https://github.com/tensorflow/tensorflow/issues/47386

PiperOrigin-RevId: 378926792
Change-Id: I4659db01915880f6fec2ef543d967b5792f198fe
2021-06-11 12:38:54 -07:00
Reed Wanderman-Milne
d3bbd2ffc9 Fix build failure with --config=dbg.
Before, the build would fail with errors such as: "relocation truncated to fit: R_X86_64_32 against .debug_info'". The issue was the debug info was too large. I believe the issue was occurring because offsets into the .debug_info section are stored as 32-bit integers, and so that section cannot exceed 4GiB. To fix, debug info is only included for files under tensorflow/, excluding kernels. This brings the size of the .debug_info section down to about 1.4GiB, well under the 4GiB limit.

Unfortunately, TF kernels and TF dependencies do not have debugging info anymore, but I suspect these are rarely debugged. Debugging info for specific kernels/dependencies can still be explicitly included by the user, e.g. by passing the bazel flags: --config=dbg --per_file_copt=+tensorflow/core/kernels/identity_op.*@-g

See https://github.com/tensorflow/tensorflow/issues/48919 for more context.

PiperOrigin-RevId: 378910826
Change-Id: I4b94e3d53bb3ca00c30d5c83d2a57e4bd390c5a8
2021-06-11 11:17:59 -07:00
Rahul Joshi
259399eeda Enable use of sandboxed strategy only for linux and android builds.
This addresses https://github.com/tensorflow/tensorflow/issues/47386

PiperOrigin-RevId: 378894693
Change-Id: Ib510432f7a9b65cffc08b351a7bb419b9c7df81c
2021-06-11 10:07:13 -07:00
Rahul Joshi
763ae9bed6 Switch to use spawn_strategy=sandboxed by default.
This addresses https://github.com/tensorflow/tensorflow/issues/47386

PiperOrigin-RevId: 378454300
Change-Id: Idd7d93c696ab935dd441df2f67a16cd867615311
2021-06-09 11:33:19 -07:00
TensorFlower Gardener
8ea7f5632a Merge pull request #49510 from hcoona:private/zhangshuai.ds/fix-bazelrc-ubsan
PiperOrigin-RevId: 378436819
Change-Id: Ib57c5948d41d1123914973a99547f7833a97af0d
2021-06-09 10:32:17 -07:00
Rahul Joshi
9171e39a2a Switch to use spawn_strategy=sandboxed by default.
This addresses https://github.com/tensorflow/tensorflow/issues/47386

PiperOrigin-RevId: 378038032
Change-Id: I73d6a02a36f2a2205411a8f5da7998d5386cfcad
2021-06-07 17:08:52 -07:00
Shuai Zhang
dbd7f29f84 Fix ubsan in bazelrc 2021-05-24 17:05:15 +08:00
A. Unique TensorFlower
fe134d6a08 Allow localized disabling of tests for asan/msan/tsan/ubsan.
Currently we can disable entire test targets via the bazel tags `noasan`,
`nomsan`, `notsan`, `noubsan`.  However, in some instances, we may expect
one particular edge-case to fail a sanitizer test (for known reason), but
still want the rest of the test suite to be analyzed. By disabling individual
test methods or parts thereof locally, we allow for this without the need to
separate the offending case into a different target.

There already was some functionality in tflite, which seems to currently
be unused (though has been previously to locally skip tests).
Refactored and moved to tf so can be more widely used.

Also added corresponding defines to the `.bazelrc` file.

PiperOrigin-RevId: 374231232
Change-Id: I62ca36ebe9edec18e0d6d358496b6d07efb43574
2021-05-17 11:05:41 -07:00
TensorFlower Gardener
ce227000bd Merge pull request #48522 from yongtang:no-aws-hdfs
PiperOrigin-RevId: 371807950
Change-Id: Id7e762945f0b5d965865220d213cb297bb565f3e
2021-05-03 17:03:56 -07:00
A. Unique TensorFlower
4e5c203b49 Enable --remote_download_minimal for Windows RBE build
A roll-forward of cl/370462021, but only on Windows.
This avoids creating runfiles tree on the local machine, which is very slow.

PiperOrigin-RevId: 370679217
Change-Id: I11f331d236575e408ceaa3b46d3830010177b386
2021-04-27 07:54:41 -07:00
Yong Tang
1a18c890b1 Disable AWS and HDFS support by default
This PR disables AWS and HDFS support by default, as was discussed
in:
https://docs.google.com/document/d/1CB51yJxns5WA4Ylv89D-a5qReiGTC0GYum6DU-9nKGo/edit#heading=h.8ifswzr0wtbp

Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
2021-04-27 13:41:28 +00:00
Chenkai Kuang
312e7818c9 Use --remote_download_minimal for RBE build
This avoids downloading remote build outputs to local machine, which can speed up build and test time, especially on Windows.

PiperOrigin-RevId: 370574647
Change-Id: Ied7b9c4df06165d0543b71bcb1e7c94219fd3d43
2021-04-26 17:19:40 -07:00
A. Unique TensorFlower
282fa073ab Don't build the python tests as self-extracting archive in the RBE build.
On Windows, py_binary & py_test are by default built as self-extracting archives of their runfiles tree. It is very time consuming, but we don't need that
for the RBE build because the runfiles tree is created in the remote worker anyway.

This significantly speeds up TF's python tests in the Windows RBE build.

Related doc: https://bazel.build/designs/2016/09/05/build-python-on-windows.html

PiperOrigin-RevId: 370464199
Change-Id: Ic91fea41f1fc1b0f5de3b6aac4d054cc43536ebf
2021-04-26 08:46:28 -07:00
A. Unique TensorFlower
84fa3052ec Use --remote_download_minimal for RBE build
This avoids downloading remote build outputs to local machine, which can speed up build and test time, especially on Windows.

PiperOrigin-RevId: 370462021
Change-Id: I5531d8a5e5e517373d74f8528e28472b199e6661
2021-04-26 08:27:46 -07:00
Deven Desai
22a512f570 Enabling MLIR generated kernels by default on ROCm 2021-04-06 14:49:22 +00:00
Christian Sigg
1e8f4666f2 Test GPU in Windows CI.
With --experimental_strict_action_env, CUDA DLLs are not in PATH, tests cannot load them, and GPU tests fall back to CPU.

--experimental_strict_action_env became the default for Windows builds in cl/283405622. This change restricts it to Windows RBE builds, which is CPU only.

PiperOrigin-RevId: 365530221
Change-Id: I09684440e7c64fc9151694da5758faa4ecf3dd7b
2021-03-29 02:20:14 -07:00
Aleksandr Nikolaev
10f7d23b9c
Merge branch 'master' into aarch64_build_patch 2021-03-24 19:58:55 +00:00
Christian Sigg
682f7806e3 Disable generated gpu kernels only for ROCm. Non-GPU builds don't include them either way.
PiperOrigin-RevId: 364653388
Change-Id: I882eb857ebffd09d705bdd652bc5247e1c21b5ea
2021-03-23 15:09:33 -07:00
Aleksandr Nikolaev
dee97cdfb1
Merge branch 'master' into aarch64_build_patch 2021-03-23 21:12:25 +00:00
Christian Sigg
25923d49d0 Fix cuda_clang CI build.
Remove duplicate expansion of rbe, rbe_linux, and short_logs configs.

PiperOrigin-RevId: 364534626
Change-Id: I7fb0c0f083a558552e48516479bb074cf812b49a
2021-03-23 05:18:13 -07:00
Amit Patankar
eda4347f3b Remove Java toolchains from tensorflow and move them to the toolchains repo.
PiperOrigin-RevId: 364422405
Change-Id: I62c0845e40cce697e20dc2fef6404734400c3bef
2021-03-22 15:33:03 -07:00
Christian Sigg
b7dda31081 Move implicit .bazelrc options to the top.
Implicit options are evaluated first no matter where they are. Following https://blog.bazel.build/2018/01/19/config-parsing-order.html, the .bazelrc should ideally be ordered by evaluation order.

PiperOrigin-RevId: 364409118
Change-Id: Iec71a709dd0c40b7de196b16bb980d247d21fdee
2021-03-22 14:36:00 -07:00
Christian Sigg
5beb3fc7eb Remove --config=release_base from build:release_gpu_base.
This avoid double-expansion of --config=release_base for --config=release_gpu_{linux,windows}.

PiperOrigin-RevId: 364358564
Change-Id: I20adf018cfaf7f02e9fdfe79dba252116f05840b
2021-03-22 11:04:13 -07:00
Aleksandr Nikolaev
1e15aa8c83
Merge branch 'master' into aarch64_build_patch 2021-03-19 19:45:25 +00:00
Christian Sigg
45f45e032e Remove --config=native_arch_linux that was accidentally to linux release builds in cl/363193986.
PiperOrigin-RevId: 363892446
Change-Id: Ia30ce941a55bdb2e7ba48ebac2968281c5da3a88
2021-03-19 08:24:03 -07:00
Christian Sigg
9afae6a56b Add test settings written by ./configure to .bazelrc release configs.
This is a preparation change to remove running ./configure in release build scripts.

PiperOrigin-RevId: 363624089
Change-Id: I88a67df4e784c489539d928641f9eda574ad1dff
2021-03-18 03:57:02 -07:00
Christian Sigg
c0b4a3a1fd Resubmit: Clean up .bazelrc for MacOS.
This time, don't move `--enable_platform_specific_config` because it affects the expanded flag order.

PiperOrigin-RevId: 363584510
Change-Id: I2290305f280f71e469fea9c64307d59b5cc336a5
2021-03-17 22:35:24 -07:00
Mihai Maruseac
a88dbe9ade Clean up .bazelrc for MacOS.
PiperOrigin-RevId: 363494734
Change-Id: Ib45cf82d6d6f45b7089b1250dd4c2547b1a175cb
2021-03-17 13:32:50 -07:00
Christian Sigg
dd372ec5e1 Do not write TF_CONFIGURE_IOS to .bazelrc, only use it in ./configure.py.
PiperOrigin-RevId: 363484115
Change-Id: I5fd6186f19d1612ef9fe85a088aa7a72dc45d26d
2021-03-17 12:48:52 -07:00
Christian Sigg
3c58f664ee Enable XLA by default except on Android/iOS.
Previously, XLA was enabled by default with --config=rbe_linux and when running ./configure.

PiperOrigin-RevId: 363433913
Change-Id: I9afa53496a9da2ae29b71a9ef38fc63b90dfba02
2021-03-17 09:20:15 -07:00
Christian Sigg
5488f543a0 Clean up .bazelrc for MacOS.
PiperOrigin-RevId: 363427363
Change-Id: Iabb12f56cf269b6070b7f43225538bf2d84806a1
2021-03-17 08:45:14 -07:00
Christian Sigg
bff4bce6a0 Fix typo in .bazelrc.
PiperOrigin-RevId: 363250153
Change-Id: I8f1a5af806919b79fafec47261ae54be9ef4a55c
2021-03-16 13:04:03 -07:00
Christian Sigg
d89ef949b7 Revert the --config=release_cpu_linux crosstool_top to the nvcc toolchain. The one I changed it to doesn't even exist, and I don't know if //third_party/toolchains/preconfig/ubuntu16.04/gcc7_manylinux2010:toolchain is functional.
PiperOrigin-RevId: 363242634
Change-Id: I6e257c2ed8312a46b2549014dd368da7c5180fd9
2021-03-16 12:31:55 -07:00
Mihai Maruseac
0736a8d14c Typo fix avx_windows -> avx_win
PiperOrigin-RevId: 363225673
Change-Id: I3807cc0b3d4aed77205aecf3b2c19554124de27d
2021-03-16 11:19:44 -07:00
Christian Sigg
2683829da9 Remove no longer needed CUDA 11.0 build configs from .bazelrc.
Use --config=avx_{linux,windows} instead of --config=opt in release builds. This is an NFC because release build scripts set build:opt to enable AVX.

PiperOrigin-RevId: 363193986
Change-Id: Ifb0788e29485873e35469d0936c1d33488315870
2021-03-16 09:03:56 -07:00
Christian Sigg
e0c681c5c7 Exclude TensorRT from Windows GPU builds, it's not supported.
PiperOrigin-RevId: 363144050
Change-Id: Ifcec1495c0a93a4d439520cca19db6979b98480b
2021-03-16 03:18:38 -07:00
Christian Sigg
3a49729a55 Remove no longer needed configs from .bazelrc
PiperOrigin-RevId: 363122154
Change-Id: I16f947155d160ff0159f330491c812965d4df059
2021-03-16 00:19:09 -07:00
Aleksandr Nikolaev
06dd30ef98 Update of --config=mkl_aarch64 with Compute Library support
This PR introduces the support of Compute Library for the Arm
Architecture (ACL) as part of Bazel build.

Signed-off-by: Aleksandr Nikolaev <aleksandr.nikolaev@arm.com>
2021-03-12 19:29:51 -05:00
Austin Anderson
a60c2180c7 Fix .bazelrc release_gpu settings
PiperOrigin-RevId: 362568547
Change-Id: I9eae930c7016bd67db37aaa9667ca59b60a88f5b
2021-03-12 11:57:16 -08:00
Amit Patankar
2b6c831659 Cuda 11.2 internal fix.
PiperOrigin-RevId: 362546466
Change-Id: I65608db392a10078b000aed1f86a4a9e75b22a17
2021-03-12 10:16:29 -08:00
Austin Anderson
b523975c16 Change TF HEAD to primarily use CUDA 11.2
PiperOrigin-RevId: 362414269
Change-Id: I8d2747b8e1a5079c775d7056566c4aab184b0553
2021-03-11 17:32:40 -08:00