This simplifies ./configure process on Windows, we can remove them when we actually support the corresponding feature on Windows.
PiperOrigin-RevId: 204804112
After this change, to build TensorFlow with GPU support on Windows, you just need to follow the same procedure as Linux.
Also re-enable remote cache since the bug in b/111106393 has been fixed by the new wrapper script.
The clean-up of the wrapper scripts also made the build around 5 mins faster than using Bazel's msvc wrapper script.
PiperOrigin-RevId: 204105368
* Add KinesisDataset support for tensorflow Dataset
This fix is an attempt to add Kinesis support
for tensorflow's Dataset. Kinesis is provided by
AWS as a managed data streaming service. It is
similiar to Apache Kafka, often used in places
where maintaining a independent Kafka cluster on AWS
is not desirable or possible.
This fix adds the Kinesis support for tensorflow Dataset.
Similiar to the Kafka integration in tensorflow,
KinesisDataset outputs tf.string for records.
Test cases have also been added, which could be invoked manually.
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
* Expose KinesisDataset in dataset_ops.cc
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
* Expose KinesisDataset in python wrapper
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
* Add test cases for KinesisDataset
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
* Update AWS library include files
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
* Add Bazel BUILD files
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
* Rename s3_crypto to aws_crypto
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
* Rename with_s3_support to with_aws_support
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
* Selectively add kinesis to tensorflow/contrib/BUILD
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
* Set different partition key and pylint fix.
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
* Add missing modules in cmake's python_modules.txt
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
* Address review feedback
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
These rules currently get written by configure.py script to WORKSPACE
file which is not ideal since (1) WORKSPACE file is tracked by git and
(2) we require users to manually delete the rules in order to
update/regenerate them.
Moving these rules into an external repo that is generated based on
several ENV variables set by the configure.py script. Modifying any
of these ENV variables will cause the rules to be updated.
PiperOrigin-RevId: 199388460
Revert #18413. Too many internal test failures due to the name scope change caused by this change.
Revert #18192. Cannot use re2::StringPiece internally. Need alternative for set call. Will pull and clean this up in a separate change.
PiperOrigin-RevId: 197991247
Since py_runtime was introduced, Bazel ignores options such as
--force_python2 and --python2_path. Deleting to clean stuff up and
make sure people are not misled.
PiperOrigin-RevId: 193127681
The configure.py script will error out on Windows GPU builds due
to NCCL attempted to be configured (and is currently Linux only).
PiperOrigin-RevId: 192461362
The nccl_configure.bzl generates two different BUILD files based on the chose NCCL version. For NCCL 1, it aliases to the existing 'nccl_archive' http_repo on GitHub. For NCCL 2, it creates a target containing the NCCL 2 library and headers from the chosen install directory.
PiperOrigin-RevId: 191718007
Previously we only allowed to download clang when doing GPU builds.
The added skylark files use bazel's autoconf scripts, which were only added in
0.10.0. To provide nice error message for older versions of bazel
(i.e. 'version is less than 0.10' vs 'can't load @bazel_tools/cpp/...'), we
move the bazel version check into WORKSPACE file from workspace.bzl.
PiperOrigin-RevId: 190050798
Moving --config=android_arm --config=mkl and --config=monolithic
build configs into tools/bazel.rc. These options are just always
written the same way to .bazelrc when configure.py is run. This
should trim down the scope of configure.py and make it easier to
build TF without running configure.py prior.
PiperOrigin-RevId: 180973131
A recent botched three-way-merge sync seems to have duplicated (more than once on Github) the set_trisycl_include_dir function. This change removes the duplicates in favor of a cleaned-up variant that I added in another recent change.
PiperOrigin-RevId: 179249451
This change teaches the configure script how to search for Android NDK
and SDK installations and create new WORKSPACE rules pointing to them.
It also refactors many similar loop-over-user-input functions into using
a reusable method (not the more complex ones).
Specifying an SDK directory will further query for the available SDK API
levels and build tools versions, but it won't perform any compatibility
checks.
Like other settings, every android-related setting can be set beforehand
via an env param. The script will not ask for any Android settings if
there are already any android repository rules in the WORKSPACE.
The script will emit a warning if using an NDK version newer than 14 due
to https://github.com/bazelbuild/bazel/issues/4068.
PiperOrigin-RevId: 177989785
