Some wheels like `jax-cuda12-plugin` have `-` symbol in the name, but `_` symbol in the filename (e.g. `jax_cuda12_plugin-0.8.0.dev0+selfbuilt-cp313-cp313t-manylinux_2_27_x86_64.whl`). Older `rules_python` implementation didn't make a difference whether the lock file had `jax-cuda12-plugin` or `jax_cuda12_plugin` records. `rules_python` v1.6 treats them differently: if there are records like below, then the local wheel doesn't override the one that should be downloaded from pypi.
```
jax-cuda12-plugin==0.7.2 ; sys_platform == "linux" and python_version < "3.14" \
--hash=sha256:05b6942985f015be82becd2cec363f0aceb25311981821d7613a51f630490e8c \
--hash=sha256:1d00f9f5c5f68ae0f41cb7b589005ed5cb556517d65bbab5a891be46ed7a781c \
--hash=sha256:23b8f1050c48b4020610fb818930d3cbe0304c6681b069687e5416ee349bd734 \
--hash=sha256:2a727a89ae69ac21c1f5093d8d5aef89a0e692e66b034fc934c8accc72e40290 \
--hash=sha256:45d5a1cbf0b9d05318722382fc71c4cede0c028bad6aa8e53f7a7032392f719c \
--hash=sha256:5e3e2aa4d721fb02dd1028262aaeaec2958e45bca5c4d3512b29151b570cb425 \
--hash=sha256:7212c12d75b7dc51275f271827df4a6d378430c06f650e6c31c162fe9579ff12 \
--hash=sha256:7ad3afc51bcbc4e8117845d359e5d02cbc5ca2b152efdebd3c55fb9e4c2f848e \
--hash=sha256:8284e7cf7f544906604f111702a6f0011a96df7f0113878b381bec0905172536 \
--hash=sha256:98a975655382858d874d6471ce97194310609d0a2a7c4283c6e07e37933b7768 \
--hash=sha256:adc924ebc7a45c8d3400ea0118dc70a7082b2a86e35711738d403dd3815d09bf \
--hash=sha256:e881b56fe27e6870db2f2e9c574b81965fe1102b1532eae60e240a40c065daf5
# via -r build/requirements.in
jax_cuda12_plugin @ file:///jax_cuda12_plugin-0.8.0.dev0+selfbuilt-cp311-cp311-manylinux_2_27_x86_64.whl
```
To fix this, `python_repo()` implementation was changed to add the local wheels with exactly the same module names as in the original lock file.
PiperOrigin-RevId: 809147267
Forked from https://github.com/openxla/xla/pull/25183
Without this change rules_python doesn't correctly install free-threaded wheels for a free-threaded Python build.
PiperOrigin-RevId: 749128015
The gist of this change (the new rules implementation) is contained within `rules_pywrap` folder. The rules are generic and not tensorflow-specific
1) (internal-specific)
2) (internal-specific)
3) It provides same linking strategy of final artifacts on all 3 supported platforms (no major differences between Linux, Mac and Windows).
4) It makes it possible to abandon usage of header-only targets to prevent ODR violations. Simply speaking you can now depend on generated protobuf message classes normally, without need to worry how that is linked afterwards.
5) The current version is backward-compatible and unless explicitly enabled is a no-op. To enable the new rules pass `--repo_env=USE_PYWRAP_RULES=True` flag to build/test command.
6) The `if_pywrap` construct is temporary and will be removed once full migration is completed. Currently if_pywrap is mainly used to pass normal dependencies (instead of header-only). The header-only stuff is kept for backward compatibility and smoother migration but will be eventually removed.
7) This CL migrates TF and the most problematic among all google ML repositories. Once TF is sabilized the other repositories, such as JAX and XLA will be migrated too (which should be way easier than migrating TF anyways)
PiperOrigin-RevId: 684324990
This test verifies whether the API v2 packages can be imported from the
current build. It utilizes the `_api/v2/api_packages.txt` list of packages from
the local wheel file specified in the `requirements_lock_<python_version>.txt`.
The test should be executed after the TF wheel was built and put into `dist` dir inside Tensorflow repository.
PiperOrigin-RevId: 676893008
637889039 by A. Unique TensorFlower<gardener@tensorflow.org>:
Remove experimental_adaptive_avx_optimization flag from XNNPACK delegate options
It's always on now.
--
637886275 by A. Unique TensorFlower<gardener@tensorflow.org>:
[XLA:GPU][IndexAnalysis] Use a flag for IsKnownEmpty instead of recomputing every time.
Right now, we would try to simplify or compose with indexing maps that have a known empty domain. That's incorrect, but checking if the domain is empty every time is expensive and can be cached.
--
637876088 by A. Unique TensorFlower<gardener@tensorflow.org>:
Internal config change
--
637864812 by A. Unique TensorFlower<gardener@tensorflow.org>:
PR #13088: [ROCm] Fix reduce_atomic_min.hlo.test
Imported from GitHub PR https://github.com/openxla/xla/pull/13088
Copybara import of the project:
--
b241e076198c03fffd8c7e3a6568070ef0223653 by mmakevic <Milica.Makevic@amd.com>:
Fix reduce_atomic_min.hlo.test
--
f894f1954513019f0ca6890a27e09e0fee9d462e by mmakevic <Milica.Makevic@amd.com>:
Remove extra space
Merging this change closes#13088
--
637860531 by A. Unique TensorFlower<gardener@tensorflow.org>:
Remove xla_gpu_normalize_layouts flag.
By now, this is really not experimental anymore.
--
637857834 by A. Unique TensorFlower<gardener@tensorflow.org>:
Add heuristic for when to treat Gather ops as coalesced.
--
637820064 by A. Unique TensorFlower<gardener@tensorflow.org>:
compat: Update forward compatibility horizon to 2024-05-28
--
637820063 by A. Unique TensorFlower<gardener@tensorflow.org>:
Update GraphDef version to 1876.
--
637756070 by A. Unique TensorFlower<gardener@tensorflow.org>:
Automated rollback of changelist 636206934.
637674999 by A. Unique TensorFlower<gardener@tensorflow.org>:
[xla:cpu] Add initial support for Thunk-based execution to CpuCompiler and CpuExecutable
Add support for compiling XLA:CPU HloModule to a ThunkSequence instead of a LLVM module and a jit-compiled function.
--
637666734 by A. Unique TensorFlower<gardener@tensorflow.org>:
Don't fuse inside computations that are already fused.
--
637657345 by A. Unique TensorFlower<gardener@tensorflow.org>:
Automated rollback of changelist 636208997.
637651034 by A. Unique TensorFlower<gardener@tensorflow.org>:
Integrate LLVM at llvm/llvm-project@fddf350f96
Updates LLVM usage to match
[fddf350f9640](https://github.com/llvm/llvm-project/commit/fddf350f9640)
--
637639233 by A. Unique TensorFlower<gardener@tensorflow.org>:
PR #12940: [ROCm] Fix dot_bf16.hlo.test on ROCm
Imported from GitHub PR https://github.com/openxla/xla/pull/12940
Added additional params for `hlo_lit_tests` as a workaround, so `mi200.txtpb` would be used in `dot_bf16.hlo.test` for rocm.
Copybara import of the project:
--
c3bb3a7349266a51ff22a2e18dab0afb6e81bad4 by mmakevic <Milica.Makevic@amd.com>:
Have dot_bf16.hlo.test use mi200.txtpb for rocm
Merging this change closes#12940
--
637632492 by A. Unique TensorFlower<gardener@tensorflow.org>:
PR #13089: Fix reduce_large_row_to_scalar.hlo.test
Imported from GitHub PR https://github.com/openxla/xla/pull/13089
Copybara import of the project:
--
ae97058c01ca57107a2566a6f190d51f5ad4ca0e by mmakevic <Milica.Makevic@amd.com>:
Fix reduce_large_row_to_scalar.hlo.test
Merging this change closes#13089
--
637623329 by A. Unique TensorFlower<gardener@tensorflow.org>:
Automated rollback of changelist 637594837.
637607386 by A. Unique TensorFlower<gardener@tensorflow.org>:
Automated rollback of changelist 636926669.
637594837 by A. Unique TensorFlower<gardener@tensorflow.org>:
[XLA:GPU] Pass CUDA_VERSION explicitly into CudnnFusedConvRewriter.
Passing the CuDNN version will be the next step.
--
637580666 by A. Unique TensorFlower<gardener@tensorflow.org>:
Remove usage of --xla_gpu_enable_triton_hopper in autotuner
--
637578573 by A. Unique TensorFlower<gardener@tensorflow.org>:
[XLA:GPU] Add documentation about RTVars.
--
637570959 by A. Unique TensorFlower<gardener@tensorflow.org>:
Update GraphDef version to 1875.
--
637570942 by A. Unique TensorFlower<gardener@tensorflow.org>:
compat: Update forward compatibility horizon to 2024-05-27
--
637561798 by A. Unique TensorFlower<gardener@tensorflow.org>:
PR #12979: [NVIDIA] Fix PGLE for latency estimation of p2p instructions
Imported from GitHub PR https://github.com/openxla/xla/pull/12979
PGLE doesn't recognize p2p instruction such as send or recv as async operations.
This adds the utility to check if instruction is a p2p communication instruction.
Copybara import of the project:
--
469b2d31ff6b0270dda28f8754462681514d0e04 by TJ Xu <tjx@nvidia.com>:
fix pgle not recognizing p2p instructions
Merging this change closes#12979
--
637560035 by A. Unique TensorFlower<gardener@tensorflow.org>:
[xla:gpu] Track loop iteration counter of a WhileThunk in thread local variable
--
637552495 by A. Unique TensorFlower<gardener@tensorflow.org>:
PR #13056: Use `operator->` with XLA FFI Result Buffers in custom call docs
Imported from GitHub PR https://github.com/openxla/xla/pull/13056
Copybara import of the project:
--
7940a1a02a0f93736a88406958edf62488bdbe19 by Andrey Portnoy <aportnoy@nvidia.com>:
Use `operator->` with XLA FFI Result Buffers in custom call docs
Merging this change closes#13056
--
637547404 by A. Unique TensorFlower<gardener@tensorflow.org>:
PR #13068: Introduce the Blackwell compute capability.
Imported from GitHub PR https://github.com/openxla/xla/pull/13068
Introduce the Blackwell compute capability. Future Blackwell-specific changes can be guarded by this capability.
Copybara import of the project:
--
cc1adebc95166b2d3979cc01de954a1895515ad4 by Dimitris Vardoulakis <dvardoulakis@nvidia.com>:
Introduce the Blackwell compute capability. Future Blackwell-specific changes can be guarded by this capability.
Merging this change closes#13068
--
637541058 by A. Unique TensorFlower<gardener@tensorflow.org>:
PR #13061: Add Tirton support for XLA clamp
Imported from GitHub PR https://github.com/openxla/xla/pull/13061
Add Triton support for XLA clamp instruction. Clamp is a common instruction found in FP8 fusions, and will be used in cuDNN fusions:
This is a fix for perviously rolled-back PR due to internal ir_emitter_triton test failure: d114eceb0a
cc @sergeykozub @sergachev
Copybara import of the project:
--
3496ba2fa86571ab290e0881dd06400c415d80b6 by Elfie Guo <elfieg@nvidia.com>:
Add Tirton support for XLA clamp.
Merging this change closes#13061
--
637366630 by A. Unique TensorFlower<gardener@tensorflow.org>:
Update GraphDef version to 1874.
--
637366295 by A. Unique TensorFlower<gardener@tensorflow.org>:
compat: Update forward compatibility horizon to 2024-05-26
--
637185396 by A. Unique TensorFlower<gardener@tensorflow.org>:
Automated Code Change
--
637168744 by A. Unique TensorFlower<gardener@tensorflow.org>:
Update GraphDef version to 1873.
--
637168421 by A. Unique TensorFlower<gardener@tensorflow.org>:
compat: Update forward compatibility horizon to 2024-05-25
--
637166714 by A. Unique TensorFlower<gardener@tensorflow.org>:
Attempt loading libOpenCL.so before libOpenCL-pixel.so
--
637137789 by A. Unique TensorFlower<gardener@tensorflow.org>:
feat: Implement hermetic Python version matching system Python version
--
637102058 by A. Unique TensorFlower<gardener@tensorflow.org>:
[IFRT] Add xla::ifrt::Sharding::IsFullyReplicated()
IFRT Sharding type gains `IsFullyReplicated()`, which quickly tells if the
sharding represents a fully-replicated sharding.
The main motivation is to make full replication information queriable at IFRT
shardings and prepare for enabling IFRT implementations to handle full
replication directly.
There are a preset of rules:
* `SingleDeviceSharding` is trivially fully replicated by its definition.
* `ConcreteSharding` and `OpaqueSharding` is not fully replicated. They have special cases where it may be fully replicated, but the user is advised to use a more specific sharding type to represent such cases.
* `ConcreteEvenSharding` may/may not fully replicated. This is controlled at creation time.
* `ShardingParamSharding` and (IFRT) `HloSharding` depend on whether their lower-level sharding represents full replication.
`ConcreteEvenSharding` is a noteworthy case where the full replication information
does not come from the existing source of the information. This is because the
creators of this sharding (e.g., JAX) typically has the information, but the
replication information is lost when coercing it into `ConcreteEvenSharding`.
This problem will be gradually less problematic once JAX uses a higher-level
IFRT sharding type (mainly (IFRT) `HloSharding`) at more places.
This change extends the `Sharding` type, but the new method is not used by any
existing code.
--
637097325 by A. Unique TensorFlower<gardener@tensorflow.org>:
Ensure delegates properly delegate models
--
637080761 by A. Unique TensorFlower<gardener@tensorflow.org>:
Add barrier logs.
--
637070664 by A. Unique TensorFlower<gardener@tensorflow.org>:
Clean up include and build file
--
637069670 by A. Unique TensorFlower<gardener@tensorflow.org>:
Use the `LoadedClientGraph`'s copy of `FunctionLibraryDefinition` instead of getting it from the `FallbackState` in the parent `GraphExecutor`
--
637069442 by A. Unique TensorFlower<gardener@tensorflow.org>:
update doc ref
--
637061122 by A. Unique TensorFlower<gardener@tensorflow.org>:
Refactor exhaustive testing of unary float32 functions into a library.
--
637046941 by A. Unique TensorFlower<gardener@tensorflow.org>:
fix profile_util's compatible_with tag typo
--
637028365 by A. Unique TensorFlower<gardener@tensorflow.org>:
[XLA] Refactor HostOffloader.
Change HostOffloader's algorithm for identifying host memory offloading. This approach supports every conceivable host memory offloading pattern (as of today).
--
637023690 by A. Unique TensorFlower<gardener@tensorflow.org>:
Simplify volumes for docker container in XLA build script
--
637018892 by A. Unique TensorFlower<gardener@tensorflow.org>:
move flatbuffer_compatibility_test target to tflite compiler
--
637008187 by A. Unique TensorFlower<gardener@tensorflow.org>:
Add copyright notice to profiler_utils.cc
--
636990162 by A. Unique TensorFlower<gardener@tensorflow.org>:
Adds a proto profile summary formatter to the TFLite benchmark.
Adds a Python script to convert benchmark profile protos to a JSON consumable by the model-explorer.
--
636976463 by A. Unique TensorFlower<gardener@tensorflow.org>:
Add profiler_util to enable flexibly tpu profiler registration for different purposes
--
PiperOrigin-RevId: 637889039
This allows users to specify a list of workspaces that contain pre-built local wheels without need to manually add them in requirements.txt files.
The wheels will be automatically processed by bazel rules and injected into the requirements_lock_<py_version>.txt on the fly (assuming `HERMETIC_PYTHON_VERSION=py_version`).
This feature is mainly inspired by pytorch/xla demand, since building pytorch/xla implies first building pytorch repo locally and then pointing to its artifacts (both raw .so files and entire .whl) in pytorch/xla build.
This also helps JAX to facilitate build_jaxlib=false case, as it would eliminate need to manually update requirements_locak.txt files in JAX CI as well.
PiperOrigin-RevId: 636691616
