Release v3.2.0

Release highlights

iree-turbine core

• #434 : iree-turbine has a new website: https://iree-turbine.readthedocs.io/. See the docs/ folder for contributing instructions.

• #373 : The deprecated shark_turbine namespace has been fully deleted, users should migrate to iree.turbine.

• #418: There are new utility APIs for preparing tensors as input arguments for IREE tools:

  # iree.turbine.support.conversions
  torch_dtyped_shape_to_iree_format(...)
  
  # iree.turbine.support.tools
  iree_tool_format_cli_input_arg(...)
  iree_tool_prepare_input_args(...)

TKW

Improved support and performance for attention kernel variants:

• #387: Added a new paged decode attention kernel.
• #412: Added a new implementation of prefill attention.
• #452: Add self_index, predicate, and selectOp to implement causal attention.
• #424: Reordering of shared load-write to minimize LDS barriers improved performance for some attention shapes by up to 10%.

Other optimizations:

• #394: A memory layout attribute for the MemoryType now allows users to specify a physical shape that differs from the logical shape. This is useful in scenarios like kv-caches where certain dimensions physically are quite large but map to fixed logical dimensions.
• #436: Use buffer ops for masked load/stores.

Development quality of life features:

• #406: Tests parameterized by shapes now have better names.
• #423: Wave pass pipelines now feature printing options.

Changelog

Full list of changes: v3.1.0...v3.2.0

Release v3.1.0

Release highlights

iree-turbine core

The core of iree-turbine remains fairly stable, with a few quality of life and infrastructure changes this release cycle.

• PyTorch is no longer a direct requirement that gets automatically installed alongside the iree-turbine package as of #343. Instead, users should install the torch package variant of their choice before importing from iree-turbine will succeed. For example:

  pip install torch --index-url https://download.pytorch.org/whl/test/cpu
  pip install iree-turbine

• Pre-release versions of the iree-turbine package are now published nightly. Install with

  pip install \
    --find-links https://iree.dev/pip-release-links.html \
    --upgrade --pre \
    iree-turbine
  

• Project tests are now running continuously across Python versions 3.10-3.12: #326.

• A new turbine_generate rule is available for the iree.build API. This can be used to programmatically export and compile pipelines of programs using iree-turbine: #249.

TKW

Activity in the TKW (turbine/kernel/wave/) subproject has been accelerating.

• Attention decode kernels are in development: #308, #340.
  • Support has also been added to target MMA and MFMA intrinsics, workings towards efficient GEMM and attention kernels using lower precision data types: #279, #269, #287.
• TKW now supports a compiled kernel cache that helps with both online and offline uses: #329, #351.
• An evoformer example was added at iree/turbine/kernel/wave/templates/evoformer.py in #307.
• A number of optimizations have been added across the project: #291, #296, #292, #304, #318, #335, and more.

Full changelog

v3.0.0...v3.1.0

dev-wheels

Automatic snapshot release of iree-turbine python wheels.

Release v3.0.0

Full changelog: v2.9.0...v3.0.0

Release v2.9.0

Release highlights

• Bumped IREE to the latest stable release using new iree-base-compiler and iree-base-runtime package names and the matching version 2.9.0.

Full Changelog: v2.5.0...v2.9.0

Release v2.5.0

Release highlights

• Bumped IREE to the latest stable release: 20241104.1068. We expect the next release to use a new version scheme and new package names. See iree-org/iree#18938 for full details.
• PyTorch versions 2.5+ are now supported thanks to 97e0517.
• The migration from the shark-turbine package namespace to iree.turbine is structurally complete: 40016ad . Downstream projects should switch to the new namespace and iree-turbine package name.

Full Changelog: v2.3.0...v2.5.0

2.3.0

Release of both shark-turbine and iree-turbine wheels for v2.3.0. This release requires torch 2.3, which was just released (we still have some uses of prior versions and it can work in carefully crafted ways on prior versions, but it is not recommended).

Notes:

• PyTorch 2.3 cut without Python 3.12 support. Recommend either sticking with 3.11 or using a nightly PyTorch. They expect that 2.4 will include Python 3.12 support properly.
• This is the last release that we will make of both iree-turbine and shark-turbine (they are currently a duplicate of each other). Future versions will only include the former.
• This is also the last release where the packages will be natively under import shark_turbine. Currently import iree.turbine aliases to that, but this will be swapped for the next release.
• This was released out of its new home at https://github.com/iree-org/iree-turbine

What's Changed

• Run pre-submit on all files. by @stellaraccident in #2
• Add device bridge support for HIP and CUDA. by @stellaraccident in #3
• Ensure that globals are always emitted in declaration order at the top. by @stellaraccident in #5
• Add initial CI by @saienduri in #4

New Contributors

• @saienduri made their first contribution in #4

Full Changelog: https://github.com/iree-org/iree-turbine/commits/v2.3.0