Skip to content

Conversation

MugundanMCW
Copy link
Contributor

  • The adoption of Windows on ARM (WoA) devices is steadily increasing, yet many Python wheels are still not available for this platform.
  • GitHub Actions now offer native CI runners for Windows on ARM devices (windows-11-arm), enabling automated builds and testing.
  • Currently, official tokenizers Python wheels are not provided for Windows ARM64 and thus users/developers were facing difficulties using popular tokenizers library natively.
  • This PR introduces support for building tokenizers wheels on Windows ARM64, improving accessibility for developers and end users on this emerging platform.

Copy link
Contributor

@Narsil Narsil left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In principle yes.

Note for @ArthurZucker : When CI was added and support for ARM added, Windows had tons of bugs making it impossible for us to release.
I hope it works better now.

with:
python-version: 3.x
architecture: ${{ matrix.platform.target }}
architecture: ${{ matrix.platform.target == 'aarch64' && 'arm64' || matrix.platform.target }}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we remove this weird condition, and have 2 target names if it's really needed ? Why isn't the target named aarch64 ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @Narsil,
The reason for using a separate Python target name for Windows on ARM is that the official Python GitHub Action only supports the architectures x86, x64, and arm64 for Windows [Reference]. Specifying aarch64 for Windows on ARM64 results in an invalid configuration, which causes the CI to fail.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I know, but then maybe we could have "platform.target" and "platform.architecture" be defined in the matrix, which should make the flow easier to read.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Narsil thanks for the suggestions, made the relevant changes

@MugundanMCW MugundanMCW requested a review from Narsil October 13, 2025 08:19
Copy link
Collaborator

@ArthurZucker ArthurZucker left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sounds good!

@HuggingFaceDocBuilderDev

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

@ArthurZucker ArthurZucker merged commit 386f3d8 into huggingface:main Oct 16, 2025
30 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants