-
-
Notifications
You must be signed in to change notification settings - Fork 315
Extract Windows build how-to from the Knowledge Base #2792
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
bd22f1b
4a5d82c
224be82
6c2ece8
87853ba
cebb030
70d2c34
88beb1f
6bd00ac
4c1901a
09e8438
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,11 @@ | ||
| --- | ||
| tags: [how-to, advanced] | ||
| --- | ||
|
|
||
| # Building packages for Windows | ||
|
|
||
| The following guides cover topics specific to building packages for Windows on conda-forge. | ||
|
|
||
| import DocCardList from '@theme/DocCardList'; | ||
|
|
||
| <DocCardList /> |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,20 @@ | ||
| --- | ||
| tags: [how-to, advanced, windows] | ||
| --- | ||
|
|
||
| # CMD/Batch scripting for Windows recipes | ||
|
|
||
| Windows recipes rely on CMD/Batch scripts (`.bat`) by default. | ||
| Batch syntax is a bit different from Bash and friends on Unix, so we have collected some tips here to help you get started if you are not familiar with this scripting language. | ||
|
|
||
| - Check if you need to write a Batch script first! | ||
| Simple recipes might not need shell-specific code and can be written in an agnostic way. | ||
| Use the `build.script` item in `meta.yaml` (see [conda-build docs](https://docs.conda.io/projects/conda-build/en/stable/resources/define-metadata.html#script)). | ||
| This item can take a string or a list of strings (one per line). | ||
| - [SS64's CMD howto pages](https://ss64.com/nt/syntax.html) are the best resource for any kind of question regarding CMD/Batch syntax. | ||
| - Search conda-forge for existing `.bat` scripts and learn with examples. | ||
| See this [example query for all Batchfiles](https://github.com/search?q=org%3Aconda-forge+language%3ABatchfile&type=code&l=Batchfile). | ||
| - You can get free trial Windows VMs from Microsoft. | ||
| Set one up with your favorite virtualization solution to debug your CMD syntax. | ||
| There are also some minimal emulators online that might get you started with the basics, even if not all CMD features are present. | ||
| For example, this [Windows 95 emulator](https://www.pcjs.org/software/pcx86/sys/windows/win95/4.00.950/) features a more or less okay MS-DOS prompt. |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,56 @@ | ||
| --- | ||
| tags: [how-to, advanced, windows] | ||
| --- | ||
|
|
||
| # Testing and debugging Windows builds locally | ||
|
|
||
| The first thing that you should know is that you can locally test Windows | ||
| builds of your packages even if you don't own a Windows machine. Microsoft | ||
| makes available free trial Windows virtual machines (VMs). If you | ||
| are unfamiliar with VM systems or have trouble installing Microsoft's VMs, please | ||
| use a general web search to explore — while these topics are beyond the | ||
| scope of this documentation, there are ample discussions on them on the broader | ||
| Internet. | ||
|
|
||
| To bootstrap a conda environment, consider | ||
| [miniforge](https://github.com/conda-forge/miniforge). | ||
|
|
||
| ## Testing using wine | ||
|
|
||
| Some degree of testing and debugging can also be performed without a Windows | ||
| system, using [wine](https://www.winehq.org/). | ||
| [miniforge](https://github.com/conda-forge/miniforge) works correctly | ||
| in the wine's `cmd` shell, and can be used to create and run Conda environments. | ||
| In fact, sometimes Wine is able to provide more insightful error messages, | ||
| for example: | ||
|
|
||
| ``` | ||
| wine: Call from 00006FFFFFF999EA to unimplemented function libblas.dll.cdotc_, aborting | ||
| ``` | ||
|
|
||
| It may be necessary to manipulate the `WINEDEBUG` variable to obtain more | ||
| debugging logs. | ||
|
|
||
| ## Debugging DLL issues | ||
|
|
||
| When debugging issues related to dynamically-linked libraries (DLLs) failing | ||
| to load, the following tools can be helpful: | ||
|
|
||
| - [Windows Debugger](https://learn.microsoft.com/en-us/windows-hardware/drivers/debugger/debugger-download-tools) | ||
| can be used when debugging the dreaded "DLL load failed" errors. For example to debug a numpy import error: | ||
| - `"C:\Program Files (x86)\Windows Kits\10\Debuggers\x64\gflags.exe" -i python.exe +sls` | ||
| - `"C:\Program Files (x86)\Windows Kits\10\Debuggers\x64\cdb.exe" -logo log.txt -g -G -o -xn av python -c "import numpy"` | ||
| The log file saved in `log.txt` will display information about where the DLLs were loaded from, | ||
| which DLLs are missing and which symbols are missing from a DLL. | ||
|
|
||
| - [Dependency Walker](https://www.dependencywalker.com/) can display a tree | ||
| diagram of all dependent modules. | ||
|
|
||
| - [Dependencies](https://github.com/lucasg/Dependencies) is a more modern | ||
| replacement for Dependency Walker, with both console and GUI interface. | ||
| It also works better on Wine. | ||
|
|
||
| - [dumpbin](https://learn.microsoft.com/en-us/cpp/build/reference/dumpbin-reference) | ||
| tool from MSVC can be used to obtain information about Windows binaries. | ||
| On Unix, [gendef](https://sourceforge.net/p/mingw-w64/wiki2/gendef) | ||
| tool can be used instead by installing the conda package. | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This one in particular feels more like an explanation at times to me, but it's also a how-to of sorts. No need to change anything, just saying that it's mixed media and maybe we will get back to it at some point. |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,147 @@ | ||
| --- | ||
| tags: [how-to, advanced, windows] | ||
| --- | ||
|
|
||
| import { RecipeTabs } from "@site/src/components/RecipeTabs"; | ||
|
|
||
| # Compiling native code on Windows with MSVC | ||
|
|
||
| In order to compile native code (C, C++, etc.) on Windows, you will need to | ||
| install Microsoft's Visual C++ build tools on your VM. You must install | ||
| particular versions of these tools — this is to maintain compatibility between | ||
| compiled libraries used in Python, [as described on this Python wiki page](https://wiki.python.org/moin/WindowsCompilers). The current relevant | ||
| versions are: | ||
|
|
||
| - For Python 3.5–3.12+: Visual C++ 14.x | ||
|
|
||
| While you can obtain these tools by installing the right version of the full | ||
| [Visual Studio](https://visualstudio.microsoft.com/) development | ||
| environment, you can save a lot of time and bandwidth by installing standalone | ||
| "build tools" packages. You can get them from [Visual Studio | ||
| Subscriptions](https://visualstudio.microsoft.com/vs/older-downloads/#visual-studio-2022-and-other-products). | ||
| To download build tools, you'll need a Microsoft account. Once on the | ||
| Visual Studio Subscriptions page, you may also need to join the Dev Essentials | ||
| program. Once that's done, you can click the "Download" tab and search for | ||
| "Build Tools for Visual Studio 2022". You can directly install VC-2022 using the | ||
| [Visual Studio Build Tools 2022 installer](https://aka.ms/vs/17/release/vs_BuildTools.exe). | ||
|
|
||
| If you need more information. Please refer [the Python wiki page on Windows compilers](https://wiki.python.org/moin/WindowsCompilers). | ||
|
|
||
| ## Simple CMake-Based `bld.bat` | ||
|
|
||
| Some projects provide hooks for CMake to build the project. The following | ||
| example `bld.bat` file demonstrates how to build a traditional, out-of-core | ||
| build for such projects. | ||
|
|
||
| ```batch title="CMake-based bld.bat" | ||
| setlocal EnableDelayedExpansion | ||
|
|
||
| :: Make a build folder and change to it. | ||
| mkdir build | ||
| cd build | ||
|
|
||
| :: Configure using the CMakeFiles | ||
| cmake -G "NMake Makefiles" ^ | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Don't we need to pass
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I think, I hope this addresses your review. |
||
| %CMAKE_ARGS% ^ | ||
| .. | ||
| if errorlevel 1 exit 1 | ||
|
|
||
| :: Build! | ||
| nmake | ||
| if errorlevel 1 exit 1 | ||
|
|
||
| :: Install! | ||
| nmake install | ||
| if errorlevel 1 exit 1 | ||
| ``` | ||
|
|
||
| The following feedstocks are examples of this build structure deployed: | ||
|
|
||
| - [Pugixml](https://github.com/conda-forge/pugixml-feedstock/blob/main/recipe/bld.bat) | ||
|
|
||
| <a id="building-for-different-vc-versions"></a> | ||
|
|
||
| ## Versions of Microsoft Visual C++ (MSVC) | ||
|
|
||
| Up until Visual Studio (VS) 2013, the MSVC compiler changed ABI on every major release, | ||
| meaning packages that communicated through a C/C++ API had to be built with a consistent | ||
| version. This meant for example that python 2.7 required `vs2008` for the duration of its lifetime. | ||
|
|
||
| Since VS2015, the ABI has been kept stable though, meaning that packages do not have to be | ||
| built with the same MSVC version. This allows us to avoid complicated migrations for ~all | ||
| compiled packages, or getting stuck on old compiler versions. | ||
|
|
||
| Speaking of versions, the situation with MSVC can be very confusing. The Visual Studio year is | ||
| often used as a shorthand for the compiler generation, but what's really relevant is the | ||
| toolchain version, which, since VS2015 has been 14.x, and which is referred to as `vc`. | ||
|
|
||
| | VS Year | VS Version | Compiler Version | Toolchain Version | Runtime Version | | ||
| | ------- | ---------- | ---------------- | ----------------- | --------------- | | ||
| | 2015 | 14.0 | 19.0 | 14.0 | 14.0.zzzzz | | ||
| | 2017 | 15.x | 19.1y | 14.1 | 14.1y.zzzzz | | ||
| | 2019 | 16.x | 19.2x | 14.2 | 14.2x.zzzzz | | ||
| | 2022 | 17.x | 19.3x | 14.3 | 14.3x.zzzzz | | ||
|
|
||
| In the table above, `x` and `y` on the same line are referring to the same digit, though there | ||
| are various deviations from this schema. For example, the minor versions for the 2022 line went | ||
| beyond 9, so 17.14 corresponds to compiler 19.44 (30+14) and runtime 14.44. | ||
|
|
||
| We are always able to use the newest runtime in an environment, so we always satisfy the respective | ||
| lower bound imposed by the compiler version. However, there are other situations where the toolchain | ||
| version becomes "viral", in the sense that certain artefacts like static libraries have to be | ||
| consumed by a toolchain that's at least as new as the one that produced it. We therefore try to | ||
| not raise our default compiler version too quickly. | ||
|
|
||
| As of June 2025, we are now on the most recent VS2022 (because VS2019 had reached end-of-life already | ||
| in mid 2024), though the next major version of Visual Studio is already on the horizon. It will | ||
| still be ABI-compatible, such that we will not have to rebuild everything. In other words, the | ||
| toolchain version will remain `14.x`. | ||
|
|
||
| If the day ever comes where MSVC breaks the ABI again (codename "vNext"), then we will have to rebuild | ||
| all compiled packages in conda-forge on windows. After such a migration has taken place, you can then | ||
| choose to skip building for one or the other by using `vc` in a selector, e.g. | ||
|
|
||
| <RecipeTabs> | ||
|
|
||
| ```recipe | ||
| build: | ||
| # to demonstrate mechanism: only build vNext | ||
| skip: true # [win and vc<15] | ||
|
|
||
| requirements: | ||
| build: | ||
| - {{ compiler('cxx') }} | ||
| ``` | ||
|
|
||
| ```recipe | ||
| build: | ||
| # to demonstrate mechanism: only build vNext | ||
| skip: win and vc<15 | ||
|
|
||
| requirements: | ||
| build: | ||
| - ${{ compiler('cxx') }} | ||
| ``` | ||
|
|
||
| </RecipeTabs> | ||
|
|
||
| <a id="using-vs2022"></a> | ||
|
|
||
| ## Using newer MSVC versions | ||
|
|
||
| Given that conda-forge is currently (June 2025) on the latest VS line, it's currently not possible | ||
| to use even newer versions. However, we expect a new Visual Studio major version within the next year | ||
| or so, and once it becomes available in the runner-images for Azure Pipelines and then in conda-forge, | ||
| it will be possible to opt into newer compilers (e.g. for C++23 support) as follows. | ||
|
|
||
| In `recipe/conda_build_config.yaml` file: | ||
|
|
||
| ```recipe title="recipe/conda_build_config.yaml" | ||
| # note: future VS version is speculative! | ||
| c_compiler: # [win] | ||
| - vs2026 # [win] | ||
| cxx_compiler: # [win] | ||
| - vs2026 # [win] | ||
| ``` | ||
|
|
||
| After making these changes don't forget to rerender with `conda-smithy` (to rerender manually use `conda smithy rerender` from the command line). | ||
Uh oh!
There was an error while loading. Please reload this page.