Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Starting on windows #149

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

aloysbaillet
Copy link
Contributor

Added windows Dockerfile which can already build a few packages, I've tested ninja and cmake for now.

N.B. that building clang from source makes little sense as binaries are already available for windows. Building python from source also makes little sense... We need to think about which packages we really want to handle properly in Conan.

Signed-off-by: Aloys Baillet <[email protected]>
Copy link
Contributor

@jfpanisset jfpanisset left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very exciting stuff, can't wait to see first officially published Conan Windows packages.

On Windows, you are using the same "build inside a container" approach as on Linux, although you aren't publishing containers (that I can tell), just the resulting Conan packages, right? For Linux building inside a container is required to get a CentOS 7 / VFX Platform like start point (instead of the Ubuntu environment provided by GHA runners). Since there's a lot less variability on the Windows platform, would it make sense instead to build directly on the VM, just installing / specifying the VS Code version you want, and thus save the non-trivial overhead of large Windows base container images?

I imagine that supporting "native" builds will be non trivial, but will likely be needed to support building Conan packages on macOS.

@@ -53,7 +53,8 @@ def configure(self):

def _configure_cmake(self):
cmake = CMake(self)
cmake.definitions["GCC_INSTALL_PREFIX"] = os.environ["GCC_INSTALL_PREFIX"]
if platform.system() == "Linux":
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

May end up being:

and platform.system() != "Windows"

with self._build_context():
autotools = self._configure_autotools()
autotools.make()
if system() == "Linux":
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

match / case when when we switch to Python 3.10 and support Darwin/macOS


[storage]
path = ../d
download_cache = /tmp/downloads
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That looks like a Linux-specific path?

@@ -56,14 +57,19 @@ def make_bake_dict(self) -> typing.Dict[str, dict]:
version = version_info.ci_common_version
major_version = utils.get_major_version(version)
versions_to_bake.add(version)
if platform.system() == "Linux":
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

match / case with Python 3.10 / macOS support


RUN choco install --no-progress --yes visualstudio2017buildtools --version=15.9.41.0
RUN choco install --no-progress --yes visualstudio2017-workload-vctools --version=1.3.1
RUN choco install --no-progress --yes --execution-timeout=0 visualstudio2017-workload-manageddesktop --version=1.2.1
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

VFX Platform now specifies Visual Studio version: 2017 for CY2021, "Visual Studio 2019 v16.9 or later" for CY2022,
so version of Visual Studio to be pulled from Chocolatey should be in a more generic CY20XX config YAML?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point, I've been trying to use the ci-common for the low-level toolchain, but the versions change at different rates for windows and linux, such a pain :(


RUN iex ((New-Object System.Net.WebClient).DownloadString('https://chocolatey.org/install.ps1')); \
$env:Path += '";C:\tools\python3;C:\tools\python3\Scripts"'; \
choco install --no-progress --yes python3 --version=${Env:ASWF_CONAN_PYTHON_VERSION} --params '"/InstallDir:C:\tools\python3"'
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should the version of Python 3 be specific to the VFX Platform CY20XX, or is this just for build purposes and won't end up in the build artifacts?

# variables available

LINUX_CONAN_USER_HOME = "/tmp/c"
WINDOWS_CONAN_USER_HOME = "C:\\tmp\\c"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On Windows the system defined %TEMP% directory is typically C:\Users\User Name\AppData\Local\Temp aka %USERPROFILE%\AppData\Local\Temp, although I've seen a lot of local installs use C:\Temp which goes back to very old versions of Windows.

@@ -405,7 +481,7 @@ def test_builder_cli_conan(self):
)
self.assertTrue(
cmds[2].endswith(
f"conan create {constants.CONAN_USER_HOME}/recipes/openexr openexr/2.3.0@aswftesting/vfx2019'"
f"conan create {os.path.join(self._docker_home, 'recipes', 'openexr')} openexr/2.3.0@aswftesting/vfx2019'"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good change to not build paths yourself. Could be interesting exercise to grep for / in the whole codebase (you may have already done that).

@aloysbaillet
Copy link
Contributor Author

Thanks JF!
Sorry I was meant to set this PR as a draft for now, still quite WIP :)

I think using a windows container has at least these 2 nice advantages:

  • more reproducibility especially with things like python installs which can only be done once per version, with secret registry keys that could change build behaviours etc... This is quite key for building things locally and having some confidence that these will build as well in the GitHub actions.
  • more symmetric code with the linux builds, which means simpler code overall

That said, once we have the packages built and uploaded to Conan, I don't see much value in using containers to download and use the packages, I don't expect client repos to use docker for windows builds.
Hopefully we can provide a GitHub action that installs Conan with the right settings on the windows GHA VMs when running windows CI.

Happy to discuss at the next CI meeting.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants