use vcpkg for Windows & macOS builds; add Qt6 builds; use sccache #107

Be-ing · 2022-03-08T04:17:48Z

This builds Qt with vcpkg and caches the C++ dependencies on GitHub Packages. The builds work on Linux, Windows, and macOS for both Qt5 and Qt6, however the tests only pass on Linux. It uses sccache for both C++ and Rust to speed up the builds.

Currently it is using GitHub Packages under my GitHub user because that is what I have write access to. GITHUB_TOKEN can only have write access to the KDAB organization's GitHub Packages for a pull request from a branch on the upstream repository; it only has read access for pull requests from forked repositories. When this is merged, Be-ing should be replaced with KDAB in the Authenticate to GitHub Packages step of the GitHub Actions workflow and a GitHub Personal Access Token with read-only permission to packages generated for anonymous access in CMakeLists.txt.

Fixes #78, #79

Be-ing · 2022-03-08T08:40:32Z

qmltestrunner fails to import the QtQuick module on macOS. I suspect this has something to do with statically linking Qt. I tried specifying the -import and -plugins command line options for qmltestrunner using the absolute path to where vcpkg installed Qt, but those did not work. For now I have disabled the one test (example_qml_extension_plugin_test) which uses that. Otherwise macOS tests are working now.

Windows tests are failing to link symbols that look like they should come from cxx-qt-gen/src/cxx_qt.cpp. I don't know what's going wrong there. Maybe not worth spending time on before #105 is merged.

Windows Qt6 tests are hanging indefinitely.

.github/workflows/github-cxx-qt-tests.yml

cmake/CxxQt.cmake

cxx-qt-build/src/lib.rs

ahayzen-kdab · 2022-03-08T10:46:49Z

qmltestrunner fails to import the QtQuick module on macOS. I suspect this has something to do with statically linking Qt. I tried specifying the -import and -plugins command line options for qmltestrunner using the absolute path to where vcpkg installed Qt, but those did not work. For now I have disabled the one test (example_qml_extension_plugin_test) which uses that. Otherwise macOS tests are working now.

Windows tests are failing to link symbols that look like they should come from cxx-qt-gen/src/cxx_qt.cpp. I don't know what's going wrong there. Maybe not worth spending time on before #105 is merged.

Windows Qt6 tests are hanging indefinitely.

So the macOS part sounds interesting, which part are we static linking to Qt ? I thought we should be statically linking the Rust library into the C++ library or executable and then that is dynamic linking to Qt ? Or is something going wrong / missing on macOS ?

For Windows, as you've managed to get things building quite far, I'll try and setup an environment so I can investigate what is failing on Windows (as none of us have tried it yet 😅 ), and as you have noted parts there are potential issues in cxx_qt.cpp are being changed in #105 so it'll likely be after that.

Be-ing · 2022-03-08T16:06:55Z

So the macOS part sounds interesting, which part are we static linking to Qt ?

vcpkg defaults to building everything statically on macOS (as well as Linux, though we're not using vcpkg for Linux builds).

Or is something going wrong / missing on macOS ?

I haven't tested static linking on another OS or dynamic linking on macOS, but that may be worth a try.

Be-ing · 2022-03-08T16:14:41Z

I'll try and setup an environment so I can investigate what is failing on Windows

Note that this branch will automatically download the dependencies with vcpkg when configuring CMake using my read-only PAT. This takes a while (several minutes) during which there isn't much feedback from vcpkg, but don't worry, it hasn't frozen.

Be-ing · 2022-03-08T16:45:17Z

qmltestrunner fails to import the QtQuick module on macOS. I suspect this has something to do with statically linking Qt.

I suspect this may be a bug in qmltestrunner's build system rather than in cxx-qt.

Be-ing · 2022-03-09T15:22:31Z

How shall we proceed with this PR? I think it would be reasonable to disable the tests for Windows, merge this, then you could fix the Windows tests in a future PR.

If you want to speed up the CI runs, manually cancel the earlier runs from https://github.com/KDAB/cxx-qt/actions. The earlier runs will take many hours before they finish because they didn't hit the GitHub Packages cache. I don't have permission to cancel CI runs on this repo.

.github/workflows/github-cxx-qt-tests.yml

ahayzen-kdab · 2022-03-09T19:03:40Z

How shall we proceed with this PR? I think it would be reasonable to disable the tests for Windows, merge this, then you could fix the Windows tests in a future PR.

If you want to speed up the CI runs, manually cancel the earlier runs from https://github.com/KDAB/cxx-qt/actions. The earlier runs will take many hours before they finish because they didn't hit the GitHub Packages cache. I don't have permission to cancel CI runs on this repo.

I'll try and have a closer look at this tomorrow, I wonder about splitting things into smaller changes as we have multiple things going on here. eg there could be the following pull requests (?)

Add Qt 6 support for Linux
Add macOS support (Qt 5 and Qt 6)
Add Windows support (Qt 5 and Qt 6) (we could then leave this one open until we figure out the changes to get the tests passing :-) ) (and probably stack this branch ontop of the macOS one, as the macOS one looks likely to land next)

Be-ing · 2022-03-09T19:09:59Z

Add Qt 6 support for Linux

The rewrite of the GitHub Actions workflow file and adding vcpkg for macOS & Windows is all done together in one commit. If you insist I could split it up, but IMO that wouldn't be worth the trouble.

ahayzen-kdab

It's looking good, left some more comments for clarifications :-)

CMakeLists.txt

.github/workflows/github-cxx-qt-tests.yml

CMakeLists.txt

cmake/CompilerCaching.cmake

vcpkg.json

cmake/CompilerCaching.cmake

Be-ing · 2022-03-10T15:53:54Z

Well, this is strange. Explicitly setting QT_DEFAULT_VERSION_MAJOR=6 with both Qt5 and Qt6 development packages installed on Fedora 35 fails with:

CMake Error at /usr/lib64/cmake/Qt5QmlImportScanner/Qt5QmlImportScannerConfig.cmake:11 (qt6_import_qml_plugins):
  Unknown CMake command "qt6_import_qml_plugins".
Call Stack (most recent call first):
  examples/qml_extension_plugin/CMakeLists.txt:44 (qt_import_qml_plugins)

qt6_import_qml_plugins is defined in /usr/lib64/cmake/Qt6Qml/Qt6QmlMacros.cmake. I am puzzled why Qt5QmlImportScannerConfig.cmake is invoked at all. Not setting QT_DEFAULT_VERSION_MAJOR works fine and defaults to Qt6 as expected. Setting QT_DEFAULT_MAJOR_VERSION=5 also works. I suppose there's a bug in Qt's CMake scripts?? Anyway, it's working now.

Be-ing · 2022-03-10T16:01:25Z

I think all review comments have been addressed at this point. I also added documentation to README.md explaining how the build system works.

ahayzen-kdab · 2022-03-10T16:38:47Z

Well, this is strange. Explicitly setting QT_DEFAULT_VERSION_MAJOR=6 with both Qt5 and Qt6 development packages installed on Fedora 35 fails with:
CMake Error at /usr/lib64/cmake/Qt5QmlImportScanner/Qt5QmlImportScannerConfig.cmake:11 (qt6_import_qml_plugins):
  Unknown CMake command "qt6_import_qml_plugins".
Call Stack (most recent call first):
  examples/qml_extension_plugin/CMakeLists.txt:44 (qt_import_qml_plugins)
qt6_import_qml_plugins is defined in /usr/lib64/cmake/Qt6Qml/Qt6QmlMacros.cmake. I am puzzled why Qt5QmlImportScannerConfig.cmake is invoked at all. Not setting QT_DEFAULT_VERSION_MAJOR works fine and defaults to Qt6 as expected. Setting QT_DEFAULT_MAJOR_VERSION=5 also works. I suppose there's a bug in Qt's CMake scripts?? Anyway, it's working now.

Right sounds like a bug to report upstream :-)

.github/workflows/github-cxx-qt-tests.yml

ahayzen-kdab · 2022-03-10T17:26:54Z

OK, lets go with this!

ahayzen-kdab · 2022-03-10T17:32:41Z

@Be-ing Thanks for your contribution! I wasn't expecting to have this working for a while! :-)

I'll try updating the tokens, I wonder if the "username" is my account or the org account if the token is from my personal account 🤔 ...

Be-ing · 2022-03-10T17:36:24Z

I wonder if the "username" is my account or the org account

Yes, it's confusing, but the user for the GitHub Packages NuGet source is the organization (KDAB), not ahayzen-kdab.

For example you can see the packages I built on my fork: https://github.com/Be-ing?tab=packages

ahayzen-kdab · 2022-03-10T17:46:45Z

Thanks :-)

Be-ing · 2022-03-10T21:34:50Z

The CI build on the update-tokens branch succeeded. You can merge that to the main branch now.

I wasn't expecting to have this working for a while! :-)

Yep, I've done this for two different cross platform C++ applications before, which took months of work, so I figured I'd help out with this cool project. In my experience this setup with vcpkg, GitHub Actions, and GitHub Packages has been the most practical way to maintain cross platform C++ applications. Don't hesitate to ping me if you have any questions maintaining it. The same build process works both on CI and for local development, so you don't get ugly surprises when developers try to get started contributing nor weird issues in CI builds that developers can't reproduce locally. Also, vcpkg works on Linux, so you can maintain packaging for Windows and macOS from Linux without needing to actually use Windows or macOS much.

P.S. I just sent an application to [email protected]. It would be great to continue working together on this project. :)

ahayzen-kdab · 2022-03-11T10:47:03Z

Thanks! It's all in main now :-) I'll tag you if we have any further issues thanks! And good luck ! :-)

Be-ing force-pushed the vcpkg branch 5 times, most recently from 2218664 to f176ea8 Compare March 8, 2022 08:38

Be-ing force-pushed the vcpkg branch 2 times, most recently from 776c64e to 20af347 Compare March 8, 2022 09:55

ahayzen-kdab reviewed Mar 8, 2022

View reviewed changes

.github/workflows/github-cxx-qt-tests.yml Show resolved Hide resolved

ahayzen-kdab reviewed Mar 8, 2022

View reviewed changes

cmake/CxxQt.cmake Show resolved Hide resolved

ahayzen-kdab reviewed Mar 8, 2022

View reviewed changes

cxx-qt-build/src/lib.rs Outdated Show resolved Hide resolved

ahayzen-kdab mentioned this pull request Mar 8, 2022

cxx-qt-lib: use cxx for bridging Qt types #105

Merged

Be-ing mentioned this pull request Mar 8, 2022

cc in build.rs fails to specify -isysroot on macOS when called from CMake dtolnay/cxx#1020

Closed

Be-ing force-pushed the vcpkg branch 3 times, most recently from f47b6ca to b1e755b Compare March 9, 2022 14:58

Be-ing commented Mar 9, 2022

View reviewed changes

.github/workflows/github-cxx-qt-tests.yml Show resolved Hide resolved

Be-ing force-pushed the vcpkg branch 5 times, most recently from c1d9ab3 to 5314485 Compare March 9, 2022 18:54

Be-ing mentioned this pull request Mar 9, 2022

build in debug mode by default #109

Merged

Be-ing added 2 commits March 9, 2022 20:04

CxxQt.cmake: hack around cxx's build.rs failing on macOS

b9d24f9

fix QML tests with statically linked Qt

902ae2d

ahayzen-kdab requested changes Mar 10, 2022

View reviewed changes

Be-ing force-pushed the vcpkg branch from 7113d73 to 1b34db9 Compare March 10, 2022 14:24

ahayzen-kdab reviewed Mar 10, 2022

View reviewed changes

cmake/CompilerCaching.cmake Show resolved Hide resolved

Be-ing force-pushed the vcpkg branch 3 times, most recently from 0731fa6 to 0f28bcb Compare March 10, 2022 15:50

Be-ing force-pushed the vcpkg branch from 0f28bcb to c4e4521 Compare March 10, 2022 16:00

ahayzen-kdab reviewed Mar 10, 2022

View reviewed changes

.github/workflows/github-cxx-qt-tests.yml Show resolved Hide resolved

Be-ing force-pushed the vcpkg branch from c4e4521 to 537e941 Compare March 10, 2022 16:50

Be-ing added 3 commits March 10, 2022 10:51

build with vcpkg on Windows & macOS; add Qt6 builds to CI

60f8719

clang_format_check.sh: skip vcpkg submodule and vcpkg_installed

cc8bfe8

CMake: automatically use ccache or sccache if available

4217468

Be-ing force-pushed the vcpkg branch from 537e941 to 4217468 Compare March 10, 2022 16:51

Be-ing changed the title ~~use vcpkg for Windows & macOS builds; add Qt6 builds~~ use vcpkg for Windows & macOS builds; add Qt6 builds; compiler caching Mar 10, 2022

Be-ing changed the title ~~use vcpkg for Windows & macOS builds; add Qt6 builds; compiler caching~~ use vcpkg for Windows & macOS builds; add Qt6 builds; use sccache Mar 10, 2022

ahayzen-kdab approved these changes Mar 10, 2022

View reviewed changes

ahayzen-kdab merged commit d203f27 into KDAB:main Mar 10, 2022

Be-ing mentioned this pull request Mar 10, 2022

Add CI for using Windows with msvc #79

Closed

7 tasks

Be-ing deleted the vcpkg branch March 10, 2022 17:34

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

use vcpkg for Windows & macOS builds; add Qt6 builds; use sccache #107

use vcpkg for Windows & macOS builds; add Qt6 builds; use sccache #107

Be-ing commented Mar 8, 2022 •

edited

Loading

Be-ing commented Mar 8, 2022 •

edited

Loading

ahayzen-kdab commented Mar 8, 2022

Be-ing commented Mar 8, 2022

Be-ing commented Mar 8, 2022

Be-ing commented Mar 8, 2022

Be-ing commented Mar 9, 2022 •

edited

Loading

ahayzen-kdab commented Mar 9, 2022

Be-ing commented Mar 9, 2022

ahayzen-kdab left a comment

Be-ing commented Mar 10, 2022

Be-ing commented Mar 10, 2022

ahayzen-kdab commented Mar 10, 2022

ahayzen-kdab commented Mar 10, 2022

ahayzen-kdab commented Mar 10, 2022

Be-ing commented Mar 10, 2022 •

edited

Loading

ahayzen-kdab commented Mar 10, 2022

Be-ing commented Mar 10, 2022

ahayzen-kdab commented Mar 11, 2022

use vcpkg for Windows & macOS builds; add Qt6 builds; use sccache #107

use vcpkg for Windows & macOS builds; add Qt6 builds; use sccache #107

Conversation

Be-ing commented Mar 8, 2022 • edited Loading

Be-ing commented Mar 8, 2022 • edited Loading

ahayzen-kdab commented Mar 8, 2022

Be-ing commented Mar 8, 2022

Be-ing commented Mar 8, 2022

Be-ing commented Mar 8, 2022

Be-ing commented Mar 9, 2022 • edited Loading

ahayzen-kdab commented Mar 9, 2022

Be-ing commented Mar 9, 2022

ahayzen-kdab left a comment

Choose a reason for hiding this comment

Be-ing commented Mar 10, 2022

Be-ing commented Mar 10, 2022

ahayzen-kdab commented Mar 10, 2022

ahayzen-kdab commented Mar 10, 2022

ahayzen-kdab commented Mar 10, 2022

Be-ing commented Mar 10, 2022 • edited Loading

ahayzen-kdab commented Mar 10, 2022

Be-ing commented Mar 10, 2022

ahayzen-kdab commented Mar 11, 2022

Be-ing commented Mar 8, 2022 •

edited

Loading

Be-ing commented Mar 8, 2022 •

edited

Loading

Be-ing commented Mar 9, 2022 •

edited

Loading

Be-ing commented Mar 10, 2022 •

edited

Loading