From e9be4a0835bfeaaccc9436138fb3a163725d3d73 Mon Sep 17 00:00:00 2001 From: Alexandre Zollinger Chohfi Date: Tue, 6 Oct 2026 14:07:23 -0700 Subject: [PATCH] parallel-checkout: default to four workers on Windows Git checks out files one at a time unless `checkout.workers` says otherwise. On Windows, replacing a file in the worktree (unlink, create, write, close) costs about 250 microseconds even on a Dev Drive (ReFS), and several times that on NTFS with real-time antivirus scanning. That cost overlaps well across processes, so parallel checkout makes a 2,000-file checkout 28-38% faster (see below). Since 373d334333 (entry: flush fscache after creating directories and writing files) and d7b8fabe04 (parallel-checkout: limit worker count to what poll() can wait on), parallel checkout works with the FSCache and with any worker count. Starting a worker is not free on Windows either, so the upstream threshold of 100 files is too low. With four workers and that threshold, switching between branches that differ in 100 files took 33ms longer than the 138ms of a sequential checkout. Default to four workers on Windows, and use them only for checkouts of at least 500 files. Both settings keep working as before; this only changes what Git does when they are not set. Switching between branches of a 40,000-file repository that differ in N files, Windows 11 on a 32-thread desktop with an NVMe drive. Medians; time saved is the median paired difference with its 95% bootstrap interval, over 12-15 interleaved runs: Dev Drive (ReFS) NTFS, Defender real-time N files before saved before saved 250 145ms (sequential) 529ms (sequential) 500 187ms 0ms [-36, 7] 671ms 123ms [95, 148] 1000 287ms 40ms [27, 66] 1024ms 282ms [219, 308] 2000 461ms 129ms [61, 180] 1668ms 633ms [603, 645] `git worktree add` of all 40,000 files went from 7.07s to 4.51s (2.56s saved [2.00, 3.13]) on the Dev Drive. Below the threshold the files are still written sequentially, and those checkouts were not slower (250 files: 10ms faster [4, 16] on the Dev Drive, 24ms faster [5, 63] on NTFS). Eight workers were no better than four at 2,000 files, and `checkout.workers=0` (one per logical CPU, 32 here) was no faster than sequential checkout. The defaults are compile-time macros, overridable per platform, in the same way as FSYNC_METHOD_DEFAULT and PROTECT_NTFS_DEFAULT, so other platforms are unchanged. A compiled default covers every Git for Windows distribution in one place. Shipping it as configuration, the way core.fscache is shipped, would mean adding it separately to the installer, the MSI, PortableGit and MinGit, and to any system gitconfig that a tool bundling Git writes for itself. Add a test that checks the default worker count on both sides of the threshold. Signed-off-by: Alexandre Zollinger Chohfi --- Documentation/config/checkout.adoc | 12 ++++---- compat/mingw.h | 9 ++++++ parallel-checkout.c | 12 ++++++-- t/t2080-parallel-checkout-basics.sh | 45 +++++++++++++++++++++++++++++ 4 files changed, 70 insertions(+), 8 deletions(-) diff --git a/Documentation/config/checkout.adoc b/Documentation/config/checkout.adoc index 45951bf38a5e3c..5b00e517434296 100644 --- a/Documentation/config/checkout.adoc +++ b/Documentation/config/checkout.adoc @@ -24,11 +24,11 @@ commands or functionality in the future. `checkout.workers`:: The number of parallel workers to use when updating the working tree. - The default is one, i.e. sequential execution. If set to a value less - than one, Git will use as many workers as the number of logical cores - available. This setting and `checkout.thresholdForParallelism` affect - all commands that perform checkout. E.g. checkout, clone, reset, - sparse-checkout, etc. + The default is one, i.e. sequential execution, except on Windows, + where it is four. If set to a value less than one, Git will use as + many workers as the number of logical cores available. This setting + and `checkout.thresholdForParallelism` affect all commands that + perform checkout. E.g. checkout, clone, reset, sparse-checkout, etc. + On Windows the number of workers is capped at 62, because the `poll()` emulation cannot wait on more worker pipes than that. A higher configured @@ -46,4 +46,4 @@ well the parallel version performs. of subprocess spawning and inter-process communication might outweigh the parallelization gains. This setting allows you to define the minimum number of files for which parallel checkout should be attempted. The - default is 100. + default is 100 (500 on Windows). diff --git a/compat/mingw.h b/compat/mingw.h index 379d7f5d079165..a83c9a53d72d5a 100644 --- a/compat/mingw.h +++ b/compat/mingw.h @@ -87,6 +87,15 @@ int is_valid_win32_path(const char *path, int allow_literal_nul); */ #define MAX_LONG_PATH 4096 +/* + * Replacing a file in the worktree (unlink, create, write, close) costs + * a few hundred microseconds on Windows, and parallel checkout overlaps + * that cost well. Starting a worker costs tens of milliseconds, though, + * so only use workers for checkouts that touch at least 500 files. + */ +#define PARALLEL_CHECKOUT_DEFAULT_WORKERS 4 +#define PARALLEL_CHECKOUT_DEFAULT_THRESHOLD 500 + /** * Handles paths that would exceed the MAX_PATH limit of Windows Unicode APIs. * diff --git a/parallel-checkout.c b/parallel-checkout.c index 4595cf4e8d9250..912e587c19f56d 100644 --- a/parallel-checkout.c +++ b/parallel-checkout.c @@ -38,8 +38,16 @@ enum pc_status parallel_checkout_status(void) return parallel_checkout.status; } -static const int DEFAULT_THRESHOLD_FOR_PARALLELISM = 100; -static const int DEFAULT_NUM_WORKERS = 1; +#ifndef PARALLEL_CHECKOUT_DEFAULT_THRESHOLD +#define PARALLEL_CHECKOUT_DEFAULT_THRESHOLD 100 +#endif + +#ifndef PARALLEL_CHECKOUT_DEFAULT_WORKERS +#define PARALLEL_CHECKOUT_DEFAULT_WORKERS 1 +#endif + +static const int DEFAULT_THRESHOLD_FOR_PARALLELISM = PARALLEL_CHECKOUT_DEFAULT_THRESHOLD; +static const int DEFAULT_NUM_WORKERS = PARALLEL_CHECKOUT_DEFAULT_WORKERS; void get_parallel_checkout_configs(int *num_workers, int *threshold) { diff --git a/t/t2080-parallel-checkout-basics.sh b/t/t2080-parallel-checkout-basics.sh index 51d4e1db4d9cbb..496c51163e889b 100755 --- a/t/t2080-parallel-checkout-basics.sh +++ b/t/t2080-parallel-checkout-basics.sh @@ -356,4 +356,49 @@ test_expect_success MINGW 'checkout caps workers at the poll limit' ' verify_checkout many-workers ' +# On Windows, parallel checkout is on by default (four workers), but only +# for checkouts of at least 500 files; elsewhere it is off by default. +test_expect_success 'default workers and threshold' ' + test_when_finished "rm -rf default-workers" && + git init default-workers && + ( + cd default-workers && + mkdir dir && + for i in $(test_seq 1 500) + do + echo "content $i" >dir/file$i || return 1 + done && + git add -A && + git commit -q -m base && + git tag base && + + git checkout -q -b few && + for i in $(test_seq 1 499) + do + echo "changed $i" >dir/file$i || return 1 + done && + git commit -q -a -m few && + + git checkout -q -b many base && + for i in $(test_seq 1 500) + do + echo "changed $i" >dir/file$i || return 1 + done && + git commit -q -a -m many && + git checkout -q base + ) && + + if test_have_prereq MINGW + then + expected_workers=4 + else + expected_workers=0 + fi && + test_checkout_workers 0 git -C default-workers checkout few && + verify_checkout default-workers && + git -C default-workers checkout -q base && + test_checkout_workers $expected_workers git -C default-workers checkout many && + verify_checkout default-workers +' + test_done