Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Tasks getting skipped with no explanation #9277

Open
1 task done
jakeleventhal opened this issue Oct 17, 2024 · 14 comments
Open
1 task done

Tasks getting skipped with no explanation #9277

jakeleventhal opened this issue Oct 17, 2024 · 14 comments
Labels
kind: bug Something isn't working

Comments

@jakeleventhal
Copy link
Contributor

jakeleventhal commented Oct 17, 2024

Verify canary release

  • I verified that the issue exists in the latest Turborepo canary release.

Link to code that reproduces this issue

See output

What package manager are you using / does the bug impact?

pnpm

What operating system are you using?

Mac

Which canary version will you have in your reproduction?

[email protected]

Describe the Bug

For some reason, I am getting tasks that seem to skip with no explanation. In a directory of my application apps/app1/api, I run turbo tsc and expect 15 total tasks to be ran. It seems that turbo randomly only does a subset of tasks. If I run turbo tsc several times in a row, I randomly will get correct output:

▶ turbo tsc -vv
2024-10-17T09:59:41.813-0400 [DEBUG] turborepo_lib::shim: Global turbo version: 2.1.3
2024-10-17T09:59:41.815-0400 [DEBUG] turborepo_lib::shim: Repository Root: /Users/jakeleventhal/Developer/rip-technologies
2024-10-17T09:59:41.815-0400 [DEBUG] turborepo_lib::shim::local_turbo_state: No local turbo binary found at: /Users/jakeleventhal/Developer/rip-technologies/node_modules/turbo-darwin-arm64/bin/turbo
2024-10-17T09:59:41.815-0400 [DEBUG] turborepo_lib::shim::local_turbo_state: No local turbo binary found at: /Users/jakeleventhal/Developer/rip-technologies/node_modules/turbo/node_modules/turbo-darwin-arm64/bin/turbo
2024-10-17T09:59:41.815-0400 [DEBUG] turborepo_lib::shim::local_turbo_state: Local turbo path: /Users/jakeleventhal/Developer/rip-technologies/node_modules/.pnpm/[email protected]/node_modules/turbo-darwin-arm64/bin/turbo
2024-10-17T09:59:41.815-0400 [DEBUG] turborepo_lib::shim::local_turbo_state: Local turbo version: 2.1.3
2024-10-17T09:59:41.815-0400 [DEBUG] turborepo_lib::shim: Running local turbo binary in /Users/jakeleventhal/Developer/rip-technologies/node_modules/.pnpm/[email protected]/node_modules/turbo-darwin-arm64/bin/turbo

2024-10-17T09:59:41.815-0400 [DEBUG] turborepo_lib::shim: supports_skip_infer_and_single_package true
2024-10-17T09:59:41.824-0400 [DEBUG] turborepo_lib::shim: Global turbo version: 2.1.3
turbo 2.1.3

2024-10-17T09:59:41.998-0400 [DEBUG] turborepo_lib::cli: pkg_inference_root set to "apps/app1/api"
2024-10-17T09:59:42.267-0400 [DEBUG] turborepo_lib::process: spawning children with pty: true
2024-10-17T09:59:42.267-0400 [DEBUG] turborepo_lib::daemon::connector: looking for pid in lockfile: AbsoluteSystemPathBuf("/var/folders/d0/lbtvrfsx7jqbqmr7dknv42jw0000gn/T/turbod/4066f6ab3656c5cb/turbod.pid")
2024-10-17T09:59:42.267-0400 [DEBUG] turborepo_lib::daemon::connector: found pid: 58100
2024-10-17T09:59:42.267-0400 [DEBUG] turborepo_lib::daemon::connector: got daemon with pid: 58100
2024-10-17T09:59:42.267-0400 [DEBUG] turborepo_lib::daemon::connector: creating AbsoluteSystemPath("/var/folders/d0/lbtvrfsx7jqbqmr7dknv42jw0000gn/T/turbod/4066f6ab3656c5cb")
2024-10-17T09:59:42.267-0400 [DEBUG] turborepo_lib::daemon::connector: watching AbsoluteSystemPath("/var/folders/d0/lbtvrfsx7jqbqmr7dknv42jw0000gn/T/turbod/4066f6ab3656c5cb")
2024-10-17T09:59:42.267-0400 [DEBUG] turborepo_lib::daemon::connector: creating AbsoluteSystemPath("/var/folders/d0/lbtvrfsx7jqbqmr7dknv42jw0000gn/T/turbod/4066f6ab3656c5cb")
2024-10-17T09:59:42.267-0400 [DEBUG] turborepo_lib::daemon::connector: watching AbsoluteSystemPath("/var/folders/d0/lbtvrfsx7jqbqmr7dknv42jw0000gn/T/turbod/4066f6ab3656c5cb")
2024-10-17T09:59:42.267-0400 [DEBUG] turborepo_lib::daemon::connector: connecting to socket: /var/folders/d0/lbtvrfsx7jqbqmr7dknv42jw0000gn/T/turbod/4066f6ab3656c5cb/turbod.sock
2024-10-17T09:59:42.267-0400 [DEBUG] tower::buffer::worker: "processing request"
2024-10-17T09:59:42.268-0400 [DEBUG] turborepo_lib::daemon::connector: connected in 757µs
2024-10-17T09:59:42.268-0400 [DEBUG] turborepo_lib::run::builder: running in daemon mode
2024-10-17T09:59:42.268-0400 [DEBUG] turborepo_repository::discovery: discovering packages using caching strategy
2024-10-17T09:59:42.268-0400 [DEBUG] turborepo_repository::discovery: discovering packages using primary strategy
2024-10-17T09:59:42.268-0400 [DEBUG] turborepo_repository::discovery: discovering packages using fallback strategy
2024-10-17T09:59:42.268-0400 [DEBUG] turborepo_repository::discovery: attempting primary strategy
2024-10-17T09:59:42.268-0400 [DEBUG] turborepo_lib::run::package_discovery: discovering packages using daemon
2024-10-17T09:59:42.268-0400 [DEBUG] tower::buffer::worker: "processing request"
2024-10-17T09:59:42.275-0400 [DEBUG] log: No cached session for DnsName(DnsName(DnsName("telemetry.vercel.com")))
2024-10-17T09:59:42.275-0400 [DEBUG] log: Not resuming any session
2024-10-17T09:59:42.280-0400 [DEBUG] turborepo_repository::discovery: primary strategy timed out, attempting fallback strategy
2024-10-17T09:59:42.280-0400 [DEBUG] turborepo_repository::discovery: discovering packages using local strategy
2024-10-17T09:59:42.281-0400 [DEBUG] globwalk: processing includes: ["apps/*/package.json", "apps/*/*/package.json", "apps-scripts/*/package.json", "chrome-extensions/*/package.json", "packages/*/*/package.json", "switch/*/package.json", "tools/*/package.json"]
2024-10-17T09:59:42.281-0400 [DEBUG] globwalk: processing excludes: ["**/node_modules/**", "**/bower_components/**"]
2024-10-17T09:59:42.281-0400 [DEBUG] globwalk: processed includes: ["/Users/jakeleventhal/Developer/rip-technologies/apps/*/package.json", "/Users/jakeleventhal/Developer/rip-technologies/apps/*/*/package.json", "/Users/jakeleventhal/Developer/rip-technologies/apps-scripts/*/package.json", "/Users/jakeleventhal/Developer/rip-technologies/chrome-extensions/*/package.json", "/Users/jakeleventhal/Developer/rip-technologies/packages/*/*/package.json", "/Users/jakeleventhal/Developer/rip-technologies/switch/*/package.json", "/Users/jakeleventhal/Developer/rip-technologies/tools/*/package.json"]
2024-10-17T09:59:42.281-0400 [DEBUG] globwalk: processed excludes: ["/Users/jakeleventhal/Developer/rip-technologies/**/node_modules/**", "/Users/jakeleventhal/Developer/rip-technologies/**/bower_components/**"]
2024-10-17T09:59:42.290-0400 [DEBUG] turborepo_repository::discovery: discovering packages using caching strategy
2024-10-17T09:59:42.290-0400 [DEBUG] turborepo_repository::discovery: discovering packages using caching strategy
2024-10-17T09:59:42.336-0400 [DEBUG] log: Using ciphersuite TLS13_AES_128_GCM_SHA256
2024-10-17T09:59:42.336-0400 [DEBUG] log: Not resuming
2024-10-17T09:59:42.336-0400 [DEBUG] log: TLS1.3 encrypted extensions: [Protocols([6832])]
2024-10-17T09:59:42.336-0400 [DEBUG] log: ALPN protocol is Some(b"h2")
2024-10-17T09:59:42.336-0400 [DEBUG] log: Ticket saved
2024-10-17T09:59:42.390-0400 [DEBUG] turborepo_repository::discovery: discovering packages using caching strategy
2024-10-17T09:59:42.390-0400 [DEBUG] turborepo_lib::run::task_access: TASK ACCESS TRACE: unable to restore config from cache
2024-10-17T09:59:42.391-0400 [DEBUG] turborepo_lib::run::scope::filter: Using apps/app1/api as a basis for selecting packages
• Packages in scope: @app1/api
• Running tsc in 1 packages
• Remote caching enabled
2024-10-17T09:59:42.432-0400 [DEBUG] globwalk: processing includes: [".nvmrc", "**/package.json", "pnpm-lock.yaml", "pnpm-workspace.yaml", "package.json", "turbo.json"]
2024-10-17T09:59:42.432-0400 [DEBUG] globwalk: processing excludes: ["**/.next/**", "**/dist/**", "**/out/**", "**/node_modules/**"]
2024-10-17T09:59:42.432-0400 [DEBUG] globwalk: processed includes: ["/Users/jakeleventhal/Developer/rip-technologies/.nvmrc", "/Users/jakeleventhal/Developer/rip-technologies/**/package.json", "/Users/jakeleventhal/Developer/rip-technologies/pnpm-lock.yaml", "/Users/jakeleventhal/Developer/rip-technologies/pnpm-workspace.yaml", "/Users/jakeleventhal/Developer/rip-technologies/package.json", "/Users/jakeleventhal/Developer/rip-technologies/turbo.json"]
2024-10-17T09:59:42.432-0400 [DEBUG] globwalk: processed excludes: ["/Users/jakeleventhal/Developer/rip-technologies/**/.next/**", "/Users/jakeleventhal/Developer/rip-technologies/**/dist/**", "/Users/jakeleventhal/Developer/rip-technologies/**/out/**", "/Users/jakeleventhal/Developer/rip-technologies/**/node_modules/**"]
2024-10-17T09:59:42.630-0400 [DEBUG] turborepo_lib::run::global_hash: global hash env vars []
2024-10-17T09:59:42.630-0400 [DEBUG] turborepo_lib::run::global_hash: external deps hash: 6bc2b0625ff27546
2024-10-17T09:59:42.642-0400 [DEBUG] turborepo_lib::run: running visitor
2024-10-17T09:59:42.643-0400 [DEBUG] turborepo_lib::task_graph::visitor: task //#package-deps hash is 674ca7f7bca69102
2024-10-17T09:59:42.643-0400 [DEBUG] tower::buffer::worker: "processing request"
//:package-deps: cache hit (outputs already on disk), suppressing logs 674ca7f7bca69102
2024-10-17T09:59:42.655-0400 [DEBUG] turborepo_lib::task_graph::visitor: task @riptech/tsconfig#tsc hash is 113440cfa794bcb4
2024-10-17T09:59:42.655-0400 [DEBUG] turborepo_lib::task_graph::visitor: task eslint-config-riptech#tsc hash is e6374f582c57bd54
2024-10-17T09:59:42.656-0400 [DEBUG] tower::buffer::worker: "processing request"
eslint-config-riptech:tsc: cache hit (outputs already on disk), suppressing logs e6374f582c57bd54
2024-10-17T09:59:42.668-0400 [DEBUG] turborepo_lib::task_graph::visitor: task @riptech/jest-config#tsc hash is 73e56b6127332706
2024-10-17T09:59:42.668-0400 [DEBUG] turborepo_lib::task_graph::visitor: task @riptech/logger#tsc hash is 180a106959d444a8
2024-10-17T09:59:42.669-0400 [DEBUG] tower::buffer::worker: "processing request"
2024-10-17T09:59:42.669-0400 [DEBUG] turborepo_lib::task_graph::visitor: task @riptech/aws#tsc hash is 4a3203efa1e4f0ce
2024-10-17T09:59:42.669-0400 [DEBUG] tower::buffer::worker: "processing request"
2024-10-17T09:59:42.669-0400 [DEBUG] turborepo_lib::task_graph::visitor: task @riptech/emailer#tsc hash is ee99aff6e0859102
2024-10-17T09:59:42.670-0400 [DEBUG] tower::buffer::worker: "processing request"
2024-10-17T09:59:42.670-0400 [DEBUG] turborepo_lib::task_graph::visitor: task @riptech/docker#tsc hash is 768ff26fbd46d6c8
2024-10-17T09:59:42.670-0400 [DEBUG] tower::buffer::worker: "processing request"
@riptech/docker:tsc: cache hit (outputs already on disk), suppressing logs 768ff26fbd46d6c8
@riptech/emailer:tsc: cache hit (outputs already on disk), suppressing logs ee99aff6e0859102
@riptech/logger:tsc: cache hit (outputs already on disk), suppressing logs 180a106959d444a8
@riptech/aws:tsc: cache hit (outputs already on disk), suppressing logs 4a3203efa1e4f0ce
2024-10-17T09:59:42.681-0400 [DEBUG] turborepo_lib::task_hash: auto detected framework for @riptech/utils
2024-10-17T09:59:42.681-0400 [DEBUG] turborepo_lib::task_hash: framework: nextjs, env_prefix: ["NEXT_PUBLIC_*"]
2024-10-17T09:59:42.682-0400 [DEBUG] turborepo_lib::task_graph::visitor: task @riptech/utils#tsc hash is fd8edfcc937f0bac
2024-10-17T09:59:42.682-0400 [DEBUG] tower::buffer::worker: "processing request"
@riptech/utils:tsc: cache hit (outputs already on disk), suppressing logs fd8edfcc937f0bac
2024-10-17T09:59:42.694-0400 [DEBUG] turborepo_lib::task_graph::visitor: task @riptech/test-utils#tsc hash is b6631d63446de342
2024-10-17T09:59:42.694-0400 [DEBUG] tower::buffer::worker: "processing request"
2024-10-17T09:59:42.694-0400 [DEBUG] turborepo_lib::task_graph::visitor: task @riptech/stripe#tsc hash is 735e468ce0e5f1a9
2024-10-17T09:59:42.695-0400 [DEBUG] tower::buffer::worker: "processing request"
@riptech/stripe:tsc: cache hit (outputs already on disk), suppressing logs 735e468ce0e5f1a9
@riptech/test-utils:tsc: cache hit (outputs already on disk), suppressing logs b6631d63446de342
2024-10-17T09:59:42.706-0400 [DEBUG] turborepo_lib::task_graph::visitor: task @app1/database#tsc hash is 102ba2a5eaa84660
2024-10-17T09:59:42.706-0400 [DEBUG] tower::buffer::worker: "processing request"
@app1/database:tsc: cache hit (outputs already on disk), suppressing logs 102ba2a5eaa84660
2024-10-17T09:59:42.718-0400 [DEBUG] turborepo_lib::task_graph::visitor: task @app1/types#tsc hash is 509f1a38a879e3a9
2024-10-17T09:59:42.719-0400 [DEBUG] tower::buffer::worker: "processing request"
@app1/types:tsc: cache hit (outputs already on disk), suppressing logs 509f1a38a879e3a9
2024-10-17T09:59:42.733-0400 [DEBUG] turborepo_lib::task_graph::visitor: task @app1/utils#tsc hash is 9d2a96573b84dbcd
2024-10-17T09:59:42.734-0400 [DEBUG] tower::buffer::worker: "processing request"
@app1/utils:tsc: cache hit (outputs already on disk), suppressing logs 9d2a96573b84dbcd
2024-10-17T09:59:42.746-0400 [DEBUG] turborepo_lib::task_graph::visitor: task @app1/product-catalog#tsc hash is 883170081095bc85
2024-10-17T09:59:42.746-0400 [DEBUG] tower::buffer::worker: "processing request"
@app1/product-catalog:tsc: cache hit (outputs already on disk), suppressing logs 883170081095bc85
2024-10-17T09:59:42.758-0400 [DEBUG] turborepo_lib::task_graph::visitor: task @app1/integrations#tsc hash is 9f0beb72c82e2692
2024-10-17T09:59:42.759-0400 [DEBUG] tower::buffer::worker: "processing request"
@app1/integrations:tsc: cache hit (outputs already on disk), suppressing logs 9f0beb72c82e2692
2024-10-17T09:59:42.771-0400 [DEBUG] turborepo_lib::task_hash: auto detected framework for @app1/api
2024-10-17T09:59:42.771-0400 [DEBUG] turborepo_lib::task_hash: framework: vite, env_prefix: ["VITE_*"]
2024-10-17T09:59:42.771-0400 [DEBUG] turborepo_lib::task_graph::visitor: task @app1/api#tsc hash is c20d43eaf7c63672
2024-10-17T09:59:42.771-0400 [DEBUG] tower::buffer::worker: "processing request"
@app1/api:tsc: cache hit (outputs already on disk), suppressing logs c20d43eaf7c63672

 Tasks:    15 successful, 15 total
Cached:    15 cached, 15 total
  Time:    516ms >>> FULL TURBO

2024-10-17T09:59:42.790-0400 [DEBUG] turborepo_lib::process: waiting for 0 processes to exit
2024-10-17T09:59:42.876-0400 [DEBUG] turborepo_telemetry: telemetry handle closed
2024-10-17T09:59:42.876-0400 [DEBUG] log: Sending warning alert CloseNotify

and randomly get incorrect output that does not download cached outputs:

▶ turbo tsc -vv
2024-10-17T09:59:43.324-0400 [DEBUG] turborepo_lib::shim: Global turbo version: 2.1.3
2024-10-17T09:59:43.326-0400 [DEBUG] turborepo_lib::shim: Repository Root: /Users/jakeleventhal/Developer/rip-technologies
2024-10-17T09:59:43.326-0400 [DEBUG] turborepo_lib::shim::local_turbo_state: No local turbo binary found at: /Users/jakeleventhal/Developer/rip-technologies/node_modules/turbo-darwin-arm64/bin/turbo
2024-10-17T09:59:43.326-0400 [DEBUG] turborepo_lib::shim::local_turbo_state: No local turbo binary found at: /Users/jakeleventhal/Developer/rip-technologies/node_modules/turbo/node_modules/turbo-darwin-arm64/bin/turbo
2024-10-17T09:59:43.326-0400 [DEBUG] turborepo_lib::shim::local_turbo_state: Local turbo path: /Users/jakeleventhal/Developer/rip-technologies/node_modules/.pnpm/[email protected]/node_modules/turbo-darwin-arm64/bin/turbo
2024-10-17T09:59:43.326-0400 [DEBUG] turborepo_lib::shim::local_turbo_state: Local turbo version: 2.1.3
2024-10-17T09:59:43.327-0400 [DEBUG] turborepo_lib::shim: Running local turbo binary in /Users/jakeleventhal/Developer/rip-technologies/node_modules/.pnpm/[email protected]/node_modules/turbo-darwin-arm64/bin/turbo

2024-10-17T09:59:43.327-0400 [DEBUG] turborepo_lib::shim: supports_skip_infer_and_single_package true
2024-10-17T09:59:43.333-0400 [DEBUG] turborepo_lib::shim: Global turbo version: 2.1.3
turbo 2.1.3

2024-10-17T09:59:43.483-0400 [DEBUG] turborepo_lib::cli: pkg_inference_root set to "apps/app1/api"
2024-10-17T09:59:43.752-0400 [DEBUG] turborepo_lib::process: spawning children with pty: true
2024-10-17T09:59:43.752-0400 [DEBUG] turborepo_lib::daemon::connector: looking for pid in lockfile: AbsoluteSystemPathBuf("/var/folders/d0/lbtvrfsx7jqbqmr7dknv42jw0000gn/T/turbod/4066f6ab3656c5cb/turbod.pid")
2024-10-17T09:59:43.752-0400 [DEBUG] turborepo_lib::daemon::connector: found pid: 58100
2024-10-17T09:59:43.752-0400 [DEBUG] turborepo_lib::daemon::connector: got daemon with pid: 58100
2024-10-17T09:59:43.752-0400 [DEBUG] turborepo_lib::daemon::connector: creating AbsoluteSystemPath("/var/folders/d0/lbtvrfsx7jqbqmr7dknv42jw0000gn/T/turbod/4066f6ab3656c5cb")
2024-10-17T09:59:43.752-0400 [DEBUG] turborepo_lib::daemon::connector: watching AbsoluteSystemPath("/var/folders/d0/lbtvrfsx7jqbqmr7dknv42jw0000gn/T/turbod/4066f6ab3656c5cb")
2024-10-17T09:59:43.752-0400 [DEBUG] turborepo_lib::daemon::connector: creating AbsoluteSystemPath("/var/folders/d0/lbtvrfsx7jqbqmr7dknv42jw0000gn/T/turbod/4066f6ab3656c5cb")
2024-10-17T09:59:43.752-0400 [DEBUG] turborepo_lib::daemon::connector: watching AbsoluteSystemPath("/var/folders/d0/lbtvrfsx7jqbqmr7dknv42jw0000gn/T/turbod/4066f6ab3656c5cb")
2024-10-17T09:59:43.752-0400 [DEBUG] turborepo_lib::daemon::connector: connecting to socket: /var/folders/d0/lbtvrfsx7jqbqmr7dknv42jw0000gn/T/turbod/4066f6ab3656c5cb/turbod.sock
2024-10-17T09:59:43.752-0400 [DEBUG] tower::buffer::worker: "processing request"
2024-10-17T09:59:43.753-0400 [DEBUG] turborepo_lib::daemon::connector: connected in 736µs
2024-10-17T09:59:43.753-0400 [DEBUG] turborepo_lib::run::builder: running in daemon mode
2024-10-17T09:59:43.753-0400 [DEBUG] turborepo_repository::discovery: discovering packages using caching strategy
2024-10-17T09:59:43.753-0400 [DEBUG] turborepo_repository::discovery: discovering packages using primary strategy
2024-10-17T09:59:43.753-0400 [DEBUG] turborepo_repository::discovery: discovering packages using fallback strategy
2024-10-17T09:59:43.753-0400 [DEBUG] turborepo_repository::discovery: attempting primary strategy
2024-10-17T09:59:43.753-0400 [DEBUG] turborepo_lib::run::package_discovery: discovering packages using daemon
2024-10-17T09:59:43.753-0400 [DEBUG] tower::buffer::worker: "processing request"
2024-10-17T09:59:43.761-0400 [DEBUG] log: No cached session for DnsName(DnsName(DnsName("telemetry.vercel.com")))
2024-10-17T09:59:43.761-0400 [DEBUG] log: Not resuming any session
2024-10-17T09:59:43.764-0400 [DEBUG] turborepo_repository::discovery: primary strategy timed out, attempting fallback strategy
2024-10-17T09:59:43.764-0400 [DEBUG] turborepo_repository::discovery: discovering packages using local strategy
2024-10-17T09:59:43.766-0400 [DEBUG] globwalk: processing includes: ["apps/*/package.json", "apps/*/*/package.json", "apps-scripts/*/package.json", "chrome-extensions/*/package.json", "packages/*/*/package.json", "switch/*/package.json", "tools/*/package.json"]
2024-10-17T09:59:43.766-0400 [DEBUG] globwalk: processing excludes: ["**/node_modules/**", "**/bower_components/**"]
2024-10-17T09:59:43.766-0400 [DEBUG] globwalk: processed includes: ["/Users/jakeleventhal/Developer/rip-technologies/apps/*/package.json", "/Users/jakeleventhal/Developer/rip-technologies/apps/*/*/package.json", "/Users/jakeleventhal/Developer/rip-technologies/apps-scripts/*/package.json", "/Users/jakeleventhal/Developer/rip-technologies/chrome-extensions/*/package.json", "/Users/jakeleventhal/Developer/rip-technologies/packages/*/*/package.json", "/Users/jakeleventhal/Developer/rip-technologies/switch/*/package.json", "/Users/jakeleventhal/Developer/rip-technologies/tools/*/package.json"]
2024-10-17T09:59:43.766-0400 [DEBUG] globwalk: processed excludes: ["/Users/jakeleventhal/Developer/rip-technologies/**/node_modules/**", "/Users/jakeleventhal/Developer/rip-technologies/**/bower_components/**"]
2024-10-17T09:59:43.776-0400 [DEBUG] turborepo_repository::discovery: discovering packages using caching strategy
2024-10-17T09:59:43.776-0400 [DEBUG] turborepo_repository::discovery: discovering packages using caching strategy
2024-10-17T09:59:43.820-0400 [DEBUG] log: Using ciphersuite TLS13_AES_128_GCM_SHA256
2024-10-17T09:59:43.820-0400 [DEBUG] log: Not resuming
2024-10-17T09:59:43.820-0400 [DEBUG] log: TLS1.3 encrypted extensions: [Protocols([6832])]
2024-10-17T09:59:43.820-0400 [DEBUG] log: ALPN protocol is Some(b"h2")
2024-10-17T09:59:43.821-0400 [DEBUG] log: Ticket saved
2024-10-17T09:59:43.874-0400 [DEBUG] turborepo_repository::discovery: discovering packages using caching strategy
2024-10-17T09:59:43.874-0400 [DEBUG] turborepo_lib::run::task_access: TASK ACCESS TRACE: unable to restore config from cache
2024-10-17T09:59:43.874-0400 [DEBUG] turborepo_lib::run::scope::filter: Using apps/app1/api as a basis for selecting packages
• Packages in scope: @riptech/app1
• Running tsc in 1 packages
• Remote caching enabled
2024-10-17T09:59:43.875-0400 [DEBUG] globwalk: processing includes: [".nvmrc", "**/package.json", "pnpm-lock.yaml", "pnpm-workspace.yaml", "package.json", "turbo.json"]
2024-10-17T09:59:43.875-0400 [DEBUG] globwalk: processing excludes: ["**/.next/**", "**/dist/**", "**/out/**", "**/node_modules/**"]
2024-10-17T09:59:43.875-0400 [DEBUG] globwalk: processed includes: ["/Users/jakeleventhal/Developer/rip-technologies/.nvmrc", "/Users/jakeleventhal/Developer/rip-technologies/**/package.json", "/Users/jakeleventhal/Developer/rip-technologies/pnpm-lock.yaml", "/Users/jakeleventhal/Developer/rip-technologies/pnpm-workspace.yaml", "/Users/jakeleventhal/Developer/rip-technologies/package.json", "/Users/jakeleventhal/Developer/rip-technologies/turbo.json"]
2024-10-17T09:59:43.875-0400 [DEBUG] globwalk: processed excludes: ["/Users/jakeleventhal/Developer/rip-technologies/**/.next/**", "/Users/jakeleventhal/Developer/rip-technologies/**/dist/**", "/Users/jakeleventhal/Developer/rip-technologies/**/out/**", "/Users/jakeleventhal/Developer/rip-technologies/**/node_modules/**"]
2024-10-17T09:59:44.073-0400 [DEBUG] turborepo_lib::run::global_hash: global hash env vars []
2024-10-17T09:59:44.073-0400 [DEBUG] turborepo_lib::run::global_hash: external deps hash: 6bc2b0625ff27546
2024-10-17T09:59:44.085-0400 [DEBUG] turborepo_lib::run: running visitor
2024-10-17T09:59:44.086-0400 [DEBUG] turborepo_lib::task_graph::visitor: task //#package-deps hash is 674ca7f7bca69102
2024-10-17T09:59:44.086-0400 [DEBUG] tower::buffer::worker: "processing request"
//:package-deps: cache hit (outputs already on disk), suppressing logs 674ca7f7bca69102
2024-10-17T09:59:44.098-0400 [DEBUG] turborepo_lib::task_graph::visitor: task @riptech/app1#tsc hash is 51a62435b5335326

 Tasks:    1 successful, 1 total
Cached:    1 cached, 1 total
  Time:    347ms >>> FULL TURBO

2024-10-17T09:59:44.104-0400 [DEBUG] turborepo_lib::process: waiting for 0 processes to exit
2024-10-17T09:59:44.222-0400 [DEBUG] turborepo_telemetry: telemetry handle closed
2024-10-17T09:59:44.222-0400 [DEBUG] log: Sending warning alert CloseNotify

Notably, see how it says:

2024-10-17T09:59:44.098-0400 [DEBUG] turborepo_lib::task_graph::visitor: task @riptech/app1#tsc hash is 51a62435b5335326

I don't even have a tsc task for @riptech/app1

Expected Behavior

In all cases, I would expect all 15 tasks to run (cached).

To Reproduce

See above

Additional context

Here is my turbo.json

{
  "$schema": "https://turbo.build/schema.json",
  "tasks": {
    "//#coverage": {
      "dependsOn": ["//#package-deps", "@riptech/lcov-merger#tsc"],
      "inputs": ["!**/node_modules/**", "**/coverage/lcov.info"],
      "outputLogs": "new-only",
      "outputs": ["coverage/_coverage_report.dat"]
    },
    "//#package-deps": {
      "inputs": [
        "!**/.next/**",
        "!**/dist/**",
        "!**/out/**",
        "!**/node_modules/**",
        ".nvmrc",
        "**/package.json",
        "pnpm-lock.yaml",
        "pnpm-workspace.yaml"
      ],
      "outputLogs": "new-only",
      "outputs": ["pnpm-lock.yaml"]
    },
    "format": {
      "dependsOn": ["//#package-deps", "eslint-config-riptech#tsc"],
      "outputLogs": "new-only"
    },
    "image": {
      "dependsOn": ["//#package-deps", "^tsc", "@riptech/docker#tsc"],
      "inputs": [".env*", "dist/**"],
      "outputLogs": "new-only"
    },
    "image-production": {
      "cache": false,
      "dependsOn": ["//#package-deps", "^tsc", "@riptech/docker#tsc"]
    },
    "image-staging": {
      "cache": false,
      "dependsOn": ["//#package-deps", "^tsc", "@riptech/docker#tsc"]
    },
    "lint": {
      "dependsOn": ["//#package-deps", "eslint-config-riptech#tsc"],
      "outputLogs": "new-only"
    },
    "pull-vercel": {
      "cache": false
    },
    "test": {
      "dependsOn": ["//#package-deps", "@riptech/jest-config#tsc"],
      "outputLogs": "new-only",
      "outputs": ["coverage/lcov.info"]
    },
    "tsc": {
      "dependsOn": ["//#package-deps", "^tsc"],
      "outputLogs": "new-only",
      "outputs": ["dist/**"]
    }
  }
}
@jakeleventhal jakeleventhal added kind: bug Something isn't working needs: triage New issues get this label. Remove it after triage labels Oct 17, 2024
@NicholasLYang
Copy link
Contributor

Hi, thanks for the issue. What directory is @riptech/app1 in? And do you have a nested workspaces, i.e. packages that contain other packages?

@jakeleventhal
Copy link
Contributor Author

jakeleventhal commented Oct 17, 2024

▶ ls -1 **/package.json
apps/app1/admin/package.json
apps/app1/api/package.json
apps/app1/client/package.json
apps/app1/landing/package.json
apps/app1/package.json
apps/app1/sls-api/package.json
apps/app2/api/package.json
apps/app2/client/package.json
apps/app2/landing/package.json
apps/app2/package.json
apps/app3/client/package.json
apps/app3/package.json
apps/app4/client/package.json
apps/app4/landing/package.json
apps/app4/package.json
package.json
packages/app1/database/package.json
packages/app1/integrations/package.json
packages/app1/process-image/package.json
packages/app1/product-catalog/package.json
packages/app1/shared-ui/package.json
packages/app1/types/package.json
packages/app1/utils/package.json
packages/app2/database/package.json
packages/app2/integrations/package.json
packages/app2/shared-ui/package.json
packages/app2/types/package.json
packages/app2/utils/package.json
packages/app3/database/package.json
packages/app3/integrations/package.json
packages/app3/types/package.json
packages/app4/database/package.json
packages/app4/shared-ui/package.json
packages/app4/types/package.json
packages/app4/utils/package.json
packages/riptech/apps-script-utils/package.json
packages/riptech/aws/package.json
packages/riptech/emailer/package.json
packages/riptech/google/package.json
packages/riptech/logger/package.json
packages/riptech/minimal-ui/package.json
packages/riptech/shared-ui/package.json
packages/riptech/stripe/package.json
packages/riptech/test-utils/package.json
packages/riptech/utils/package.json
tools/bulk-product-create/package.json
tools/docker/package.json
tools/eslint/package.json
tools/jest-config/package.json
tools/lcov-merger/package.json
tools/tsconfig/package.json

Here is @riptech/app1 package.json:

{
  "name": "@riptech/app1",
  "private": true,
  "scripts": {
    "admin": "pnpm --dir=./admin run start",
    "api": "pnpm --dir=./api run start",
    "client": "pnpm --dir=./client run start",
    "landing": "pnpm --dir=./landing run start",
    "onboard-demo": "rm -rf $(pwd)/localDBData/demoData && pnpm --dir=./api run onboard-demo",
    "start": "concurrently --raw \"pnpm run start-database\" \"pnpm run client\"",
    "start-admin": "concurrently --raw \"pnpm run start-database\" \"pnpm run admin\"",
    "start-database": "docker-compose up --build",
    "stop": "docker-compose down && killall -9 node"
  }
}

@NicholasLYang
Copy link
Contributor

Thanks, just to confirm, @riptech/app1 is in which directory? I'm not exactly sure from seeing those directories which one it is.

Judging by your layout, this is likely an issue with nested workspaces. Turborepo doesn't support nested workspaces, i.e. having apps/app1 as a package but also apps/app1/admin as a package. In that case, we recommend either consolidating app1 into a single package, or moving apps/app1/admin into a separate, not nested package like apps/app1-admin

@jakeleventhal
Copy link
Contributor Author

jakeleventhal commented Oct 17, 2024

The directory is apps/app1

I don't mean for @riptech/app1 to even be a recognized "package". I just have a package.json there for the sake of running scripts like "start" that can run several apps locally at once. My pnpm-workspace.yaml file defines my packages like so:

packages:
  - 'apps/*'
  - 'apps/*/*'
  - 'packages/*/*'
  - 'tools/*'

If I remove apps/* it actually works just fine, but then the problem I have is that if I run turbo tsc from the apps/app1 directory, I would expect to run tsc for all packages downstream (in this case apps/app1/admin, apps/app1/api, apps/app1/client, apps/app1/landing, and apps/app1/sls-api). Instead, if I remove apps/* from the yaml file and run turbo tsc from apps/app1 I get this output:

rip-technologies/apps/artelo  art-10567-implement-webhook-execution ✗                                                                                                                                     5m ⚑  ⍉
▶ npx turbo tsc
 WARNING  No locally installed `turbo` found. Using version: 2.1.3.
turbo 2.1.3

  × missing packageManager field in package.json

even though packageManager is defined in the repo root package.json

@NicholasLYang
Copy link
Contributor

Interesting. The behavior you described of running all packages downstream is what should happen. I'll look into that. And possibly improve the error message to show which package json it's trying to pull from

@NicholasLYang
Copy link
Contributor

Is this repository public? Or is it possible to get access to a smaller version of it without any code?

@jakeleventhal
Copy link
Contributor Author

My repo is private, and very large. If you're willing, I can invite you to the repo temporarily.

@NicholasLYang
Copy link
Contributor

That would be great, thanks!

@jakeleventhal
Copy link
Contributor Author

@NicholasLYang added you as a collaborator. We can continue discussion in this thread.

@NicholasLYang
Copy link
Contributor

Hi @jakeleventhal, thanks for giving me access! I tried running npx turbo tsc in the apps/app1 directory, and I got the correct output, i.e. it builds all the packages that are in that directory. Can you check if the TURBO_BINARY_PATH env variable is set in your environment? And what OS are you using?

@jakeleventhal
Copy link
Contributor Author

jakeleventhal commented Oct 21, 2024

The bug I mentioned is when you remove 'apps/*' from the pnpm-workspace.yaml
am using macOS

@NicholasLYang
Copy link
Contributor

Gotcha. If you delete the package.json in apps/app1, then it should work. Our workspace detection algorithm seems to think that is the workspace root.

@jakeleventhal
Copy link
Contributor Author

Right, thats the problem though, I use that package.json as a hub for my project for running npm scripts. I feel like turbo should be able to have ignore paths that I can just specify somehow

@anthonyshew anthonyshew removed the needs: triage New issues get this label. Remove it after triage label Oct 31, 2024
@jakeleventhal
Copy link
Contributor Author

@anthonyshew @NicholasLYang is this going to be implemented as a feature?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind: bug Something isn't working
Projects
None yet
Development

No branches or pull requests

3 participants