Is there a way to depend on a subpackage of a monorepo via a git url?

[jclaveau]

Summary

To target subpackages of monorepos via github urls I had to write a little script run on prepare. Is my approach correct? Is there a better one?

Context

• I'm presently working on a project using Directus
• Some Directus packages are dependencies of my two apps (backend and frontend) and my domain package which is a directus extension
  • directus (the cli) for the backend
  • @directus/extension-sdk for the domain package
  • @directus/sdk for the frontend
  • @directus/errors
  • ...
• I patched Directus to enable code coverage for my backend code (the domain layer injected as an extension done with @directus/extension-sdk).
• My Directus dependencies is not up to date and i'd prefer to raise my code coverage to 80% before upgrading it.

My issue

• To use my patch I used github links like "@directus/errors": "jclaveau/directus#my-patch-branch"
• When I run pnpm i and try to build, I now have dependencies errors as the symlink in my node_modules point to the monorepo root instead of its subpackage
• As a workaround I wrote a little script that
  • look for dependencies using links (https, git, github) as version
  • check if the package name matches the name in the package.json
  • if it's not the case look for a subpackage having this name
  • if it finds one, updates the symlink to point to the right subfolder
  • When I run this script during the prepare step of my root package, it works \ :D / !
    • I tried to run this script from .pnpmfile.cjs during afterAllResolved() but it's too early

My questions

• Is there a better way to achieve this?
  • Did I miss a way to point to a monorepo's subpackage with links?
  • Did I miss another dev workflow?
  • Is my approach of finding the right subpackage location with the same version robust or having a caveat I didn't anticipate?
  • Is there a better hook than prepare in the dependencies lifecycle to run this script?
• If not, should I file a bug or a feature request (I didn't find my issue in the 1500 already opened ones)?

Other tracks

• I also tried with
  • https://gitpkg.vercel.app but it breaks workspace:* dependencies inside monorepos
  • Using git submodules but my cloud provider (Railway.app) do not support them so it was a mess
  • Using pnpm patch but it blocks the process of discussions / pull requests
• Also, using remote links has the advantage to be easily compatible with pnpm link to a local clone without requiring another layer of complexity to handle versions like Nx which I'd like to keep out of scope for now
• Looking for solutions I discovered the bit.dev project wich seems very interesting and I wonder how this case would be handled without package.json file

Ok, this seems enough for now, thanks in advance for your time and thank you very much for sharing your work!

[jclaveau]

Digging more I found a much better workaround:

• Even if I can't find it in the docs, we can target a specific path of a git dependency version: b13d2dc
• Using this doesn't work with monorepos as worspace:* deps are not handled but this can be handle in .pnpmfile.cjs

Here is the working result:
In a package.json

{
  "name": "...",
  "devDependencies": {
    "@directus/extensions-sdk": "github:jclaveau/directus#my_branch&path:packages/extensions-sdk",
  },
  "dependencies": {
    "@directus/errors": "github:jclaveau/directus#my_branch&path:packages/errors",
    "@directus/sdk": "github:jclaveau/directus#my_branch&path:packages/sdk",
    "//": "..."
  }
}

In .pnpmfile.cjs

const directusBranch = 'my_branch'

function readPackage(pkg, context) {

  if (pkg.name.match(/^@directus\//) || pkg.name.match(/^directus$/)) {
    // context.log(JSON.stringify(pkg, null, 2))
    for (const depEntry of ['dependencies', 'devDependencies', 'peerDependencies']) {
      if (!pkg[depEntry]) {
        continue
      }
      pkg[depEntry] = Object.fromEntries(
        Object.entries(pkg[depEntry])
        .map(([key, value]) => fixForkedDependencies(key, value, context))
      )
    }

    function fixForkedDependencies(depName, depVersion, context) {
      if (depName.match(/^@directus\//) && depVersion.match(/^workspace:\*/)) {
        const nameParts = depName.split('/')
        const newVersion = ['api', 'directus', 'app'].includes(nameParts[1]) // sdk will be moved here in a later directus version 
        ? 'github:jclaveau/directus#' + directusBranch + '&path:' + nameParts[1]
        : 'github:jclaveau/directus#' + directusBranch + '&path:packages/' + nameParts[1]
        context.log(`fixForkedDependencies of ${pkg.name}: ${depName}: ${depVersion} => ${newVersion}`)
        depVersion = newVersion
      }
      
      return [depName, depVersion];
    }
  }

  return pkg
}

module.exports = {
  hooks: {
    readPackage
  }
}

Pros:

• faster
• respects the regular pnpm flow
• the graph of depedencies is valid

Remaining issues:

• The same tarball is downloaded multiple times
• Updating the deps may require updating .pnpmfile.cjs
• The target branch requires the dist to be committed

[jclaveau]

I opened a bug to track the support for monorepos btw #8243