Skip to content

Preserve structured app validation issues internally#7066

Open
dmerand wants to merge 1 commit intomainfrom
dlm-app-validate-errors
Open

Preserve structured app validation issues internally#7066
dmerand wants to merge 1 commit intomainfrom
dlm-app-validate-errors

Conversation

@dmerand
Copy link
Contributor

@dmerand dmerand commented Mar 20, 2026

What

Preserve structured app validation issues internally during app loading.

This keeps the current CLI behavior the same while retaining structured issue metadata alongside rendered validation messages.

Why

app config validate --json needs file-oriented issue data, but the validation pipeline still flattens schema issues into strings too early.

To support repair loops and future automation work, the CLI needs to retain structured validation details before deciding how to render them publicly.

How

  • add structured issue parsing alongside the existing human-readable formatter
  • preserve structured issue metadata at the loader/report boundary
  • update AppErrors to retain both rendered messages and structured issues
  • introduce LocalConfigError as a shared lower-layer local configuration error
  • keep current text output and existing public behavior unchanged

Copy link
Contributor Author

dmerand commented Mar 20, 2026

@github-actions
Copy link
Contributor

github-actions bot commented Mar 20, 2026

Coverage report

St.
Category Percentage Covered / Total
🟢 Statements 82.3% 15132/18386
🟡 Branches 74.86% 7470/9979
🟢 Functions 81.34% 3806/4679
🟢 Lines 82.69% 14304/17299

Test suite run success

4001 tests passing in 1529 suites.

Report generated by 🧪jest coverage report action from a7602f8

@dmerand dmerand force-pushed the dlm-app-validate-json branch from 59c4766 to efd8c3d Compare March 20, 2026 18:24
@dmerand dmerand force-pushed the dlm-app-validate-errors branch 2 times, most recently from b6ee50f to 113bc0e Compare March 20, 2026 18:31
@dmerand dmerand force-pushed the dlm-app-validate-json branch from efd8c3d to a078db7 Compare March 20, 2026 19:46
@dmerand dmerand force-pushed the dlm-app-validate-errors branch from 113bc0e to 3c313f8 Compare March 20, 2026 19:46
@dmerand dmerand changed the base branch from dlm-app-validate-json to graphite-base/7066 March 23, 2026 21:30
@dmerand dmerand force-pushed the dlm-app-validate-errors branch from 3c313f8 to 0158180 Compare March 23, 2026 23:17
@dmerand dmerand force-pushed the graphite-base/7066 branch from a078db7 to d22a6e8 Compare March 23, 2026 23:17
@dmerand dmerand changed the base branch from graphite-base/7066 to dlm-app-validate-json March 23, 2026 23:17
@dmerand dmerand force-pushed the dlm-app-validate-errors branch from 0158180 to 9d58583 Compare March 24, 2026 15:22
@github-actions
Copy link
Contributor

Differences in type declarations

We detected differences in the type declarations generated by Typescript for this branch compared to the baseline ('main' branch). Please, review them to ensure they are backward-compatible. Here are some important things to keep in mind:

  • Some seemingly private modules might be re-exported through public modules.
  • If the branch is behind main you might see odd diffs, rebase main into this branch.

New type declarations

We found no new type declarations in this PR

Existing type declarations

packages/cli-kit/dist/private/node/constants.d.ts
@@ -8,7 +8,6 @@ export declare const environmentVariables: {
     env: string;
     firstPartyDev: string;
     noAnalytics: string;
-    cliToken: string;
     partnersToken: string;
     runAsUser: string;
     serviceEnv: string;
packages/cli-kit/dist/public/node/environment.d.ts
@@ -10,12 +10,17 @@
  */
 export declare function getEnvironmentVariables(): NodeJS.ProcessEnv;
 /**
- * Returns the value of the SHOPIFY_CLI_TOKEN environment variable,
- * falling back to the deprecated SHOPIFY_CLI_PARTNERS_TOKEN.
+ * Returns the value of the SHOPIFY_CLI_PARTNERS_TOKEN environment variable.
  *
- * @returns The CLI token value, or undefined if neither env var is set.
+ * @returns Current process environment variables.
+ */
+export declare function getPartnersToken(): string | undefined;
+/**
+ * Check if the current proccess is running using the partners token.
+ *
+ * @returns True if the current proccess is running using the partners token.
  */
-export declare function getCliToken(): string | undefined;
+export declare function usePartnersToken(): boolean;
 /**
  * Returns the value of the organization id from the environment variables.
  *
packages/cli-kit/dist/private/node/session/exchange.d.ts
@@ -25,7 +25,7 @@ export declare function exchangeAccessForApplicationTokens(identityToken: Identi
  */
 export declare function refreshAccessToken(currentToken: IdentityToken): Promise<IdentityToken>;
 /**
- * Given a custom CLI token passed as ENV variable, request a valid Partners API token.
+ * Given a custom CLI token passed as ENV variable, request a valid Partners API token
  * This token does not accept extra scopes, just the cli one.
  * @param token - The CLI token passed as ENV variable 
  * @returns An instance with the application access tokens.
@@ -35,7 +35,7 @@ export declare function exchangeCustomPartnerToken(token: string): Promise<{
     userId: string;
 }>;
 /**
- * Given a custom CLI token passed as ENV variable, request a valid App Management API token.
+ * Given a custom CLI token passed as ENV variable, request a valid App Management API token
  * @param token - The CLI token passed as ENV variable 
  * @returns An instance with the application access tokens.
  */
@@ -44,7 +44,7 @@ export declare function exchangeCliTokenForAppManagementAccessToken(token: strin
     userId: string;
 }>;
 /**
- * Given a custom CLI token passed as ENV variable, request a valid Business Platform API token.
+ * Given a custom CLI token passed as ENV variable, request a valid Business Platform API token
  * @param token - The CLI token passed as ENV variable 
  * @returns An instance with the application access tokens.
  */

@dmerand dmerand force-pushed the dlm-app-validate-json branch from d22a6e8 to 5828af2 Compare March 25, 2026 15:15
@dmerand dmerand force-pushed the dlm-app-validate-errors branch from 9d58583 to 46515e4 Compare March 25, 2026 15:15
@dmerand dmerand force-pushed the dlm-app-validate-json branch from 5828af2 to 7a4c5d2 Compare March 25, 2026 15:50
@dmerand dmerand force-pushed the dlm-app-validate-errors branch from 46515e4 to 365b143 Compare March 25, 2026 15:50
@dmerand dmerand force-pushed the dlm-app-validate-json branch from 7a4c5d2 to bcfb6e3 Compare March 26, 2026 01:11
@dmerand dmerand force-pushed the dlm-app-validate-errors branch 2 times, most recently from 15b52cf to 964455a Compare March 26, 2026 14:02
@dmerand dmerand force-pushed the dlm-app-validate-json branch from bcfb6e3 to b9064bc Compare March 26, 2026 14:02
@dmerand dmerand marked this pull request as ready for review March 26, 2026 14:08
@dmerand dmerand requested a review from a team as a code owner March 26, 2026 14:08
@github-actions
Copy link
Contributor

We detected some changes at packages/*/src and there are no updates in the .changeset.
If the changes are user-facing, run pnpm changeset add to track your changes and include them in the next release CHANGELOG.

Caution

DO NOT create changesets for features which you do not wish to be included in the public changelog of the next CLI release.

Copy link
Contributor

@ryancbahan ryancbahan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we chat about path forward? I just removed the app loader's mode and have been working on a stack to untangle error handling.

Base automatically changed from dlm-app-validate-json to main March 26, 2026 14:56
Copy link
Contributor Author

dmerand commented Mar 26, 2026

Thanks, I somehow missed your changes. I'm refactoring now, I'll put the PR back into draft.

@dmerand dmerand marked this pull request as draft March 26, 2026 15:02
@dmerand dmerand force-pushed the dlm-app-validate-errors branch 2 times, most recently from 123cec3 to 30acf6b Compare March 26, 2026 18:18
@dmerand dmerand marked this pull request as ready for review March 26, 2026 18:36
Copilot AI review requested due to automatic review settings March 26, 2026 18:36
Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR updates the app-loading validation pipeline to retain structured Zod validation issue metadata (file/path-aware) internally while keeping existing human-readable CLI output behavior unchanged.

Changes:

  • Added structured issue parsing (parseStructuredErrors) alongside the existing human-readable formatter.
  • Introduced LocalConfigError and updated ConfigurationError/loader code paths to carry structured issues through the loader boundary.
  • Updated AppErrors to store both rendered messages and structured issues, with new accessors and structured export.

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
packages/app/src/cli/models/app/local-config-error.ts Introduces a shared local configuration error that carries configuration path + structured issues.
packages/app/src/cli/models/app/loader.ts Threads structured issues through configuration parsing and error aggregation (AppErrors).
packages/app/src/cli/models/app/loader.test.ts Adds coverage for structured issues on thrown configuration errors and AppErrors behavior.
packages/app/src/cli/models/app/error-parsing.ts Adds structured issue shaping (AppValidationIssue / parseStructuredErrors) and shared path formatting helpers.
packages/app/src/cli/models/app/error-parsing.test.ts Adds unit tests for parseStructuredErrors, including union handling.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines +121 to +129
const fallbackIssues = issue.unionErrors.flatMap((unionError) => unionError.issues ?? [])
if (fallbackIssues.length > 0) {
// Preserve any concrete nested issues we were able to recover before
// falling back to a synthetic root issue. This structured path is still
// internal, and retaining leaf issues is more actionable than erasing
// them behind a generic union failure.
return fallbackIssues.map((nestedIssue) => toStructuredIssue(filePath, nestedIssue))
}

Copy link

Copilot AI Mar 26, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

parseStructuredErrors contains a fallback branch that appears unreachable with the current findBestMatchingVariant implementation: if any union variant has non-empty issues, findBestMatchingVariant will always return a variant with issues.length > 0, so this fallbackIssues path will never execute. Consider either removing this dead code, or updating findBestMatchingVariant to return null when it can’t confidently pick a single variant (so the fallback becomes meaningful).

Suggested change
const fallbackIssues = issue.unionErrors.flatMap((unionError) => unionError.issues ?? [])
if (fallbackIssues.length > 0) {
// Preserve any concrete nested issues we were able to recover before
// falling back to a synthetic root issue. This structured path is still
// internal, and retaining leaf issues is more actionable than erasing
// them behind a generic union failure.
return fallbackIssues.map((nestedIssue) => toStructuredIssue(filePath, nestedIssue))
}

Copilot uses AI. Check for mistakes.
Comment on lines +29 to +33
export interface AppValidationFileIssues {
filePath: string
message: OutputMessage
issues: AppValidationIssue[]
}
Copy link

Copilot AI Mar 26, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AppValidationFileIssues.message is typed as OutputMessage (which can be a TokenizedString). If this structure is intended to be JSON-serialized (as the name suggests), TokenizedString will serialize as an object ({"value": ...}) rather than the rendered string. Consider changing message to a plain string (produced via stringifyMessage) or renaming this type/method to avoid implying JSON-safe output.

Copilot uses AI. Check for mistakes.
Comment on lines +3486 to 3489
function setupParsing(errorObj: zod.ZodIssue | {}, webhookConfigOverrides: WebhooksConfig) {
const toParse = getWebhookConfig(webhookConfigOverrides)
try {
const parsedConfiguration = parseConfigurationObject(WebhooksSchema, 'tmp', toParse)
Copy link

Copilot AI Mar 26, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

setupParsing takes an errorObj argument but never uses it. This makes the test helper signature misleading and keeps a lot of unused error objects in the tests. Consider removing the parameter (and updating call sites) or asserting against result.error using the provided errorObj to validate the expected Zod issue details.

Copilot uses AI. Check for mistakes.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants