GraphQl Breaking Schema Change Detector

.Net tool which can detect breaking schema changes in a GraphQl schema.
Includes support for automatic comments creation in Azure Devops Pr pipelines.

Why

When maintaining a GraphQl api the convention is to not version the graph but instead extend it in a backwards compatibel way.
The possibility to solve api versioning this way is one of the great strenghts of GraphQl but one needs to ensure that the graph is indeed always backwards compatibel. As the name implies the 'GraphQl Breaking Schema Change Detector' allows for automatically detecting such breaking changes. It can either compare two GraphQl schemas and report breaking changes or it allows to validate the schema during pull request stage.

How to compare two GraphQl Schemas via Cli

1. Make sure you have installed the dotnet sdk
2. Install the Breaking Change Detector as a global dotnet tool: dotnet tool install -g graphql-breaking-schema-change-detector
3. Compare two schemas with the following command
   breaking-change file --oldSchema <old schema file> --newSchema <new schema file>

Setup Azure Devops PR Validation

The azure devops integration allows you to monitor a graphql schema file for backward incompatibel changes. This is valuable when your schema is automatically generated based on your source code. If one of your unit test ensures that the schema of your GraphQl service is corresponding to a GraphQl schema file under source control then the GraphqlBreakingSchemaChangeDetector can monitor this file for breaking changes (see example).

To enable Azure DevOps integration 4 steps are needed.

1. Create a new yaml file containing the definition of a new PR validation build
2. Create a new pipeline based on the new yaml file.
3. Add the new pipeline as a PR Build validation step
4. Give the Build Agent the needed permissions to create comments on the pull request

Create a new yaml file

Create a new yaml file in your source repository with the following content.

pr:
  - main

pool:
  vmImage: ubuntu-latest

steps:
  - task: UseDotNet@2
    inputs:
      version: "6.0.x"
      includePreviewVersions: false
  - script: |
      dotnet tool install --tool-path . graphql-breaking-schema-change-detector
      ./breaking-change pr --file <PATH TO YOUR SCHEMA FILE>
    env:
      SYSTEM_ACCESSTOKEN: $(System.AccessToken)
    displayName: "Run Breaking Change Detector"

Make sure to replace <PATH TO YOUR SCHEMA FILE> with the file path to the schema file you want to monitor. The file path must be relative to the root of your repository. Example: test/foo/bar/schema.graphql.

Create a new Pipeline.

1. Choose your source provider and connect to your repository
2. In "Configure your pipeline" choose the "Existing Azure Pipelines YAML file" and select the azure-pipelines.yml file
3. Save your new pipeline

Add the new pipeline as a PR Build Validation step

1. Go to your repository
2. Choose Repos -> Branches -> Press the three dots on your main branch -> Branch policies
3. Build Validation -> Plus Sign -> Choose your new pipeline from the "Build pipeline" dropdown
4. Save

Allow the Build Agent to "Contribute to pull requests"

1. Go to Project settings (bottom left)
2. Repositories -> Choose your repository -> Security Tab -> Users -> Build Agend -> Contribute to pull requests -> Set to Allow

Troubleshooting

Issues with Nuget restore

Problem

/opt/hostedtoolcache/dotnet/sdk/6.0.201/NuGet.targets(130,5): error : Unable to load the service index for source <...>/nuget/v3/index.json. [/tmp/rsb0lugh.s3e/restore.csproj]
/opt/hostedtoolcache/dotnet/sdk/6.0.201/NuGet.targets(130,5): error :   Response status code does not indicate success: 401 (Unauthorized). [/tmp/rsb0lugh.s3e/restore.csproj]
The tool package could not be restored.
Tool 'graphql-breaking-schema-change-detector' failed to install. This failure may have been caused by:

Solution

It seems that you have configured a private nuget feed for your repository. You need to either add the required authorization before installing the tool, for example with the nuget-authenticate task or falling back to nuget.org by adding the two parameters --ignore-failed-sources and --add-source to the instalation command.

dotnet tool install --ignore-failed-sources --add-source https://api.nuget.org/v3/index.json --tool-path . graphql-breaking-schema-change-detector

Issue with file not found

Problem

{"$id":"1","innerException":null,"message":"TF400714: File ID 293829685 not found.","typeName":"Microsoft.TeamFoundation.Framework.Server.FileIdNotFoundException, Microsoft.TeamFoundation.Framework.Server","typeKey":"FileIdNotFoundException","errorCode":0,"eventId":4005}
Unhandled exception: System.Exception: Reading file <...>.csv from ado pr failed

Solution

It seems that the path to your schema file is in an incorrect format. Make sure that your path does not start with a dot (.) or slash (/) and use forward slashes to seperate your folders. Good exampel: test/__snapshots__/mySchemaSnapshot.snap. Bad example: .\src\schema.graphql