Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

If no experiments are specified, use the GitHub defaults #1514

Open
wants to merge 2 commits into
base: main
Choose a base branch
from
Open

If no experiments are specified, use the GitHub defaults #1514

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
8c5e5e4
If no experiments are specified, use the GitHub service defaults
rhyskoedijk Dec 14, 2024
7658f41
Update documentation
rhyskoedijk Dec 16, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
103 changes: 51 additions & 52 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,10 +10,10 @@ In this repository you'll find:

1. Azure DevOps [Extension](https://marketplace.visualstudio.com/items?itemName=tingle-software.dependabot), [source code](./extension) and [docs](./docs/extension.md).
1. Dependabot Server, [source code](./server/) and [docs](./docs/server.md).
1. Dependabot Updater image, [Dockerfile](./updater/Dockerfile), [source code](./updater/) and [docs](./docs/updater.md). **(Deprecated since v2.0)**
1. Dependabot Updater image, [Dockerfile](./updater/Dockerfile), [source code](./updater/) and [docs](./docs/updater.md). **(deprecated)**

> [!IMPORTANT]
> This project is currently undergoing a major version increment (V1 → V2); See the [migration guide](./docs/migrations/v1-to-v2.md#summary-of-changes-v1--v2) for more details and progress updates.
> The Azure pipelines task is currently undergoing a major version increment (V1 → V2); See the [migration guide](./docs/migrations/v1-to-v2.md#summary-of-changes-v1--v2) for more details and progress updates.

## Table of Contents
- [Getting started](#getting-started)
Expand All @@ -24,8 +24,8 @@ In this repository you'll find:
- [Configuring assignees and reviewers](#configuring-assignees-and-reviewers)
- [Unsupported features and configurations](#unsupported-features-and-configurations)
* [Dependabot Task](#dependabot-task)
+ [dependabot@V2](#dependabotv2)
+ [dependabot@V1](#dependabotv1)
+ [dependabot@2](#dependabot2)
+ [dependabot@1](#dependabot1)
* [Dependabot Updater Docker image](#dependabot-updater-docker-image)
* [Dependabot Server](#dependabot-server)
- [Migration Guide](#migration-guide)
Expand All @@ -36,10 +36,37 @@ In this repository you'll find:

## Getting started

Unlike the GitHub-hosted version, Dependabot for Azure DevOps must be explicitly setup in your organisation; creating a `dependabot.yml` file alone is **not** enough to enable updates. There are two ways to enable Dependabot, using:
Dependabot for Azure DevOps must be explicitly configured to run in your organisation; creating a `dependabot.yml` file alone is **not** enough to enable updates. There are two ways to enable Dependabot, using:

- [Azure DevOps Extension](https://marketplace.visualstudio.com/items?itemName=tingle-software.dependabot) - Ideal if you want to get Dependabot running with minimal administrative effort. The extension can run directly inside your existing pipeline agents and doesn't require hosting of any additional services. Because the extension runs in pipelines, this option does **not** scale well if you have a large number of projects and repositories.

<details>
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I read an issue or two recently that suggested maybe people weren't finding the "getting started" pipeline example from /extension/README.MD, so though it might be worthwhile pasting it at the top of the root read me too for visibility. Thoughts?

<summary>Example:</summary>

```yaml
trigger: none # Disable CI trigger

schedules:
- cron: '0 0 * * 0' # weekly on sunday at midnight UTC
always: true # run even when there are no code changes
branches:
include:
- master
batch: true
displayName: Weekly

pool:
vmImage: 'ubuntu-latest' # requires macos or ubuntu (windows is not supported)

steps:
- task: dependabot@2
inputs:
mergeStrategy: 'squash'
```
See [task requirements](/extension/README.md#task-requirements) and [task parameters](/extension/README.md#task-parameters) for more information.

</details>

- [Hosted Server](./docs/server.md) - Ideal if you have a large number of projects and repositories or prefer to run Dependabot as a managed service instead of using pipeline agents. See [why should I use the server?](./docs/server.md#why-should-i-use-the-server) for more info.

> [!NOTE]
Expand All @@ -49,15 +76,17 @@ Unlike the GitHub-hosted version, Dependabot for Azure DevOps must be explicitly

Similar to the GitHub-hosted version, Dependabot is configured using a [dependabot.yml file](https://docs.github.com/en/code-security/dependabot/dependabot-version-updates/configuration-options-for-the-dependabot.yml-file) located at `.azuredevops/dependabot.yml` or `.github/dependabot.yml` in your repository.

Most [official configuration options](https://docs.github.com/en/code-security/dependabot/dependabot-version-updates/configuration-options-for-the-dependabot.yml-file) are supported since V2; Earlier versions have several limitations, see [unsupported features and configurations](#unsupported-features-and-configurations) for more.
Most [official configuration options](https://docs.github.com/en/code-security/dependabot/dependabot-version-updates/configuration-options-for-the-dependabot.yml-file) are supported; See [unsupported features and configurations](#unsupported-features-and-configurations) for more details.

## Configuring private feeds and registries

Besides accessing the repository, sometimes private feeds/registries may need to be accessed. For example a private NuGet feed or a company internal docker registry.

Private registries are configured in `dependabot.yml`, refer to the [official documentation](https://docs.github.com/en/code-security/dependabot/dependabot-version-updates/configuration-options-for-the-dependabot.yml-file#configuration-options-for-private-registries).

Examples:

<details open>
<summary>Example:</summary>

```yml
version: 2
Expand Down Expand Up @@ -93,6 +122,8 @@ updates:
...
```

</details>

Note when using authentication secrets in configuration files:

> [!IMPORTANT]
Expand All @@ -104,41 +135,6 @@ BUT the values will be used from pipeline environment variables. Template variab
> 1. Have `Packaging (Read)` permission.
> 2. Be issued by a user with permission to the feed either directly or via a group. An easy way for this is to give `Contributor` permissions the `[{project_name}]\Contributors` group under the `Feed Settings -> Permissions` page. The page has the url format: `https://dev.azure.com/{organization}/{project}/_packaging?_a=settings&feed={feed-name}&view=permissions`.

> [!NOTE]
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I thought it would be good to drop this from to reduce confusion and complexity given using V1 is discouraged now, any concerns?

> When using `dependabot@V1` with a private feed/registry secured with basic auth, the `username`, `password`, **and** `token` properties are all required. The token format must be `${{ USERNAME }}:${{ PASSWORD }}`.

> [!NOTE]
> When using `dependabot@V1` with a repository containing a `nuget.config` file configured with custom package sources, the `key` property is required for each registry. The key must match between `dependabot.yml` and `nuget.config` otherwise the package source will be duplicated, package source mappings will be ignored, and auth errors will occur during dependency discovery. If your `nuget.config` looks like this:
> ```xml
> <?xml version="1.0" encoding="utf-8"?>
> <configuration>
> <packageSources>
> <clear />
> <add key="nuget.org" value="https://api.nuget.org/v3/index.json" />
> <add key="my-organisation1-nuget" value="https://dev.azure.com/my-organization/_packaging/my-nuget-feed/nuget/v3/index.json" />
> </packageSources>
> <packageSourceMapping>
> <packageSource key="nuget.org">
> <package pattern="*" />
> </packageSource>
> <packageSource key="my-organisation-nuget">
> <package pattern="Organisation.*" />
> </packageSource>
> </packageSourceMapping>
> </configuration>
> ```
>
> Then your `dependabot.yml` registry should look like this:
> ```yml
> version: 2
> registries:
> my-org:
> type: nuget-feed
> key: my-organisation1-nuget
> url: https://dev.azure.com/my-organization/_packaging/my-nuget-feed/nuget/v3/index.json
> token: PAT:${{ MY_DEPENDABOT_ADO_PAT }}
> ```

## Configuring security advisories and known vulnerabilities

Security-only updates is a mechanism to only create pull requests for dependencies with vulnerabilities by updating them to the earliest available non-vulnerable version. [Security updates are supported in the same way as the GitHub-hosted version](https://docs.github.com/en/code-security/dependabot/dependabot-security-updates/configuring-dependabot-security-updates#overriding-the-default-behavior-with-a-configuration-file) provided that a GitHub access token with `public_repo` access is provided in the `gitHubAccessToken` or `gitHubConnection` task inputs.
Expand All @@ -150,11 +146,10 @@ Dependabot uses an internal feature flag system called "experiments". Typically,

Experiments vary depending on the package ecyosystem used; They can be enabled using the `experiments` task input with a comma-seperated list of key/value pairs representing the experiments e.g. `experiments: 'tidy=true,vendor=true,goprivate=*'`.

> [!NOTE]
> Dependabot experinment names are not [publicly] documented. For convenience, some known experiments are listed below; However, **be aware that this may be out-of-date at the time of reading.**
By default, the enabled experiments will mirror the GitHub-hosted version of Dependabot, which can be found [here](/extension/tasks/dependabotV2/utils/dependabot/experiments.ts). Specifying experiments in the task input parameters will override all defaults.

<details>
<summary>List of known experiments from [email protected]</summary>
<details open>
<summary>List of known experiments:</summary>

|Package Ecosystem|Experiment Name|Value Type|More Information|
|--|--|--|--|
Expand All @@ -177,19 +172,23 @@ Experiments vary depending on the package ecyosystem used; They can be enabled u
| NuGet | nuget_legacy_dependency_solver | true/false | https://github.com/dependabot/dependabot-core/pull/10671 |
| NuGet | nuget_use_direct_discovery | true/false | https://github.com/dependabot/dependabot-core/pull/10597 |

> To find the latest list of Dependabot experiments, search the `dependabot-core` GitHub repository using queries like ["enabled?(x)"](https://github.com/search?q=repo%3Adependabot%2Fdependabot-core+%2Fenabled%5CW%5C%28.*%5C%29%2F&type=code) and ["options.fetch(x)"](https://github.com/search?q=repo%3Adependabot%2Fdependabot-core+%2Foptions%5C.fetch%5C%28.*%2C%2F&type=code).
> [!NOTE]
> Dependabot experiment names are not [publicly] documented and these may be out-of-date at the time of reading. To find the latest list of experiments, search the `dependabot-core` GitHub repository using queries like ["enabled?(x)"](https://github.com/search?q=repo%3Adependabot%2Fdependabot-core+%2Fenabled%5CW%5C%28.*%5C%29%2F&type=code) and ["options.fetch(x)"](https://github.com/search?q=repo%3Adependabot%2Fdependabot-core+%2Foptions%5C.fetch%5C%28.*%2C%2F&type=code).

</details>

## Configuring assignees and reviewers
Dependabot allows for the configuration of both [`assignees`](https://docs.github.com/en/code-security/dependabot/dependabot-version-updates/configuration-options-for-the-dependabot.yml-file#assignees) and [`reviewers`](https://docs.github.com/en/code-security/dependabot/dependabot-version-updates/configuration-options-for-the-dependabot.yml-file#reviewers). However, Azure DevOps does not have the concept of pull request assignees. Because of this, `assignees` will be treated as **required** reviewers and `reviewers` will be treated as **optional** reviewers.
Dependabot supports [`assignees`](https://docs.github.com/en/code-security/dependabot/dependabot-version-updates/configuration-options-for-the-dependabot.yml-file#assignees) and [`reviewers`](https://docs.github.com/en/code-security/dependabot/dependabot-version-updates/configuration-options-for-the-dependabot.yml-file#reviewers). However, Azure DevOps does not have the concept of pull request assignees. To work around this:

- [`assignees`](https://docs.github.com/en/code-security/dependabot/dependabot-version-updates/configuration-options-for-the-dependabot.yml-file#assignees) are treated as **required** pull request reviewers.
- [`reviewers`](https://docs.github.com/en/code-security/dependabot/dependabot-version-updates/configuration-options-for-the-dependabot.yml-file#reviewers) are treated as **optional** pull request reviewers.

Reviewers can be any of the following values:
The following values can be used as assignees or reviewers:

- User GUID
- User username
- User email address
- User full [display] name
- User full display name
- Group name
- Team name

Expand All @@ -198,11 +197,11 @@ We aim to support all [official configuration options](https://docs.github.com/e

### Dependabot Task

#### `dependabot@V2`
#### `dependabot@2`
- [`schedule`](https://docs.github.com/en/code-security/dependabot/dependabot-version-updates/configuration-options-for-the-dependabot.yml-file#scheduleinterval) is ignored, use [pipeline scheduled triggers](https://learn.microsoft.com/en-us/azure/devops/pipelines/process/scheduled-triggers?view=azure-devops&tabs=yaml#scheduled-triggers) instead.
- [`securityAdvisoriesFile`](#configuring-security-advisories-and-known-vulnerabilities) task input is not yet supported.

#### `dependabot@V1`
#### `dependabot@1`
- [`schedule`](https://docs.github.com/en/code-security/dependabot/dependabot-version-updates/configuration-options-for-the-dependabot.yml-file#scheduleinterval) is ignored, use [pipeline scheduled triggers](https://learn.microsoft.com/en-us/azure/devops/pipelines/process/scheduled-triggers?view=azure-devops&tabs=yaml#scheduled-triggers) instead.
- [`directories`](https://docs.github.com/en/code-security/dependabot/dependabot-version-updates/configuration-options-for-the-dependabot.yml-file#directories) are only supported if task input `useUpdateScriptVNext: true` is set.
- [`groups`](https://docs.github.com/en/code-security/dependabot/dependabot-version-updates/configuration-options-for-the-dependabot.yml-file#groups) are only supported if task input `useUpdateScriptVNext: true` is set.
Expand Down
43 changes: 37 additions & 6 deletions docs/extension.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,8 +10,8 @@
* [Running the task locally](#running-the-task-locally)
* [Running the unit tests](#running-the-unit-tests)
- [Architecture](#architecture)
* [Task V2 high-level update process diagram](#task-v2-high-level-update-process-diagram)

* [dependabot@2 versioned update process diagram](#dependabot2-versioned-update-process-diagram)
* [dependabot@2 hsecurity-only update process diagram](#dependabot2-security-only-update-process-diagram)

# Using the extension

Expand Down Expand Up @@ -65,8 +65,8 @@ npm start

To run a specific task version:
```bash
npm run start:V1 # runs dependabotV1 task
npm run start:V2 # runs dependabotV2 task
npm run start:V1 # runs dependabot@1 task
npm run start:V2 # runs dependabot@2 task
```

## Running the unit tests
Expand All @@ -78,8 +78,8 @@ npm test

# Architecture

## Task V2 high-level update process diagram
High-level sequence diagram illustrating how the `dependabotV2` task performs updates using [dependabot-cli](https://github.com/dependabot/cli). For more technical details, see [how dependabot-cli works](https://github.com/dependabot/cli?tab=readme-ov-file#how-it-works).
## dependabot2 versioned update process diagram
High-level sequence diagram illustrating how the `dependabot@2` task performs versioned updates using [dependabot-cli](https://github.com/dependabot/cli). For more technical details, see [how dependabot-cli works](https://github.com/dependabot/cli?tab=readme-ov-file#how-it-works).

```mermaid
sequenceDiagram
Expand Down Expand Up @@ -128,3 +128,34 @@ High-level sequence diagram illustrating how the `dependabotV2` task performs up
end

```

## dependabot2 security-only update process diagram
High-level sequence diagram illustrating how the `dependabot@2` task performs security-only updates using [dependabot-cli](https://github.com/dependabot/cli).

```mermaid
sequenceDiagram
participant ext as TaskV2
participant cli as Dependabot CLI
participant gha as GitHub Advisory Database

ext->>ext: Write `list-dependencies-job.yml`
Note right of ext: The job file contains `ignore: [ 'dependency-name': '*' ]`.<br>This will make Dependabot to discover all dependencies, but not update anything.<br>We can then extract the dependency list from the "depenedency_list" output.
ext->>+cli: Execute `dependabot update -f list-dependencies-job.yml -o output.yml`
cli->>cli: Run update job
cli->>cli: Write `output.yaml`
cli-->>-ext: Update completed

ext->>ext: Read and parse `output.yaml`, extract "dependency_list"
loop for each dependency
ext->>gha: Check security advisories for dependency
end
ext->>ext: Filter dependency list to only ones containing security advisories
ext->>ext: Write `security-only-update-job.yml`
Note right of ext: The job file contains the list of `dependency-names` and `security-advisories`.<br>This will make Dependanbot only update the dependencies named in the job file.
ext->>+cli: Execute `dependabot update -f security-only-update-job-job.yml -o output.yml`
cli->>cli: Run update job
cli->>cli: Write `output.yaml`
cli-->>-ext: Update completed
ext->>ext: Read and parse `output.yaml`
Note right of ext: Normal update logic resumes from this point.<br/>Outputs are parsed, pull requests are created/updated/closed based on the outputs
```
18 changes: 9 additions & 9 deletions docs/migrations/v1-to-v2.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@

> [!WARNING]
> **:construction: Work in progress;** `dependabot@V2` is still under development and this document may change without notice up until general availability (GA).
> **:construction: Work in progress;** `dependabot@2` is still under development and this document may change without notice up until general availability (GA).

# Table of Contents
- [Summary of changes V1 → V2](#summary-of-changes-v1-v2)
Expand All @@ -10,10 +10,10 @@
# Summary of changes V1 → V2
V2 is a complete re-write of the Dependabot task; It aims to:

- Resolve the [numerous private feed/registry authentication issues](https://github.com/tinglesoftware/dependabot-azure-devops/discussions/1317) that currently exist in V1;
- Resolve the [private feed/registry authentication issues](https://github.com/tinglesoftware/dependabot-azure-devops/discussions/1317) that exist in V1;
- More closely align the update logic with the GitHub-hosted Dependabot service;

The task now uses [Dependabot CLI](https://github.com/dependabot/cli) to perform dependency updates, which is the _[currently]_ recommended approach for running Dependabot. See [extension task architecture](../extension.md#architecture) for more details on the technical changes and impact to the update process.
The task now uses [Dependabot CLI](https://github.com/dependabot/cli) to perform dependency updates, which is the _current_ recommended approach for running Dependabot. See [extension task architecture](../extension.md#architecture) for more details on the technical changes and impact to the update process.

# Breaking changes V1 → V2

Expand Down Expand Up @@ -62,12 +62,12 @@ The following environment variables have been removed entirely; the feature is n

| Removed Environment Variable | Reason |
|--|--|
|`DEPENDABOT_PR_NAME_PREFIX_STYLE`| Feature is not supported; It is not an official configuration |
|`DEPENDABOT_COMPATIBILITY_SCORE_BADGE`| Feature is not supported; It is not an official configuration |
|`DEPENDABOT_MESSAGE_HEADER`| Feature is not supported; It is not an official configuration |
|`DEPENDABOT_MESSAGE_FOOTER`| Feature is not supported; It is not an official configuration |
|`DEPENDABOT_SIGNATURE_KEY`| Feature is not supported; It is not an official configuration |
|`DEPENDABOT_JOB_ID`| Set automatically by extension |
|`DEPENDABOT_COMPATIBILITY_SCORE_BADGE`| Feature is now enabled by default, no configuration required. |
|`DEPENDABOT_PR_NAME_PREFIX_STYLE`| Feature is not supported; It is not an official configuration. Use [`commit-message.prefix`](https://docs.github.com/en/code-security/dependabot/dependabot-version-updates/configuration-options-for-the-dependabot.yml-file#commit-message) instead. |
|`DEPENDABOT_MESSAGE_HEADER`| Feature is not supported; It is not an official configuration. |
|`DEPENDABOT_MESSAGE_FOOTER`| Feature is not supported; It is not an official configuration. |
|`DEPENDABOT_SIGNATURE_KEY`| Feature is not supported; It is not an official configuration. |
|`DEPENDABOT_JOB_ID`| Set automatically, no configuration required. |

## Todo before general availability
Before removing the preview flag from V2 `task.json`, we need to:
Expand Down
Loading
Loading