Skip to content

GitHub Action for side-effect free deployment of Azure Data Factory resources

License

Notifications You must be signed in to change notification settings

sep/data-factory-deploy-action

 
 

Repository files navigation

Azure Data Factory Deploy Action

GitHub Action that performs a side-effect free deployment of Azure Data Factory entities in a Data Factory instance.

How it works

The GitHub Action uses pre and post-deployment scripts to prevent the deployment from potential side effects, such as:

  • Execution of active triggers during the deployment process that could corrupt resources relationships or have pipelines in undesired states.
  • Availability of unused resources that could bring confusion to data engineers and reduce maintainability.

Architecture Design

It is designed to run the following steps sequentially:

  1. A pre-deployment task checks for all active triggers and stop them.
  2. An ARM template deployment task is executed.
  3. A post-deployment task deletes all resources that have been removed from the ARM template (triggers, pipelines, dataflows, datasets, linked services, Integration Runtimes) and restarts the active triggers.

When to use

The action is useful on Continuous Deployment (CD) scenarios, where a step can be added in a workflow to deploy the Data Factory resources.

Getting Started

Prerequisites

If your GitHub Actions workflows are running on a self-hosted runner, ensure you have installed:

Example Usage

steps:
  - name: Login via Az module
    uses: azure/login@v1
    with:
      creds: ${{ secrets.AZURE_CREDENTIALS }}
      enable-AzPSSession: true 

  - name: Deploy resources
    uses: Azure/[email protected]
    with:
      resourceGroupName: myResourceGroup
      dataFactoryName: myDataFactory
      armTemplateFile: myArmTemplate.json
      # armTemplateParametersFile: myArmTemplateParameters.json [optional]
      # additionalParameters: 'key1=value key2=value keyN=value' [optional]
      # skipAzModuleInstallation: true [optional]
      # cleanupResources: true [optional]
      # deleteDeployment: true [optional]

Inputs

Name Description Required Default value
resourceGroupName Data Factory resource group name true
dataFactoryName Data Factory name true
armTemplateFile Data Factory ARM template file false ARMTemplateForFactory.json
armTemplateParametersFile Data Factory ARM template parameters file false ARMTemplateParametersForFactory.json
additionalParameters Data Factory custom parameters. Key-values must be splitted by space. false
skipAzModuleInstallation Skip Az powershell module installation. false false
cleanupResources Whether or not to run the cleanup step to remove deleted factory elements. false true
deleteDeployment Whether or not to delete the deployment after completion. false true

Contributing

This project welcomes contributions and suggestions. Most contributions require you to agree to a Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us the rights to use your contribution. For details, visit https://cla.opensource.microsoft.com.

When you submit a pull request, a CLA bot will automatically determine whether you need to provide a CLA and decorate the PR appropriately (e.g., status check, comment). Simply follow the instructions provided by the bot. You will only need to do this once across all repos using our CLA.

This project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact [email protected] with any additional questions or comments.

Trademarks

This project may contain trademarks or logos for projects, products, or services. Authorized use of Microsoft trademarks or logos is subject to and must follow Microsoft's Trademark & Brand Guidelines. Use of Microsoft trademarks or logos in modified versions of this project must not cause confusion or imply Microsoft sponsorship. Any use of third-party trademarks or logos are subject to those third-party's policies.

About

GitHub Action for side-effect free deployment of Azure Data Factory resources

Resources

License

Code of conduct

Security policy

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages

  • PowerShell 100.0%