[HUDI-7438] Reimplement Azure CI report check with open PRs

[yihua]

Change Logs

This PR redesigns and implements the way of surfacing Azure CI report status of a PR and fixes bugs in checking Azure CI report based on issue_comment event triggers.

There are a few things to be aware of:

• The GitHub token used by the GitHub workflows triggered by PRs from a fork (e.g., yihua/hudi instead of apache/hudi) has read-only permissions, and no write permissions are granted (Apache restriction). Only the GitHub workflows triggered from master and branches on apache/hudi contain GitHub token with write permissions, including scheduled tasks and workflows triggered by issue/PR comments. Write permissions are required for creating commit status, commenting on the PR, adding labels to the PR, add related projects, etc.
• The Azure CI report is included in the comments from hudi-bot user. Such comments are put by the bot some time after the PR is created.
• The workflow triggered by issue_comment, e.g., PR comments, is not associated with the PR, so from the PR page, we cannot directly see the workflow result, whether the check is successful or not.

This PR aims to create a commit status to the PR (see sample screenshots below) and maintain that based on the Azure CI report so it's easier to check the Azure CI status. Changes are:

• After each PR is created, a pending Azure CI commit status is created to indicate author to wait for the CI. Given that pull_request event-triggered workflows do not have write permissions, a new scheduled workflow, which has write permissions, is created in scheduled_workflow.yml to run every 5 minutes to check on recently created and updated PRs, and create or update commit status for Azure CI to the corresponding PR.
• The workflow in azure_ci_checks.yml is fixed to also create or update the commit status of the corresponding PR based on the Azure CI report status.
• PRs that are close, or containing HOTFIX: SKIP AZURE CI magic phrase in the description, are skipped for the check.
• The schedule workflow can be further extended for any other periodic actions that require write permissions, e.g., adding PR size labeling, issue labeling, attaching projects, etc.

How the commit status looks like in the PR:
The "Details" links to the Azure CI run.
Azure CI pending

Azure CI failing

Azure CI success

The core logic implemented is tested in a pushed branch to apache/hudi with this PR since we cannot test the logic from a fork: #10743.

Impact

Makes Azure CI status easy to read.

Risk level

none

Documentation Update

N/A

Contributor's checklist

• Read through contributor's guide
• Change Logs and Impact were stated clearly
• Adequate tests were added if applicable
• CI passed

[hudi-bot]

CI report:

• 9474d74 Azure: PENDING

Bot commands

@hudi-bot supports the following commands:

• @hudi-bot run azure re-run the last Azure build