Skip to content

[FLINK-38976][runtime] Support multiple batch jobs in an application#27765

Merged
zhuzhurk merged 2 commits intoapache:masterfrom
eemario:FLIP560-4
Mar 26, 2026
Merged

[FLINK-38976][runtime] Support multiple batch jobs in an application#27765
zhuzhurk merged 2 commits intoapache:masterfrom
eemario:FLIP560-4

Conversation

@eemario
Copy link
Copy Markdown
Contributor

@eemario eemario commented Mar 13, 2026

What is the purpose of the change

This pull request relaxes the restriction on multiple jobs in HA mode, allowing an application to submit more than one batch job. It also removes the restriction to support cases where no jobs need to be submitted.

Brief change log

  • Update the job count limit validation logic in StreamContextEnvironment to support multiple jobs
  • Update the JarRunApplicationHandler and ApplicationDispatcherGatewayServiceFactory to enable submission of multiple batch jobs in HA mode
  • Update the PackagedProgramApplication to allow applications with no jobs

Verifying this change

This change added tests and can be verified as follows:

  • Added tests for PackagedProgramApplication to validate multiple job support

Does this pull request potentially affect one of the following parts:

  • Dependencies (does it add or upgrade a dependency): (no)
  • The public API, i.e., is any changed class annotated with @Public(Evolving): (no)
  • The serializers: (no)
  • The runtime per-record code paths (performance sensitive): (no)
  • Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: (no)
  • The S3 file system connector: (no)

Documentation

  • Does this pull request introduce a new feature? (yes)
  • If yes, how is the feature documented? (docs)

@flinkbot
Copy link
Copy Markdown
Collaborator

flinkbot commented Mar 13, 2026

CI report:

Bot commands The @flinkbot bot supports the following commands:
  • @flinkbot run azure re-run the last Azure build

@eemario eemario marked this pull request as ready for review March 13, 2026 06:40
@eemario eemario force-pushed the FLIP560-4 branch 2 times, most recently from 0068cb6 to e9d0df9 Compare March 18, 2026 08:22
@eemario eemario force-pushed the FLIP560-4 branch 2 times, most recently from e77bd90 to b9cb395 Compare March 26, 2026 02:07
Copy link
Copy Markdown
Contributor

@zhuzhurk zhuzhurk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me except for a few minor comments.

@zhuzhurk zhuzhurk merged commit ee64584 into apache:master Mar 26, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants