[Feature]: CRUD API for schedules

[wouterhhuisman]

What should we build?

The schedules can be viewed, but not yet edited. The UI should report per task whether or not it is scheduled and when the next run is planned.

Besides the viewing option we would like to be able to add single occurence schedules and recurring schedules with a certain interval. Frontend design is described in workfloworchestrator/orchestrator-ui-library#2215

Overview

• Linker table for workflows and scheduled tasks
• Manage scheduled tasks through API
  • Create scheduled task (REST)
  • Read schedule for a task (GraphQL)
  • Update scheduled task (REST)
  • Delete schedule task (REST)
• Rework existing @scheduler.scheduled_job

Details

Create linker table workflows_apscheduler_jobs in database to link workflows and apscheduler_jobs.

• Table columns:
  • workflow_id: FK to workflows.workflow_id with "ON DELETE CASCADE"
  • schedule_id: FK to apscheduler_jobs.id with "ON DELETE CASCADE"
• Add a unique constraint

Create scheduled task:

• Get apscheduler object and use it's interface to add a job. Follow the example https://github.com/agronholm/apscheduler/tree/master/examples/separate_worker to see how to implement this properly with an event broker.
  • id: uuid4
  • name: schedule description (can be defaulted to task description in UI)
  • trigger: support all triggers (date, interval, cron)
• Wrapper function around orchestrator.services.processes.start_process() that accepts the name of the task to start.
• Insert entry in linker table

Read scheduled task:

• Already exists and implements retrieving schedule.
• Add workflow_id to ScheduledTask model so that it is visible in the API. The value should be retrieved from the linker table.

Update scheduled task:

• Same as Create with exceptions:
  • Only allow update of name and trigger
  • Job id must be that of an existing job
  • Do not insert in linker table - it should already exist.

Delete scheduled task:

• Delete scheduled task by the given job id. Use the scheduler interface for removing jobs.

Rework existing @scheduler.scheduled_job:

• Will no longer be needed as users can schedule tasks themselves through the API.
• For core schedules that run a task with a precondition: create a generic task (i.e. "task_run_task_if_not_active") that checks the precondition, and if true runs the actual task with start_process()
• For core schedules that have no precondition: remove the entire file.
• In the documentation add a section on recommended schedules.

[Mark90]

Rework existing @scheduler.scheduled_job may be split off to a follow-up story if needed

[Mark90]

A little bit of scope creep... :)

In order to avoid race conditions when the API modifies a job that the scheduler is working on, our plan is to make all schedule changes through the scheduler process.

To achieve this:

• The API has to push a message onto a redis queue for creating/updating/deleting schedules
• The scheduler process (in the run() method) has to listen for redis messages and call the corresponding schedule.add_job() (or modify/remove)

Out of scope is sending websocket events to the UI, created #1222 for this.