feat: support independent task queue for workforce

[a7m-1st]

Description

First part support for #3215 , which is an independent task queue.

New functions:

• get_independent_tasks - get _indepedent_task_queue
• add_task - now also adds to _indepedent_task_queue if flag is set, otherwise just treats as another sub_task
• remove_task - either removes from _indepedent_task_queue or _pending_task_queue given Task.id
• _handle_skip_task - handles skipping tasks situations
• handle_decompose_append_task - abstract the task decompositon, validation and return into a separate function thus simplifying process_task_async & _process_task_with_snapshot
• handle_independent_tasks - decompose the task from queue and manage it (add to completed tasks if no error)

Skipping (skip_gracefully & _async_skip_gracefully => triggers _handle_skip_task)

1. Run in graceful manner
2. If _pending_tasks
   2.1.1 Mark task as complete
   2.1.2 Transfer tasks to confirmed queue
   if task is in channel:
   2.2.1 Extract task from channel (thus get_in_flight_tasks (SENT, RETURNED or PROCESSING) from task_channel)
   2.2.2 Run 2.1.1 & 2.1.2 for each task
3. If _independent_task_queue is not empty
   3.1 Get the next task from _independent_task_queue then decompose it and continue loop.
   else:
   stop the loop.

Discussion, Doubts and Future enhancements:

1. Those are in my opinion the most important states to track for this PR. One issue I realized is that currently I am extracting the tasks which are only (SENT, RETURNED or PROCESSING) by the publisher and not Archived.
   I didn't get the chance to test if it has been Archived for now but will review it.
2. I am assuming the channel can only have tasks related to only and only to the current task in focus, OTHERWISE the publisher_id is new with each Task instance. Bcz I am essentially wiping it and marking as completed.

Todos

1. Need to support long lived loop natively (i.e. keep the loop running), by perhaps adding 'multi_turn' flag to the loop and stopping on timeout (after no _independent_tasks)
2. Question Agent will be implemented in a separate PR

Checklist

Go over all the following points, and put an x in all the boxes that apply.

• I have read the CONTRIBUTION guide (required)
• I have linked this PR to an issue using the Development section on the right sidebar or by adding Fixes #issue-number in the PR description (required)
• I have checked if any dependencies need to be added or updated in pyproject.toml and uv lock
• I have updated the tests accordingly (required for a bug fix or a new feature)
• I have updated the documentation if needed:
• I have added examples if this is a new feature

If you are unsure about any of these, don't hesitate to ask. We are here to help!

[coderabbitai]

Important

Review skipped

Auto reviews are disabled on this repository.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

✨ Finishing touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Post copyable unit tests in a comment
• Commit unit tests in branch feat-3215-multi-turn

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}