ADR Record Evolving current setup to using 2 services: TaskScheduler and GraphQL/gRPC Architecture

[codecakes]

Does it superscede an existing ADR?

Yes

Status

accepted

Context

A transition to using a more lean version of the project using graphql is proposed here: #135

Originally, xcov19’s TaskScheduler was implemented synchronously. Tasks were added via methods like add_task_on_startup and executed using blocking calls (often via ThreadPoolExecutor with run_in_executor).

This “fire‑and‑forget” design allowed the app server to start but came with major downsides:

• Synchronous Behavior: The blocking implementation limited scalability and hindered integration with an ASGI server (such as FastAPI) when true concurrency was required.
• Limited gRPC Integration: Our initial gRPC services were tightly coupled with synchronous processing, making it difficult to evolve into a modern, non‑blocking architecture.

The need to support a high‑performance GraphQL API and robust gRPC inter‑service communication motivated us to re‑architect our TaskScheduler as a non‑blocking, asynchronous component.

Decision

We re‑designed the TaskScheduler to run asynchronously using Python’s asyncio framework. Key changes include:

• Non‑Blocking Scheduling:
  • Replacing asyncio.run_coroutine_threadsafe() with asyncio.create_task() to schedule long‑running background tasks.
  • Storing background task references so that they can be cancelled and awaited during graceful shutdown.
• Asynchronous Task Processing:
  • Converting methods like _run_task to async def and ensuring that asynchronous operations (e.g. await asyncio.gather(...) and await asyncio.sleep(5)) are correctly used.
• Synchronous Callbacks Handling:
  • Offloading synchronous callbacks to a ThreadPoolExecutor via loop.run_in_executor(), using functools.partial when keyword arguments are needed.
• Graceful Shutdown:
  • Implementing an on_shutdown() method that cancels and awaits all background tasks, ensuring no unobserved exceptions occur.
• GraphQL Integration:
  • Decoupling TaskScheduler startup from the ASGI server startup by scheduling the scheduler as a background task via asyncio.create_task(), allowing the server to begin serving immediately.

Consequences

What becomes easier:
• Scalability and Responsiveness: The asynchronous design avoids blocking the event loop, allowing the GraphQL API to remain highly responsive even under load.
• Graceful Shutdown: Tracking and managing background tasks enables a cleaner shutdown procedure that prevents unobserved exceptions and resource leaks.
• Integration: The decoupled, non‑blocking scheduler integrates smoothly with modern ASGI frameworks and the grpc.aio API, providing a robust architecture for inter‑service communication.

What becomes more difficult:
• Complexity: The transition to async code introduces new challenges in debugging, testing, and ensuring that all synchronous code is properly offloaded.
• Resource Management: Properly managing background tasks and ensuring graceful cancellation requires careful design and thorough testing.