Workflow Checkpoints Limitations

[jefhughe]

Workflow CheckPoints

It seems the framework has a limitation of where checkpoint data can be stored. With workflows that may take several human-in-the-loop interactions, checkPoint state being held in the framework with no means to store in distributed cache seems to be a hard limitation. Unless I am not fully understanding how it works,* and if so please enlighten me.

Scenario

• I am running a stateless API with multiple pods to server requests.
• I am not using sessionAffinity and don't want to.
• I need to be able kill pods and not break workflows
• I have a Agent Framework workflow process that has human in the loop (multi-turn) conversations supporting multiple utterances.
• if the agent needs information from the user and the subsequent request is routed to another pod there will be no checkpoint data.
• I was looking at storing the checkpoint data in a distributed cache but the Agent Framework does not allow access to state data.

Workflow Process

• orchestration triage agent (fields the initial information request)
• handoff to 1-n agents

Question

• It would be very helpfull if we could store this state out of band so it can be re-inflated on any pod to complete the work.

[moonbox3]

We don't yet support a distributed agent runtime. That is coming in the future.

[lokitoth]

Hi @jefhughe - I was wondering if you are in .NET or Python. In .NET we support externalizing the checkpoint state by providing an implementation of ICheckpointStore. There is also JsonCheckpointStore (link) providing a nicer set of capabilities for the case of using JSON to serialize the state.

When you have one of those, you can create an externalizing CheckpointManager via the CreateJson static method.

I hope I understood your use case correctly here, but please let me know if this does not address it.

[jefhughe]

Jabob,
Thanks for the response.

I am using .NET and yes I think we are on the same page. The delimia I am having is I run the agent API in AKS and it will need to scale out to several pods as load increases and I don't want to use sessionAffinity. I want to externalize the CheckPoint Store but could not figure out how. I did look at the CheckPoint manager which did show the InMemory store. I am looking to externalize this.

[jefhughe]

One thing I notice with going out of band now is it works but I never see a parent id. I'm always getting a new one even on human in the loop where the Agent needs more information.

[lokitoth]

I did look at the CheckPoint manager which did show the InMemory store. I am looking to externalize this.

Take a look at the JsonCheckpointStore class - and as an example, the FileSystemJsonCheckpointStore implementation here.

You create a CheckpointManager from it using CheckpointManager.CreateJson (link).

Then when you are ready to create a connector for your own custom backend, you can switch it out.

I never see a parent id

That may be a bug - will take a look.

[jefhughe]

I have a CreateLocal and CreateRedis. Local allows for development and sessionAffinity where the redis enables full stateless APIs.
I believe I have it correct. It all works but never see a parent id and the handoffs are still a bit shaky. FYI running net10.0

Agent Versions

    <PackageReference Include="Microsoft.Agents.AI" Version="1.0.0-preview.251104.1" />
    <PackageReference Include="Microsoft.Agents.AI.OpenAI" Version="1.0.0-preview.251104.1" />
    <PackageReference Include="Microsoft.Agents.AI.Workflows" Version="1.0.0-preview.251104.1" />

 public static DistributedCheckpointManager CreateLocal(
        ILogger<DistributedCheckpointManager> logger,
        ILogger<LocalCheckpointStore> localStoreLogger,
        JsonSerializerOptions? customOptions = null)
    {
        var localStore = new LocalCheckpointStore(localStoreLogger);
        var checkpointManager = CheckpointManager.CreateJson(localStore, customOptions);
        
        logger.LogInformation("[DISTRIBUTED CHECKPOINT] Created Local-backed checkpoint manager");
        
        return new DistributedCheckpointManager(checkpointManager, logger, "Local");
    }

    private DistributedCheckpointManager(
        CheckpointManager checkpointManager, 
        ILogger<DistributedCheckpointManager> logger, 
        string storeType)
    {
        _checkpointManager = checkpointManager;
        _logger = logger;
        _storeType = storeType;
    }

localCheckPointStore

public class LocalCheckpointStore : JsonCheckpointStore
{
    private readonly ILogger<LocalCheckpointStore> _logger;
    private readonly ConcurrentDictionary<string, JsonElement> _checkpoints;
    private readonly ConcurrentDictionary<string, List<CheckpointInfo>> _indexes;
    private static int _checkpointCounter = 0;
.....
}

[jefhughe]

@lokitoth you stated "That may be a bug - will take a look." in reference to the parentId not being present. We're you able to determine if this was a bug or not?

[cecilphillip]

@jefhughe have you taking a look at this discussion? I ask a similar question a few weeks ago and soon after (by coincidence) the Durable extension came out.

#1092 (comment)