How to send progress notification from server to client

[shaojun]

Hi,
I'm implementing an lang-graph app, as some steps inside are accessing external services which typically take times, I want the client side to get notified for such server side operations, then the user would not panic in waiting.
I believed in web application, the web socket might be a way for this scenario, but i just want to confirm if is there anyway in langserve framework can simplify.
I noticed the Client callbacks are not yet supported for events that originate on the server in
https://python.langchain.com/docs/langserve#limitations
does this the feature that most satisfies my case? if this feature is missed, any other suggestion?

[rever1and]

sounds like you'd like to have realtime callback events.

[eyurtsev]

I'd recommend using astream events API for this purpose rather than callback events.

https://python.langchain.com/docs/expression_language/streaming#using-stream-events

The API is here and was just added to langserve. Let me know if you have questions.

[philbo]

"Hi, I have a similar problem to the OP.

Essentially at points in a chain(s) I'd like to stream back a custom message to the client. Ideally (but not necessarily) something syntactically like:

chain = ( prompt | llm | message("finished step 1") | prompt | llm | message("finished step 2) )

I've looked at the astream_events output but I'm not sure how I would inject a message into the response stream.

The callbacks interface looked like it could help here by overriding functions like on_chain_start but I could only get that to log output on the server-side - I'm not sure how to write a message back to the response stream.

Would something like the LogStreamCallbackHandler work? Or possibly the with_listeners method?

Thanks

[eyurtsev]

The easiest way to do this is using astream_events. Read over the documentation for astream events on LangChain to see how to assign a name for the runnable. Then client side you can filter the messages based on the run name and use "on_x_start" and "on_x_end" events to notify about progress.

I strongly recommend against using callbacks for this since you'll be picking up a bunch of extra complexity that's already solved by astream events

[philbo]

thanks for the pointers, I'll stick with the event stream. I ended up adding the message I wanted to the metadata of the Runnable so I can parse it client-side