Feature Request: Synchronous Calls

[JonathanRayner]

I would love it if I had the option use the library entirely with synchronous calls. This way, I have full freedom to enable or disable this, pursue other concurrency models, and would have less issues interfacing with other libraries that are opinionated about open event loops.

Unfortunately, run.sync doesn't solve this. I'd actually prefer if synchronous calls were a 1st class citizen and there was a thin wrapper or arg to make things async instead of vice versa. Unfortunately, I understand that this isn't necessarily easy to express in python, meaning it often leads to two full copies of the api for sync and async. Still, I think there's a good reason this is available in e.g. OpenAI's client.

Addendum (feel free to ignore)

More broadly (please excuse if I'm overstepping my bounds), but I wonder a bit if this is a downstream issue of a broader problem with what the identity of this library is. When I read:

Agent Framework / shim to use Pydantic with LLMs

PydanticAI is a Python agent framework designed to make it less painful to build production grade applications with Generative AI.

Every agent framework and LLM library in Python uses Pydantic, yet when we began to use LLMs in Pydantic Logfire, we couldn't find anything that gave us the same feeling.

I think "Oh cool, the Pydantic team is making an agent framework. I'll use that if I need to do something involving agentic workflows, probably overkill otherwise."

But then when I hear:

PydanticAI

PydanticAI is a Python Agent Framework designed to make it less painful to build production grade applications with Generative AI.

We built PydanticAI with one simple aim: to bring that FastAPI feeling to GenAI app development.

My thought is, "Oh! Amazing! The Pydantic team is going bring that 'minimal, just works, clean' feel compared to lots of other recent general AI frameworks."

Personally I would love a philosophy that is something like

1. PydanticAI: makes input/output data validation easy, standardize ways of calling apis, standardize logging/observability (or connect to other observability tools)
2. PydanticAgents or PydanticWorkflows w/e: build complex workflows, graphs, etc. on top of PydanticAI
3. ???

Overall, I'm sort of hoping for Pydantic <-> PydanticAI and FastAPI <-> PydanticAgents or something like this. But right now they're blending together. I think the tension between these two viewpoints leads to some design consequences:

• As a user, it's easier for me to build agent workflows on top of what I call PydanticAI + some features of PydanticAgents being implemented, but it's much harder to build on top of what I call fully-featured PydanticAgents with only some of PydanticAI implemented. A key example of this would be the decision prioritize more agentic features, while multi-modal support still doesn't feel particularly clean. If thinking generally about AI, multi-modal input would be one of the first things I think should be supported, before more complex features.
• Maybe if you're thinking agents, then it might be "obvious" that async is a reasonable default (maybe not). But if thinking just "good foundation for ai" this would be an optional extra feature request.
• Certainly the default usage of just passing things into an Agent works great and feels clean. But also there are situations when it seems like you need to create a Client just to pass it into a Model just to pass that into an Agent. Maybe this is outdated docs, but if not, that doesn't feel right. Using a custom endpoint in the OpenAI client outside of this library requires just passing the different endpoint and api key as args, not instantiating 3 classes. It's not a big deal, but I wonder if more things like this will happen if Agent is the first class citizen, instead of viewed as built "on top of" PydanticAI.
• The choice to have a simple centralized way to call loads of providers (as opposed to "bring your own client"), great data validation on inputs and outputs, and logfire makes it feel like this is a general foundation for AI, not just a way to run agent workflows (and that's great!).
• Having more of a separation between the philosophy of PydanticAI and PydanticAgents also helps with possible paradigm shifts. Maybe in a year or two, we won't really be thinking about "agents," maybe we will. But we'll always want full-featured standards for calling AI apis, data validation in and out, logging, observability, etc.
• Said another way: what defines an agent? Is it a client + state? Model + state? Or just a client or a model? A useful coordination point of many different abilities? I think it might be useful to have building blocks to create many different types of agents in the long run.

Thanks for all the hard work, curious what you think. I wouldn't write this if I wasn't super excited about this library!

[dmontagu]

Can you expand on this:

I'd actually prefer if synchronous calls were a 1st class citizen and there was a thin wrapper or arg to make things async instead of vice versa. Unfortunately, I understand that this isn't necessarily easy to express in python, meaning it often leads to two full copies of the api for sync and async. Still, I think there's a good reason this is available in e.g. OpenAI's client.

I think we can do a better job than we have so far of powering this workflow better, but I also think we might be able to do that while wrapping the async usage as much as possible.

I think we're more likely to run into issues with the .iter() API and graph run iteration, etc., introduced in #833, so that may ultimately force our hand into a second, fully-sync implementation.

But it would definitely be helpful to better understand where the dislike for sync-wrapper-around-async is coming from, if there's anything more to it than just "it's kind of buggy". In particular, I don't want to take on a big maintenance burden of duplicating all code paths in a sync and async way just because there are bugs today that we can fix e.g. by creating a new event loop for every sync call (instead of using the existing one like we currently do).

[dmontagu]

I'll also note that the approach described here, which is how psycopg maintains a sync and async version of the codebase, may be feasible for us. Just need to adapt this surprisingly-not-huge script.

That said, I think we'll keep the Async version of the code with the non-prefixed names, and add prefixes for Sync code. (Since in most real world usage scenarios, the Async version is what you probably should use.)

---

I think you've raised some good questions in your addendum, I want to address them, but that will take a bit more time and wanted to get a response out to the part of the issue tied to the issue title sooner.

[Kludex]

We agreed on working on this.

The idea is that we are going to generate the sync API via an CST script.

Whenever I work on my priorities, I'll get back to this.

[cspiecker]

Hi @Kludex ,
Just checking in to see if there’s any update on the synchronous API feature. Is it currently being worked on, or is there an estimated timeline for its release? Appreciate all the work you’re doing

[fortzi]

where the dislike for sync-wrapper-around-async is coming from

@dmontagu Not sure about the OP's use case, but in mine, I've integrated pydantic-ai into a larger application that uses threading for concurrency.
Incorporating pydantic-ai into this application proves to be more painful that we've expected, as we're getting many errors around event-loop usage/management.

[fortzi]

Hi @Kludex,
Are you able to share if this is planned to be implemented in the near future, or if there's any timeline? 🙏

[Kludex]

I don't have any plans to work on this.