Skip to content

Commit

Permalink
Added example how to use ollama hosted models (#536)
Browse files Browse the repository at this point in the history
Co-authored-by: Gerrit Grotewold <[email protected]>
Co-authored-by: mskarlin <[email protected]>
  • Loading branch information
3 people authored Oct 9, 2024
1 parent 62d4275 commit 16c1af4
Showing 1 changed file with 30 additions and 0 deletions.
30 changes: 30 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -459,6 +459,36 @@ answer = ask(
)
```

Models hosted with `ollama` are also supported.
To run the example below make sure you have downloaded llama3.2 and mxbai-embed-large via ollama.

```python
from paperqa import Settings, ask

local_llm_config = {
"model_list": [
{
"model_name": "ollama/llama3.2",
"litellm_params": {
"model": "ollama/llama3.2",
"api_base": "http://localhost:11434",
},
}
]
}

answer = ask(
"What manufacturing challenges are unique to bispecific antibodies?",
settings=Settings(
llm="ollama/llama3.2",
llm_config=local_llm_config,
summary_llm="ollama/llama3.2",
summary_llm_config=local_llm_config,
embedding="ollama/mxbai-embed-large",
),
)
```

### Changing Embedding Model

PaperQA2 defaults to using OpenAI (`text-embedding-3-small`) embeddings, but has flexible options for both vector stores and embedding choices. The simplest way to change an embedding is via the `embedding` argument to the `Settings` object constructor:
Expand Down

0 comments on commit 16c1af4

Please sign in to comment.