Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Consolidating Memory tests under client-sdk #703

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open

Conversation

vladimirivic
Copy link
Contributor

@vladimirivic vladimirivic commented Jan 1, 2025

Summary:
Part of #651

Requirements

  • add more integration tests in tests/client-sdk covering functionalities in llama-stack-apps

Porting tests from

  • llama_stack/providers/tests/memory/test_memory.py

Ensuring we cover some basic functions

  • MemoryResource src/llama_stack_client/resources/memory.py
  • MemoryBanksResource src/llama_stack_client/resources/memory_banks.py

Test Plan:
Run against the stack as lib

LLAMA_STACK_CONFIG=tests/client-sdk/memory/resources/run.yaml pytest tests/client-sdk/memory -v

tests/client-sdk/memory/test_memory.py::test_memory_bank_retrieve PASSED                                                                                                                 [ 16%]
tests/client-sdk/memory/test_memory.py::test_memory_bank_list PASSED                                                                                                                     [ 33%]
tests/client-sdk/memory/test_memory.py::test_memory_bank_register PASSED                                                                                                                 [ 50%]
tests/client-sdk/memory/test_memory.py::test_memory_bank_unregister PASSED                                                                                                               [ 66%]
tests/client-sdk/memory/test_memory.py::test_memory_bank_insert_inline_and_query PASSED                                                                                                  [ 83%]
tests/client-sdk/memory/test_memory.py::test_memory_bank_insert_from_url_and_query PASSED                                                                                                [100%]

Run agianst the local server

LLAMA_STACK_BASE_URL=http://localhost:5000 pytest tests/client-sdk/memory -v


tests/client-sdk/memory/test_memory.py::test_memory_bank_list PASSED                                                                                                                     [ 20%]
tests/client-sdk/memory/test_memory.py::test_memory_bank_register PASSED                                                                                                                 [ 40%]
tests/client-sdk/memory/test_memory.py::test_memory_bank_unregister PASSED                                                                                                               [ 60%]
tests/client-sdk/memory/test_memory.py::test_memory_bank_insert_inline_and_query PASSED                                                                                                  [ 80%]
tests/client-sdk/memory/test_memory.py::test_memory_bank_insert_from_url_and_query PASSED                                                                                                [100%]

@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Meta Open Source bot. label Jan 1, 2025
@vladimirivic vladimirivic marked this pull request as ready for review January 1, 2025 21:49
@@ -0,0 +1,108 @@
version: '2'
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remove? This is a duplicate of https://github.com/meta-llama/llama-stack/blob/main/llama_stack/templates/ollama/run.yaml?

We should be able to run tests on all template/*/run.yaml configs.

@@ -0,0 +1,28 @@
# Using Llama Stack as Library
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I will consolidate docs in the last PR for this issue and I can add instructions there.

Summary:
Part of #651

Requirements
* add more integration tests in tests/client-sdk covering functionalities in llama-stack-apps

Porting tests from
* llama_stack/providers/tests/memory/test_memory.py

Ensuring we cover some basic functions
* MemoryResource src/llama_stack_client/resources/memory.py
* MemoryBanksResource src/llama_stack_client/resources/memory_banks.py

Test Plan:
Run against the stack as lib
```
LLAMA_STACK_CONFIG=tests/client-sdk/memory/resources/run.yaml pytest tests/client-sdk/memory -v

tests/client-sdk/memory/test_memory.py::test_memory_bank_retrieve PASSED                                                                                                                 [ 16%]
tests/client-sdk/memory/test_memory.py::test_memory_bank_list PASSED                                                                                                                     [ 33%]
tests/client-sdk/memory/test_memory.py::test_memory_bank_register PASSED                                                                                                                 [ 50%]
tests/client-sdk/memory/test_memory.py::test_memory_bank_unregister PASSED                                                                                                               [ 66%]
tests/client-sdk/memory/test_memory.py::test_memory_bank_insert_inline_and_query PASSED                                                                                                  [ 83%]
tests/client-sdk/memory/test_memory.py::test_memory_bank_insert_from_url_and_query PASSED                                                                                                [100%]
```


Run agianst the local server
```
LLAMA_STACK_BASE_URL=http://localhost:5000 pytest tests/client-sdk/memory -v


tests/client-sdk/memory/test_memory.py::test_memory_bank_list PASSED                                                                                                                     [ 20%]
tests/client-sdk/memory/test_memory.py::test_memory_bank_register PASSED                                                                                                                 [ 40%]
tests/client-sdk/memory/test_memory.py::test_memory_bank_unregister PASSED                                                                                                               [ 60%]
tests/client-sdk/memory/test_memory.py::test_memory_bank_insert_inline_and_query PASSED                                                                                                  [ 80%]
tests/client-sdk/memory/test_memory.py::test_memory_bank_insert_from_url_and_query PASSED                                                                                                [100%]
```
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CLA Signed This label is managed by the Meta Open Source bot.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants