feat: Add search/cloud-function/python #861

zeroasterisk · 2024-07-11T01:07:46Z

This is a long awaited update of a now closed PR #314. (blast from the past)

Ready:

This should be very easy to deploy and use locally, based on Cloud Functions as a standardization
There is minimal response simplification based on the search configuration, which should help people get started and
There are unit tests which mock the APIs
There are integration tests which do very basic functionality validation, calling the API (more in the future)

Future:

Right now this is python only, but I intend to make the same abstraction for a few other languages (polyglot FTW)
Creation of private integration tests based on this framework validating combinations of features
- Summarization + Unstructured data + Chunk mode
- Summarization + Structured data
- Summarization + Blended data
There's no UX today but we could consider adding one
There's a placeholder config to do Generation based on Vertex Search retrieval via GEMINI or GENERATE_GROUNDED_ANSWERS, which might be added to the python client before creating variants (ref
go/vertex-ai-search-retrieval-gemini-generate)

Co-authored-by: Holt Skinner <[email protected]>

[![Mend Renovate](https://app.renovatebot.com/images/banner.svg)](https://renovatebot.com) This PR contains the following updates: | Package | Change | Age | Adoption | Passing | Confidence | |---|---|---|---|---|---| | [flowbite](https://flowbite.com) ([source](https://togithub.com/themesberg/flowbite)) | [`^1.8.1` -> `^2.0.0`](https://renovatebot.com/diffs/npm/flowbite/1.8.1/2.3.0) | [![age](https://developer.mend.io/api/mc/badges/age/npm/flowbite/2.3.0?slim=true)](https://docs.renovatebot.com/merge-confidence/) | [![adoption](https://developer.mend.io/api/mc/badges/adoption/npm/flowbite/2.3.0?slim=true)](https://docs.renovatebot.com/merge-confidence/) | [![passing](https://developer.mend.io/api/mc/badges/compatibility/npm/flowbite/1.8.1/2.3.0?slim=true)](https://docs.renovatebot.com/merge-confidence/) | [![confidence](https://developer.mend.io/api/mc/badges/confidence/npm/flowbite/1.8.1/2.3.0?slim=true)](https://docs.renovatebot.com/merge-confidence/) | --- ### Release Notes <details> <summary>themesberg/flowbite (flowbite)</summary> ### [`v2.3.0`](https://togithub.com/themesberg/flowbite/releases/tag/v2.3.0) [Compare Source](https://togithub.com/themesberg/flowbite/compare/v2.2.1...v2.3.0) - added new [copy to clipboard](https://flowbite.com/docs/components/clipboard/) component and examples - added new JavaScript tab to the documentation examples - update dependency to Tailwind CSS v3.4.1 - fixed carousel component behavior when there’s only one image - added new data attribute to set active and inactive classes for tabs - Ruby on Rails integration fix for turbo load - minor bug fixes and improvements ### [`v2.2.1`](https://togithub.com/themesberg/flowbite/releases/tag/v2.2.1) [Compare Source](https://togithub.com/themesberg/flowbite/compare/v2.2.0...v2.2.1) - relased new [phone number input](https://flowbite.com/docs/forms/phone-input/) component - relased new [chat bubble](https://flowbite.com/docs/components/chat-bubble/) component - updated `z-index` of the datepicker to allow usage with modals and drawers ### [`v2.2.0`](https://togithub.com/themesberg/flowbite/releases/tag/v2.2.0) [Compare Source](https://togithub.com/themesberg/flowbite/compare/v2.1.1...v2.2.0) - released new [number input](https://flowbite.com/docs/forms/number-input/) component examples - created a new `InputCounter` object class in the Flowbite JS ### [`v2.1.1`](https://togithub.com/themesberg/flowbite/releases/tag/v2.1.1) [Compare Source](https://togithub.com/themesberg/flowbite/compare/v2.1.0...v2.1.1) - fix the `InstanceOptions` and `EventListenerInstance` type exports for TypeScript ### [`v2.1.0`](https://togithub.com/themesberg/flowbite/releases/tag/v2.1.0) [Compare Source](https://togithub.com/themesberg/flowbite/compare/v2.0.0...v2.1.0) This update adds [full RTL support](https://flowbite.com/docs/customize/rtl/) to all of the UI components from Flowbite and also provides and update to the Instance Manager API which now allows you to set custom id's as the 4th parameter of each object class. - updated to Tailwind CSS `v3.3.5` - added full RTL support to all UI components - updated the JS API and Instance Manager to allow setting custom id's - added new vertical tabs component example - added three next modal component examples - fixed responsive wrapping issues for the table components - uses Tailwind CSS v3.0 classes for background opacity fixing [themesberg/flowbite#565 - other minor bug fixes and improvements Breaking changes: - `data-modal-toggle` alone won't initialise the modal anymore, you need to use `data-modal-target` for that ### [`v2.0.0`](https://togithub.com/themesberg/flowbite/releases/tag/v2.0.0) [Compare Source](https://togithub.com/themesberg/flowbite/compare/v1.8.1...v2.0.0) This update introduces a new instance manager feature that allows you manage all of the interactive components from a single place. This is useful if you want to programmatically show or hide a modal, popover, or tooltip. This also works if you use the data attributes interface. This change introduces a few breaking changes to the JavaScript API when creating objects: - new `$carouselEl` HTML element when creating a `Carousel` object as the first parameter - new `$accordion` HTML element when creating a `Accordion` object as the first parameter - new `$tabsEl` HTML element when creating a `Tabs` object as the first parameter The breaking changes do not apply to the data attributes interface, only for objects created programmatically via JavaScript. Learn more about the new [JavaScript API on the docs](https://flowbite.com/docs/getting-started/javascript/). </details> --- ### Configuration 📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] If you want to rebase/retry this PR, check this box --- This PR has been generated by [Mend Renovate](https://www.mend.io/free-developer-tools/renovate/). View repository job log [here](https://developer.mend.io/github/GoogleCloudPlatform/generative-ai).  Co-authored-by: Holt Skinner <[email protected]>

add root level README the goal is for a polyglot set of examples in multiple languages

- Split functionality: - main.py does Cloud Functions - vertex_search_client.py does Vertex AI search - implemented .env configuration for settings - add unit test_vertex_search_client (mocked) - add integration test_integration_vertex_search_client (only 1 for now) Tested and working.

search/cloud-function/python/setup_env.sh

search/cloud-function/python/main.py

Moved content to README from file comment. Added docblocks to all functions and classes in main.py and vertex_search_client.py

The flexible enums were just complicated enough that they could be more trouble than they are worth - added a test file.

See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md

Enums are better, but this is already a pretty complicated example and I thought this would make it simpler. Also switched to a typed config object vs class properties on VertexSearchClient

holtskinner

I like this overall. Tbh, a lot of what has been written here could be a good basis for a handwritten SDK

holtskinner · 2024-07-23T16:49:49Z

search/cloud-function/README.md

+[Your RAG powered by Google Search](https://cloud.google.com/blog/products/ai-machine-learning/rags-powered-by-google-search-technology-part-1)).
+
+If you want even more control see
+[Vertex AI APIs for building search and RAG experiences](https://cloud.google.com/generative-ai-app-builder/docs/builder-apis),


Suggested change

[Vertex AI APIs for building search and RAG experiences](https://cloud.google.com/generative-ai-app-builder/docs/builder-apis),

[Vertex AI Search Component APIs](https://cloud.google.com/generative-ai-app-builder/docs/builder-apis),

holtskinner · 2024-07-23T16:51:04Z

search/cloud-function/README.md

+```mermaid
+flowchart LR
+    A[fa:fa-search Vertex Search AI] --> B(Google Cloud Function)
+    B --> C[My App Server]
+    C -->|One| D[fa:fa-laptop web]
+    C -->|Two| E[fa:fa-mobile mobile]
+```


This is so neat! I've never seen this text based flowchart used in GitHub markdown.

holtskinner · 2024-07-23T16:51:18Z

search/cloud-function/README.md

+quality the better the agent response based on your source data.
+
+For more on agents, check out
+[Agent Builder Use Cases](https://cloud.google.com/products/agent-builder?hl=en#common-uses)


Suggested change

[Agent Builder Use Cases](https://cloud.google.com/products/agent-builder?hl=en#common-uses)

[Vertex AI Agent Builder Use Cases](https://cloud.google.com/products/agent-builder?hl=en#common-uses)

holtskinner · 2024-07-23T16:51:51Z

search/cloud-function/python/README.md

+If you require more extensive customization, consider using an orchestration
+framework like [LangChain](https://www.langchain.com/).
+
+**Take a look at other implementations in [search/cloud-functions/](../)**.


Not sure this is needed yet

holtskinner · 2024-07-23T16:52:50Z

search/cloud-function/python/main.py

+
+
+@functions_framework.http
+def vertex_search(request: Request) -> Tuple[str, int, Dict[str, str]]:


Nit: Can this use consistent naming?

Suggested change

def vertex_search(request: Request) -> Tuple[str, int, Dict[str, str]]:

def vertex_ai_search(request: Request) -> Tuple[str, int, Dict[str, str]]:

holtskinner · 2024-07-23T16:53:15Z

search/cloud-function/python/test_integration_vertex_search_client.py

+from typing import Generator
+
+import pytest
+from vertex_search_client import VertexSearchClient, VertexSearchConfig


Nit: Can this consistently be called "Vertex AI Search"?

holtskinner · 2024-07-23T16:55:21Z

search/cloud-function/python/vertex_search_client.py

+        self.client = self._create_client()
+        self.serving_config = self._get_serving_config()


Is there a reason to bring these out into separate methods? They're only used the one time

holtskinner · 2024-07-23T16:58:00Z

Just fix the remaining lint errors or add ignores when appropriate, then this can merge.

zeroasterisk and others added 10 commits July 10, 2024 20:50

Add example cloud function + search

86be786

Update search/example-cloud-function/main.py

2d9595e

Co-authored-by: Holt Skinner <[email protected]>

Update search/example-cloud-function/main.py

7880c57

Co-authored-by: Holt Skinner <[email protected]>

ran nox -s format

f97af78

renamed dir to search/cloud-function

39bd511

fork python implementation into subdir

a874196

add root level README the goal is for a polyglot set of examples in multiple languages

add local runtime instructions for cloud functions

b5ed397

ran nox -s format

06b597c

zeroasterisk requested a review from a team as a code owner July 11, 2024 01:07

zeroasterisk requested a review from lavinigam-gcp July 11, 2024 01:07

github-actions bot added the owlbot:run Add this label to trigger the Owlbot post processor. label Jul 11, 2024

gcf-owl-bot bot removed the owlbot:run Add this label to trigger the Owlbot post processor. label Jul 11, 2024

This comment has been minimized.

Sign in to view

zeroasterisk changed the title ~~Add search/cloud-function/python (repeat of #314)~~ Add search/cloud-function/python Jul 11, 2024

zeroasterisk changed the title ~~Add search/cloud-function/python~~ feat: Add search/cloud-function/python Jul 11, 2024

fix: minor spelling fixes re:owlbot

460cb74

github-actions bot added the owlbot:run Add this label to trigger the Owlbot post processor. label Jul 11, 2024

gcf-owl-bot bot removed the owlbot:run Add this label to trigger the Owlbot post processor. label Jul 11, 2024

This comment has been minimized.

Sign in to view

holtskinner reviewed Jul 11, 2024

View reviewed changes

search/cloud-function/python/setup_env.sh Show resolved Hide resolved

holtskinner reviewed Jul 11, 2024

View reviewed changes

search/cloud-function/python/main.py Outdated Show resolved Hide resolved

Merge branch 'main' into example-cloud-function

b24e049

github-actions bot added the owlbot:run Add this label to trigger the Owlbot post processor. label Jul 11, 2024

gcf-owl-bot bot removed the owlbot:run Add this label to trigger the Owlbot post processor. label Jul 11, 2024

This comment has been minimized.

Sign in to view

Fix spelling/minor lint errors

34a15c0

github-actions bot added the owlbot:run Add this label to trigger the Owlbot post processor. label Jul 11, 2024

gcf-owl-bot bot removed the owlbot:run Add this label to trigger the Owlbot post processor. label Jul 11, 2024

Alan Blount added 2 commits July 16, 2024 23:46

fix linter errors

3cf5e86

Added docblocks and shuffled content

e3dd9a4

Moved content to README from file comment. Added docblocks to all functions and classes in main.py and vertex_search_client.py