Add Spring Boot Project Starter for Google Gemini API model - ChatLangauge, Streaming model and Embedding Model #74

Suhas-Koheda · 2024-11-13T22:16:24Z

Closes langchain4j/langchain4j#2103

Suhas-Koheda · 2024-11-13T22:17:44Z

@langchain4j hey can you please have a look at this

langchain4j

@Suhas-Koheda thanks a lot!

langchain4j-google-ai-gemini/pom.xml

langchain4j · 2024-11-14T07:45:38Z

langchain4j-google-ai-gemini/src/main/java/dev/langchain4j/googleaigemini/AutoConfig.java

+                .topP(chatModelProperties.getTopP())
+                .topK(chatModelProperties.getTopK())
+                .maxOutputTokens(chatModelProperties.getMaxOutputTokens())
+                .responseFormat(ResponseFormat.JSON)


why is it json?

Hi. Related to your question. AutoConfig it's not injecting logging properties and json format from the properties -> https://docs.langchain4j.dev/tutorials/logging/ is not working when this AutoConfig is used. I will wait until this PR is merged to create a PR to avoid conflicts or If you want to add this configuration in this PR. Whatever you consider better.
Thank you!

Hey! can you provide me with some references
As i did not find the logging used in other starters
Or maybe i overlooked
Thank you!
@franglopez

langchain4j-google-ai-gemini/src/main/java/dev/langchain4j/googleaigemini/AutoConfig.java

...ain4j-google-ai-gemini/src/main/java/dev/langchain4j/googleaigemini/ChatModelProperties.java

.../src/test/java/dev/langchain4j/googleaigemini/Langchain4jGoogleAiGeminiApplicationTests.java

langchain4j-google-ai-gemini/src/main/resources/application.properties.local

Suhas-Koheda · 2024-11-14T13:02:35Z

@langchain4j also in the timeout values i had doubt
Will it be set by the user? Or should i keep the condition if its not set then set to 0

At present i have hardcoded used 60 secs

langchain4j · 2024-11-14T13:48:31Z

@Suhas-Koheda you can see how timeouts are handled in OpenAI starter here, the same for .logRequests(chatModelProperties.logRequests()).

ddobrin · 2024-11-14T16:19:09Z

@Suhas-Koheda
when you are looking at logging requests and responses, please be aware that the GogleAiGeminiChatModel in the module langchain4j-google-ai-gemini does not make a distinction in the constructor between logRequests() and log Responses() and uses a single Boolean to enable/disable the logging functionality link

Suhas-Koheda · 2024-11-14T16:29:48Z

Yeah sure I'll take care of it!
Thank you!
@ddobrin

Suhas-Koheda · 2024-11-15T21:03:47Z

@ddobrin hey for the response format i have kept the type of prop as ResponseFormat only which is user defined - not a primitive type?
is this ok ? or does it cause any problem?

…gaugeModel

…urces/application.properties

Suhas-Koheda · 2024-11-15T23:32:37Z

@langchain4j
Hey! Hi!
I've completed writing tests and both of the tests passed!
Also review the code once and let me know if any changes are to be made :)

ddobrin · 2024-11-16T17:43:44Z

@Suhas-Koheda on a 📱 let me come back to it at the beginning of the week

langchain4j · 2024-12-20T11:19:10Z

@Suhas-Koheda I would love to include your PR in this release (today), but there are still some issues left (please see my comments above). Since this is an independent module, we can release it a bit later separately, once this PR is ready.

Suhas-Koheda · 2024-12-21T08:01:38Z

@langchain4j
Yeah yeha sure!
I'll be working on it
I was a bit busy with my exams
Now I'll be completely on it!

Suhas-Koheda · 2024-12-21T13:22:40Z

@langchain4j hey i have written the NPE checks can you please have a look over this
Thank you!

langchain4j · 2024-12-22T07:21:46Z

...mini-spring-boot-starter/src/main/java/dev/langchain4j/googleaigemini/spring/AutoConfig.java

+            return defaultMap;
+        }
+        return Map.of(
+                safetySetting.geminiHarmCategory(),


Are these settings supposed to have only a single key-value pair?

The safety settings builder takes the map of geminiharmcategory as key and geminiharmblockthreshold as value

No it can take any number of values
It changes them into list

But for a single model thre would be only one harmcategory and blockthreshold right?

If that's not the case we can write such that the harmcategory and blockthreshold takes comma separated values and we manually convert then into map in the autoconfig?

@langchain4j

private Map<GeminiHarmCategory,GeminiHarmBlockThreshold> checkSafetySettingForNull(GeminiSafetySetting safetySetting) { if(safetySetting==null){ Map<GeminiHarmCategory,GeminiHarmBlockThreshold> defaultMap= new HashMap<>(); defaultMap.put(HARM_CATEGORY_CIVIC_INTEGRITY,HARM_BLOCK_THRESHOLD_UNSPECIFIED); return defaultMap; } Map<GeminiHarmCategory,GeminiHarmBlockThreshold> userMap= new HashMap<>(); safetySetting.geminiHarmCategory().forEach(category -> userMap.put(category,safetySetting.geminiHarmBlockThreshold().get(safetySetting.geminiHarmCategory().indexOf(category)))); return userMap; }

this can be done if there are multiple GeminiHarmCategory and GeminiHarmBlockThreshold

and the test cases run properly too

the code is ready to be commited
thank you!

Hi @ddobrin, could you please help with the review?

It seems to me that current logic in checkSafetySettingForNull is not correct. If settings are not specified, defaultMap.put(HARM_CATEGORY_CIVIC_INTEGRITY,HARM_BLOCK_THRESHOLD_UNSPECIFIED); is set (is it correct?). Also, chatModelProperties can have only a single setting...

@Suhas-Koheda I guess ChatModelProperties should have a Map<GeminiHarmCategory, GeminiHarmBlockThreshold> safetySettings instead of GeminiSafetySetting safetySetting.

Or List<GeminiSafetySetting> safetySettings, the same way as it is in the BaseGeminiChatModel

Hi @Suhas-Koheda

The safety filters settings, with their defaults, are available at his link

For flash and pro 1.5 the "default" block method is "SEVERITY" with the threshold by default at "BLOCK_MEDIUM_AND_ABOVE".

The HARM_CATEGORY_CIVIC_INTEGRITY filter is off by default.

Can I suggest that you do not set a value at all in
private GeminiMode checkGeminiModeForNull(GeminiFunctionCallingConfig geminiFunctionCallingConfig)?

To simplify, you can use the GeminiSafetySetting class which has a pair of <category, threshold?

Just in case: I would not duplicate the default settings for Gemini in LC4j code. If user did not configure anything explicitly, we should not set the defaults, but let Gemini backend do it

Hey i have a doubt
Like we are writing the safety settings in builder right
How can we leave it empty or such?
Probably there isn't any function in the backend with withSafetySettings() as such!
I am not sure of what to do
Like we should include safetysettings in builder pattern but pass null value since the user did not configure any in settings

@langchain4j
@ddobrin

Yes, agreed.
If the user does not specify any settings, then none are to be set.

Gemini has a default set which it will use if no other setting is modifying those defaults.
Link above just shows you the settings and why civic integrity should not be set

You can just change the condition when safetySettings is null or empty

Suhas-Koheda · 2024-12-24T04:39:56Z

@langchain4j hey! :-)

Suhas-Koheda · 2025-01-03T19:29:25Z

@langchain4j this logic looks perfect to me now!
jus that default values have to be sorted

ddobrin · 2025-01-04T01:03:26Z

Let me have a look tomorrow

Suhas-Koheda · 2025-01-07T17:12:39Z

@ddobrin
@langchain4j
hey can you please check this
i have solved the issue
all the tests run
and there are no defautl values too!

ddobrin · 2025-01-08T01:45:44Z

Hi @Suhas-Koheda
thanks for making the change.

The test code runs as is, and, if safetySettings are not set, it passes, so that part is solved.

While looking at the code and running it, you can see that there is no test which would address the use case when a user wishes to actually set in the configuration the safetySettings, which would close the testing for this area.

Do you want to add a test as well for setting actual values?

ddobrin · 2025-01-08T03:03:56Z

Hi @Suhas-Koheda
Setting the safety settings has to be tested out, as the values are not plain Strings, but enum values.
There are multiple ways to resolve this, from conversion to fully qualified enum values in the config.

I'll share some code here, as non-intrusive as possible, and you can consider merging it to your code, as you see fit:

Added a new test:

    @Test
    void provide_chat_mode_and_safety_settings() {
        contextRunner.withPropertyValues(
            "langchain4j.google-ai-gemini.chat-model.api-key=" + API_KEY,
                "langchain4j.google-ai-gemini.chat-model.model-name=gemini-1.5-flash",
                "langchain4j.google-ai-gemini.chat-model.temperature=1.0",
                "logging.level.org.springframework.boot.context.properties=DEBUG",
                "langchain4j.google-ai-gemini.chat-model.safety-settings.HARM_CATEGORY_DANGEROUS_CONTENT=BLOCK_LOW_AND_ABOVE"
            )
            .run(context -> {
                ChatLanguageModel chatLanguageModel = context.getBean(ChatLanguageModel.class);
                assertThat(context.getBean(GoogleAiGeminiChatModel.class)).isSameAs(chatLanguageModel);

                String response = chatLanguageModel.generate("What is the capital of India");
                assertThat(response).contains("Delhi");

                String newResponse = chatLanguageModel.generate("Calculate the Fibonacci of 22 and give me the result as an integer value along with the code. ");
                assertThat(newResponse).contains("17711");
            });
    }

Changed in ChatModelProperties the settings to be plain Strings:
Map<String, String> safetySettings

Last, in AutoConfig:

...
        if (chatModelProperties.safetySettings() != null
            && !chatModelProperties.safetySettings().isEmpty()) {
            builder.safetySettings(convertSafetySettings(chatModelProperties.safetySettings()));
        }
...

The small conversion method maps the Strings to the actual enums:

    private Map<GeminiHarmCategory, GeminiHarmBlockThreshold> convertSafetySettings(Map<String, String> map) {
        return map.entrySet().stream()
            .collect(Collectors.toMap(
                e -> GeminiHarmCategory.valueOf(e.getKey()),
                e -> GeminiHarmBlockThreshold.valueOf(e.getValue())
            ));
    }

Suhas-Koheda · 2025-01-08T03:37:34Z

@ddobrin yeah sure the map conversion would help
The other tests for the safetysettings and toolconfig
I have tests written before
I would be adding it by evening

Suhas-Koheda · 2025-01-08T16:40:33Z

@ddobrin
hey i have written the detailed tests!
and made the changes required

ddobrin · 2025-01-08T18:05:08Z

Thank you for making the changes @Suhas-Koheda. Nice to add the individual properties to the tests.

The only small thing you might wish to adjust is L.58 in the AutoConfigIT test, as the test is not running correctly due to:
"langchain4j.google-ai-gemini.chat-model.safety-settings.HARM_CATEGORY_DANGEROUS_CONTENT=BLOCK_LOW_AND_ABOVE",

should be
"langchain4j.google-ai-gemini.chat-model.safety-setting.HARM_CATEGORY_DANGEROUS_CONTENT=BLOCK_LOW_AND_ABOVE",

"setting" instead of "settings"

Suhas-Koheda · 2025-01-08T18:15:04Z

@ddobrin
Done! :-)

ddobrin · 2025-01-08T18:56:41Z

Just small things:

streamingChatModel is using lower-case C in a test and thus fails
flash returns sometimes 17,711 and sometimes 17711 - to avoid that search only for 711 - this is non-deterministic by the model
same for using the gemini-2.0-flash-exp model - sometimes the streamingChatTest fails, and 2.0 seems consistent
Thanks @Suhas-Koheda

Suhas-Koheda · 2025-01-08T19:08:25Z

@ddobrin

solved
Is that okay? Like just checking for 711? Actually i would change the prompt in a way to return just the integer along with code to eliminate any commas in between
I did not get this? What does fail? The 17711 one? It can be solved by requesting just the integer
If it is any other you can specify

I have run all the tests with gemini 2 flash exp model and works for me fine!

Please mention if there are any other things!
Thank you!

ddobrin · 2025-01-08T19:24:06Z

@Suhas-Koheda - yes, any change to get a deterministic text works for 17,711 vs 17711

The tests failing on the model are the streamingChatModel() tests

Suhas-Koheda · 2025-01-08T19:26:31Z

@ddobrin
Can you check the latest commit?
All works well for me!

ddobrin · 2025-01-08T19:49:13Z

Looks good @Suhas-Koheda

Suhas-Koheda · 2025-01-11T05:30:38Z

@langchain4j can you have a look over this!

Suhas-Koheda added 9 commits November 14, 2024 02:23

Base Commit for Google Gemini AI Model

89fa967

Added Configuration files for Google Gemini AI - ChatModel

cb8544f

Added Configuration files for Google Gemini AI - ChatModel

3a8765c

Check

0dfbc0d

Adding Streaming Google AI Chat Model and Removing Lombok uses

8335e67

Adding application.properties.local for easy usage

84bc5e3

Ignore all the .idea folders in subfolders

6d7c4cf

Ignore all the .idea folders in subfolders

300cfa2

Ignore all the .idea folders in subfolders

9618410

langchain4j reviewed Nov 14, 2024

View reviewed changes

Changing pom.xml headers

7ee36c6

Suhas-Koheda force-pushed the main branch from 8581056 to 7ee36c6 Compare November 14, 2024 11:46

Suhas-Koheda added 3 commits November 14, 2024 17:19

Resolving API_KEY to apikey

f696e82

Resolving API_KEY to apikey

4b8f2ec

Resolving API_KEY to apikey

9453ced

Suhas-Koheda added 2 commits November 16, 2024 02:28

Change Response Format to User Defined Props

4f267ff

Change Response Format to User Defined Props

affefe3

Suhas-Koheda added 5 commits November 16, 2024 03:59

Write test to check working of ChatLangaugeModel and StreamingCHatLan…

76053d9

…gaugeModel

Complete Test for Chat Language Model

6ee971e

Complete Test for Chat Language Model

637cd0e

Implement test for Streaming Chat Language Model

3e5fca5

Delete langchain4j-google-ai-gemini-spring-boot-starter/src/main/reso…

cfc5cae

…urces/application.properties

Suhas-Koheda added 2 commits December 21, 2024 18:18

Merge branch 'langchain4j:main' into main

9b52b6e

Adding NPE check for Safety Setting and Tool Config

9c167d1

langchain4j reviewed Dec 22, 2024

View reviewed changes

Changing safetySettings to map of harm category and harm threshold

816652c

Suhas-Koheda added 2 commits January 7, 2025 22:40

Checking Safety Settings and Tool Config

692c243

Remove unused imports

fd1a2f4

Add detailed tests

3ee4b30

-

ecd46fd

-

5cc8465

Change in tests

bf04308

Merge branch 'langchain4j:main' into main

320d596

Add Spring Boot Project Starter for Google Gemini API model - ChatLangauge, Streaming model and Embedding Model #74

Are you sure you want to change the base?

Add Spring Boot Project Starter for Google Gemini API model - ChatLangauge, Streaming model and Embedding Model #74

Conversation

Suhas-Koheda commented Nov 13, 2024 • edited by langchain4j Loading

Suhas-Koheda commented Nov 13, 2024

langchain4j left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

franglopez Nov 14, 2024 • edited Loading

Choose a reason for hiding this comment

Suhas-Koheda Nov 14, 2024 • edited Loading

Choose a reason for hiding this comment

Suhas-Koheda commented Nov 14, 2024

langchain4j commented Nov 14, 2024

ddobrin commented Nov 14, 2024 • edited Loading

Suhas-Koheda commented Nov 14, 2024 • edited Loading

Suhas-Koheda commented Nov 15, 2024

Suhas-Koheda commented Nov 15, 2024

ddobrin commented Nov 16, 2024

langchain4j commented Dec 20, 2024

Suhas-Koheda commented Dec 21, 2024

Suhas-Koheda commented Dec 21, 2024

Choose a reason for hiding this comment

Suhas-Koheda Dec 22, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Suhas-Koheda Jan 7, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Suhas-Koheda commented Dec 24, 2024

Suhas-Koheda commented Jan 3, 2025

ddobrin commented Jan 4, 2025

Suhas-Koheda commented Jan 7, 2025

ddobrin commented Jan 8, 2025

ddobrin commented Jan 8, 2025

Suhas-Koheda commented Jan 8, 2025

Suhas-Koheda commented Jan 8, 2025

ddobrin commented Jan 8, 2025

Suhas-Koheda commented Jan 8, 2025

ddobrin commented Jan 8, 2025

Suhas-Koheda commented Jan 8, 2025 • edited Loading

ddobrin commented Jan 8, 2025

Suhas-Koheda commented Jan 8, 2025

ddobrin commented Jan 8, 2025

Suhas-Koheda commented Jan 11, 2025

Suhas-Koheda commented Nov 13, 2024 •

edited by langchain4j

Loading

franglopez Nov 14, 2024 •

edited

Loading

Suhas-Koheda Nov 14, 2024 •

edited

Loading

ddobrin commented Nov 14, 2024 •

edited

Loading

Suhas-Koheda commented Nov 14, 2024 •

edited

Loading

Suhas-Koheda Dec 22, 2024 •

edited

Loading

Suhas-Koheda Jan 7, 2025 •

edited

Loading

Suhas-Koheda commented Jan 8, 2025 •

edited

Loading