Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Spring Boot Project Starter for Google Gemini API model - ChatLangauge, Streaming model and Embedding Model #74

Open
wants to merge 55 commits into
base: main
Choose a base branch
from
Open
Changes from 2 commits
Commits
Show all changes
55 commits
Select commit Hold shift + click to select a range
89fa967
Base Commit for Google Gemini AI Model
Suhas-Koheda Nov 13, 2024
cb8544f
Added Configuration files for Google Gemini AI - ChatModel
Suhas-Koheda Nov 13, 2024
3a8765c
Added Configuration files for Google Gemini AI - ChatModel
Suhas-Koheda Nov 13, 2024
0dfbc0d
Check
Suhas-Koheda Nov 13, 2024
8335e67
Adding Streaming Google AI Chat Model and Removing Lombok uses
Suhas-Koheda Nov 13, 2024
84bc5e3
Adding application.properties.local for easy usage
Suhas-Koheda Nov 13, 2024
6d7c4cf
Ignore all the .idea folders in subfolders
Suhas-Koheda Nov 13, 2024
300cfa2
Ignore all the .idea folders in subfolders
Suhas-Koheda Nov 13, 2024
9618410
Ignore all the .idea folders in subfolders
Suhas-Koheda Nov 13, 2024
7ee36c6
Changing pom.xml headers
Suhas-Koheda Nov 14, 2024
f696e82
Resolving API_KEY to apikey
Suhas-Koheda Nov 14, 2024
4b8f2ec
Resolving API_KEY to apikey
Suhas-Koheda Nov 14, 2024
9453ced
Resolving API_KEY to apikey
Suhas-Koheda Nov 14, 2024
4f267ff
Change Response Format to User Defined Props
Suhas-Koheda Nov 15, 2024
affefe3
Change Response Format to User Defined Props
Suhas-Koheda Nov 15, 2024
76053d9
Write test to check working of ChatLangaugeModel and StreamingCHatLan…
Suhas-Koheda Nov 15, 2024
6ee971e
Complete Test for Chat Language Model
Suhas-Koheda Nov 15, 2024
637cd0e
Complete Test for Chat Language Model
Suhas-Koheda Nov 15, 2024
3e5fca5
Implement test for Streaming Chat Language Model
Suhas-Koheda Nov 15, 2024
cfc5cae
Delete langchain4j-google-ai-gemini-spring-boot-starter/src/main/reso…
Suhas-Koheda Nov 15, 2024
66c828e
Changing Java Version in pom.xml from 21 to 17 to support maven (mvn)…
Suhas-Koheda Nov 16, 2024
11d7989
Merge remote-tracking branch 'refs/remotes/origin/main'
Suhas-Koheda Nov 16, 2024
ec1275b
Changing Java version back to 21 :) in pom.xml
Suhas-Koheda Nov 16, 2024
0fd6868
Resolving other issues except Embedding model
Suhas-Koheda Nov 18, 2024
070a71e
Adding env variable to main.yaml and release.yaml
Suhas-Koheda Nov 18, 2024
7aa79cb
Merge branch 'langchain4j:main' into main
Suhas-Koheda Nov 18, 2024
889aa2f
Update release.yaml to add Google AI Gemini API Key
Suhas-Koheda Nov 18, 2024
db889db
Update main.yaml to add Google AI Gemini API Key
Suhas-Koheda Nov 18, 2024
b626e12
Merge branch 'main' of https://github.com/Suhas-Koheda/langchain4j-sp…
Suhas-Koheda Nov 18, 2024
4ae025a
Changing the version of dependencies to project version
Suhas-Koheda Nov 18, 2024
9aad39f
adding GeminiSafetyProperties and GeminiSafetySettings to googleGemin…
Suhas-Koheda Nov 19, 2024
3c32174
Add tests for chatlanguagemodel with updated properties
Suhas-Koheda Nov 19, 2024
c4ba686
Adding Embedding model configuration and test for Embedding model
Suhas-Koheda Nov 20, 2024
ef71908
Merge branch 'langchain4j:main' into main
Suhas-Koheda Nov 20, 2024
b0a8f9a
Resolving issues in spring boot starter for google ai gemini
Suhas-Koheda Nov 26, 2024
89694f5
Update main.yaml for google ai gemini key
Suhas-Koheda Nov 26, 2024
907424b
Update release.yaml for google ai gemini key
Suhas-Koheda Nov 26, 2024
6159165
refactoring resources
Suhas-Koheda Nov 26, 2024
4a27ef1
Merge remote-tracking branch 'refs/remotes/origin/main'
Suhas-Koheda Nov 26, 2024
7554b25
Changing the safteySetting and toolConfig and writing test
Suhas-Koheda Nov 30, 2024
4afe606
Merge branch 'langchain4j:main' into main
Suhas-Koheda Dec 2, 2024
69660a4
Resolving silly mistake
Suhas-Koheda Dec 3, 2024
3107cb9
Resolving issues
Suhas-Koheda Dec 7, 2024
56d7546
Changing bean properties
Suhas-Koheda Dec 9, 2024
ed5afe0
cosmetics
Dec 16, 2024
9b52b6e
Merge branch 'langchain4j:main' into main
Suhas-Koheda Dec 21, 2024
9c167d1
Adding NPE check for Safety Setting and Tool Config
Suhas-Koheda Dec 21, 2024
816652c
Changing safetySettings to map of harm category and harm threshold
Suhas-Koheda Jan 3, 2025
692c243
Checking Safety Settings and Tool Config
Suhas-Koheda Jan 7, 2025
fd1a2f4
Remove unused imports
Suhas-Koheda Jan 7, 2025
3ee4b30
Add detailed tests
Suhas-Koheda Jan 8, 2025
ecd46fd
-
Suhas-Koheda Jan 8, 2025
5cc8465
-
Suhas-Koheda Jan 8, 2025
bf04308
Change in tests
Suhas-Koheda Jan 8, 2025
320d596
Merge branch 'langchain4j:main' into main
Suhas-Koheda Jan 11, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -1,16 +1,17 @@
package dev.langchain4j.googleaigemini.spring;

import dev.langchain4j.model.googleai.GoogleAiEmbeddingModel;
import dev.langchain4j.model.googleai.GoogleAiGeminiChatModel;
import dev.langchain4j.model.googleai.GoogleAiGeminiStreamingChatModel;
import dev.langchain4j.model.googleai.*;
import org.springframework.boot.autoconfigure.AutoConfiguration;
import org.springframework.boot.autoconfigure.condition.ConditionalOnProperty;
import org.springframework.boot.context.properties.EnableConfigurationProperties;
import org.springframework.context.annotation.Bean;

import java.util.HashMap;
import java.util.Map;

import static dev.langchain4j.googleaigemini.spring.Properties.PREFIX;
import static dev.langchain4j.model.googleai.GeminiHarmBlockThreshold.HARM_BLOCK_THRESHOLD_UNSPECIFIED;
import static dev.langchain4j.model.googleai.GeminiHarmCategory.HARM_CATEGORY_CIVIC_INTEGRITY;

@AutoConfiguration
@EnableConfigurationProperties(Properties.class)
Expand All @@ -32,15 +33,10 @@ GoogleAiGeminiChatModel googleAiGeminiChatModel(Properties properties) {
.maxOutputTokens(chatModelProperties.maxOutputTokens())
.responseFormat(chatModelProperties.responseFormat())
.logRequestsAndResponses(chatModelProperties.logRequestsAndResponses())
.safetySettings(Map.of(
// TODO NPE?
chatModelProperties.safetySetting().geminiHarmCategory(),
chatModelProperties.safetySetting().geminiHarmBlockThreshold()
))
.safetySettings(checkSafetySettingForNull(chatModelProperties.safetySetting()))
.toolConfig(
// TODO NPE?
chatModelProperties.functionCallingConfig().geminiMode(),
chatModelProperties.functionCallingConfig().allowedFunctionNames().toArray(new String[0])
checkGeminiModeForNull(chatModelProperties.functionCallingConfig()),
checkFunctionNamesForNull(chatModelProperties.functionCallingConfig())
)
.build();
}
Expand All @@ -60,15 +56,10 @@ GoogleAiGeminiStreamingChatModel googleAiGeminiStreamingChatModel(Properties pro
.topK(chatModelProperties.topK())
.responseFormat(chatModelProperties.responseFormat())
.logRequestsAndResponses(chatModelProperties.logRequestsAndResponses())
.safetySettings(Map.of(
// TODO NPE?
chatModelProperties.safetySetting().geminiHarmCategory(),
chatModelProperties.safetySetting().geminiHarmBlockThreshold()
))
.safetySettings(checkSafetySettingForNull(chatModelProperties.safetySetting()))
.toolConfig(
// TODO NPE?
chatModelProperties.functionCallingConfig().geminiMode(),
chatModelProperties.functionCallingConfig().allowedFunctionNames().toArray(new String[0])
checkGeminiModeForNull(chatModelProperties.functionCallingConfig()),
checkFunctionNamesForNull(chatModelProperties.functionCallingConfig())
)
.build();
}
Expand All @@ -91,4 +82,31 @@ GoogleAiEmbeddingModel googleAiEmbeddingModel(Properties properties) {
.titleMetadataKey(embeddingModelProperties.titleMetadataKey())
.build();
}

private String[] checkFunctionNamesForNull(GeminiFunctionCallingConfig geminiFunctionCallingConfig) {
if(geminiFunctionCallingConfig==null){
return new String[0];
}
return geminiFunctionCallingConfig.allowedFunctionNames().toArray(new String[0]);
}

private GeminiMode checkGeminiModeForNull(GeminiFunctionCallingConfig geminiFunctionCallingConfig) {
if(geminiFunctionCallingConfig==null){
return GeminiMode.NONE;
}
return geminiFunctionCallingConfig.geminiMode();
}

private Map<GeminiHarmCategory,GeminiHarmBlockThreshold> checkSafetySettingForNull(GeminiSafetySetting safetySetting) {
if(safetySetting==null){
Map<GeminiHarmCategory,GeminiHarmBlockThreshold> defaultMap= new HashMap<>();
defaultMap.put(HARM_CATEGORY_CIVIC_INTEGRITY,HARM_BLOCK_THRESHOLD_UNSPECIFIED);
return defaultMap;
}
return Map.of(
safetySetting.geminiHarmCategory(),
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are these settings supposed to have only a single key-value pair?

Copy link
Author

@Suhas-Koheda Suhas-Koheda Dec 22, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The safety settings builder takes the map of geminiharmcategory as key and geminiharmblockthreshold as value

No it can take any number of values
It changes them into list

But for a single model thre would be only one harmcategory and blockthreshold right?

If that's not the case we can write such that the harmcategory and blockthreshold takes comma separated values and we manually convert then into map in the autoconfig?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@langchain4j

private Map<GeminiHarmCategory,GeminiHarmBlockThreshold> checkSafetySettingForNull(GeminiSafetySetting safetySetting) {
        if(safetySetting==null){
            Map<GeminiHarmCategory,GeminiHarmBlockThreshold> defaultMap= new HashMap<>();
            defaultMap.put(HARM_CATEGORY_CIVIC_INTEGRITY,HARM_BLOCK_THRESHOLD_UNSPECIFIED);
            return defaultMap;
        }
        Map<GeminiHarmCategory,GeminiHarmBlockThreshold> userMap= new HashMap<>();
        safetySetting.geminiHarmCategory().forEach(category -> userMap.put(category,safetySetting.geminiHarmBlockThreshold().get(safetySetting.geminiHarmCategory().indexOf(category))));
        return userMap;
    }

this can be done if there are multiple GeminiHarmCategory and GeminiHarmBlockThreshold

and the test cases run properly too

the code is ready to be commited
thank you!

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @ddobrin, could you please help with the review?

It seems to me that current logic in checkSafetySettingForNull is not correct. If settings are not specified, defaultMap.put(HARM_CATEGORY_CIVIC_INTEGRITY,HARM_BLOCK_THRESHOLD_UNSPECIFIED); is set (is it correct?). Also, chatModelProperties can have only a single setting...

@Suhas-Koheda I guess ChatModelProperties should have a Map<GeminiHarmCategory, GeminiHarmBlockThreshold> safetySettings instead of GeminiSafetySetting safetySetting.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Or List<GeminiSafetySetting> safetySettings, the same way as it is in the BaseGeminiChatModel

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @Suhas-Koheda

The safety filters settings, with their defaults, are available at his link

For flash and pro 1.5 the "default" block method is "SEVERITY" with the threshold by default at "BLOCK_MEDIUM_AND_ABOVE".

The HARM_CATEGORY_CIVIC_INTEGRITY filter is off by default.

Can I suggest that you do not set a value at all in
private GeminiMode checkGeminiModeForNull(GeminiFunctionCallingConfig geminiFunctionCallingConfig)?

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To simplify, you can use the GeminiSafetySetting class which has a pair of <category, threshold?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just in case: I would not duplicate the default settings for Gemini in LC4j code. If user did not configure anything explicitly, we should not set the defaults, but let Gemini backend do it

Copy link
Author

@Suhas-Koheda Suhas-Koheda Jan 7, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey i have a doubt
Like we are writing the safety settings in builder right
How can we leave it empty or such?
Probably there isn't any function in the backend with withSafetySettings() as such!
I am not sure of what to do
Like we should include safetysettings in builder pattern but pass null value since the user did not configure any in settings

@langchain4j
@ddobrin

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, agreed.
If the user does not specify any settings, then none are to be set.

Gemini has a default set which it will use if no other setting is modifying those defaults.
Link above just shows you the settings and why civic integrity should not be set

You can just change the condition when safetySettings is null or empty

safetySetting.geminiHarmBlockThreshold()
);
}

}