-
Notifications
You must be signed in to change notification settings - Fork 4
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Streaming] Fix issues when not using try-with-resource
#49
base: main
Are you sure you want to change the base?
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You can remove try-with-resources in the controller and readme since it doesn't do anything anymore
e2e-test-app/src/main/java/com/sap/ai/sdk/app/controllers/OpenAiController.java
Outdated
Show resolved
Hide resolved
The close handler is still there
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Left some comments. I would consider un-nesting the HandledIterator
and merge it with the StreamConverter
into just one class.
e2e-test-app/src/main/java/com/sap/ai/sdk/app/controllers/OpenAiController.java
Outdated
Show resolved
Hide resolved
...tion-models/openai/src/main/java/com/sap/ai/sdk/foundationmodels/openai/StreamConverter.java
Outdated
Show resolved
Hide resolved
...tion-models/openai/src/main/java/com/sap/ai/sdk/foundationmodels/openai/StreamConverter.java
Outdated
Show resolved
Hide resolved
...tion-models/openai/src/main/java/com/sap/ai/sdk/foundationmodels/openai/StreamConverter.java
Outdated
Show resolved
Hide resolved
...tion-models/openai/src/main/java/com/sap/ai/sdk/foundationmodels/openai/StreamConverter.java
Outdated
Show resolved
Hide resolved
sample-code/spring-app/src/main/java/com/sap/ai/sdk/app/controllers/OpenAiController.java
Outdated
Show resolved
Hide resolved
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I get a timeout on http://localhost:8080/streamChatCompletionDeltas
final var entity = new InputStreamEntity(inputStream, ContentType.TEXT_PLAIN); | ||
|
||
final var sut = IterableStreamConverter.lines(entity); | ||
assertThatCode(sut::count) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
assertThatCode(sut::count) | |
assertThatThrownBy(sut::count) |
The main difference with assertThatCode(ThrowingCallable) is that this method fails if no exception was thrown.
package com.sap.ai.sdk.foundationmodels.openai; | ||
|
||
import static org.assertj.core.api.Assertions.assertThat; | ||
import static org.assertj.core.api.Assertions.assertThatCode; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
import static org.assertj.core.api.Assertions.assertThatCode; | |
import static org.assertj.core.api.Assertions.assertThatThrownBy; |
@@ -42,20 +36,8 @@ Stream<D> handleResponse(@Nonnull final ClassicHttpResponse response) | |||
@SuppressWarnings("PMD.CloseResource") | |||
private Stream<D> parseResponse(@Nonnull final ClassicHttpResponse response) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The stream is closed by the user of the Stream
This is false now. Could you update this comment and duplicate it in IterableStreamConverter.lines()
Alternatively just merge parseResponse
inside of handleResponse
since parseResponse
only contains a return
and nothing else
} catch (final Exception e) { | ||
isDone = true; | ||
stopHandler.run(); | ||
throw new IllegalStateException("Iterator stopped unexpectedly.", e); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What can be thrown here?
Make the streaming API lenient towards
try-with-resources
.Before:
After:
The suggested
HandledIterator
understands whenBufferedReader#readLine()
returnsnull
, then the underlyingInputStream
shall be closed [left]. Also when the consumingStream
is closed or finished iteration [right].See JavaDoc of
String BufferedReader#readLine()
to understandnull
contract.See JavaDoc and code of
Stream BufferedReader#lines()
for comparison.