Skip to content
This repository has been archived by the owner on Sep 4, 2024. It is now read-only.

Commit

Permalink
timeout within circuit breaker
Browse files Browse the repository at this point in the history 
Signed-off-by: Pawel Leszczynski <[email protected]>
  • Loading branch information
@pawel-big-lebowski
pawel-big-lebowski committed Apr 19, 2024
1 parent d029f92 commit 95809f6
Showing 1 changed file with 20 additions and 10 deletions.
30 changes: 20 additions & 10 deletions docs/client/java/partials/java_circuit_breaker.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,8 @@ Simple circuit breaker which is working based only on free memory within JVM. Co
contain free memory threshold limit (percentage). Default value is `20%`. The circuit breaker
will close within first call if free memory is low. `circuitCheckIntervalInMillis` parameter is used
to configure a frequency circuit breaker is called. Default value is `1000ms`, when no entry in config.
`timeoutInSeconds` is optional. If set, OpenLineage code execution is terminated when a timeout
is reached (added in version 1.13).

<Tabs groupId="integrations">
<TabItem value="yaml" label="Yaml Config">
Expand All @@ -23,24 +25,27 @@ circuitBreaker:
type: simpleMemory
memoryThreshold: 20
circuitCheckIntervalInMillis: 1000
timeoutInSeconds: 90
```
</TabItem>
<TabItem value="spark" label="Spark Config">
| Parameter | Definition | Example |
--------------------------------------|---------------------------------------|-------------
| spark.openlineage.circuitBreaker.type | Circuit breaker type selected | simpleMemory |
| spark.openlineage.circuitBreaker.memoryThreshold | Memory threshold | 20 |
| spark.openlineage.circuitBreaker.circuitCheckIntervalInMillis | Frequency of checking circuit breaker | 1000 |
| Parameter | Definition | Example |
--------------------------------------|----------------------------------------------------------------|--------------
| spark.openlineage.circuitBreaker.type | Circuit breaker type selected | simpleMemory |
| spark.openlineage.circuitBreaker.memoryThreshold | Memory threshold | 20 |
| spark.openlineage.circuitBreaker.circuitCheckIntervalInMillis | Frequency of checking circuit breaker | 1000 |
| spark.openlineage.circuitBreaker.timeoutInSeconds | Optional timeout for OpenLineage execution (Since version 1.13)| 90 |
</TabItem>
<TabItem value="flink" label="Flink Config">
| Parameter | Definition | Example |
--------------------------------------|---------------------------------------|-------------
| openlineage.circuitBreaker.type | Circuit breaker type selected | simpleMemory |
| openlineage.circuitBreaker.memoryThreshold | Memory threshold | 20 |
| openlineage.circuitBreaker.circuitCheckIntervalInMillis | Frequency of checking circuit breaker | 1000 |
| Parameter | Definition | Example |
--------------------------------------|---------------------------------------------|-------------
| openlineage.circuitBreaker.type | Circuit breaker type selected | simpleMemory |
| openlineage.circuitBreaker.memoryThreshold | Memory threshold | 20 |
| openlineage.circuitBreaker.circuitCheckIntervalInMillis | Frequency of checking circuit breaker | 1000 |
| spark.openlineage.circuitBreaker.timeoutInSeconds | Optional timeout for OpenLineage execution (Since version 1.13) | 90 |
</TabItem>
</Tabs>
Expand All @@ -55,6 +60,8 @@ as GC threshold is computed since the previous circuit breaker call.
`circuitCheckIntervalInMillis` parameter is used
to configure a frequency circuit breaker is called.
Default value is `1000ms`, when no entry in config.
`timeoutInSeconds` is optional. If set, OpenLineage code execution is terminated when a timeout
is reached (added in version 1.13).

<Tabs groupId="integrations">
<TabItem value="yaml" label="Yaml Config">
Expand All @@ -65,6 +72,7 @@ circuitBreaker:
memoryThreshold: 20
gcCpuThreshold: 10
circuitCheckIntervalInMillis: 1000
timeoutInSeconds: 90
```
</TabItem>
<TabItem value="spark" label="Spark Config">
Expand All @@ -75,6 +83,7 @@ circuitBreaker:
| spark.openlineage.circuitBreaker.memoryThreshold | Memory threshold | 20 |
| spark.openlineage.circuitBreaker.gcCpuThreshold | Garbage Collection CPU threshold | 10 |
| spark.openlineage.circuitBreaker.circuitCheckIntervalInMillis | Frequency of checking circuit breaker | 1000 |
| spark.openlineage.circuitBreaker.timeoutInSeconds | Optional timeout for OpenLineage execution (Since version 1.13)| 90 |


</TabItem>
Expand All @@ -86,6 +95,7 @@ circuitBreaker:
| openlineage.circuitBreaker.memoryThreshold | Memory threshold | 20 |
| openlineage.circuitBreaker.gcCpuThreshold | Garbage Collection CPU threshold | 10 |
| openlineage.circuitBreaker.circuitCheckIntervalInMillis | Frequency of checking circuit breaker | 1000 |
| spark.openlineage.circuitBreaker.timeoutInSeconds | Optional timeout for OpenLineage execution (Since version 1.13) | 90 |


</TabItem>
Expand Down

0 comments on commit 95809f6

Please sign in to comment.