Releases: jina-ai/serve
💫 Patch v3.11.2
Release note
This release contains 1 hot fix.
🐞 Bug Fix
Unpin jina-hubble-sdk
To avoid conflicting dependency versions with docarray, unpin the jina-hubble-sdk version dependency
🤟 Contributors
Joan Fontanals (@JoanFM )
Contributors
Assets 2
💫 Release v3.11.0
Release Note (3.11.0)
Release time: 2022-10-24 14:42:35
This release contains 6 new features, 1 bug fix and 10 documentation improvements.
🆕 Features
Add OpenTelemetry tracing and metrics with basic configuration (#5175)
Jina now supports OpenTelemetry Tracing and Metrics libraries for increased observability and instrumentation of Jina Runtimes. Read the docs (#5291) for details. We also provide a migration guide for a smooth transition from the previously-supported Prometheus-only based metrics instrumentation.
You can integrate Jaeger or other distributed tracing tools to collect and visualize request-level and application level service operation attributes. This helps you analyze request-response lifecycle, application behavior and performance.
from jina import Executor, requests, DocumentArray
class MyExec(Executor):
@requests
def encode(self, docs: DocumentArray, **kwargs):
with self.tracer.start_as_current_span(
'encode', context=tracing_context
) as span:
with self.monitor(
'preprocessing_seconds', 'Time preprocessing the requests'
):
docs.tensors = preprocessing(docs)
with self.monitor(
'model_inference_seconds', 'Time doing inference the requests'
):
docs.embedding = model_inference(docs.tensors)Record existing Prometheus metrics into OpenTelemetry histograms (#5275)
Prometheus-only based metrics are also available as OpenTelemetry supported metrics. You can switch to OpenTelemetry metrics with little effort.
Add default tracing interceptors to head gRPC connection pool (#5271)
Head Runtime supports default traces when forwarding requests to the shard replica.
Add gRPC metadata to Executors and Gateway deployments (#5221)
You can now provide HTTP header access tokens as client metadata for accessing external and authentication protected services.
from jina import Client
client = Client(host='api.clip.jina.ai', port=2096, tls=True, protocol='grpc')
client.post(on='/encode', metadata=(('authorization', '<your access token>'),))Show local location of Executors in Hub (#5282)
Use the list sub command to list the locations of local Hub Executors:
jina hub list
Dump to Statefulset in K8s when volumes are passed to Executor (#5265)
Jina will generate a Kubernetes Statefulset manifest for Executors if you provide volumes to the Flow.to_kubernets_yaml() method. Executors can persist data on mounted volumes and correctly scale horizontally if required.
🐞 Bug Fixes
Invalid input raises exception (#5141)
- The
client.post()method was not raising an exception if any of the underlying async requests raised an exception. - The new version correctly raises the exception raised by any async request. The user can now correctly catch any exceptions and act upon the exception.
📗 Documentation Improvements
- Docs for OpenTelemetry instrumentation (#5291)
- Add Jina AI Cloud restructure Jcloud and Hub (#5298)
- Standardize naming conventions (#5285)
- Fix metrics naming (#5277)
- Misc clean up (#5255, #5287, #5284, #5290, #5289, #5288)
🤟 Contributors
We would like to thank all contributors to this release:
- AlaeddineAbdessalem (@alaeddine-13)
- Anthony Le (@AnthonyLe93)
- felix-wang (@numb3r3)
- Girish Chandrashekar (@girishc13)
- zhangkai (@floralatin)
- Joan Fontanals (@JoanFM)
- Johannes Messner (@JohannesMessner)
- samsja (@samsja)
- Han Xiao (@hanxiao)
- Alex Cureton-Griffiths (@alexcg1)
- Andrei Ungureanu (@Andrei997)
Contributors
Assets 2
💫 Patch v3.10.1
Release Note (3.10.1)
Release time: 2022-10-06 15:05:48
This release contains 1 new feature, 1 refactor, 3 bug fixes and 7 documentation improvements.
🆕 Features
Distributed replicas across different hosts (#5217)
You can now start different Executors on different machines and use them as replicas by passing their addresses in the host parameter as a list, together with the external flag set to True:
f.add(host='localhost:12345,91.198.174.192:12346', external=True)The Flow ensures the requests are load balanced between Executor instances.
⚙ Refactoring
Move Hubble related code to hubble-sdk (#5227)
All code related to hubble and communication with Jina Hub has been moved to the jina-hubble-sdk repo and has been added as an external dependency.
🐞 Bug Fixes
Disable rich traceback in logging (#5242)
We disabled the use of rich logging when printing tracebacks for Exceptions. This makes it easier to debug issues as the lines are not broken.
Previously, an Exception would be reported to the console as:
ERROR executor0/rep-0@183551 ZeroDivisionError('division by [10/05/22 11:26:23]
zero')
add "--quiet-error" to suppress the exception
details
╭──────── Traceback (most recent call last) ────────╮
│ /home/xxx/Documents/workspace/Jina/jina/jina/se… │
│ in process_data │
│ │
│ 179 │ │ │ │ if self.logger.debug_enable │
│ 180 │ │ │ │ │ self._log_data_request( │
│ 181 │ │ │ │ │
│ ❱ 182 │ │ │ │ result = await self._data_r │
│ 183 │ │ │ │ if self._successful_request │
│ 184 │ │ │ │ │ self._successful_reques │
│ 185 │ │ │ │ return result │
│ │
│ /home/xxx/Documents/workspace/Jina/jina/jina/se… │
│ in handle │
│ │
│ 161 │ │ ) │
│ 162 │ │ │
│ 163 │ │ # executor logic │
│ ❱ 164 │ │ return_data = await self._executor. │
│ 165 │ │ │ req_endpoint=requests[0].header │
│ 166 │ │ │ docs=docs, │
│ 167 │ │ │ parameters=params, │
│ │
│ /home/xxx/Documents/workspace/Jina/jina/jina/se… │
│ in __acall__ │
│ │
│ 290 │ │ if req_endpoint in self.requests: │
│ 291 │ │ │ return await self.__acall_endpo │
│ 292 │ │ elif __default_endpoint__ in self.r │
│ ❱ 293 │ │ │ return await self.__acall_endpo │
│ 294 │ │
│ 295 │ async def __acall_endpoint__(self, req_ │
│ 296 │ │ func = self.requests[req_endpoint] │
│ │
│ /home/xxx/Documents/workspace/Jina/jina/jina/se… │
│ in __acall_endpoint__ │
│ │
│ 311 │ │ │ if iscoroutinefunction(func): │
│ 312 │ │ │ │ return await func(self, **k │
│ 313 │ │ │ else: │
│ ❱ 314 │ │ │ │ return func(self, **kwargs) │
│ 315 │ │
│ 316 │ @property │
│ 317 │ def workspace(self) -> Optional[str]: │
│ │
│ /home/xxx/Documents/workspace/Jina/jina/jina/se… │
│ in arg_wrapper │
│ │
│ 159 │ │ │ │ def arg_wrapper( │
│ 160 │ │ │ │ │ executor_instance, *arg │
│ 161 │ │ │ │ ): # we need to get the su │
│ the self │
│ ❱ 162 │ │ │ │ │ return fn(executor_inst │
│ 163 │ │ │ │ │
│ 164 │ │ │ │ self.fn = arg_wrapper │
│ 165 │
│ │
│ /home/USER/.config/JetBrains/PyCharmCE2022.2/scr… │
│ in foo │
│ │
│ 12 │ │
│ 13 │ @requests │
│ 14 │ def foo(self, docs, **kwargs): │
│ ❱ 15 │ │ 1/0 │
│ 16 │
│ 17 with Flow().add(uses=MyExec) as f: │
│ 18 │
╰───────────────────────────────────────────────────╯
ZeroDivisionError: division by zero
Traceback (most recent call last):
File "/home/USER/.config/JetBrains/PyCharmCE2022.2/scratches/scratch_6.py", line 19, in <module>
f.search(inputs=[Document()])
File "/home/USER/Documents/workspace/Jina/jina/jina/clients/mixin.py", line 271, in post
return run_async(
File "/home/USER/Documents/workspace/Jina/jina/jina/helper.py", line 1334, in run_async
return asyncio.run(func(*args, **kwargs))
File "/home/USER/.pyenv/versions/3.9.10/lib/python3.9/asyncio/runners.py", line 44, in run
return loop.run_until_complete(main)
File "/home/USER/.pyenv/versions/3.9.10/lib/python3.9/asyncio/base_events.py", line 642, in run_until_complete
return future.result()
File "/home/USER/Documents/workspace/Jina/jina/jina/clients/mixin.py", line 262, in _get_results
async for resp in c._get_results(*args, **kwargs):
File "/home/USER/Documents/workspace/Jina/jina/jina/clients/base/grpc.py", line 122, in _get_results
callback_exec(
File "/home/USER/Documents/workspace/Jina/jina/jina/clients/helper.py", line 81, in callback_exec
raise BadServer(response.header)
jina.excepts.BadServer: request_id: "c50a57014fb948ccbd8065ada487a136"
status {
code: ERROR
description: "ZeroDivisionError(\'division by zero\')"
exception {
name: "ZeroDivisionError"
args: "division by zero"
stacks: "Traceback (most recent call last):\n"
stacks: " File \"/home/USER/Documents/workspace/Jina/jina/jina/serve/runtimes/worker/__init__.py\", line 182, in process_data\n result = await self._data_request_handler.handle(requests=requests)\n"
stacks: " File \"/home/USER/Documents/workspace/Jina/jina/jina/serve/runtimes/request_handlers/data_request_handler.py\", line 164, in handle\n return_data = await self._executor.__acall__(\n"
stacks: " File \"/home/USER/Documents/workspace/Jina/jina/jina/serve/executors/__init__.py\", line 293, in __acall__\n return await self.__acall_endpoint__(__default_endpoint__, **kwargs)\n"
stacks: " File \"/home/USER/Documents/workspace/Jina/jina/jina/serve/executors/__init__.py\", line 314, in __acall_endpoint__\n return func(self, **kwargs)\n"
stacks: " File \"/home/USER/Documents/workspace/Jina/jina/jina/serve/executors/decorators.py\", line 162, in arg_wrapper\n return fn(executor_instance, *args, **kwargs)\n"
stacks: " File \"/home/USER/.config/JetBrains/PyCharmCE2022.2/scratches/scratch_6.py\", line 15, in foo\n 1/0\n"
stacks: "ZeroDivisionError: division by zero\n"
executor: "MyExec"
}
}
exec_endpoint: "/search"
target_executor: ""
...
Contributors
Assets 2
💫 Release v3.10.0
Release Note (v3.10.0)
Release time: 2022-09-29 16:18:19
This release contains 10 new features, 9 bug fixes and 12 documentation improvements.
🆕 Features
Enable jina ping as readiness probe on local and Kubernetes (#5200)
This feature lets you check the readiness of an entire Flow or its individual components:
jina ping flow grpc://localhost:12345
jina ping executor localhost:12346
jina ping gateway grpc://localhost:12345Successful output:
INFO JINA@28600 readiness check succeeded 1 times!!! Unsuccessful output:
This command exits with code 1 when the readiness check is not successful.
Therefore, it is a good choice to use as a readiness probe for Executor and Gateway when
deployed in Kubernetes.
Add argument to specify exceptions that will exit the runtime (#5165)
You can specify which Exceptions should trigger termination of an affected Executor:
Flow().add(uses=MyExecutor, exit_on_exceptions: ['Exception', 'RuntimeException'])This ensures an Executor is not stuck in an unusable but technically alive state, and that the Executor can be restarted.
This is especially useful when Executor restarts are automated through Kubernetes.
Retry mechanism for client.post (#5176)
client.post() accepts max_attempts, initial_backoff, max_backoff and backoff_multiplier parameters to control the capacity to retry requests, when a transient connectivity error occurs, using an exponential backoff strategy:
client.post(
on='/',
inputs=docs,
max_attempts=5,
initial_backoff=0.1,
max_backoff=0.5,
backoff_multiplier=1.5
)Add monitoring of requests size everywhere (#5111)
The size of requests (jina_request_size_bytes) is now monitored everywhere: On every Executor, every Gateway, and every sharding Head.
Consequently, this new metric can now be accessed via Prometheus and Grafana.
JSON logging for JCloud (#5201)
It is now possible to create logs in JSON format, instead of the standard line-by-line text output.
Sample output:
{"created": 1663930387.5850368, "filename": "data_request_handler.py", "funcName": "_load_executor", "levelname": "DEBUG", "lineno": 98, "module": "data_request_handler", "msg": "<jina.serve.executors.BaseExecutor object at 0x7fc1951f78d0> is successfully loaded!", "name": "WorkerRuntime", "pathname": "/home/foo/jina/serve/runtimes/request_handlers/data_request_handler.py", "process": 13386, "processName": "Pod", "thread": 140469860276032, "threadName": "MainThread"}To enable this, set the environment variable 'JINA_LOG_CONFIG to 'json'.
Support list-like syntax to specify visible cuda devices (#5187)
You can now use device id's to specify visible CUDA devices that should be used in the round-robin GPU assignment:
`CUDA_VISIBLE_DEVICES=RR1,3`This will assign CUDA devices 1 and 3 to the different Executor replicas in a round-robin fashion.
Make jina hub push non-blocking (#5129)
jina hub push is now a non-blocking operation, meaning that the user does not have to wait for the entire push to be completed on the server side.
While the push is being processed in the background, its status can be checked with the following command:
jina hub status [<path_to_executor_folder>] [--id TASK_ID] [--verbose] [--replay]
Warning messages in jina hub push (#5156)
jina hub push can now display helpful warning messages, such as deprecation warnings:
Add duration info in events (#5157)
Telemetry data now also includes the duration of every event that is reported.
🐞 Bug Fixes
Fix compatibility with Protobuf Python backend (#5222)
This bug caused an exception when Python was used as a backend for Protobuf, rather than cpp or udp.
With this fix, Python can also be used as a backend for Protobuf.
Move significant params to sequential (#5205)
This bug (occasionally) caused docker.errors.NullResource: Resource ID was not provided when pulling from Jina Hub.
This was due to a faulty local caching implementation, which is now fixed.
Fix missing secret when logged-in user with --force-update (#5180)
This bug caused a failure for logged in users when force pushing to Jina Hub: Their secret would not be passed. This is now fixed and force push works as expected.
Use default gRPC parameters for gRPC connection pool connection (#5211)
We brought our gRPC client in line with the recommended configuration at https://github.com/grpc/grpc/blob/master/doc/keepalive.md
Bump DocArray version for new column syntax (#5171)
Some changes to the column syntax in DocArray required a version bump.
Increase minimum Protobuf version (#5166)
Some changes in the Protobuf library required a version bump.
Remove leftover prints (#5199)
Some unwanted print statements had sneaked their way into our codebase, and we kicked them right back out!
📗 Documentation Improvements
- Change setenv into environ (#5223)
- Mention prefetch (#5220)
- Fix install instructions docker (#5213)
- Fix grammar, wording, punctuation (#5209)
- Fix grammar, punctuation (#5208)
- Clarify exec endpoint usage in http (#5202)
- Document grpc client limitation (#5193)
- Update faq and lifetime (#5191)
- Improve warning about config file in custom docker (#5190)
- Clarify return behavior in parameters (#5179)
- Add section for exit_on_exceptions argument (#5172)
- Labels in flow yaml (#5164)
🤟 Contributors
We would like to thank all contributors to this release:
- AlaeddineAbdessalem (@alaeddine-13)
- Girish Chandrashekar (@girishc13)
- Andrei Ungureanu (@Andrei997)
- Michael Günther (@guenthermi)
- Deepankar Mahapatro (@deepankarm)
- Han Xiao (@hanxiao)
- zhangkai (@floralatin)
- Joan Fontanals (@JoanFM)
- Alex Cureton-Griffiths (@alexcg1)
- Yanlong Wang (@nomagick)
- Nikolas Pitsillos (@npitsillos)
- samsja (@samsja)
Contributors
Assets 2
💫 Patch v3.9.3
Release Note (3.9.3)
Release time: 2022-09-29 16:05:04
🙇 We'd like to thank all contributors for this new release! In particular,
Joan Fontanals, AlaeddineAbdessalem, Michael Günther, Yanlong Wang, samsja, Alex Cureton-Griffiths, Deepankar Mahapatro, Nikolas Pitsillos, Han Xiao, zhangkai, Jina Dev Bot, 🙇
🆕 New Features
- [
11533962] - add gateway option to jina ping and use ping in readinessProbe (#5200) (Joan Fontanals) - [
d0838d37] - add multiple attempts options to client.post API (#5176) (Joan Fontanals) - [
13edf908] - use default grpc parameters for grpc connection pool connection (#5211) (samsja) - [
216b4bf0] - monitoring: add monitoring of requests size in bytes at all level (#5111) (samsja) - [
737875f5] - logs: json logging for jcloud (#5201) (Deepankar Mahapatro) - [
fd4c0347] - support list-like syntax to round robin CUDA devices (#5187) (Joan Fontanals) - [
9dff4c88] - add duration info in events (#5157) (Joan Fontanals)
🐞 Bug fixes
- [
3683fccf] - fix compatibility with protobuf python backend (#5222) (AlaeddineAbdessalem) - [
e932d6f9] - hubio.fetch_meta: move significant params to sequential (#5205) (Yanlong Wang) - [
60e9b8de] - remove leftover prints (#5199) (Joan Fontanals) - [
da186a23] - provide logger and streamer (#5188) (AlaeddineAbdessalem) - [
7b5c0316] - fix get-openapi-schemas script (#5185) (AlaeddineAbdessalem) - [
caf4a3d6] - fix missing secret when logged-in user with --force-update and … (#5180) (zhangkai)
🧼 Code Refactoring
- [
9a6079ef] - separate gateway and asyncio runtime readiness checks (#5224) (AlaeddineAbdessalem) - [
243639dd] - extract gateway app logic into custom gateway class (#5153) (AlaeddineAbdessalem)
📗 Documentation
- [
147317db] - change setenv into environ (#5223) (Michael Günther) - [
ad37bdd2] - mention prefetch (#5220) (Joan Fontanals) - [
b9c8e44e] - fix install instructions docker (#5213) (Joan Fontanals) - [
7cbf3471] - what-is-jina: fix grammar, wording, punctuation (#5209) (Alex Cureton-Griffiths) - [
7c7d2cb0] - what-is-modality: fix grammar, punctuation (#5208) (Alex Cureton-Griffiths) - [
1399e36c] - clarify exec endpoint usage in http (#5202) (Joan Fontanals) - [
842a585b] - document grpc client limitation (#5193) (AlaeddineAbdessalem) - [
96c5e7c1] - jcloud: update faq and lifetime (#5191) (Nikolas Pitsillos) - [
c17e6416] - improve warning about config file in custom docker (#5190) (Joan Fontanals) - [
22330fcb] - clarify return behavior in parameters (#5179) (Joan Fontanals)
🍹 Other Improvements
- [
b5977f47] - update contributing to remove labeling (#5225) (Joan Fontanals) - [
45c7c6e1] - update pull request template link (#5214) (Joan Fontanals) - [
1eab9ce6] - fix doc template (Han Xiao) - [
8c2d0758] - docs: update TOC (Jina Dev Bot) - [
139b068e] - version: the next version will be 3.9.3 (Jina Dev Bot)
💫 Patch v3.9.2
Release Note (3.9.2)
Release time: 2022-09-15 14:00:36
🙇 We'd like to thank all contributors for this new release! In particular,
Girish Chandrashekar, zhangkai, AlaeddineAbdessalem, Joan Fontanals, Jina Dev Bot, Andrei Ungureanu, Deepankar Mahapatro, 🙇
🆕 New Features
- [
c51f9014] - hubble async push (#5129) (zhangkai) - [
3f39ed46] - runtime: add argument to specify exceptions that will exit the runtime (#5165) (Girish Chandrashekar) - [
27e1f779] - hubio: display warning messages from hubble request (jina hub push) (#5156) (Andrei Ungureanu)
🐞 Bug fixes
- [
29ad1750] - exit on exception args only applies to executors (#5169) (AlaeddineAbdessalem) - [
55188165] - pin docarray version for new column syntax (#5171) (Joan Fontanals) - [
fa83955c] - increase minimum protobuf version (#5166) (AlaeddineAbdessalem)
📗 Documentation
- [
28aeac8e] - add section for exit_on_exceptions argument (#5172) (Girish Chandrashekar) - [
33891c46] - jcloud: labels in flow yaml (#5164) (Deepankar Mahapatro)
🏁 Unit Test and CICD
🍹 Other Improvements
💫 Patch v3.9.1
Release Note (3.9.1)
Release time: 2022-09-08 14:10:44
🙇 We'd like to thank all contributors for this new release! In particular,
Han Xiao, Jina Dev Bot, 🙇
🧼 Code Refactoring
🍹 Other Improvements
- [
0a71009b] - version: the next version will be 3.9.1 (Jina Dev Bot)
💫 Release v3.9.0
Highlights 🌟
Update to new protobuf version
We have updated our protobuf dependency to use the latest protobuf version leveraging all the performance improvement that comes with it.
Merge manifest.yml and config.yml for Jina Hub Executors
We have made it easier for the users to manage their Hub Executors by merging the information from manifest.yml to config.yml which makes the experience much better.
Expose grpc server parameters to Executor
We expose all the parameters that grpc server can accept for users to be able to configure their services in the most robust and reliable way.
Other changes
- Add more Dockerfile templates options in
jina hub new#5104 - Add failed and successfull requests metrics #5079
Bug fixes
- Fix reading empty list from environment variables #5143
💫 Patch v3.8.4
Release Note (3.8.4)
Release time: 2022-09-08 08:51:32
🙇 We'd like to thank all contributors for this new release! In particular,
Deepankar Mahapatro, AlaeddineAbdessalem, Joan Fontanals, Johannes Messner, Andrei Ungureanu, samsja, Jina Dev Bot, 🙇
🆕 New Features
🐞 Bug fixes
- [
e501ddfd] - version: adapt to docker hub v2 (#5146) (Deepankar Mahapatro) - [
d76e69ac] - fix parsing empty list (#5143) (AlaeddineAbdessalem) - [
3ec33c08] - gpu dockerfile template include now jina install (#5124) (samsja) - [
e2967f98] - set minimal docarray dependency (#5133) (Joan Fontanals) - [
ff803686] - linkerd cd (#5131) (AlaeddineAbdessalem)
🧼 Code Refactoring
🏁 Unit Test and CICD
- [
ddf90efb] - remove outdated test (#5139) (Johannes Messner) - [
a4ea7215] - increase wait time (#5137) (AlaeddineAbdessalem) - [
478e53f6] - fix k8s tests return responses params (#5127) (Joan Fontanals)
🍹 Other Improvements
💫 Patch v3.8.3
Release Note (3.8.3)
Release time: 2022-09-05 14:48:30
🙇 We'd like to thank all contributors for this new release! In particular,
AlaeddineAbdessalem, Jina Dev Bot, 🙇