-
Notifications
You must be signed in to change notification settings - Fork 17
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
Showing
2 changed files
with
275 additions
and
6 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -7,7 +7,7 @@ | |
msgid "" | ||
msgstr "" | ||
"Project-Id-Version: PACKAGE VERSION\n" | ||
"POT-Creation-Date: 2023-10-02 08:09+0000\n" | ||
"POT-Creation-Date: 2023-10-09 15:15+0000\n" | ||
"PO-Revision-Date: YEAR-MO-DA HO:MI+ZONE\n" | ||
"Last-Translator: FULL NAME <EMAIL@ADDRESS>\n" | ||
"Language-Team: LANGUAGE <[email protected]>\n" | ||
|
@@ -587,30 +587,35 @@ msgstr "" | |
|
||
#. type: Plain text | ||
#: upstream/_posts/2023-09-19-virtual-thread-1.adoc:361 | ||
msgid "How to build a native executable when using virtual threads (_to be published_)" | ||
msgid "https://quarkus.io/blog/virtual-threads-4/[How to process Kafka messages using virtual threads]" | ||
msgstr "" | ||
|
||
#. type: Plain text | ||
#: upstream/_posts/2023-09-19-virtual-thread-1.adoc:362 | ||
msgid "How to containerize an application using virtual threads (in JVM mode) (_to be published_)" | ||
msgid "How to build a native executable when using virtual threads (_to be published_)" | ||
msgstr "" | ||
|
||
#. type: Plain text | ||
#: upstream/_posts/2023-09-19-virtual-thread-1.adoc:363 | ||
msgid "How to containerize an application using virtual threads in native mode (_to be published_)" | ||
msgid "How to containerize an application using virtual threads (in JVM mode) (_to be published_)" | ||
msgstr "" | ||
|
||
#. type: Plain text | ||
#: upstream/_posts/2023-09-19-virtual-thread-1.adoc:364 | ||
msgid "How to process Kafka messages using virtual threads (_to be published_)" | ||
msgid "How to containerize an application using virtual threads in native mode (_to be published_)" | ||
msgstr "" | ||
|
||
#. type: Plain text | ||
#: upstream/_posts/2023-09-19-virtual-thread-1.adoc:365 | ||
msgid "How to process Kafka messages using virtual threads (_to be published_)" | ||
msgstr "" | ||
|
||
#. type: Plain text | ||
#: upstream/_posts/2023-09-19-virtual-thread-1.adoc:366 | ||
msgid "What are we exploring to improve the virtual thread support in Quarkus (_to be published_)" | ||
msgstr "" | ||
|
||
#. type: Plain text | ||
#: upstream/_posts/2023-09-19-virtual-thread-1.adoc:367 | ||
#: upstream/_posts/2023-09-19-virtual-thread-1.adoc:368 | ||
msgid "To know more about the virtual thread support in Quarkus, check the https://quarkus.io/guides/virtual-threads[Virtual thread reference guide]." | ||
msgstr "" |
264 changes: 264 additions & 0 deletions
264
l10n/po/zh_CN/_posts/2023-09-30-virtual-threads-4.adoc.po
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,264 @@ | ||
# SOME DESCRIPTIVE TITLE | ||
# Copyright (C) YEAR Free Software Foundation, Inc. | ||
# This file is distributed under the same license as the PACKAGE package. | ||
# FIRST AUTHOR <EMAIL@ADDRESS>, YEAR. | ||
# | ||
#, fuzzy | ||
msgid "" | ||
msgstr "" | ||
"Project-Id-Version: PACKAGE VERSION\n" | ||
"POT-Creation-Date: 2023-10-09 15:15+0000\n" | ||
"PO-Revision-Date: YEAR-MO-DA HO:MI+ZONE\n" | ||
"Last-Translator: FULL NAME <EMAIL@ADDRESS>\n" | ||
"Language-Team: LANGUAGE <[email protected]>\n" | ||
"Language: \n" | ||
"MIME-Version: 1.0\n" | ||
"Content-Type: text/plain; charset=UTF-8\n" | ||
"Content-Transfer-Encoding: 8bit\n" | ||
|
||
#. type: YAML Front Matter: author | ||
#: upstream/_posts/2023-09-30-virtual-threads-4.adoc:1 | ||
#, no-wrap | ||
msgid "cescoffier" | ||
msgstr "" | ||
|
||
#. type: YAML Front Matter: date | ||
#: upstream/_posts/2023-09-30-virtual-threads-4.adoc:1 | ||
#, no-wrap | ||
msgid "2023-10-09" | ||
msgstr "" | ||
|
||
#. type: YAML Front Matter: layout | ||
#: upstream/_posts/2023-09-30-virtual-threads-4.adoc:1 | ||
#, no-wrap | ||
msgid "post" | ||
msgstr "" | ||
|
||
#. type: YAML Front Matter: synopsis | ||
#: upstream/_posts/2023-09-30-virtual-threads-4.adoc:1 | ||
#, no-wrap | ||
msgid "Learn about the virtual threads integration in Quarkus messaging (Kafka, AMQP, Pulsar...)." | ||
msgstr "" | ||
|
||
#. type: YAML Front Matter: tags | ||
#: upstream/_posts/2023-09-30-virtual-threads-4.adoc:1 | ||
#, no-wrap | ||
msgid "virtual-threads reactive redis kafka messaging" | ||
msgstr "" | ||
|
||
#. type: YAML Front Matter: title | ||
#: upstream/_posts/2023-09-30-virtual-threads-4.adoc:1 | ||
#, no-wrap | ||
msgid "Processing Kafka records on virtual threads" | ||
msgstr "" | ||
|
||
#. type: Plain text | ||
#: upstream/_posts/2023-09-30-virtual-threads-4.adoc:15 | ||
msgid "In https://quarkus.io/blog/virtual-threads-2/[another blog post], we have seen how you can implement a CRUD application with Quarkus to utilize virtual threads. The virtual threads support in Quarkus is not limited to REST and HTTP. Many other parts support virtual threads, such as gRPC, scheduled tasks, and messaging. In this post, we will see how you can process Kafka records on virtual threads, increasing the concurrency of the processing." | ||
msgstr "" | ||
|
||
#. type: Plain text | ||
#: upstream/_posts/2023-09-30-virtual-threads-4.adoc:17 | ||
msgid "## Processing messages on virtual threads" | ||
msgstr "" | ||
|
||
#. type: Plain text | ||
#: upstream/_posts/2023-09-30-virtual-threads-4.adoc:20 | ||
msgid "The Quarkus Reactive Messaging extension supports virtual threads. Similarly to HTTP, to execute the processing on a virtual thread, you need to use the `@RunOnVirtualThread` annotation:" | ||
msgstr "" | ||
|
||
#. type: delimited block - | ||
#: upstream/_posts/2023-09-30-virtual-threads-4.adoc:29 | ||
#, no-wrap | ||
msgid "" | ||
"@Incoming(\"input-channel\")\n" | ||
"@Outgoing(\"output-channel\")\n" | ||
"@RunOnVirtualThread\n" | ||
"public Fraud detect(Transaction tx) {\n" | ||
" // Run on a virtual thread\n" | ||
"}\n" | ||
msgstr "" | ||
|
||
#. type: Plain text | ||
#: upstream/_posts/2023-09-30-virtual-threads-4.adoc:33 | ||
msgid "The processing of each message runs on separate virtual threads. So, for each message from the `input-channel`, a new virtual thread is created (as seen in https://quarkus.io/blog/virtual-thread-1/[this blog post], virtual thread creation is cheap)." | ||
msgstr "" | ||
|
||
#. type: Positional ($1) AttributeList argument for macro 'image' | ||
#: upstream/_posts/2023-09-30-virtual-threads-4.adoc:34 | ||
#, no-wrap | ||
msgid "Threading model of the messaging application" | ||
msgstr "" | ||
|
||
#. type: Target for macro image | ||
#: upstream/_posts/2023-09-30-virtual-threads-4.adoc:34 | ||
#, no-wrap | ||
msgid "virtual-thread-messaging.png" | ||
msgstr "" | ||
|
||
#. type: Plain text | ||
#: upstream/_posts/2023-09-30-virtual-threads-4.adoc:39 | ||
msgid "This execution model can be used with any Quarkus reactive messaging connector, including AMQP 1.0, Apache Pulsar, and Apache Kafka. The concurrency of this processing is no longer limited by the number of worker threads, as it would with the `@Blocking` annotation. Thus, this novel execution model simplifies the development of highly concurrent data streaming applications." | ||
msgstr "" | ||
|
||
#. type: Plain text | ||
#: upstream/_posts/2023-09-30-virtual-threads-4.adoc:44 | ||
msgid "As we will see later, such high-level concurrency can cause problems. To keep this concurrency controllable, Quarkus limits the number of concurrent message processing to `1024` (This default value is https://quarkus.io/guides/messaging-virtual-threads[configurable]). One of the main benefits of this limit is preventing the application from polling millions of messages, which would be very expensive in terms of memory. Without this limit, a Kafka application would poll all the records from the assigned topics-partitions and consume a large amount of memory." | ||
msgstr "" | ||
|
||
#. type: Plain text | ||
#: upstream/_posts/2023-09-30-virtual-threads-4.adoc:50 | ||
msgid "Also, you may wonder why we do not use virtual threads by default. The reasons have been explained in https://quarkus.io/blog/virtual-thread-1/#five-things-you-need-to-know-before-using-virtual-threads-for-everything[a previous blog post]. There are limitations that can make virtual threads dangerous. You need to make sure your virtual threads usage is safe before using it. We will see a few examples in this post." | ||
msgstr "" | ||
|
||
#. type: Plain text | ||
#: upstream/_posts/2023-09-30-virtual-threads-4.adoc:52 | ||
msgid "## Processing Kafka records on virtual threads" | ||
msgstr "" | ||
|
||
#. type: Plain text | ||
#: upstream/_posts/2023-09-30-virtual-threads-4.adoc:60 | ||
msgid "To illustrate how to process Kafka records on virtual threads, let's consider a simple application. This application is a fake fraud detector. It analyzes banking transactions, and if the transaction amount for a given account in a given period of time reaches a threshold, we consider there is fraud. The code is available in this https://github.com/quarkusio/virtual-threads-demos/tree/main/kafka-example[GitHub repository]. Of course, you can use more complex detection algorithms, and even use AI/ML. In this case, we use the https://redis.io/docs/data-types/timeseries/[Redis time series] commands inefficiently to introduce more I/O than necessary. It is done purposefully to utilize the virtual thread's ability to block:" | ||
msgstr "" | ||
|
||
#. type: delimited block - | ||
#: upstream/_posts/2023-09-30-virtual-threads-4.adoc:68 | ||
#, no-wrap | ||
msgid "" | ||
"@Incoming(\"tx\")\n" | ||
"@Outgoing(\"frauds\")\n" | ||
"@RunOnVirtualThread\n" | ||
"public Fraud detect(Transaction tx) {\n" | ||
" String key = \"account:transactions:\" + tx.account;\n" | ||
msgstr "" | ||
|
||
#. type: delimited block - | ||
#: upstream/_posts/2023-09-30-virtual-threads-4.adoc:73 | ||
#, no-wrap | ||
msgid "" | ||
" // Add sample\n" | ||
" long timestamp = tx.date.toInstant(ZoneOffset.UTC).toEpochMilli();\n" | ||
" timeseries.tsAdd(key, timestamp, tx.amount, new AddArgs()\n" | ||
" .onDuplicate(DuplicatePolicy.SUM));\n" | ||
msgstr "" | ||
|
||
#. type: delimited block - | ||
#: upstream/_posts/2023-09-30-virtual-threads-4.adoc:79 | ||
#, no-wrap | ||
msgid "" | ||
" // Retrieve the last sum.\n" | ||
" var range = timeseries.tsRevRange(key, TimeSeriesRange.fromTimeSeries(),\n" | ||
" // 1 min for demo purpose.\n" | ||
" new RangeArgs().aggregation(Aggregation.SUM, Duration.ofMinutes(1))\n" | ||
" .count(1));\n" | ||
msgstr "" | ||
|
||
#. type: delimited block - | ||
#: upstream/_posts/2023-09-30-virtual-threads-4.adoc:91 | ||
#, no-wrap | ||
msgid "" | ||
" if (!range.isEmpty()) {\n" | ||
" // Analysis\n" | ||
" var sum = range.get(0).value;\n" | ||
" if (sum > 10_000) {\n" | ||
" Log.warnf(\"Fraud detected for account %s: %.2f\", tx.account, sum);\n" | ||
" return new Fraud(tx.account, sum);\n" | ||
" }\n" | ||
" }\n" | ||
" return null;\n" | ||
"}\n" | ||
"---- \n" | ||
msgstr "" | ||
|
||
#. type: delimited block - | ||
#: upstream/_posts/2023-09-30-virtual-threads-4.adoc:98 | ||
#, no-wrap | ||
msgid "" | ||
"If you run this application and have a burst of transactions, it will not work. \n" | ||
"The processing is correctly executed on virtual threads. \n" | ||
"However, the Redis connection pool has not been tuned to handle that concurrency level. \n" | ||
"Very quickly, no Redis connections are available, and it starts enqueuing the commands into a waiting list. \n" | ||
"When this queue is full, it starts rejecting the commands. \n" | ||
"Fortunately, you can configure the max size of the waiting queue with:\n" | ||
msgstr "" | ||
|
||
#. type: delimited block - | ||
#: upstream/_posts/2023-09-30-virtual-threads-4.adoc:100 | ||
#, no-wrap | ||
msgid "[source, properties]\n" | ||
msgstr "" | ||
|
||
#. type: Plain text | ||
#: upstream/_posts/2023-09-30-virtual-threads-4.adoc:104 | ||
msgid "# Increase Redis pool size (and waiting queue size) as we will have a lot of concurrency quarkus.redis.max-pool-size=100 # Number of connection in the pool quarkus.redis.max-pool-waiting=10000 # Waiting queue max size" | ||
msgstr "" | ||
|
||
#. type: delimited block - | ||
#: upstream/_posts/2023-09-30-virtual-threads-4.adoc:108 | ||
#, no-wrap | ||
msgid "" | ||
"While we use Redis in this application, you will face identical problems with many other clients (including HTTP clients). \n" | ||
"So, configure them properly to handle this new level of concurrency.\n" | ||
msgstr "" | ||
|
||
#. type: delimited block - | ||
#: upstream/_posts/2023-09-30-virtual-threads-4.adoc:110 | ||
#, no-wrap | ||
msgid "If you run the application and open the UI, you will see that the concurrency reached a maximum of 1024, as expected.\n" | ||
msgstr "" | ||
|
||
#. type: delimited block - | ||
#: upstream/_posts/2023-09-30-virtual-threads-4.adoc:112 | ||
#, no-wrap | ||
msgid "image::fraud-detection-screenshot.png[The application reached 1024 as top concurrency,800,float=\"right\",align=\"center\"]\n" | ||
msgstr "" | ||
|
||
#. type: delimited block - | ||
#: upstream/_posts/2023-09-30-virtual-threads-4.adoc:114 | ||
#, no-wrap | ||
msgid "## A note about pinning and monopolization\n" | ||
msgstr "" | ||
|
||
#. type: delimited block - | ||
#: upstream/_posts/2023-09-30-virtual-threads-4.adoc:118 | ||
#, no-wrap | ||
msgid "" | ||
"Our messaging connectors have been tailored to avoid pinning. \n" | ||
"It is also the case for the Quarkus Redis client. \n" | ||
"Thus, this application does not pin the carrier thread.\n" | ||
msgstr "" | ||
|
||
#. type: delimited block - | ||
#: upstream/_posts/2023-09-30-virtual-threads-4.adoc:125 | ||
#, no-wrap | ||
msgid "" | ||
"But pinning is not the only problem that can arise.\n" | ||
"While virtual threads can be appealing, you must be careful not to monopolize the carrier thread.\n" | ||
"If, for example, you implemented a complex and CPU-intensive detection algorithm instead of relying on Redis, you would likely monopolize the carrier thread, defeating the purpose of virtual threads.\n" | ||
"It will force the JVM to create new carrier threads, ultimately increasing memory usage. \n" | ||
"The JVM will limit the number of created carrier threads. \n" | ||
"When this happens, your application will under-perform as your tasks will be enqueued until a carrier thread is available.\n" | ||
msgstr "" | ||
|
||
#. type: delimited block - | ||
#: upstream/_posts/2023-09-30-virtual-threads-4.adoc:127 | ||
#, no-wrap | ||
msgid "## Summary\n" | ||
msgstr "" | ||
|
||
#. type: delimited block - | ||
#: upstream/_posts/2023-09-30-virtual-threads-4.adoc:131 | ||
#, no-wrap | ||
msgid "" | ||
"This post explains how you can execute message processing on virtual threads.\n" | ||
"While the example uses Kafka, you can use the same approach with the other messaging connectors provided by Quarkus. \n" | ||
"Do not forget that such kind of application:\n" | ||
msgstr "" | ||
|
||
#. type: delimited block - | ||
#: upstream/_posts/2023-09-30-virtual-threads-4.adoc:133 | ||
#, no-wrap | ||
msgid "" | ||
"* requires tuning connection pools, as the concurrency is much higher than before\n" | ||
"* can lead to monopolization if your processing is CPU-intensive\n" | ||
msgstr "" |