This repository was archived by the owner on Jan 29, 2024. It is now read-only.
This repository was archived by the owner on Jan 29, 2024. It is now read-only.
Low throughput #11
Open
Description
I'm trying to use the library to process messages in the subscription with big number of messages. I have this code:
#include <chrono>
#include <memory>
#include <list>
#include <spdlog/spdlog.h>
#include <spdlog/sinks/stdout_sinks.h>
#include "pubsuber/Pubsuber.h"
#include "base/logging.h"
using namespace std::chrono;
int main() {
pubsuber::ClientOptions opts = pubsuber::ClientOptions::CreateDefault("my-team");
auto sink = std::make_shared<spdlog::sinks::stdout_sink_mt>();
sink->set_pattern("%D-%T.%f [%n] [%t] [%L]: %v");
opts.SetLogSink(sink);
opts.SetLogLevel(spdlog::level::info);
pubsuber::ClientPtr client;
pubsuber::RetryCountPolicy countPolicy{4};
pubsuber::MaxRetryTimePolicy timePolicy{30s};
pubsuber::ExponentialBackoffPolicy backoffPolicy{250ms, 3s, 2.0};
client = pubsuber::Client::Create(std::move(opts), std::move(timePolicy), std::move(countPolicy), std::move(backoffPolicy));
client->SetLogLevel(spdlog::level::info);
auto subscription = client->GetSubscription("MY-CHANNEL");
std::list<pubsuber::MessagePtr> messages;
std::mutex mmm;
size_t count = 0;
subscription->Receive([&messages, &count, &mmm](pubsuber::MessagePtr&& m) {
std::unique_lock l(mmm);
if (++count % 1000 == 0) {
LOG(Info, "count: " << count << " message_size: " << m->Payload().size());
messages.clear();
}
messages.push_back(std::move(m));
messages.back()->Ack();
});
while (true) {}
return 0;
}
When I start the application there is a long queue of unacked messages (~100K) in the subscription.
I see that throughput doesn't allow to read them fast: I see throughput 1K messages in a minute.
Is this an expected speed to process messages? Is it possible to setup library to get a higher throughput?
Metadata
Metadata
Assignees
Labels
No labels