Kafka C++ client taking a long time to receive a message - c++

Question

I am using the cppkafka library, a wrapper of the librdkafka, in turn a C++ Kafka client for a very simple message streaming task. My consumer class is behaving weiredly, because it takes a rather long time to receive a message. More precisely, every time the receiving executable is run and kept running, the consumer can receive the first batch of messages correctly, but subsequent messages will take roughly 15 seconds to arrive. Anyone understand what possibility can lead to something like this (kafka configurations, library specific problems or my stupid faults)? A million thanks.
My receving thread is as follows
configuration_.set("group.id", 0);
consumer_ = std::make_unique<cppkafka::Consumer>(configuration_);
consumer_->subscribe({TopicTraits<trade::OrderRequest>::topic, TopicTraits<trade::CancelRequest>::topic});
std::thread([this] {
while (working_) {
cppkafka::Message msg = consumer_->poll();
if (msg) {
if (msg.get_error()) {
if (!msg.is_eof()) {
ERROR("error occurred while polling message: {}", msg.get_error());
}
} else {
try {
Json j = Json::parse(msg.get_payload());
if (msg.get_topic() == TopicTraits<trade::OrderRequest>::topic) {
INFO("received [order_req], {}", msg.get_payload());
ReceiveOrderRequest(j.get<trade::OrderRequest>());
} else if (msg.get_topic() == TopicTraits<trade::CancelRequest>::topic) {
INFO("received [cancel_req], {}", msg.get_payload());
ReceiveCancelRequest(j.get<trade::CancelRequest>());
}
} catch (const std::exception &e) {
ERROR("error occurred while handling incoming message, {}", e.what());
}
}
}
}
}).detach();

Answer 1

Two consumers with the same group id subscribing to different topics blocked poll()
After some research, I found the problem related to one of the more fundamental configuration options of kafka. The problem is that my consumer was blocked in the call to poll(), and the direct cause of it is two consumers with the same group id subscribing to differing topics. I reassigned the group id and the problem vanished.