Why thread stuck? [closed] - c++

Question

Closed. This question needs details or clarity. It is not currently accepting answers.
Want to improve this question? Add details and clarify the problem by editing this post.
Closed last year.
Improve this question
I'm implementing a multiple threads download manager on Windows using C++.
The main thread starts download manage thread M, M starts several download threads D.
Each D will do the HTTP data transfer using library cpr which is a wrapper around libcurl.
After starts some D threads, M enters a loop, keep watching the download progress.
The strange thing: once the second D started, or I abort the first D by return error code(an integer other then 0, I return 1.) from the libcurl's CURLOPT_XFERINFOFUNCTION callback, M's loop will stop. (There's one debugging output inside M's loop. I notice its stop by the disappear of that output from console. Maybe it's not stop, just going into some waiting state instead...)
Both M and D thread are started by STL's std::thread.
Have been scratched by this problem a whole day. Any clue will be appreciated...
This is the M thread entrance:
void HttpDownloader1::MasterThreadFunc_()
{
int loop_count = 0;
// start the first download thread
if (!SplitDownload_(nullptr))
{
status_ = Status::ERRONEOUS;
return;
}
uint64_t prev_downloaded_bytes = recorder_->GetDownloadedBytes();
for (loop_count = 0; download_threads_.size() > 0; loop_count++)
//while (true)
{
loop_count++;
#ifdef _DEBUG
Debug_("main loop, threads: " + std::to_string(download_threads_.size()));
#endif
std::this_thread::sleep_for(std::chrono::milliseconds(500)); // 0.5s
DownloadThread* splitable_segment = nullptr;
auto it = download_threads_.begin();
while (it != download_threads_.end())
{
DownloadThread* thread = *it;
if (thread->status_ == Status::FINISH)
{
delete thread;
it = download_threads_.erase(it);
}
else
{
switch (thread->status_)
{
case Status::RUNNING:
default:
recorder_->MarkFinish(thread->begin_, thread->pos_ - 1);
// part of this segment may have been splited to other download thread
//thread->end_ = recorder_->GetSegmentEnd(thread->pos_);
break;
case Status::SUCCESSFUL:
if (recorder_->IsInitialized())
{
thread->CloseFile();
recorder_->MarkFinish(thread->begin_, thread->pos_ - 1);
}
else
{
if (!PrepareFile_(*it))
status_ = Status::ERRONEOUS;
}
splitable_segment = *it;
break;
case Status::ERRONEOUS:
if (++retry_ > kMaxHttpRetry)
{
status_ = Status::ERRONEOUS;
}
else
{
thread->CloseFile();
recorder_->MarkFailed(thread->pos_, thread->end_);
splitable_segment = *it;
}
break;
}
it++;
}
}
// break out if error occured
if (status_ == Status::ERRONEOUS)
{
break;
}
// if download completed
if (recorder_->IsFinish())
{
status_ = Status::SUCCESSFUL;
break;
}
// calculate download speed every 1 second
if ((loop_count & 1) == 1)
{
auto bytes = recorder_->GetDownloadedBytes();
bytes_per_second_ = bytes - prev_downloaded_bytes;
prev_downloaded_bytes = bytes;
}
// save progress info every 2 seconds
if ((loop_count & 3) == 3)
{
recorder_->Save();
}
// split download when any thread is available or every 4 seconds
if (splitable_segment || (loop_count & 7) == 7)
{
if (splitable_segment != nullptr)
SplitDownload_(splitable_segment);
else if (download_threads_.size() < max_threads_)
SplitDownload_(nullptr);
}
}
master_thread_.detach();
status_ = status_ != Status::ERRONEOUS ? Status::SUCCESSFUL : status_;
}
This is how M start D thread:
bool HttpDownloader1::SplitDownload_(DownloadThread* thread)
{
if (!recorder_->IsInitialized())
{
if (!thread)
thread = CreateDownloadThread_();
thread->begin_ = 0;
thread->end_ = 0;
}
else
{
int64_t begin, end;
if (recorder_->GetTask(&begin, &end))
{
// initialize this segment
if (!thread)
thread = CreateDownloadThread_();
thread->begin_ = begin;
thread->end_ = end;
thread->pos_ = thread->begin_;
if (thread->file_ == nullptr)
{
//errno_t e = fopen_s(&thread->file_, target_.GetPath().c_str(), "rb+");
thread->file_ = fopen(target_.GetPath().c_str(), "rb+");
//if (e == 0 && thread->file_)
if (thread->file_)
{
fseek(thread->file_, (long)thread->begin_, SEEK_SET);
}
else
{
thread->status_ = Status::ERRONEOUS;
return false;
}
}
}
else
{
// no more segment to download or split, remove this thread if it exists.
if (thread)
thread->status_ = Status::FINISH;
}
}
if (thread && thread->status_ != Status::FINISH)
{
thread->status_ = Status::RUNNING;
thread->thread_ = std::thread(&HttpDownloader1::DownloadThreadFunc_, this, thread);
thread->thread_.detach();
}
return true;
}
This is the D thread entrance:
void HttpDownloader1::DownloadThreadFunc_(DownloadThread* thread)
{
cpr::Response rsp;
if (thread->file_ == nullptr)
{
rsp = cpr::Get(
cpr::Url(target_.url_.c_str()),
cpr::ConnectTimeout(std::chrono::seconds(kConnectionTimeout)),
cpr::Timeout(std::chrono::seconds(kTransmitTimeout)),
cpr::VerifySsl(false),
cpr::Header{ { "Range", thread->GetRangeHeaderString().c_str() } },
cpr::CurlOption({ CURLOPT_NOPROGRESS, 1 }),
cpr::CurlOption({ CURLOPT_WRITEDATA, nullptr }),
cpr::CurlOption({ CURLOPT_WRITEFUNCTION, &HttpDownloader1::WriteCallback_ })
);
}
else
{
rsp = cpr::Get(
cpr::Url(target_.url_.c_str()),
cpr::ConnectTimeout(std::chrono::seconds(kConnectionTimeout)),
cpr::Timeout(std::chrono::seconds(kTransmitTimeout)),
cpr::VerifySsl(false),
cpr::Header{ { "Range", thread->GetRangeHeaderString().c_str() } },
cpr::CurlOption({ CURLOPT_NOPROGRESS, 0 }),
cpr::CurlOption({ CURLOPT_XFERINFODATA, thread }),
cpr::CurlOption({ CURLOPT_XFERINFOFUNCTION, &HttpDownloader1::ProgressCallback_ }),
cpr::CurlOption({ CURLOPT_WRITEDATA, thread->file_ }),
cpr::CurlOption({ CURLOPT_WRITEFUNCTION, fwrite })
);
}
if (rsp.status_code == 0)
{
thread->status_ = Status::ERRONEOUS;
Log_("thread:" + std::to_string(thread->id_) + " error: HTTP status code 0");
}
else if (rsp.status_code >= 400)
{
thread->status_ = Status::ERRONEOUS;
Log_("thread:" + std::to_string(thread->id_) + " erorr: HTTP status code " + std::to_string(rsp.status_code));
}
else if (rsp.error.code != cpr::ErrorCode::OK)
{
thread->status_ = Status::ERRONEOUS;
Log_("thread:" + std::to_string(thread->id_) + "error: " + rsp.error.message);
}
else
{
if (thread->file_ == nullptr)
thread->response_header_ = rsp.header;
thread->status_ = Status::SUCCESSFUL;
}
}
This is libcurl's progress callback:
int HttpDownloader1::ProgressCallback_(
void* clientp, std::uint64_t dltotal, std::uint64_t dlnow, std::uint64_t ultotal, std::uint64_t ulnow
) {
auto thread = (DownloadThread*)clientp;
if (dlnow > 0)
{
thread->pos_ = thread->begin_ + dlnow;
if (thread->pos_ > thread->end_)
return 1;
}
return 0;
}

Answer 1

Answer: Because I made a stupid mistake...
Details:
The M thread didn't got stuck(halt abnormally), it's just in a dead loop.
The loop isn't in the main loop of M, but inside the recorder_->MarkFinish(...) call, where I forgot incrementing the iterator at one of the if branch when loop a list container by while(...) {...}.
So when I placed a breakpoint at the recorder_->MarkFinish() line, it wouldn't be caught.
How I found that eventually:
I don't have much experience at multiple thread programming. After scratching my head for nearly a whole day, I thought there must be some low level details unfamiliar to me. So I post this question.
#Raymond comments brought back my confidence. I begun to add debug outputs line by line inside the M thread's main loop trying to find out what it was doing when it seemed stuck. And noticed that each time the error occurred, the outputs would disappear right after that recorder_->MarkFinish(...) call. So I went into that function and caught the carelessly code bellow:
while (it != segments_.end())
{
Segment& seg = *it;
if (seg.begin_ > end) break;
if (seg.end_ < start) continue; // ！
if (start <= seg.begin_ && seg.begin_ <= end)
seg.begin_ = end + 1;
if (start <= seg.end_ && seg.end_ <= end)
seg.end_ = start - 1;
if (seg.end_ < seg.begin_)
it = segments_.erase(it);
else
it++;
}
Hope this might help reducing some of the mysterious sense about the problem you meet.💀