I've written a c++ program that performs time consuming calculations and i want the user to be able to see the progress while the program is running in the background (minimized).
I'd like to use the same effect as chrome uses when downloading a file:
How do i access this feature? Can i use it in my c++ program?
If the time consuming operation can be performed inside a loop, and depending on whether or not it is a count controlled loop, you may be able to use thread and atomic to solve your problem.
If your processor architecture supports multithreading you can use threads to run calculations concurrently. The basic use of a thread is to run a function in parallel with the main thread, these operations may be effectively done at the same time, meaning you would be able to use the main thread to check the progress of your time consuming calculations. With parallel threads comes the problem of data races, wherein if two threads try to access or edit the same data, they could do so incorrectly and corrupt the memory. This can be solved with atomic. You could use an atomic_int to make sure two actions are never cause a data race.
A viable example:
#include <thread>
#include <mutex>
#include <atomic>
#include <iostream>
//function prototypes
void foo(std::mutex * mtx, std::atomic_int * i);
//main function
int main() {
//first define your variables
std::thread bar;
std::mutex mtx;
std::atomic_int value;
//store initial value just in case
value.store(0);
//create the thread and assign it a task by passing a function and any parameters of the function as parameters of thread
std::thread functionalThread;
functionalThread = std::thread(foo/*function name*/, &mtx, &value/*parameters of the function*/);
//a loop to keep checking value to see if it has reached its final value
//temp variable to hold value so that operations can be performed on it while the main thread does other things
int temp = value.load();
//double to hold percent value
double percent;
while (temp < 1000000000) {
//calculate percent value
percent = 100.0 * double(temp) / 1000000000.0;
//display percent value
std::cout << "The current percent is: " << percent << "%" << std::endl;
//get new value for temp
temp = value.load();
}
//display message when calculations complete
std::cout << "Task is done." << std::endl;
//when you join a thread you are essentially waiting for the thread to finish before the calling thread continues
functionalThread.join();
//cin to hold program from completing to view results
int wait;
std::cin >> wait;
//end program
return 0;
}
void foo(std::mutex * mtx, std::atomic_int * i) {
//function counts to 1,000,000,000 as fast as it can
for (i->store(0); i->load() < 1000000000; i->store(i->load() + 1)) {
//keep i counting
//the first part is the initial value, store() sets the value of the atomic int
//the second part is the exit condition, load() returns the currently stored value of the atomic
//the third part is the increment
}
}
Related
I have a task - to write a multithreaded webcrawler (actually I have a local set.html files that I need to parse and move to another directory). The main condition for this task is to make it possible to enter an arbitrary number of threads and determine at what number the program will stop adding in performance.
#include <iostream>
#include <fstream>
#include <thread>
#include <mutex>
#include <queue>
#include <ctime>
#include <set>
#include <chrono>
#include <atomic>
using namespace std;
class WebCrawler{
private:
const string start_path = "/";
const string end_path = "/";
int thread_counts;
string home_page;
queue<string> to_visit;
set<string> visited;
vector<thread> threads;
mutex mt1;
int count;
public:
WebCrawler(int thread_counts_, string root_)
:thread_counts(thread_counts_), home_page(root_) {
to_visit.push(root_);
visited.insert(root_);
count = 0;
}
void crawler(){
for(int i = 0; i<thread_counts; i++)
threads.push_back(thread(&WebCrawler::start_crawl, this));
for(auto &th: threads)
th.join();
cout<<"Count: "<<count<<endl;
}
void parse_html(string page_){
cout<<"Thread: "<<this_thread::get_id()<<" page: "<<page_<< endl;
ifstream page;
page.open(start_path+page_, ios::in);
string tmp;
getline(page, tmp);
page.close();
for(int i = 0; i<tmp.size(); i++){
if( tmp[i] == '<'){
string tmp_num ="";
while(tmp[i]!= '>'){
if(isdigit(tmp[i]))
tmp_num+=tmp[i];
i++;
}
tmp_num+= ".html";
if((visited.find(tmp_num) == visited.end())){
mt1.lock();
to_visit.push(tmp_num);
visited.insert(tmp_num);
mt1.unlock();
}
}
}
}
void move(string page_){
mt1.lock();
count++;
ofstream page;
page.open(end_path+page_, ios::out);
page.close();
mt1.unlock();
}
void start_crawl(){
cout<<"Thread started: "<<this_thread::get_id()<< endl;
string page;
while(!to_visit.empty()){
mt1.lock();
page = to_visit.front();
to_visit.pop();
mt1.unlock();
parse_html(page);
move(page);
}
}
};
int main(int argc, char const *argv\[])
{
int start_time = clock();
WebCrawler crawler(7, "0.html");
crawler.crawler();
int end_time = clock();
cout<<"Time: "<<(float)(end_time -start_time)/CLOCKS_PER_SEC<<endl;
cout<<thread::hardware_concurrency()<<endl;
return 0;
}
1 thread = Time: 0.709504
2 thread = Time: 0.668037
4 thread = Time: 0.762967
7 thread = Time: 0.781821
I've been trying to figure out for a week why my program is running slower even on two threads. I probably don't fully understand how mutex works, or perhaps the speed is lost during the joining of threads. Do you have any ideas how to fix it?
There are many ways to protect things in multithreading, implicit or explicit.
In addition to the totally untested code, there are also some implicit assumptions, for example of that int is large enough for your task, that must be considered.
Lets make a short analysis of what is needing protection.
Variables that are accessed from multiple threads
things that are const can be excluded
unless you const cast them
part of them are mutable
global objects like files or cout
could be overwritten
written from multiple threads
streams have their own internal locks
so you can write to a stream from multiple threads to cout
but you don't want it for the files in this case.
if multiple threads want to open the same file, you will get an error.
std::endl forces an synchronization, so change it to "\n" like a commenter noted.
So this boils down to:
queue<string> to_visit;
set<string> visited; // should be renamed visiting
int count;
<streams and files>
count is easy
std::atomic<int> count;
The files are implicit protected by your visited/visiting check, so they are good too. So the mutex in move can be removed.
The remaining needs an mutex each as they could be independently updated.
mutex mutTovisit, // formerly known as mut1.
mutVisiting.
Now we have the problem that we could deadlock with two mutexes, if we try to lock in different order in two places. You need to read up on all the lock stuff if you add more locks, scoped_lock and lock are good places to start.
Changing the code to
{
scoped_lock visitLock(mutVisiting); // unlocks at end of } even if stuff throws
if((visited.find(tmp_num) == visited.end())){
scoped_lock toLock(mutTo);
to_visit.push(tmp_num);
visited.insert(tmp_num);
}
}
And in this code there are multiple errors, that are hidden by the not thread safe access to to_visit and the randomness of the thread starts.
while(!to_visit.empty()){ // 2. now the next thread starts and sees its empty and stops
// 3. or worse it starts then hang at lock
mt1.lock();
page = to_visit.front(); // 4. and does things that are not good here with an empty to_visit
to_visit.pop(); // 1. is now empty after reading root
mt1.unlock();
parse_html(page);
move(page);
}
To solve this you need an (atomic?) counter, found(Pages) of current known unvisited pages so we know if are done. Then to start threads when there is new work that needs to be done we can use std::condition_variable(_any)
The general idea of the plan is to have the threads wait until work is available, then each time a new page is discovered notify_one to start work.
To Startup, set the found to 1 and notify_one once the threads have started, when a thread is done with the work decrease found. To stop when found is zero, the thread that decrease it to zero notify_all so they all can stop.
What you will find is that if the data is on a single slow disk, it is unlikely you will see much effect from more than 2 threads, if all files are currently cached in ram, you might see more effect.
I think there's a bottle neck on your move function. Each thread takes the same amount of time to go through that function. You could start with that
I'd like to create some threads (with c++11) and collect them in a 'vector'. Then I'd like to fire them up, not immediately upon construction of the associated thread object, but after a specified time. And this is my problem, how can I delay the execution of thread?
It is possible to do something like that? I appreciate any hints.
I'd like to create some threads (with c++11) and collect them in a 'vector'. Then I'd like to fire them up, not immediately upon construction of the associated thread object
You can default construct the std::thread objects:
std::vector<std::thread> threads(some_number);
but after specified time
You can sleep for some time:
std::this_thread::sleep_for(some_time);
Once you're ready to start the execution, create a thread with a function and assign the ones in the vector:
threads[i] = std::thread(some_callable);
That said, creating the empty std::thread objects doesn't necessarily make a lot of sense since you could easily delay creating them until you actually want to execute something.
This approach does make sense when you want to use a constant length array of threads instead of a vector.
My contribution is to provide you a simple working code as all the explanations were already provided by #eerorika perfectly. Please, feel free to look ak the comments within the code
#include <thread>
#include <iostream>
#include <vector>
#include <chrono>
#include <ctime>
/**
* action:
* #param: wait : duration of wating time before launchin an action
* index: index of the thread
**/
void action(int wait, int index)
{
// compute current time
std::time_t result = std::time(nullptr);
// and display it
std::string mytime(std::asctime(std::localtime(&result)));
// only remove the end of line char from the time
mytime.pop_back();
std::cout << "Time is "<< mytime << " Create thread "<< index << " and wait " << (int) (wait/1000) << " seconds "<< std::endl;
// force the thread to wait for the provided amount of time
std::this_thread::sleep_for(std::chrono::milliseconds(wait));
// re-compute current time
std::time_t result2 = std::time(nullptr);
mytime=std::asctime(std::localtime(&result2));
mytime.pop_back();
// and display it
std::cout << "launch action "<< index << " "<< mytime << " after "<< (int)(wait/1000) << " seconds" << std::endl;
}
int main()
{
// a vector composed of 3 threads
std::vector<std::thread> vecOfThreads(3);
vecOfThreads[0]=std::thread(action,5000, 0);
//sleep 15 ms to give the first thread to display his starting message
std::this_thread::sleep_for(std::chrono::milliseconds(15));
// create a thread and provide its runnable (function) and the two arguments of the function
// and assign it to an element of your vector
vecOfThreads[1]=std::thread(action,10000, 1);
//sleep 15 ms to give the first thread to display his starting message
std::this_thread::sleep_for(std::chrono::milliseconds(15));
vecOfThreads[2]=std::thread(action,15000, 2);
//sleep 15 ms to give the first thread to display his starting message
std::this_thread::sleep_for(std::chrono::milliseconds(15));
//waiting for the end of each thread
vecOfThreads[0].join();
vecOfThreads[1].join();
vecOfThreads[2].join();
}
std::async is for such purposes.
// Enable lazy and asynchronous evaluation of func(x, y) on a new thread.
auto a = std::async(std::launch::deferred | std::launch::async, &func, x, y);
// Do what you need.
// It is time to execute func(x, y), wait_for returns immediately without
// waiting regardless of timeout_duration value.
a.wait_for(0s);
// Do other work you need.
// Get the result, wait for a until it is finished if needed.
auto r = a.get();
I want to parallelize the execution of a randomized algorithm in the following way: I have a number of threads which execute the same randomized operations in a loop and return in case of success. I want to start multiple threads and return once at least one of them stops (returns a value). As a minimum example, consider the following code snippet:
#include <iostream>
#include <stdlib.h> /* srand, rand */
#include <future>
#include <vector>
int random_algorithm(){
while(true) {
int random_number = rand() % 10 + 1;
if (random_number > 5){
return random_number;
}
}
}
int main(){
std::vector<std::future<int>> thread_vec;
for(int i=0;i<5;++i){
std::future<int> t = std::async(std::launch::async, random_algorithm);
thread_vec.push_back(std::move(t));
}
**//Instead of the following loop, I want to**
**//continue execution as soon as one of the threads returned.**
for(auto& th: thread_vec){
th.wait();
std::cout << "thread returned " << th.get() << std::endl;
}
return 0;
}
Basically, instead of calling th.wait() on every thread, I just want to wait here until one of the threads in thread_vec has finished its work and then get that threads return value. How would I achieve this?
Note: I saw this question, but this does not seem to reveal which of the threads finished its work.
Ok, let's start discussing your code:
rand() is not re-entrant safe. You must never use it in multiple threads concurrently. Also, it's typically a really bad random number generator.
You're using C++11 or later, so use std::random instead.
Solving your problem: instead of waiting on a future, you should simply share a condition variable with all threads, and the first thread to notify the variable and thus the main thread ends the computation.
Result returning can be implemented through atomic variables, for example (std::atomic).
Question
I want to know if it is possible to wait in the main-Thread without any while(1)-loop.
I launch a few threads via std::async() and do calculation of numbers on each thread. After i start the threads i want to receive the results back. I do that with a std::future<>.get().
My problem
When i receive the result i call std::future.get(), which blocks the main thread until the calculation on the thread is done. This leads to some slower execution time, if one thread needs considerably more time then the following, where i could do some calculation with the finished results instead and then when the slowest thread is done i maybe have some some further calculation.
Is there a way to idle the main thread until ANY of the threads has finished running? I have thought of a callback function which wakes the main thread up, but i still don't know how to idle the main function without making it unresponsive for i.e. a second and not running a while(true) loop instead.
Current code
#include <iostream>
#include <future>
uint64_t calc_factorial(int start, int number);
int main()
{
uint64_t n = 1;
//The user entered number
uint64_t number = 0;
// get the user input
printf("Enter number (uint64_t): ");
scanf("%lu", &number);
std::future<uint64_t> results[4];
for (int i = 0; i < 4; i++)
{
// push to different cores
results[i] = std::async(std::launch::async, calc_factorial, i + 2, number);
}
for (int i = 0; i < 4; i++)
{
//retrieve result...I don't want to wait here if one threads needs more time than usual
n *= results[i].get();
}
// print n or the time needed
return 0;
}
uint64_t calc_factorial(int start, int number)
{
uint64_t n = 1;
for (int i = start; i <= number; i+=4) n *= i;
return n;
}
I prepared a code snippet which runs fine, I am using the GMP Lib for the big results, but the code runs with uint64_t instead if you enter small numbers.
Note
If you have compiled the GMP library for whatever reason on your PC already you could replace every uint64_t with mpz_class
I'd approach this somewhat differently.
Unless I have a fairly specific reason to do otherwise, I tend to approach most multithreaded code the same general way: use a (thread-safe) queue to transmit results. So create an instance of a thread-safe queue, and pass a reference to it to each of the threads that's doing to generate the data. The have whatever thread is going to collect the results grab them from the queue.
This makes it automatic (and trivial) that you create each result as it's produced, rather than getting stuck waiting for one after another has produced results.
So I'm trying to create a program that implements a function that generates a random number (n) and based on n, creates n threads. The main thread is responsible to print the minimum and maximum of the leafs. The depth of hierarchy with the Main thread is 3.
I have written the code below:
#include <iostream>
#include <thread>
#include <time.h>
#include <string>
#include <sstream>
using namespace std;
// a structure to keep the needed information of each thread
struct ThreadInfo
{
long randomN;
int level;
bool run;
int maxOfVals;
double minOfVals;
};
// The start address (function) of the threads
void ChildWork(void* a) {
ThreadInfo* info = (ThreadInfo*)a;
// Generate random value n
srand(time(NULL));
double n=rand()%6+1;
// initialize the thread info with n value
info->randomN=n;
info->maxOfVals=n;
info->minOfVals=n;
// the depth of recursion should not be more than 3
if(info->level > 3)
{
info->run = false;
}
// Create n threads and run them
ThreadInfo* childInfo = new ThreadInfo[(int)n];
for(int i = 0; i < n; i++)
{
childInfo[i].level = info->level + 1;
childInfo[i].run = true;
std::thread tt(ChildWork, &childInfo[i]) ;
tt.detach();
}
// checks if any child threads are working
bool anyRun = true;
while(anyRun)
{
anyRun = false;
for(int i = 0; i < n; i++)
{
anyRun = anyRun || childInfo[i].run;
}
}
// once all child threads are done, we find their max and min value
double maximum=1, minimum=6;
for( int i=0;i<n;i++)
{
// cout<<childInfo[i].maxOfVals<<endl;
if(childInfo[i].maxOfVals>=maximum)
maximum=childInfo[i].maxOfVals;
if(childInfo[i].minOfVals< minimum)
minimum=childInfo[i].minOfVals;
}
info->maxOfVals=maximum;
info->minOfVals=minimum;
// we set the info->run value to false, so that the parrent thread of this thread will know that it is done
info->run = false;
}
int main()
{
ThreadInfo info;
srand(time(NULL));
double n=rand()%6+1;
cout<<"n is: "<<n<<endl;
// initializing thread info
info.randomN=n;
info.maxOfVals=n;
info.minOfVals=n;
info.level = 1;
info.run = true;
std::thread t(ChildWork, &info) ;
t.join();
while(info.run);
info.maxOfVals= max<unsigned long>(info.randomN,info.maxOfVals);
info.minOfVals= min<unsigned long>(info.randomN,info.minOfVals);
cout << "Max is: " << info.maxOfVals <<" and Min is: "<<info.minOfVals;
}
The code compiles with no error, but when I execute it, it gives me this :
libc++abi.dylib: terminating with uncaught exception of type
std::__1::system_error: thread constructor failed: Resource
temporarily unavailable Abort trap: 6
You spawn too many threads. It looks a bit like a fork() bomb. Threads are a very heavy-weight system resource. Use them sparingly.
Within the function void Childwork I see two mistakes:
As someone already pointed out in the comments, you check the info level of a thread and then you go and create some more threads regardless of the previous check.
Within the for loop that spawns your new threads, you increment the info level right before you spawn the actual thread. However you increment a freshly created instance of ThreadInfo here ThreadInfo* childInfo = new ThreadInfo[(int)n]. All instances within childInfo hold a level of 0. Basically the level of each thread you spawn is 1.
In general avoid using threads to achieve concurrency for I/O bound operations (*). Just use threads to achieve concurrency for independent CPU bound operations. As a rule of thumb you never need more threads than you have CPU cores in your system (**). Having more does not improve concurrency and does not improve performance.
(*) You should always use direct function calls and an event based system to run pseudo concurrent I/O operations. You do not need any threading to do so. For example a TCP server does not need any threads to serve thousands of clients.
(**) This is the ideal case. In practice your software is composed of multiple parts, developed by independent developers and maintained in different modes, so it is ok to have some threads which could be theoretically avoided.
Multithreading is still rocket science in 2019. Especially in C++. Do not do it unless you know exactly what you are doing. Here is a good series of blog posts that handle threads.