I'm writing a server(mainly for windows, but it would be cool if i could keep it multiplatform) and i just use a normal console window for it. However, I want the server to be able to do commands like say text_to_say_here or kick playername, etc. How can i have a asynchronous input/output? I allready tried some stuff with the normal printf() and gets_s but that resulted in some really.... weird stuff.
I mean something like this 1
thanks.
Quick code to take advantage of C++11 features (i.e. cross-platform)
#include <atomic>
#include <thread>
#include <iostream>
void ReadCin(std::atomic<bool>& run)
{
std::string buffer;
while (run.load())
{
std::cin >> buffer;
if (buffer == "Quit")
{
run.store(false);
}
}
}
int main()
{
std::atomic<bool> run(true);
std::thread cinThread(ReadCin, std::ref(run));
while (run.load())
{
// main loop
}
run.store(false);
cinThread.join();
return 0;
}
You can simulate asynchronous I/O using threads, but more importantly, you must share a mutex between the two read/write threads in order to avoid any issues with a thread stepping on another thread, and writing to the console on top of the output of another thread. In other words std::cout, std::cin, fprintf(), etc. are not multi-thread safe, and as a result, you will get an unpredictable interleaving pattern between the two operations where a read or write takes place while another read or write was already happening. You could easily end up with a read trying to take place in the middle of a write, and furthermore, while you're typing an input on the console, another writing thread could start writing on the console, making a visual mess of what you're trying to type as input.
In order to properly manage your asynchronous read and write threads, it would be best to setup two classes, one for reading, and another for writing. In each class, setup a message queue that will either store messages (most likely std::string) for the main thread to retrieve in the case of the read thread, and for the main thread to push messages to in the case of the write thread. You may also want to make a special version of your read thread that can print a prompt, with a message pushed into its message queue by the main thread that will print a prompt before reading from stdin or std::cin. Both classes will then share a common mutex or semaphore to prevent unpredictable interleaving of I/O. By locking the common mutex before any iostream calls (an unlocking it afterwards), any unpredictable interleaving of I/O will be avoided. Each thread will also add another mutex that is unique to each thread that can be used to maintain exclusivity over access to the class's internal message queue. Finally, you can implement the message queues in each class as a std::queue<std::string>.
If you want to make your program as cross-platform as possible, I would suggest implementing this with either Boost::threads, or using the new C++0x std::threads libraries.
If you ditch the console window and use TCP connections for command and control, your server will be much easier to keep multi-platform, and also simpler and more flexible.
You can try placing the input and output on separate threads. I'm not quite sure why you want to do this, but threading should do the job.
:)
http://en.wikibooks.org/wiki/C++_Programming/Threading
Related
I am trying to introduce logging to a multithreaded application. Currently, I am just using std::cout from the different threads. However, in that case, the order of the output is getting jumbled, even though one thread logged early, its output in stdout is coming after the log of another thread.
So, one solution can be to move all logging to an extra thread, but I don't want to manage one more thread. So I am thinking of using std::async to do the logging from the different threads. Is this possible? Are there any suggested ways to do this? Also is the order of execution of std::async guaranteed?
#include <iostream>
#include <vector>
#include <algorithm>
#include <numeric>
#include <future>
#include <string>
#include <mutex>
void print(int i)
{
std::cout << i << std::endl;
}
int main()
{
auto a1 = std::async(std::launch::async, print, 1);
auto a2 = std::async(std::launch::async, print, 2);
auto a3 = std::async(std::launch::async, print, 3);
a3.wait();
a2.wait();
a1.wait();
}
For the above code, is it guaranteed that the order of output will be
1
2
3
?
The whole point of std::async(std::launch::async, ...) is that it's asynchronous (thus using not only "async" in the name, but repeating it in the first parameter).
You're not guaranteed much of anything about the relative order of things happening in threads created with std::async, unless you force synchronization using something like an std::mutex, std::condition_variable, or some of the synchronizing primitives in <atomic>.
You say you don't want to manage one more thread, but then you create and manage not only one, but three more threads. I don't quite understand how this makes sense.
My own tendency would be to create a type to handle logging. Whether it uses a separate thread or not is its own internal affair. The threads doing the real work just do something like: log(error) << "error 12345"; and it's up to the logging object to implement that efficiently. Yes, if you have a lot of other threads contending for use of the single logging object, it's likely to be best off running in a thread of its own--but they should neither know nor care one way or the other about that.
For the above code, is it guaranteed that the order of output will be
1
2
3
4
?
No. wait() waits until the value is available. It doesn't start the execution. It merely blocks until its done.
It may as well happen that all of those futures are ready even before you call any wait()s. In that case it's quite obvious that the order is not guaranteed (and completely unrelated to the order of those wait() calls).
A way to offload logging from the current thread to elsewhere is to have a queue between the thread wanting to log and a thread actually writing the log to disk (or whatever). A queue is a good object to use, because it preserves order.
There's probably way better ways of doing this than what I'm about to describe (and I did this quite a while ago, so its bound to have been bettered). It's possible to adapt log4cpp so as to have a thread accepting logging requests submitted via std::queue. That's not a multithreaded thing, so what I've done before is to create a log event class, manage those using shared_ptrs, put the shared_ptrs to log event objects on the std::queue (so the posting to the queue is small, fast, and a minimum lock time on the necessary mutex). Then I've added in ZeroMQ with a PUSH/PULL pattern to allow multiple PUSHERs (posters onto the std::queue) to send 1 byte long messages to wake up a single PULL thread (that polls the zeroMQ and pulls from the std::queue). So logging consisted of the creation of a log event object, acquiring a mutex, pushing a shared_ptr to the log event object onto a std::queue, releasing the mutex and finally pushing a 1 byte message into a ZeroMQ socket.
Yes, it's a fairly horrific blend of std::queue and ZeroMQ, but it was quick to dispatch an arbitrary long log event without having to serialise the log event data in the thread send it.
A possible embelishment would be to turn off the mutex locking on the shared_ptr (it's not needed), or just use a raw pointer instead and try to remember to call delete.
This question already has answers here:
Is cout synchronized/thread-safe?
(4 answers)
Closed 5 years ago.
Recently I started learning C++ 11. I only studied C/C++ for a brief period of time when I was in college.I come from another ecosystem (web development) so as you can imagine I'm relatively new into C++.
At the moment I'm studying threads and how could accomplish logging from multiple threads with a single writer (file handle). So I wrote the following code based on tutorials and reading various articles.
My First question and request would be to point out any bad practices / mistakes that I have overlooked (although the code works with VC 2015).
Secondly and this is what is my main concern is that I'm not closing the file handle, and I'm not sure If that causes any issues. If it does when and how would be the most appropriate way to close it?
Lastly and correct me if I'm wrong I don't want to "pause" a thread while another thread is writing. I'm writing line by line each time. Is there any case that the output messes up at some point?
Thank you very much for your time, bellow is the source (currently for learning purposes everything is inside main.cpp).
#include <iostream>
#include <fstream>
#include <thread>
#include <string>
static const int THREADS_NUM = 8;
class Logger
{
public:
Logger(const std::string &path) : filePath(path)
{
this->logFile.open(this->filePath);
}
void write(const std::string &data)
{
this->logFile << data;
}
private:
std::ofstream logFile;
std::string filePath;
};
void spawnThread(int tid, std::shared_ptr<Logger> &logger)
{
std::cout << "Thread " + std::to_string(tid) + " started" << std::endl;
logger->write("Thread " + std::to_string(tid) + " was here!\n");
};
int main()
{
std::cout << "Master started" << std::endl;
std::thread threadPool[THREADS_NUM];
auto logger = std::make_shared<Logger>("test.log");
for (int i = 0; i < THREADS_NUM; ++i)
{
threadPool[i] = std::thread(spawnThread, i, logger);
threadPool[i].join();
}
return 0;
}
PS1: In this scenario there will always be only 1 file handle open for threads to log data.
PS2: The file handle ideally should close right before the program exits... Should it be done in Logger destructor?
UPDATE
The current output with 1000 threads is the following:
Thread 0 was here!
Thread 1 was here!
Thread 2 was here!
Thread 3 was here!
.
.
.
.
Thread 995 was here!
Thread 996 was here!
Thread 997 was here!
Thread 998 was here!
Thread 999 was here!
I don't see any garbage so far...
My First question and request would be to point out any bad practices / mistakes that I have overlooked (although the code works with VC 2015).
Subjective, but the code looks fine to me. Although you are not synchronizing threads (some std::mutex in logger would do the trick).
Also note that this:
std::thread threadPool[THREADS_NUM];
auto logger = std::make_shared<Logger>("test.log");
for (int i = 0; i < THREADS_NUM; ++i)
{
threadPool[i] = std::thread(spawnThread, i, logger);
threadPool[i].join();
}
is pointless. You create a thread, join it and then create a new one. I think this is what you are looking for:
std::vector<std::thread> threadPool;
auto logger = std::make_shared<Logger>("test.log");
// create all threads
for (int i = 0; i < THREADS_NUM; ++i)
threadPool.emplace_back(spawnThread, i, logger);
// after all are created join them
for (auto& th: threadPool)
th.join();
Now you create all threads and then wait for all of them. Not one by one.
Secondly and this is what is my main concern is that I'm not closing the file handle, and I'm not sure If that causes any issues. If it does when and how would be the most appropriate way to close it?
And when do you want to close it? After each write? That would be a redundant OS work with no real benefit. The file is supposed to be open through entire program's lifetime. Therefore there is no reason to close it manually at all. With graceful exit std::ofstream will call its destructor that closes the file. On non-graceful exit the os will close all remaining handles anyway.
Flushing a file's buffer (possibly after each write?) would be helpful though.
Lastly and correct me if I'm wrong I don't want to "pause" a thread while another thread is writing. I'm writing line by line each time. Is there any case that the output messes up at some point?
Yes, of course. You are not synchronizing writes to the file, the output might be garbage. You can actually easily check it yourself: spawn 10000 threads and run the code. It's very likely you will get a corrupted file.
There are many different synchronization mechanisms. But all of them are either lock-free or lock-based (or possibly a mix). Anyway a simple std::mutex (basic lock-based synchronization) in the logger class should be fine.
The first massive mistake is saying "it works with MSVC, I see no garbage", even moreso as it only works because your test code is broken (well it's not broken, but it's not concurrent, so of course it works fine).
But even if the code was concurrent, saying "I don't see anything wrong" is a terrible mistake. Multithreaded code is never correct unless you see something wrong, it is incorrect unless proven correct.
The goal of not blocking ("pausing") one thread while another is writing is unachieveable if you want correctness, at least if they concurrently write to the same descriptor. You must synchronize properly (call it any way you like, and use any method you like), or the behavior will be incorrect. Or worse, it will look correct for as long as you look at it, and it will behave wrong six months later when your most important customer uses it for a multi-million dollar project.
Under some operating systems, you can "cheat" and get away without synchronization as these offer syscalls that have atomicity guarantees (e.g. writev). That is however not what you may think, it is indeed heavyweight synchronization, only just you don't see it.
A better (more efficient) strategy than to use a mutex or use atomic writes might be to have a single consumer thread which writes to disk, and to push log tasks onto a concurrent queue from how many producer threads you like. This has minimum latency for threads that you don't want to block, and blocking where you don't care. Plus, you can coalesce several small writes into one.
Closing or not closing a file seems like a non-issue. After all, when the program exits, files are closed anyway. Well yes, except, there are three layers of caching (four actually if you count the physical disk's caches), two of them within your application and one within the operating system.
When data has made it at least into the OS buffers, all is good unless power fails unexpectedly. Not so for the other two levels of cache!
If your process dies unexpectedly, its memory will be released, which includes anything cached within iostream and anything cached within the CRT. So if you need any amount of reliability, you will either have to flush regularly (which is expensive), or use a different strategy. File mappying may be such a strategy because whatever you copy into the mapping is automatically (by definition) within the operating system's buffers, and unless power fails or the computer explodes, it will be written to disk.
That being said, there exist dozens of free and readily available logging libraries (such as e.g. spdlog) which do the job very well. There's really not much of a reason to reinvent this particular wheel.
Hello and welcome to the community!
A few comments on the code, and a few general tips on top of that.
Don't use native arrays if you do not absolutely have to.
Eliminating the native std::thread[] array and replacing it with an std::array would allow you to do a range based for loop which is the preferred way of iterating over things in C++. An std::vector would also work since you have to generate the thredas (which you can do with std::generate in combination with std::back_inserter)
Don't use smart pointers if you do not have specific memory management requirements, in this case a reference to a stack allocated logger would be fine (the logger would probably live for the duration of the program anyway, hence no need for explicit memory management). In C++ you try to use the stack as much as possible, dynamic memory allocation is slow in many ways and shared pointers introduce overhead (unique pointers are zero cost abstractions).
The join in the for loop is probably not what you want, it will wait for the previously spawned thread and spawn another one after it is finished. If you want parallelism you need another for loop for the joins, but the preferred way would be to use std::for_each(begin(pool), end(pool), [](auto& thread) { thread.join(); }) or something similar.
Use the C++ Core Guidelines and a recent C++ standard (C++17 is the current), C++11 is old and you probably want to learn the modern stuff instead of learning how to write legacy code. http://isocpp.github.io/CppCoreGuidelines/CppCoreGuidelines
C++ is not java, use the stack as much as possible - this is one of the biggest advantages to using C++. Make sure you understand how the stack, constructors and destructors work by heart.
The first question is subjective so someone else would want to give an advice, but I don't see anything awful.
Nothing in C++ standard library is thread-safe except for some rare cases. A good answer on using ofstream in a multithreaded environment is given here.
Not closing a file is indeed an issue. You have to get familiar with RAII as it is one of the first things to learn. The answer by Detonar is a good piece of advice.
I have a program in c++ with several threads in it. I want one of the threads to be able to read/get commands from the console while others continue running, for example: "play", "stop", "pause",...
something like:
while (1)
{
std::string str;
getline(std::cin, str);
/* do something */
}
Will it work? Any suggestions?
Thanks in advance.
Short Answer: Yes.
Long Answer: It depends of what you call 'work', there is nothing that prevent you from calling a blocking function/method from a thread while other threads are running.
However, threads share memory and resources. On an UNIX machine (and it's more or less the same on Windows), stdin and stdout are shared between threads. std::cin will manipulate stdin under the hood at some point, and you should ensure that only one thread can manipulate a given resource at a time.
You can do that by either make sure that only one thread can reach code using std::cin, or use synchronization, with a mutex/semaphore.
i have created a class which reads a file and does some operations on the contents and saves a new file with time stamp. But, i am in a requirement to perform in such a way that , a code should check every one min whether the file is present. If yes, it should process the file. It need to work on cross platform.
I am novice in c++ and need to know what approach i need to follow for this. Do i need to create process or something. I am completely blank .
class inputHandler
{
public:
void readInput();
void performTask();
void saveFile();
};
since the code implementation is too large, just i am posting the structure. I am ready to spend time on this. So, i need a sample tutorial which can guide me to achieve this .
This is not addressed by the C++ standard. Thus, you'll have to implement code for each supported system, or use a library.
As far as I understood, the most general solution is to create a thread which loops every minute, checking file timestamps. Naturally, depending on your code, you could do it another way, avoiding threads whatsoever. Using a notification system such as inotify could be much better. Also, you could use alarm() on POSIX-compatible systems, being alarmed whenever a minute has passed.
Anyway, if you go with the thread solution, in POSIX-compatible systems, check out pthread_create() and stat(). In Windows, check out CreateThread() and GetFileTime(). To have a one-minute delay, sleep(60000) or Sleep(60000) respectively should do the trick.
Just to clarify, "to create a process" is system's programming jargon meaning roughly "to launch a new program" (or "thread", sometimes). In that sense, if you follow the above you'll be creating a new thread.
The simple part is checking if a file exists: when you open an std::ifstream it will be in good state only if the file exists:
std::ifstream in(filename);
if (in) {
// the file exists and can be processed here
}
The more interesting part is to do something in regular intervals. The basic idea is to set up a timer in some form. Depending on whether anything else needs to be done you may need a separate thread: if the program just waits until the file exists and doesn't do anything in the mean time, you can just sleep and there is no need to spawn another thread. Otherwise, you probably want to spawn a thread which is just sleeping.
Assuming you need to use a separate thread, you probably want to be able to interrupt it from waiting, e.g., to exit in a clean way upon condition from a separate thread. thus, I would use a condition variable with a timed wait, i.e., something like this:
std::mutex guard;
std::condition_variable condition;
bool done(false);
std::unique_lock<std::mutex> lock(guard);
while (!done) {
condition.wait_for(lock, std::chrono::minutes(n));
if (!done) {
do_whatever_needs_to_be_done_once_every_n_minutes();
}
}
The code above uses C++ 2011 facilities. If you can't use the corresponding classes, you can use suitable alternatives, e.g., the Boost classes.
This is a pretty basic scenario but I'm not finding too many helpful resources. I have a C++ program running in Linux that does file processing. Reads lines, does various transformations, writes data into a database. There's certain variables (stored in the database) that affect the processing which I'm currently reading at every iteration because I want processing to be as up to date as possible, but a slight lag is OK. But those variables change pretty rarely, and the reads are expensive over time (10 million plus rows a day). I could space out the reads to every n iterations or simply restart the program when a variable changes, but those seem hackish.
What I would like to do instead is have the program trigger a reread of the variables when it receives a SIGHUP. Everything I'm reading about signal handling is talking about the C signal library which I'm not sure how to tie in to my program's classes. The Boost signal libraries seem to be more about inter-object communication rather than handling OS signals.
Can anybody help? It seems like this should be incredibly simple, but I'm pretty rusty with C++.
I would handle it just like you might handle it in C. I think it's perfectly fine to have a stand-alone signal handler function, since you'll just be posting to a semaphore or setting a variable or some such, which another thread or object can inspect to determine if it needs to re-read the settings.
#include <signal.h>
#include <stdio.h>
/* or you might use a semaphore to notify a waiting thread */
static volatile sig_atomic_t sig_caught = 0;
void handle_sighup(int signum)
{
/* in case we registered this handler for multiple signals */
if (signum == SIGHUP) {
sig_caught = 1;
}
}
int main(int argc, char* argv[])
{
/* you may also prefer sigaction() instead of signal() */
signal(SIGHUP, handle_sighup);
while(1) {
if (sig_caught) {
sig_caught = 0;
printf("caught a SIGHUP. I should re-read settings.\n");
}
}
return 0;
}
You can test sending a SIGHUP by using kill -1 `pidof yourapp`.
I'd recommend checking out this link which gives the details on registering a signal.
Unless I'm mistaken, one important thing to remember is that any function inside an object expects a referent parameter, which means non-static member functions can't be signal handlers. I believe you'll need to register it either to a static member function, or some kind of global function. From there, if you have a specific object function you want to take care of your update, you'll need a way to reference that object.
There are several possibilities; it would not necessarily be overkill to implement all of them:
Respond to a specific signal, just like C does. C++ works the same way. See the documentation for signal().
Trigger on the modification timestamp of some file changing, like the database if it is stored in a flat file.
Trigger once per hour, or once per day (whatever makes sense).
You can define a Boost signal corresponding to the OS signal and tie the Boost signal to your slot to invoke the respective handler.