glib has a data structure called GAsyncQueue, which allows inter-thread communication with no semaphores/locks/etc., and even makes trivial the task of implementing a producer/consumer solution. If two different threads push data to a GAsyncQueue structure, the push function internally implements the mutually exclusive access to the queue; more awesomely, if a thread calls the pop function, and there is no data there, the calling thread blocks until some data is pushed into the queue by some other thread. All of this is done in a thread-safe manner, transparently to the developer.
As much as I like it, though, this library was built for C, and there might be better alternatives for higher level languages. I'm thinking about using glib anyway, but it feels odd to use a C library in a C++ code...
So, the question is: is there a C++ recommended equivalent for glib? More specifically, is there a more recommended C++ library that provides the same functionality as GAsyncQueue?
There is absolutely nothing wrong with using C in a C++ program (after all, C++ implementation is heavily based on C runtime, for example C++11 thread support cannot live without pthread library, at least on UNIX®-like platforms). I would definitely not choose the tool/library only and entirely basing on the language it is written in. But if you must use something else, then glib is not the only library in the world that provides provides asynchronous message passing (by the way, it doesn't really look like it supports IPC). Anyhow, here is a list of C++ frameworks that immediately come to my mind (in random order, as random as my thoughts):
Intel Threading Building Blocks
Boost MPI
Boost.ASIO
Qt
Each one has its own strengths and weaknesses, and which one to use really depends on what exactly your requirements are. I can only recommend you to pay attention to overall application architecture and how well the asynchronous message passing would fit into all of the components of your application. For example, in more or less complex applications that involve more than simple message passing, such asynchronous queues are oftentimes integrated with the event notification mechanisms in use (for example, OSX is built around kqueue/GCD).
Hope it helps. Good Luck!
Related
Is the new C++11 going to contain any socket library? So that one could do something std::socket-ish?
Seeing as how std::thread will be added, it feels as if sockets should be added as well. C-style sockets are a pain... They feel extremely counter-intuitive.
Anyways: Will there be C++ sockets in C++11 (googled it but couldn't find an answer)? If not, are their any plans on adding this? Why (/ why not)?
No, it is not. As for the near future, the C++ standards committee has created a study group that is developing a networking layer proposal. It looks like they're going for a bottom-up approach, starting with a basic socket layer, then building HTTP/etc support on top of that. They're looking to present the basic socket proposal at the October committee meeting.
As for why they didn't put this into C++11, that is purely speculative.
If you want my opinion on the matter, it's for this reason.
If you are making a program that does something, that has a specific functionality to it, then you can pick libraries for one of two reasons. One reason is because that library does something that is necessary to implement your code. And the other is because it does something that is helpful in implementing code in general.
It is very difficult for a design for a particular program to say, "I absolutely must use a std::vector to hold this list of items!" The design for a program isn't that specific. If you're making a web browser, the idea of a browser doesn't care if it holds its tabs in a std::vector, std::list, or a user-created object. Now, some design can strongly suggest certain data structures. But rarely does the design say explicitly that something low-level like a std::list is utterly essential.
std::list could be used in just about any program. As can std::vector, std::deque, etc.
However, if you're making a web browser, bottled within that design is networking. You must either use a networking library or write a networking layer yourself. It is a fundamental requirement of the idea.
The term I use for the former type, for libraries that could be used in anything, is "utility" libraries.
Threading is a utility library. Design might encourage threading through the need to respond to the user, but there are ways to be responsive without preemptive multithreading. Therefore, in most cases, threading is an implementation choice. Threading is therefore a utility.
Networking is not. You only use networking if your design specifically calls for it. You don't decide to just dump networking into a program. It isn't an implementation detail; it is a design requirement.
It is my opinion that the standard C/C++ library should only implement utilities. It's also why I'm against other heavyweight ideas like XML parsers, etc. It isn't wrong for other libraries to have these things, but for C and C++, these are not good choices.
I think it should, since a lot of other popular languages support socket operations as a part of the language (they don't force the user to use any OS-specific API). If we already have file streams to read/write local files, I don't see why we can't have some method of transferring data with sockets.
There will be no sockets in C++11. The difference between threads and sockets is that threads involves making more guarantees about ordering, if your program involves threads. For a platform with just one core, then C++11 doesn't mandate that your CPU springs an extra core. Sockets, on the other hand, would be... difficult to implement portably and fail gracefully on systems that don't have them.
This is so weird that in 2022, there is still no standard for a basic OS construct as sockets in C++.
The closest I found is kissnet (Apparently exists since 2019).
It's small (~1500 lines), runs on Windows and Linux, uses OpenSSL, and requires C++ 17 (Which is a plus in my book), basically everything I needed.
There will not be in C++0x. There are proposals to add them in a future version.
The amount of new stuff in C++0x had to be limited to give the committee time to deal with it all thoroughly.
The wikipedia page for C++0x is usually pretty up to date and the section on library changes doesn't seem to mention sockets.
I've been working on a few C++ projects now which involve doing some simple IPC using window messages. In a number of cases, some extra data is passed with the window messages by putting the data into a shared memory segment and then passing the pointer into the shared memory with the SendMessage call. Re-doing this all the time is annoying, but before resolving this dull repetition by inventing yet another IPC system I'd like to ask:
Is there an existing framework which satisfies the criteria?:
Written in C or C++ (we're using MSVC here)
As few dependencies as possible; in the best case, it's just a few source files which use plain C++ and Windows standard libraries and which can be compiled directly into the application/library.
Works on Windows XP and newer
Is built on window messages plus a shared memory segment
Proper error reporting would be highly desireable (remote process is gone, remote process doesn't understand given message, argument mismatch, etc.)
For what it's worth, COM is not really an option for us since it's so painful to work with it (unless you start using all kinds of wrappers on top of it which we'd like to avoid). I don't really need interfaces and all that stuff; just a convenient way to send messages with (in the best case arbitrary) arguments back and forth with a bit of error handling. Also, I discarded DBUS for doing so much more than I need.
I've had success using a memory mapped file for interprocess communication. I like it mainly because it's simple, fast, and available on any version of windows you're likely to come across (well, i doubt it will work on Win9x, but....)
There's a basic article at http://msdn.microsoft.com/en-us/library/ms810613.aspx (writtin in 1993!) that shows how to use them.
Although does not meet all of your criteria, ZeroMQ (http://www.zeromq.org/) might be worth looking at. It is small, simple and fast. Also it gives you message passing semantics which may help depending on the type of applications you are using
This question is almost 3 years old at the time of this answer, but I can't believe nobody has formally suggested Boost.Interprocess.
Light-weight, IPC, and C++ wrappings for WINAPI mechanisms are available.
I'd like to start learning multithreading in C++. I'm learning it in Java as well. In Java, if I write a program which uses multithreading, it will work anywhere. However in C++, doesn't multithreading rely on platform-specific API's? If so, that would seem to get in the way of portability.
How can I do multithreading in C++ without causing portability issues? Is boost's thread library a good solution?
As a sidenote - how is it even possible to implement multithreading as a library? Isn't that something that has to be done by the compiler?
If you do not have a compiler that supports C++0x yet (available with visual studio c++ 2010 for example), use boost threads. (Unless you use a framework that already supports threading, which is not the case - you wouldn't ask the question otherwise -). These boost threads became actually the standard in the brand new C++. Before that time C++ itself was thread unaware.
TBB Threading Building Blocks might also be interesting to you if you want to learn other aspects in parallel programming.
Regarding Qt: if you only want threading support it is complete overkill. It has horribly slow round trip times from compiling to result. It is really well designed thought. But not an official standard like the C++0x threads from boost. Therefore I would not take it as a first choice.
In C++, yes threading is platform specific. However, many threading libraries encapsulate the nuances of threading on various platforms, providing a uniform API with which to write your threaded app, so you don't have to worry about platform specific details.
Boost threading library is a very good solution.
I also recommending checking out ACE as well.
Let's start backward:
How is it possible to implement threading in a library ?
It isn't, at least not in (pure) C++. This requires language support (the compiler is only an implementation).
At the moment 2 things are used:
assembly code for some parts (like in the pthread library)
specific compiler instructions for others (dependent on the compiler and platform)
Both are brittle and require a tremendous amount of work for portability. Basically it means lot of #ifdef portions in the code to test for the compiler and targetted architecture, test for the support of some directives etc...
That is why it was deemed necessary to add threading support in C++0x.
How do I do multithreading ?
Even before you choose a library, you should choose a method. There are 2 ways of programming multithreaded applications (and you can combine them):
Communicate by sharing: this means using mutexes, atomic operations, etc... you can use pthread on Linux platforms, but I would recommend Boost.Thread (among others) for its portability.
Share by communicating: more recent, and adapted to distributed computations, this stems from the functional languages. This means passing messages from one thread to another and not sharing any resources. You can use FastFlow or Intel's Thread Building Blocks aka TBB.
You can conceivably merge the two, but it would be better not to. Personally I have found the description of FastFlow totally awesome: it encourages lock-free programming. Also, the main advantage of the second method is that it's better adapted to multi-processes programming and scales to distributed environments.
To begin with, I would recommend focusing on either one and build some applications with it. When you're comfortable you may try out the other, but be ready to start anew, they are that different.
//This program explains how pthread works, here 5 thread are trying to update a global variable simultaneously but with locking synchronization in maintained
#include<iostream>
#include<pthread.h>
using namespace std ;
#define MAX_NO_THREAD 5
int global_sum = 0 ;
pthread_mutex_t lock ; //Declared global lock mutex object
void *print_fun(void *arg)
{
cout<<"\nThread id : "<<(int)arg;
pthread_mutex_lock(&lock) ; //aquiring lock on piece of code
for ( int j=0; j<100000000; j++)
{
global_sum++ ;
}
pthread_mutex_unlock(&lock) ; //reomving lock on peice of code
cout<<"\nGlobal Sum : "<<global_sum ;
}
int main()
{
int i = 0 ;
pthread_t threads_obj[MAX_NO_THREAD] ; //Initializing object array for thread
pthread_mutex_init(&lock, NULL) ; //Initalinzing lock object for thread
int st ;
for ( i=0; i<5; i++)
{
pthread_create(&threads_obj[i], NULL, *print_fun, (void *)i) ;//Initializing threads calling function print_fun
pthread_join(threads_obj[i], 0) ; //Forcing main thread to main until these thread complete
}
pthread_mutex_destroy(&lock) ; //Destroying lock object
}
//compile this program using -lpthread option with g++
//g++ thread.cc -lpthread
also check out
Qt
To offer a suggestion different from Boost, I use Pthreads (or Pthreads-Win32 on Windows). It's a very do-it-yourself barebones library, but provides you with all you need and nothing else. It's very lightweight compared to Boost and you can easily find C++ wrappers around it to give you higher level abstractions.
you might also consider openmp http://openmp.org. Many compilers support it, including MS, GCC/G++ and Intel. Although you don't get explicit control of threads, its higher level abstraction of parallelism is sometimes more efficient (at coding time as well as runtime), and the code is much easier to understand. It's not going to help you much if you're doing GUI work, but for scalable computation it's worth a look.
The Boost Threading Library is probably the best place to start for C++. Gives you threading contructs, as well as all the mutexes and control objects you need to write a real working multithreaded application.
With C++11, "Thread support library" is introduced under <thread> header.
More info could be found from below links
https://en.cppreference.com/w/cpp/thread
http://www.cplusplus.com/reference/thread/
If you're doing this out of interest to improve your knowledge of different programming models and language skills then the Boost library would be a fine way to go. However I would think long and hard about actually building any production applications using multi-threaded C++.
C++ is challenging enough at times to attain correctness without adding the considerable complexity of shared memory multi-threading. Even the most seasoned programmers would agree that multi-threaded programs are extremely hard to reason about and get right. Even the simplest programs can quickly become hard to test and debug when multi-threaded.
Imperative languages such as C++ or Java or C# (with their mutable variables, shared memory and locking/signalling primitives) are very often the least approachable way to try to build multi-threaded applications. There are usually perfectly good single threaded implementation options to accomplish most user-space (as opposed to kernel or embedded) application problems, including on multi-core machines.
If you really want to build reliable "multi-threaded" applications I would suggest you check out functional languages like Erlang, Haskell, F# or Clojure.
I'm looking forward to an interview in C++ in the coming weeks. (yay) So I have been relearning C++ and studying up. Unfortunately I have realized that I've never implemented threads in C++, and am somewhat concerned about a quiz on concurrency.
As far as I can tell, C++ uses pthreads in Linux and some other device in Windows. Is this correct? Is there another industry standard, more OO way to handle threads in C++ that I should be expected to know? And are there any good web resources that you can point me to for practicing and learning threads in C++?
Thanks!
There is a boost threads library which is probably the closest to a standard.
Generally threads are supplied by the OS so you get whatever the OS supplied. Also peoples first exposure to threading is often in a GUI, to allow a background calculation to not block the GUI, and so people tend to use the thread functions supplied by the particular GUI framework (MFC/Qt etc)
Currently C++ is entirely unaware that threads exist. Different OSes provide threading libraries to make them available. The next version of C++, so called C++0x, is going to make a thread library standard. If I were to start a multithreaded app today I would go with either boost threads or the threads that were a part of any package I might be using i.e. QT or WxWidgets.
Well, until C++0x gets here, there is no standard way to do threading in C++. You can use whatever facilities your operating system provides. So yes, if you are on a UNIX-like operating system, you can use pthreads. On Windows, you can use the Windows API.
There are 3rd party toolkits out there that attempt to provide a uniform and portable threading API, e.g. boost threads and QT.
It is also not difficult to write your own portable abstraction layer either. We did this because the boost API didn't have everything we needed several years ago (e.g. no way to set priority).
In windows the only way to create threads is using the win32 API. Every library you may have that creates threads on windows eventually uses win32 CreateThread()
QT Contains a nice C++ wrapper around a thread that is cross platform. Usually a good practice is to have a MyThread class that contains all the nitty-gritty details of setting up a thread, checking for error codes, getting it's exit code and the like. The MyThread class would have a pure virtual function called run() which is intended to actually do whatever you'd want the thread to do. The user of the MyThread class is expected to inherit from it and implement run() this way you can isolate the user of the class from the details for actually creating the thread.
MyThead also needs to have a method start() which initiates the thread. The thread would start at some entry point inside the class (this usually needs to be a static method) and then this eventually leads to the user's run() method to be invoked.
Beyond Qt, wxWidgets, Boost and native, OS-provided threading facilities, you can just Google around for thread libraries for C++. They are probably more portable and lightweight as well (if all you are looking for is threading). However, if you have a need for more facilities and the aforementioned libraries provide them, then go ahead and use them. Boost is especially good, it has other facilities as well, but admittedly it's threading library, as Brian Neal said, is limited in some regards.
For your interview, neither boost nor qt is helpful at all. you could just use them as the high level libs and interfaces, and no one would ask you how to use boost or qt in such an interview.
For understanding threading and mutex etc, see a document from http://code.google.com/p/effoaddon/downloads/list, named EffoAddons.pdf.
Raw source
http://effoaddon.googlecode.com/svn/trunk/devel/effo/codebase/addons/thrd/src/thrd/thrd.cpp,
high level abstract interface
http://effoaddon.googlecode.com/svn/trunk/devel/effo/codebase/addons/thrd/include/thrd_i.h,
and code support wait and signal
http://effoaddon.googlecode.com/svn/trunk/devel/effo/codebase/addons/queue/include/iqb_ops_i.h.
you'd better study something underlying, not just high level interfaces, though write C++.
I've seen several 'real-world' implementations of threading using C++, and they have all been implemented by someone's writing a Thread class to wrap the underlying O/S API
For example, on Windows there's a CreateThread API: your Thread class would pass its this value to the void* lpParameter parameter of the CreateThread API; and your LPTHREAD_START_ROUTINE, which you implement as a private static method of the Thread class, then needs to cast void* back to Thread*' in order to get theThread` instance.
I am involved in a venture that will port some communications, parsing, data handling functionality from Win32 to Linux and both will be supported. The problem domain is very sensitive to throughput and performance.
I have very little experience with performance characteristics of boost and ACE. Specifically we want to understand which library provides the best performance for threading.
Can anyone provide some data -- documented or word-of-mouth or perhaps some links -- about the relative performance between the two?
EDIT
Thanks all. Confirmed our initial thoughts - we'll most likely choose boost for system level cross-platform stuff.
Neither library should really have any overhead compared to using native OS threading facilities. You should be looking at which API is cleaner. In my opinion the boost thread API is significantly easier to use.
ACE tends to be more "classic OO", while boost tends to draw from the design of the C++ standard library. For example, launching a thread in ACE requires creating a new class derived from ACE_Task, and overriding the virtual svc() function which is called when your thread runs. In boost, you create a thread and run whatever function you want, which is significantly less invasive.
Do yourself a favor and steer clear of ACE. It's a horrible, horrible library that should never have been written, if you ask me. I've worked (or rather HAD to work with it) for 3 years and I tell you it's a poorly designed, poorly documented, poorly implemented piece of junk using archaic C++ and built on completely brain-dead design decisions ... calling ACE "C with classes" is actually doing it a favor. If you look into the internal implementations of some of its constructs you'll often have a hard time suppressing your gag reflex.
Also, I can't stress the "poor documentation" aspect enough. Usually, ACE's notion of documenting a function consists of simply printing the function's signature. As to the meaning of its arguments, its return value and its general behavior, well you're usually left to figure that out on your own. I'm sick and tired of having to guess which exceptions a function may throw, which return value denotes success, which arguments I have to pass to make the function do what I need it to do or whether a function / class is thread-safe or not.
Boost on the other hand, is simple to use, modern C++, extremely well documented, and it just WORKS! Boost is the way to go, down with ACE!
Don't worry about the overhead of an OS-abstraction layer on threading and synchronization objects. Threading overhead literally doesn't matter at all (since it only applies to thread creation, which is already enormously slow compared to the overhead of a pimpl-ized pointer indirection). If you find that mutex ops are slowing you down, you're better off looking at atomic operations or rearranging your data access patterns to avoid contention.
Regarding boost vs. ACE, it's a matter of "new-style" vs. "old-style" programming. Boost has a lot of header-only template-based shenanigans (that are beautiful to work with, if you can appreciate it). If, on the other hand, you're used to "C with classes" style of C++, ACE will feel much more natural. I believe it's mostly a matter of personal taste for your team.
I've used ACE for numerous heavy duty production servers. It never failed me. It is rock solid and do the work for many years now. Tried to learn BOOST's ASIO network framework-Couldn't get the hang of it. While BOOST is more "modern" C++, it also harder to use for non trivial tasks - and without a "modern" C++ experience and deep STL knowledge it is difficult to use correctly
Even if ACE is a kind of old school C++, it still has many thread oriented features that boost doesn't provide yet.
At the moment I see no reason to not use both (but for different purposes). Once boost provide a simple mean to implement message queues between tasks, I may consider abandoning ACE.
When it comes to ease-of-use, boost is way better than ACE. boost-asio has a more transparent API, its abstractions are simpler and can easily provide building blocks to your application. The compile-time polymorphism is judiciously used in boost to warn/prevent illegal code. ACE's uses of templates, on the other hand, is limited to generalization and is hardly ever user-centric enough to disallow illegal operations. You're more likely to discover problems at run-time with ACE.
A simple example which I can think of is ACE_Reactor - a fairly scalable and decoupled interface- but you must remember to call its "own" function if you're running its event loop in a thread different from where it was created. I spent hours to figure this out for the first time and could've easily spent days. Ironically enough its object model shows more details than it hides - good for learning but bad for abstraction.
https://groups.google.com/forum/?fromgroups=#!topic/comp.soft-sys.ace/QvXE7391XKA
Threading is really only a small part of what boost and ACE provide, and the two aren't really comparable overall. I agree that boost is easier to use, as ACE is a pretty heavy framework.
I wouldn't call ACE "C with classes." ACE is not intuitive, but if you take your time and use the framework as intended, you will not regret it.
From what I can tell, after reading Boost's docs, I'd want to use ACE's framework and Boost's container classes.
Use ACE and boost cooperatively. ACE has better communication API, based on OO design patterns, whereas boost has like "modern C++" design and works well with containers for example.
We started to use ACE believing that it would hide the platform differences present between windows and unix in TCP sockets and the select call. Turns out, it does not. Ace's take on select, the reactor pattern, cannot mix sockets and stdin on windows, and the semantic differences between the platforms concerning socket writablility notifications are still present at the ACE level.
By the time we realized this we were already using the thread and process features of ACE (the latter of which again does not hide the platform differences to the extent we would have liked) so that our code is now tied to a huge library that actually prevents the porting of our code to 64 Bit MinGW!
I can't wait for the day when the last ACE usage in our code is finally replaced with something different.
I've been using ACE for many years (8) but I have just started investigating the use of boost again for my next project. I'm considering boost because it has a bigger tool bag (regex, etc) and parts of it are getting absorbed into the C++ standard so long term maintenance should be easier.
That said, boost is going to require some adjustment. Although Greg mentions that the thread support is less invasive as it can run any (C or static) function, if you're used to using thread classes that are more akin to the Java and C# thread classes which is what ACE_Task provides, you have to use a little finesse to get the same with boost.