I've created a model for executing worker tasks in a server application using a thread pool associated with an IO completion port such as shown in the posts below:
http://weblogs.asp.net/kennykerr/archive/2008/01/03/parallel-programming-with-c-part-4-i-o-completion-ports.aspx
http://blogs.msdn.com/larryosterman/archive/2004/03/29/101329.aspx
Are there any classes in boost that can help with this programming model?
Not really, at least, not last time I looked. I mean, boost::thread_group might make things marginally tidier in places, but not so as would make much of a difference, I don't think.
Boost's thread support seems marginally useful when writing something that's cross-platform, but given that what you're writing is going to be Win32-specific anyway (due to the use of IOCPs) there doesn't really seem to be much benefit from that.
You might want to check out the threadpool project, which looks like a nice threadpool implementation on top of boost. I haven't tried it myself, but it looks fairly nice.
I haven't seen anything in boost that helps with the structure that you tend to end up with when using IO Completion Ports, but then I haven't looked that recently... However, slightly off-topic, you might like to take a look at the IOCP based thread pool that is part of my free IOCP server framework. It might give you some ideas if nothing else. You can find the code here. The thread pool supports expansion and contraction based on demand and has been in use in production systems for over 6 years.
ACE has some reactors that you can use to model things around your IOCPs. Some of these could have been added to boost, but boost makes building them pretty easy.
Related
I am coding network stuff via tcp/ip.
Specifically I have been using boost::asio.
Recently, to ease coding, I started using boost::asio::ip::tcp::iostream.
It can be useful for fast developing! But I am not sure whether it uses async_read or async_write.
Does anybody know whether it does?
No it cannot. That's most of what makes it so simple.
However, you can force a full-duplex experience using boost::asio::ip::tcp::iostream with some tweaks. You'll need to run the reading/writing tasks on separate threads to achieve the parallellism without having asynchrony.
An examples of this - what I consider to be a - hack, is here:
How to avoid data race with `asio::ip::tcp::iostream`?
That answer does also give the asynchronous approach, so that may help remove the intimidation factor because you can simply compare side-by-side both approaches.
I have a homework task that requires me to learn how to use the Guava concurrency library.
In the task I have several thread pools, where each one is controlled by an individual object.
Each pool has several working threads that perform simple tasks (mostly emulating doing stuff by using Thread.sleep(long)) and all those simple tasks are stored in a container object that emulates a messageboard.
Each simple task has a dependency list of other tasks, and it cannot be executed until all of those tasks are completed.
How can I benefit from the Guava library using the ListenableFuture and the Futures.chain()?
I have searched everywhere for some extensive example code online, but didn't find anything that I understand how to use.
As Louis mentions, I think that Futures.allAsList etc. could be useful for you. However, I think that Futures.chain does seem useful and appropriate for the situation you describe. Since this is an assignment meant to challenge you, I'm not going to say any more than this: Futures.chain allows you to submit a task for execution upon completion of another task, and it returns a new ListenableFuture representing the result of that task. How does that apply to what you're trying to do?
You might be interested in reading the presentation slides on Guava util.concurrent, linked on the Guava homepage (slide 11 and onward). They really helped me crystallize my understanding of ListenableFuture and why it's so useful.
My guess is that the goal of your assignment is to understand how Futures work, and how Guava's ListenableFuture and Futures.chain() simplifies their use when coordinating multiple tasks.
The only open source code that comes to mind that uses ListenableFuture is sitebricks-mail:
MailClient interface
NettyImapClient implementation
I don't know if it uses Futures.chain(), though.
I don't think Futures.chain() is the answer here, if the primary issue is dealing with task dependency lists. More likely is Futures.allAsList or Futures.successfulAsList, which take multiple futures, and return a future that returns only after all of the input futures have succeeded or failed.
I am currently looking for a discrete event simulator written for C++. I did not find much on the web written specifically in OO-style; there are some, but outdated. Some others, such as Opnet, Omnet and ns3 are way too complicated for what I need to do. And besides, I need to simulate agent-based algorithms capable of simulating systems of thousands of nodes.
Does anybody know anything suitable for my needs?
Others have good direct answers, but I'm going to suggest an alternative. If I understand you right, you want a system in C++ or such where you can post events that fire in the future, and code is run when those events fire.
I had a project to do like this, and I started out trying to write such an event system in C++ and then quickly realized I had a better solution.
Have you considered writing your program in behavioral Verilog? That may seem strange to write software in a hardware description language, but a Verilog simulator is an event-based system underneath, and behavioral Verilog is a very convenient way to express events, timing, triggers, etc. There is a free Verilog simulator (which is what I used) called Icarus Verilog. If you're not using Ubuntu or some Linux distro with Icarus already in a package, building from source is straightforward.
I would recommend having a second look to OmNet++. At first sight it may look quite complex, but if you look it into more detail you will find that most of the complexity is in the network add-on (the INET Framework). Unless you are going to do a detailed network simulation you do not need the INET.
Using OmNet++ core is not specially difficult and it may be simpler than other similar tools.
You may want to have a look to an intro.
One of the things that makes OmNet++ attractive to me is its scalability. Is possible to run large simulations in a desktop. Besides, it is possible to scale the same simulation to a cluster without rewriting the code.
You should consider SystemC, although I'd also recommend taking a second look at OmNet++.
We use SIMLIB at my school. It is very fast, easy to understand, object oriented, discrete and continuous simulator. It might look outdated but it is still maintained.
There is CSIM from Mesquite Software which supports developing models in C, C++ and Java. However, it is paid-commercial, AFAIK.
Take a look at GBL library. It's written in modern C++ and even supports C++0x features like move semantics and lambda functions. It offers several modeling mechanisms: synchronous and asynchronous event handlers, preemptive threads, and fibers. You can create purely behavioral, cycle accurate, and real-time models, or any mixture of those.
I'm fairly new to advanced C++ program techniques such as templates,
but I am developing a simple API for a project I'm working on.
The function or method that you call can take a long time to complete.
Essentially it's transferring a file over the network.
It looks a bit like this.
Client
{
int WriteFile();
int ReadFile();
}
But I want to have a couple of options here.
call WriteFile and have it block.
Call WriteFileAsync and not have it block.
In the async version be flexible about how I know the task is done.
Be able to poll the client to find out where it's up to with my current Read or Write
operation.
I'm at a bit of a loss as to how to design this nicely the C++ way.
It's a requirement to avoid using boost, but I could use a boost-like approach.
Although, I looked through some of the headers and got very much confused. Anything beyond
basic template programming for me I find confusing.
What I'm after is a nice way of being notified of event completion and be able to wait for
an event to complete.
My advice would be looking at the docs and tutorial for boost::asio (which you can use as part of boost or as part of the independent asio project, but I guess that the requirement is no external libs, not just no boost).
Usually blocking calls are simple to define, while non-blocking operations require some callback mechanism as to notify the user of the result of the operation whenever that completes. Again, take a look at the tutorials and docs to get an idea of a clean interface, that will be much easier to browse over than the headers.
EDIT: ASIO has support for different protocols, so it might be more complex than what you need, read one of the examples and get the ideas of how to use callback mechanisms.
Regarding the use of asynchronous calls, I would suggest reading about the design of the future for C++0x.
Basically, the idea is to hand a proxy to the user, instead of a plain type. This proxy is aware of the threading and can be used to:
poll about the completion
get the result
You can also add clever mechanisms like trying to get the result for a fixed duration or up to a fixed point in time and abandon (for the moment) if the task hasn't completed in time (for example to do something else and try again later, or to simple go forward and forget about this).
The new threading API of C++0x has been very cleverly designed (taking mainly after Boost.Threads) and would give you much insight as to how to design for multi-threading.
QtConcurrent is awesome.
I'll let the Qt docs speak for themselves:
QtConcurrent includes functional programming style APIs for parallel list processing, including a MapReduce and FilterReduce implementation for shared-memory (non-distributed) systems, and classes for managing asynchronous computations in GUI applications.
For instance, you give QtConcurrent::map() an iterable sequence and a function that accepts items of the type stored in the sequence, and that function is applied to all the items in the collection. This is done in a multi-threaded manner, with a thread pool equal to the number of logical CPU's on the system.
There are plenty of other function in QtConcurrent, like filter(), filteredReduced() etc. The standard CompSci map/reduce functions and the like.
I'm totally in love with this, but I'm starting work on an OSS project that will not be using the Qt framework. It's a library, and I don't want to force others to depend on such a large framework like Qt. I'm trying to keep external dependencies to a minimum (it's the decent thing to do).
I'm looking for a generic C++ framework that provides the same/similar high-level primitives that QtConcurrent does, and that works with STL collections. AFAIK boost has nothing like this (I may be wrong though). boost::thread is very low-level compared to what I'm looking for (but if the requested lib used boost::thread for the low-level work, that would be great).
I know C# has something very similar with their Parallel Extensions so I know this isn't a Qt-only idea.
What do you suggest I use?
I've heard good things about Intel's Threaded Building Blocks, though I haven't used it
As of Oct 2009, it doesn't seem to have map-reduce specifically. But people have expressed interest and suggested they were going to come up with something:
http://software.intel.com/en-us/forums/showthread.php?t=65053
"map reduce looks like a simple combination of a filter, a sort, and a reduction but it might need some magic to get it to be efficient"
Can you use Boost? I don't think it provides quite as high-abstraction a layer as Qt, but it should be possible to make one as a reasonably thin facade on top of Boost's primitives (indeed, maybe some of the existing add-ons already provide what you require -- I have to admit I'm not familiar with them in detail, which is why I say "maybe";-).
If you find out that existing add-ons are unsuitable, your facade would be an excellent add-on to contribute to the Boost Vault (or other open-source repo) yourself, "giving back" a useful reusable open-source contribution... I hope this motivates you to do this work if needed!-)