QtConcurrent is awesome.
I'll let the Qt docs speak for themselves:
QtConcurrent includes functional programming style APIs for parallel list processing, including a MapReduce and FilterReduce implementation for shared-memory (non-distributed) systems, and classes for managing asynchronous computations in GUI applications.
For instance, you give QtConcurrent::map() an iterable sequence and a function that accepts items of the type stored in the sequence, and that function is applied to all the items in the collection. This is done in a multi-threaded manner, with a thread pool equal to the number of logical CPU's on the system.
There are plenty of other function in QtConcurrent, like filter(), filteredReduced() etc. The standard CompSci map/reduce functions and the like.
I'm totally in love with this, but I'm starting work on an OSS project that will not be using the Qt framework. It's a library, and I don't want to force others to depend on such a large framework like Qt. I'm trying to keep external dependencies to a minimum (it's the decent thing to do).
I'm looking for a generic C++ framework that provides the same/similar high-level primitives that QtConcurrent does, and that works with STL collections. AFAIK boost has nothing like this (I may be wrong though). boost::thread is very low-level compared to what I'm looking for (but if the requested lib used boost::thread for the low-level work, that would be great).
I know C# has something very similar with their Parallel Extensions so I know this isn't a Qt-only idea.
What do you suggest I use?
I've heard good things about Intel's Threaded Building Blocks, though I haven't used it
As of Oct 2009, it doesn't seem to have map-reduce specifically. But people have expressed interest and suggested they were going to come up with something:
http://software.intel.com/en-us/forums/showthread.php?t=65053
"map reduce looks like a simple combination of a filter, a sort, and a reduction but it might need some magic to get it to be efficient"
Can you use Boost? I don't think it provides quite as high-abstraction a layer as Qt, but it should be possible to make one as a reasonably thin facade on top of Boost's primitives (indeed, maybe some of the existing add-ons already provide what you require -- I have to admit I'm not familiar with them in detail, which is why I say "maybe";-).
If you find out that existing add-ons are unsuitable, your facade would be an excellent add-on to contribute to the Boost Vault (or other open-source repo) yourself, "giving back" a useful reusable open-source contribution... I hope this motivates you to do this work if needed!-)
Related
I'm recently learn F# asynchronous workflows, which is an important feature of F# concurrency. What confused me is that how many approaches to write concurrent code in F#? I read Except F#, and some blog about F# concurrency, I know things like background workers; IAsyncResult; If programming on local machine, there is shard-memory concurrency in F#; If programming on distributed system, there is message-passing concurrency. But I really not sure what is the relationship between these techniques, and how to classify them. I understand it is quite a "big" question cannot be answer with one or two sentences, so I would definitely appreciate if anyone can give me specific answer or recommend me some useful references.
I'm also rather new to F#, so I hope more answers come to complement this one :)
The first thing is you need to distinguish between .NET classes (which can be used from any .NET language) and F# unique ways to deal with asynchronous operations. In the first case as you mention and among others, you have:
System.ComponentModel.BackgroundWorker: This was used mainly in the first .NET versions with Windows Forms and it's not recommended anymore.
System.IAsyncResult: This is also an old .NET interface implemented by several classes (also Task) but I don't usually use it directly.
Windows.Foundation.IAsyncOperation: Another interface but used only in Windows Store apps. Most of the times you translate it directly to Task, so you don't have to worry too much about it.
System.Threading.Tasks.Task: This is the recommended way now to handle .NET asynchronous and parallel (with the Parallel Task Library) operations. It's the hidden force behind C# async/await keywords, which are just syntactic sugar to pass continuations to Tasks.
So now with F# unique ways: Asynchronous Workflows and MailboxProcessor. It can roughly be said the former corresponds to parallelism while the latter deals with concurrency.
Asynchronous Workflows: This is just a computation expression (aka monad) which happens to deal with asynchrony: operations that run in the background to prevent blocking the UI thread or parallel operations to get the maximum performance in multi-core systems.
It's more or less the equivalent to C# async/await but we F# fans like to think it's a more elegant solution because it uses a more generic and flexible mechanism (computation expressions) which can be adapted for example to asynchronous sequences, events or even Javascript callbacks. It has also other advantages as Thomas Petricek explains here.
Within an asynchronous workflow most of the time you'll be using the methods in Control.Async or the extensions to .NET classes (like WebRequest.AsyncGetResponse) from the F# Core Library. If necessary, you can also interact directly with .NET Tasks (Async.AwaitTask and Async.StartAsTask) or even easily create your own async operations with Async.StartWithContinuations.
To learn more about asynchronous workflows you can consult the MSDN documentation, the magnificent Scott Wlaschin's site, Tomas Petricek's blog or the F# Wikibook.
Control.MailboxProcessor: Designed to deal with concurrency, that is, several processes running at the same time which usually need to share some information. The traditional .NET way to prevent memory corruption when several threads try to write a variable at the same time was the lock statement. Besides the fact that functional style prefers to use immutable values, memory locks are complicated to use properly and can also have a high performance penalty. So instead of this, MailboxProcessor uses an Erlang-like message-based (or actor-based) approach to concurrency.
I have not used MailboxProcessor myself that much, but for more info you can check Scott Wlaschin's site or the F# Wikibook.
I hope this helps! If someone sees something not completely correct in this answer, please feel free to edit it.
Cheers!
I am currently looking for a discrete event simulator written for C++. I did not find much on the web written specifically in OO-style; there are some, but outdated. Some others, such as Opnet, Omnet and ns3 are way too complicated for what I need to do. And besides, I need to simulate agent-based algorithms capable of simulating systems of thousands of nodes.
Does anybody know anything suitable for my needs?
Others have good direct answers, but I'm going to suggest an alternative. If I understand you right, you want a system in C++ or such where you can post events that fire in the future, and code is run when those events fire.
I had a project to do like this, and I started out trying to write such an event system in C++ and then quickly realized I had a better solution.
Have you considered writing your program in behavioral Verilog? That may seem strange to write software in a hardware description language, but a Verilog simulator is an event-based system underneath, and behavioral Verilog is a very convenient way to express events, timing, triggers, etc. There is a free Verilog simulator (which is what I used) called Icarus Verilog. If you're not using Ubuntu or some Linux distro with Icarus already in a package, building from source is straightforward.
I would recommend having a second look to OmNet++. At first sight it may look quite complex, but if you look it into more detail you will find that most of the complexity is in the network add-on (the INET Framework). Unless you are going to do a detailed network simulation you do not need the INET.
Using OmNet++ core is not specially difficult and it may be simpler than other similar tools.
You may want to have a look to an intro.
One of the things that makes OmNet++ attractive to me is its scalability. Is possible to run large simulations in a desktop. Besides, it is possible to scale the same simulation to a cluster without rewriting the code.
You should consider SystemC, although I'd also recommend taking a second look at OmNet++.
We use SIMLIB at my school. It is very fast, easy to understand, object oriented, discrete and continuous simulator. It might look outdated but it is still maintained.
There is CSIM from Mesquite Software which supports developing models in C, C++ and Java. However, it is paid-commercial, AFAIK.
Take a look at GBL library. It's written in modern C++ and even supports C++0x features like move semantics and lambda functions. It offers several modeling mechanisms: synchronous and asynchronous event handlers, preemptive threads, and fibers. You can create purely behavioral, cycle accurate, and real-time models, or any mixture of those.
From some browsing on net, I just understood that any framework is set of libraries provided by the framework and we can simply use those library functions to develop the application.
I would like to know more about
what is a framework with respect to C++.
How are C++ frameworks designed?
How can we use them and develop applications.
Can someone provide me some links to understand the concept of "framework" in C++
A "framework" is something designed to provide the structure of a solution - much as the steel frame of a skyscraper gives it structure, but needs to be fleshed out with use-specific customisations. Both assume some particular problem space - whether it's multi-threaded client/server transactions, or a need for air-conditioned office space, and if your needs are substantively different - e.g. image manipulation or a government art gallery - then trying to use a poorly suited framework is often worse than using none. Indeed, if the evolving needs of your system pass beyond what the framework supports, you may find your options for customising the framework itself are insufficient, or the design you adopted to use it just doesn't suit the re-architected solution you later need. For example, a single-threaded framework encourages you to program in a non-threadsafe fashion, which may be a nightmare to make efficiently multi-threaded post-facto.
They're designed by observing that a large number of programs require a similar solution architecture, and abstracting that into a canned solution framework with facilities for those app-specific customisations.
How they're used depends on the problems they're trying to solve. A framework for transaction dispatch/handling will typically define a way to list IP ports to listen on, nominate functions to be called when connections are made and new data arrives, register timer events that call back to arbitrary functions. XML document, image manipulation, A.I. etc. frameworks would be totally different.... The whole idea is that they each provide a style of use that is simple and intuitive for the applications that might wish to use them.
A big hassle with many frameworks is that they assume ownership of the applications that use them, and relegate the application to a secondary role of filling in some callbacks. If the application needs to use several frameworks, or even one framework with some extra libraries doing e.g. asynchronous communications, then the frameworks may make that very difficult. A good framework is designed more like a set of libraries that the client can control, but need not be confined by. Good frameworks are rare.
More often than not, a framework (as opposed to "just" a library or set of libraries), in OOP languages (including C++), implies a software subsystem that, among other things, supplies classes you're supposed to inherit from, overriding certain methods to specialize the class's functionality for your application's needs, in your application code. If it was just some collection of functions and typedefs it should more properly be called a library, rather than a framework.
I hope this addresses your points 1 and 3. Regarding point 2, ideally, the designers of a framework have a lot of experience designing applications in a certain area, and they "distill" their experience and skill into a framework that lets (possibly less-experienced) developers build their own applications in that area more easily and expeditiously. In the real world, of course, such ideals are not always followed.
With a tool like CppDepend you can analyze any C++ framework, reverse engineer its design in a minute, but also have an accurate idea of the overall code quality of the framework.
An application framework (regardless of language) is a library that attempts to provide a complete framework within which you plug in functionality for your specific application.
The idea is that things like web applications and GUI applications typically require quite a bit of boilerplate to get working at all. The application framework provides all that boilerplate code, and some sort of organization (typically some variation of model-view-controller) where you can plug in the logic specific to your particular application, and it handles most of the other stuff like automatically routing messages and such as needed.
I hear a lot about boost here and I am beginning to think it could help a lot with my software development. More so in concurrency and memory management in my particular case as we have had a lot of bugs in this area.
What are the key language features I need to polish up on to effectively benefit from using boost and to shorten the learning curve? I have seen that function objects are commonly used so I would probably need to polish up on that.
Additionally, are there any tutorials and 101 resources I can quickly look at to just get a feel and understanding on using boost.
I realise there is a lot boost offers and I have to pick the right tools for the right job but any leads will help.
Related
How to learn boost (no longer valid; HTTP return status 404)
Boost has an unimaginable number of libraries.
Easy ones to get started on are
noncopyable
array
circular_buffer
foreach
operators (one of my personal favorites)
smart_ptr
date_time
More advanced ones include
lambda
bind
iostreams
serialization
threads
Getting used to boost takes time, but I assure you it will make your life much better. Plus, looking at how the boost libraries are coded will help you get better at c++ coding, especially templates.
You mentioned what should you look up before trying boost. I agree that function objects are a great thing to research. Also, make sure to look up about template programming. A common problem to make sure you know is when to use the typename qualifier for dependent types. For the most part, however, the libraries are very well documented, with examples and reference materials.
Learning boost is discussed here. As for language features that are useful? All of them. C++ is a dangerous language to use if you don't know enough of it. RAII, functors/function objects and templates probably cover the most important aspects. Boost is designed similarly to the STL, so knowing your standard library is essential. Boost itself uses a lot of template metaprogramming, but as a library user, you won't often need that (unless you start playing with Boost.MPL)
Bugs related to memory management are a good indicator that it's C++, rather than Boost you need to brush up on. The techniques for handling memory safely are well known, and not specific to Boost. (With the obvious exception of Boost's smart pointers). RAII is probably the most important concept to understand to deal with this kind of issues.
What are the key language features I need to polish up on to effectively benefit from using boost and to shorten the learning curve?
Templates
Functors
Exceptions
STL
Iterators
Algorithms
Containers
... among others.
are there any tutorials and 101 resources I can quickly look at to just get a feel and understanding on using boost.
Boost is well documented. Start here.
There are too many libraries to get lost. I'd say start with something simple, maybe smart pointers or Boost.Test (Unit Test framework) -- which will quickly help you get started. Also, try to think of a problem you cannot solve with the STL easily. Then look up Boost documentation or post here.
If you are comfortable with functional programming look at MPL/Lambda libraries.
The first ting IMO are smart pointers. Integration into new code is simple, and usually not a problem for existing code. They make memory management easy, and work for many other ressources, too.
C++ gives you the power to manage your own memory, smart pointers let you (mostly) wing it when you don't need to.
The second would be - as you mentioned - function objects, they close a big gap within C++ that is traditionally solved through inheritance, which is to strong of a coupling in many cases.
I have only little experience with boost outside these two, but most of the remainder is fairly "situational" - you may or may not need it. Get an overview over the libraries, and see what you need.
boost::any and boost::variant are good of you need a variant data type, with two different approaches.
boost::regex if you need some text parsing.
boost::thread and boost::filesystem help you write portable code. If you already have good platform specific libraries, you might not need them - but they are better than API or C++ level in any case.
Maybe you like my introduction to boost smart pointers, and a rather unorthodox use for them.
Try Björn Karlsson's book: Beyond the C++ Standard Library: An Introduction to Boost. Its pretty straightforward and easy to grasp. I read this after I'd finished Scott Meyers three c++ books (effective series).
After reading Beyond the C++ Standard Library: An Introduction to Boost, I would recommend casually browsing the documentation on boost.org, just to get an idea of what's available. You can do a deep dive into a specific boost library when it looks like a good fit for a particular application.
I think shared_ptr sould be the easiest place to start .
Start using it inplaces of simple pointer or auto_ptr data types.
You can also look into weak_ptr.
Do you use Luabind, toLua++, or some other library (if so, which one) or none at all?
For each approach, what are the pro's and con's?
I can't really agree with the 'roll your own' vote, binding basic types and static C functions to Lua is trivial, yes, but the picture changes the moment you start dealing with tables and metatables; things go trickier very quickly.
LuaBind seems to do the job, but I have a philosophical issue with it. For me it seems like if your types are already complicated the fact that Luabind is heavily templated is not going to make your code any easier to follow, as a friend of mine said "you'll need Herb Shutter to figure out the compilation messages". Plus it depends on Boost, plus compilation times get a serious hit, etc.
After trying a few bindings, Tolua++ seems the best. Tolua doesn't seem to be very much in development, where as Tolua++ seems to work fine (plus half the 'Tolua' tutorials out there are, in fact, 'Tolua++' tutorials, trust me on that:) Tolua does generate the right stuff, the source can be modified and it seems to deal with complicated cases (like templates, unions, nameless structs, etc, etc)
The biggest issue with Tolua++ seems to be the lack of proper tutorials, pre-set Visual Studio projects, or the fact that the command line is a bit tricky to follow (you path/files can't have white spaces -in Windows at least- and so on) Still, for me it is the winner.
To answer my own question in part:
Luabind: once you know how to bind methods and classes via this awkward template syntax, it's pretty straightforward and easy to add new bindings. However, luabind has a significant performance impact and shouldn't be used for realtime applications. About 5-20 times more overhead than calling C functions that manipulate the stack directly.
I don't use any library. I have used SWIG to expose a C library some time ago, but there was too much overhead, and I stop using it.
The pros are better performance and more control, but its takes more time to write.
Use raw Lua API for your bindings -- and keep them simple. Take inspiration in the API itself (AUX library) and libraries by Lua authors.
With some practice raw API is the best option -- maximum flexibility and minimum of unneeded overhead. You've got what you want and no more, the way you need it to be.
If you must bind large third-party libraries use automated generators like tolua, tolua++ (or even roll your own for the specific case). It would free you from manual work.
I would not recommend using Luabind. At the moment it's development stalled (however starting to come back to life), and if you would meet some corner case, you may be on your own. Also Luabind heavily uses template metaprogramming. This may (and may not) be unacceptable, depending on the point of view.