In the introduction to The Art of Prolog, Sterling and Shapiro defer a discussion of parallelism, concurrency, and logic programming to another book. My question is whether there is such a resource:
[the] promise of parallel computers, combined with the parallelism that seems to be available in the logic programming model, have lead to numerous attempts, which are still ongoing, to execute Prolog in parallel, and to devise novel concurrent programming languages based on the logic programming computation model. This, however, is a subject for another book (The Art of Prolog, p. xx).
Searching on Google, I found a parallel implementation of Prolog and concurrency libraries for Mercury, in addition to hundreds of research papers and dissertations. But it's harder to locate resources on the second part of the paragraph, about concurrent programming and programming languages based on the execution models of logic programming languages. Is there a good resource on these topics? I'm particularly interested in references on compiling and writing parallel and concurrent logic programs.
In Prolog you can have both non-determinism and concurrency. Non-determinsim is what is usually described as unification and backtracking. You can imagine that a Prolog clause is full of implicit amb statements. It is less known that concurrency is also supported by logic-programming.
But today we might just go with treads inside logic programming. Here is an example to implement a findall via threads. This can also be modded to perform all kinds of tasks on the collection, or maybe even produce agent networks towards distributed artificial intelligence.
There is a even a proposal for a set of ISO standard predicates that support threading inside Prolog. These predicates also cover synchronization and queueing primitives. But more important, light weighted Prolog based web servers wouldn't work, there werent multi-threaded Prolog systems around:
ISO/IEC DTR 13211–5:2007 Prolog multi-threading support
http://logtalk.org/plstd/threads.pdf
Related
I'm recently learn F# asynchronous workflows, which is an important feature of F# concurrency. What confused me is that how many approaches to write concurrent code in F#? I read Except F#, and some blog about F# concurrency, I know things like background workers; IAsyncResult; If programming on local machine, there is shard-memory concurrency in F#; If programming on distributed system, there is message-passing concurrency. But I really not sure what is the relationship between these techniques, and how to classify them. I understand it is quite a "big" question cannot be answer with one or two sentences, so I would definitely appreciate if anyone can give me specific answer or recommend me some useful references.
I'm also rather new to F#, so I hope more answers come to complement this one :)
The first thing is you need to distinguish between .NET classes (which can be used from any .NET language) and F# unique ways to deal with asynchronous operations. In the first case as you mention and among others, you have:
System.ComponentModel.BackgroundWorker: This was used mainly in the first .NET versions with Windows Forms and it's not recommended anymore.
System.IAsyncResult: This is also an old .NET interface implemented by several classes (also Task) but I don't usually use it directly.
Windows.Foundation.IAsyncOperation: Another interface but used only in Windows Store apps. Most of the times you translate it directly to Task, so you don't have to worry too much about it.
System.Threading.Tasks.Task: This is the recommended way now to handle .NET asynchronous and parallel (with the Parallel Task Library) operations. It's the hidden force behind C# async/await keywords, which are just syntactic sugar to pass continuations to Tasks.
So now with F# unique ways: Asynchronous Workflows and MailboxProcessor. It can roughly be said the former corresponds to parallelism while the latter deals with concurrency.
Asynchronous Workflows: This is just a computation expression (aka monad) which happens to deal with asynchrony: operations that run in the background to prevent blocking the UI thread or parallel operations to get the maximum performance in multi-core systems.
It's more or less the equivalent to C# async/await but we F# fans like to think it's a more elegant solution because it uses a more generic and flexible mechanism (computation expressions) which can be adapted for example to asynchronous sequences, events or even Javascript callbacks. It has also other advantages as Thomas Petricek explains here.
Within an asynchronous workflow most of the time you'll be using the methods in Control.Async or the extensions to .NET classes (like WebRequest.AsyncGetResponse) from the F# Core Library. If necessary, you can also interact directly with .NET Tasks (Async.AwaitTask and Async.StartAsTask) or even easily create your own async operations with Async.StartWithContinuations.
To learn more about asynchronous workflows you can consult the MSDN documentation, the magnificent Scott Wlaschin's site, Tomas Petricek's blog or the F# Wikibook.
Control.MailboxProcessor: Designed to deal with concurrency, that is, several processes running at the same time which usually need to share some information. The traditional .NET way to prevent memory corruption when several threads try to write a variable at the same time was the lock statement. Besides the fact that functional style prefers to use immutable values, memory locks are complicated to use properly and can also have a high performance penalty. So instead of this, MailboxProcessor uses an Erlang-like message-based (or actor-based) approach to concurrency.
I have not used MailboxProcessor myself that much, but for more info you can check Scott Wlaschin's site or the F# Wikibook.
I hope this helps! If someone sees something not completely correct in this answer, please feel free to edit it.
Cheers!
Community Wiki
I don't care about the reputation points, I just want good answers. Feel free to remark this question as community wiki.
Context
I'm been working through The Reasoned Schemer, and have found the following observations:
Logic programming is very interesting.
Logic programming is sometimes counter-intuitive
Logic programming is often "inefficient" (or at least the code I write).
It seems like in going from
Assembly -> C++, I "give up" control of writing my own machine code
C++ -> Clojure, I give up control of memory management
Clojure -> core.logic/prolog/minikanren, I lose partial control of how computations are done
Question:
Besides (1) solving logic puzzles and (2) type inference, what are the domains of problems that logic programming dominates?
Thanks!
Constraint logic programming can be really useful for solving various scheduling, resource allocation, and other nontrivial constraint satisfaction / combinatorial optimization problems. All you have is declarative: the constraints (e.g. only one aircraft can be on the runway at a time), and maybe something you want to minimize/maximize (throughput/waiting).
There are various well-known flavors of that in Prolog, including CLP(FD), which works in finite integer domain, and CLP(R), which works in real domain. At least CLP(FD) seems to be in core.logic's immediate roadmap.
I believe such Prolog-derived solutions are actively being used in air traffic control and other logistics tasks, although it's hard to get precise info what technologies exactly such mission- and life-critical companies are using under the hood.
Research in artificial intelligence, and in particular cognitive robotics and other application of logic-based knowledge representation, are areas where Prolog is used a lot for its close relation to logic theory. This relation is very useful because it basically brings theory to life. Theorems can be proven on paper, and then implemented almost trivially in prolog and executed, and the executing programs have the proven properties. This allows for programs that are "correct by construction", which is the opposite of first writing programs and then trying to prove properties about them (as is done in formal methods, using, e.g., model checking).
The semantic web is another place where logic programming plays a growing role.
I want to implement my particle filtering algorithm in parallel in Common Lisp. Particle Filtering and sampling can be parallelized and I want to do this for my 4-core machine. My question is whether programming in parallel is feasible in CL or not and if it is feasible are there any good readings, tutorials about getting started to parallel computing in CL.
Definitely feasible!
The Bordeaux Threads project provides thread primitives for a number of implementations; I would suggest using it instead of SBCL's implementation-specific primitives (especially if you aren't on SBCL!).
The thread primitives are provided by bt are, however, quite primitive. I've used and enjoyed Eager Future2 which builds on bt to provide concurrency features using futures. You can create futures that are computed lazily, eagerly (immediately), or speculatively. The speculative futures are computed by a thread pool whose size can be customized.
I started a little project to provide parallel versions of CL functions using EF2, but it's only about three functions so far, so it won't be of much use to anyone. I do of course welcome other coders to hack on it and submit pull requests, and I hope to do more work on it in the future.
There are many other libraries listed on Cliki that I haven't tried myself.
As far as tutorials, I don't know of any, but the concurrency features provided are found in other languages as well and good algorithms and practices are not generally language-specific.
If you're interested in reading a book, I recommend The Concurrent C Programming Language. The authors describe a new programming language, based on C, with concurrency as a language feature. Of course, due to the nature of CL, it would likely be possible to implement these features without resorting to creating a new compiler. In my opinion the book presents excellent concurrency concepts, and addresses many of the problems you may encounter or fail to consider in writing concurrent programs.
SBCL has some multithreading support. It is too low level and, to my knowledge, does not include any parallel algorithms. It has just the posibility of creating threads that execute some lambda function and test afterwards if the thread has finished (joining it). I used that support to generate my blog pages with great speedup (each page or set of pages in a different thread). You can see the code here:
https://github.com/dsevilla/functional-mind-blog/blob/master/blog/process.lisp
For eample, generating a thread for each page was something like:
#+sbcl
(defun generate-post-pages ()
(map nil
#'(lambda (post)
(make-thread (lambda () (page-generation-function post))))
*posts*))
You can also join-thread, and have mutexes, etc. You can read the documentation here: SBCL Threading. It is too low-level, though. You'll end missing the fantastic features of Clojure for concurrency...
Check out bordeaux threads if you're looking for a single POSIX-threads-style interface to multi-threading primitives for different Lisps.
If I were looking for a reliable free Lisp implementation, I'd start with CCL and then try SBCL. I use CCL for almost all of my testing and SBCL and LispWorks for the remainder.
Sedach's futures library should provide a higher level interface. There are also some other contributions from various users in SBCL's contrib directory.
This coming from someone who has used neither bordeaux-threads nor Sedach's futures library and has written his own version of both of those. I could send you my implementation, but these two packages are also supposed to be good, and they're probably a better starting point.
LispWorks 6 comes with a nice set of primitives for concurrent programming.
Note though that to my knowledge none of the usual Common Lisp implementations has a concurrent Garbage Collector.
Documentation for LispWorks 6 and Multiprocessing
The MP Package
Multiprocessing
I'm interested in using OCaml for a project, however I'm not sure about where its parallelization capabilities are anymore. Is there a message passing ability in OCaml? Is OCaml able to efficiently use more than 1 CPU?
Most of what I have read on the subject was written in 2002-2006, and I haven't seen anything more recent.
Thanks!
This 2009 issue of the Caml weekly news ("CWN", a digest of interesting messages from the caml list) shows that:
the official party line on threads and Ocaml hasn't changed. A notable quote:
(...) in general, the whole standard library is not thread-safe. Probably that should be stated in the
documentation for the threads library, but there isn't much point in documenting it per standard library module. -- X. Leroy
(for how Ocaml threads can still be useful, see a remark by the culprit himself in another question on SO)
the most frequently adopted paradigm for parallelism is message-passing, and of note is X. Leroy's OcamlMPI, providing bindings for programming in SPMD style against the MPI standard. The same CWN issue I pointed to above provides references to examples, and numerous other related projects.
another message-passing solution is JoCaml, pioneering new style of concurrent communications known as join calculus. Note that it is binary-compatible with OCaml compilers.
that did not prevent the confection of a runtime whose GC is ok with parallelism, though: see a discussion of OCAML4MC in this other issue of the CWN.
There is also:
Netmulticore - multi-processing sharing ocaml values via mapped shared memory.
CamlP3l - compiler for Caml parallel programs.
OCaml-Java - an OCaml compiler that emits Java bytecode
I haven't followed more recent discussions about Ocaml & parallel programming, though. I'm leaving this CW so that others can update what I mention. It would be great if this question could reach the same level of completeness as the analogous one for Haskell.
At present, the OCaml runtime does not support running across multiple cores in parallel, so a single OCaml process cannot take advantage of multiple cores. This is unlikely to change directly; the direction the OCaml developers are most interested in taking for increased parallelism seems to be allowing multiple OCaml runtimes to run in parallel in a single process; this will allow for very fast message passing, but will not allow multiple threads to run in parallel in a shared-memory configuration. The major hangup is the garbage collector; some years ago, the team experimented with a concurrent GC, but it introduced unacceptable slowdowns in the single-threaded case.
There are a couple of projects, namely Functory and OCamlnet, which provide multicore-happy parallelism by using multiple processes.
In general, the OCaml community tends to favor message passing approaches, which can be done across process boundaries (like OCamlnet does), over single-process shared-memory multithreading. If your program can be split into multiple processes (many can!), then yes, you can efficiently use multiple CPUs.
BSMLlib provides a simplified programming interface for data-parallel programming in OCaml.
Its execution amounts to BSP-style message passing but it is deterministic and even declarative for a subset of OCaml.
The key concept is the 'a par type which corresponds to a vector of values, one per process.
http://traclifo.univ-orleans.fr/BSML/
http://fr.wikipedia.org/wiki/Bulk_Synchronous_Parallel_ML
Gaétan Hains
University Paris-Est
I'm currently writing a large multi threaded C++ program (> 50K LOC).
As such I've been motivated to read up alot on various techniques for handling multi-threaded code. One theory I've found to be quite cool is:
http://en.wikipedia.org/wiki/Communicating_sequential_processes
And it's invented by a slightly famous guy, who's made other non-trivial contributions to concurrent programming.
However, is CSP used in practice? Can anyone point to any large application written in a CSP style?
Thanks!
CSP, as a process calculus, is fundamentally a theoretical thing that enables us to formalize and study some aspects of a parallel program.
If you instead want a theory that enables you to build distributed programs, then you should take a look to parallel structured programming.
Parallel structural programming is the base of the current HPC (high-performance computing) research and provides to you a methodology about how to approach and design parallel programs (essentially, flowcharts of communicating computing nodes) and runtime systems to implements them.
A central idea in parallel structured programming is that of algorithmic skeleton, developed initially by Murray Cole. A skeleton is a thing like a parallel design pattern with a cost model associated and (usually) a run-time system that supports it. A skeleton models, study and supports a class of parallel algorithms that have a certain "shape".
As a notable example, mapreduce (made popular by Google) is just a kind of skeleton that address data parallelism, where a computation can be described by a map phase (apply a function f to all elements that compose the input data), and a reduce phase (take all the transformed items and "combine" them using an associative operator +).
I found the idea of parallel structured programming both theoretical sound and practical useful, so I'll suggest to give a look to it.
A word about multi-threading: since skeletons addresses massive parallelism, usually they are implemented in distributed memory instead of shared. Intel has developed a tool, TBB, which address multi-threading and (partially) follows the parallel structured programming framework. It is a C++ library, so probably you can just start using it in your projects.
Yes and no. The basic idea of CSP is used quite a bit. For example, thread-safe queues in one form or another are frequently used as the primary (often only) communication mechanism to build a pipeline out of individual processes (threads).
Hoare being Hoare, however, there's quite a bit more to his original theory than that. He invented a notation for talking about the processes, defined a specific set of signals that can be sent between the processes, and so on. The notation has since been refined in various ways, quite a bit of work put into proving various aspects, and so on.
Application of that relatively formal model of CSP (as opposed to just the general idea) is much less common. It's been used in a few systems where high reliability was considered extremely important, but few programmers appear interested in learning (yet another) formal design notation.
When I've designed systems like this, I've generally used an approach that's less rigorous, but (at least to me) rather easier to understand: a fairly simple diagram, with boxes representing the processes, and arrows representing the lines of communication. I doubt I could really offer much in the way of a proof about most of the designs (and I'll admit I haven't designed a really huge system this way), but it's worked reasonably well nonetheless.
Take a look at the website for a company called Verum. Their ASD technology is based on CSP and is used by companies like Philips Healthcare, Ericsson and NXP semiconductors to build software for all kinds of high-tech equipment and applications.
So to answer your question: Yes, CSP is used on large software projects in real-life.
Full disclosure: I do freelance work for Verum
Answering a very old question, yet it seems important that one
There is Go where CSPs are a fundamental part of the language. In the FAQ to Go, the authors write:
Concurrency and multi-threaded programming have a reputation for difficulty. We believe this is due partly to complex designs such as pthreads and partly to overemphasis on low-level details such as mutexes, condition variables, and memory barriers. Higher-level interfaces enable much simpler code, even if there are still mutexes and such under the covers.
One of the most successful models for providing high-level linguistic support for concurrency comes from Hoare's Communicating Sequential Processes, or CSP. Occam and Erlang are two well known languages that stem from CSP. Go's concurrency primitives derive from a different part of the family tree whose main contribution is the powerful notion of channels as first class objects. Experience with several earlier languages has shown that the CSP model fits well into a procedural language framework.
Projects implemented in Go are:
Docker
Google's download server
Many more
This style is ubiquitous on Unix where many tools are designed to process from standard in to standard out. I don't have any first hand knowledge of large systems that are build that way, but I've seen many small once-off systems that are
for instance this simple command line uses (at least) 3 processes.
cat list-1 list-2 list-3 | sort | uniq > final.list
This system is only moderately sized, but I wrote a protocol processor that strips away and interprets successive layers of protocol in a message that used a style very similar to this. It was an event driven system using something akin to cooperative threading, but I could've used multithreading fairly easily with a couple of added tweaks.
The program is proprietary (unfortunately) so I can't show off the source code.
In my opinion, this style is useful for some things, but usually best mixed with some other techniques. Often there is a core part of your program that represents a processing bottleneck, and applying various concurrency increasing techniques there is likely to yield the biggest gains.
Microsoft had a technology called ActiveMovie (if I remember correctly) that did sequential processing on audio and video streams. Data got passed from one filter to another to go from input to output format (and source/sink). Maybe that's a practical example??
The Wikipedia article looks to me like a lot of funny symbols used to represent somewhat pedestrian concepts. For very large or extensible programs, the formalism can be very important to check how the (sub)processes are allowed to interact.
For a 50,000 line class program, you're probably better off architecting it as you see fit.
In general, following ideas such as these is a good idea in terms of performance. Persistent threads that process data in stages will tend not to contend, and exploit data locality well. Also, it is easy to throttle the threads to avoid data piling up as a fast stage feeds a slow stage: just block the fast one if its output buffer grows too big.
A little bit off-topic but for my thesis I used a tool framework called TERRA/LUNA which aims for software development for Embedded Control Systems but is used heavily for all sorts of software development at my institute (so only academical use here).
TERRA is a graphical CSP and software architecture editor and LUNA is both the name for a C++ library for CSP based constructs and the plugin you'll find in TERRA to generate C++ code from your CSP models.
It becomes very handy in combination with FDR3 (a CSP refinement checker) to detect any sort of (dead/life/etc) lock or even profiling.