Related
Will Move (on the Diem platform), like Ethereum, be turing complete? I have read the white paper but have been unable to decide.
This is the white paper: https://developers.libra.org/docs/assets/papers/libra-move-a-language-with-programmable-resources.pdf
It is a Turing-complete programming language. It supports while loops and Recursion.
See the Move Basics documentation for more information.
It is either a turing complete or not a turing complete smart contract programming language.
Until now, there is no documentation mentioned this.
Yes, Move is Turing complete which means that Libra is Turing complete. Because of this, determining if a given Move program terminates cannot be decided statically. However, by ensuring that (1) every bytecode instruction has a non-zero cost, and (2) the amount of gas that any program can be started with is bounded, we get this termination property for programs almost free of cost.
Check out more details in the official blog: https://www.diem.com/en-us/blog/how-gas-works-on-libra-blockchain/
I often see people say if you can do X in some language you can do Y in another language which is the Turing Complete argument. So You'll have often (usually in a snide comment) "sure you can do t with y because y is also Turing complete.
I took CS theory a long time ago but I don't think this is always true because I'm not sure where Turing fits into concurrency. For example there are programming languages with the right hardware you can execute things to happen exactly at the same time but others where that is not possible.
I understand this is probably more of a hardware/driver issue than the language but I'm curious if or how does concurrency change what it is to be Turing Complete? Can be you be more than Turing Complete?
EDIT:
The original reason that I asked this question was in large part due to quantum computing. Although the accepted answer doesn't say this but quantum computing is (ostensible) a subset of turing.
This is a confusing topic for many people; you're not alone. The issue is that there are two different definitions of "possible" in play here. One definition of "possible" is how you're using it: is it possible to do concurrency, is it possible to operate a giant robot using the language, is it possible to make the computer enjoy strawberries, etc. This is the layman's definition of "possible".
Turing completeness has nothing to do with what's possible in the above sense. Certainly, concurrency isn't possible everywhere because (for at least some definition of concurrency) it needs to be the case that the language can produce code that can run on two or more different processors simultaneously. A language that can only compile to code that will run on a single processor thus would not be capable of concurrency. It could still be Turing-complete, however.
Turing completeness has to do with the kinds of mathematical functions that can be computed on a machine given enough memory and running time. For purposes of evaluating mathematical functions, a single processor can do everything multiple processors can because it can emulate multiple processors by executing one processor's logic after the other. The (unproven and unprovable, though falsifiable) statement that all mathematical functions that could be computed on any device are computable using a Turing machine is the so-called Church-Turing thesis.
A programming language is called Turing-complete if you can prove that you can emulate any Turing machine using it. Combining this with the Church-Turing thesis, this implies that the programming language is capable of evaluating every type of mathematical function that any device could evaluate, given enough time and memory. Most languages are Turing-complete because this only requires the capacity to allocate dynamic arrays and use loops and if-statements as well as some basic data operations.
Going in the other direction, since a Turing machine can be constructed to emulate any processor we currently know of and programming languages do what they do using processors, no current programming language has more expressive power than a Turing machine, either. So the computation of mathematical functions is equally possible across all common programming languages. Functions computable in one are computable in another. This says nothing about performance - concurrency is essentially a performance optimization.
Yes and no. There is no known model of computation that can do things that Turing machines can do and still be called computation, as opposed to magic¹. Hence, in that sense, there is nothing beyond Turing completeness.
On the other hand, you may be familiar with the saying that “there is no problem that cannot be solved by adding a layer of indirection”. So we might want to distinguish between models of computation that map directly to Turing machines and models of computation that require a level of indirection. “Adding a level of indirection” is not a precise mathematical concept in general, but on many specific cases you can observe the level of indirection. Often the easiest way to prove that some paradigm of computation is Turing-computable is to write an interpreter for it on a Turing machine, and that is exactly a level of indirection.
So let's have a look at what it means to model concurrency. You mention the ability to “execute things to happen exactly at the same time”. That's a specific kind of concurrency, called parallelism, and as far as concurrency goes it's a highly restricted model. The world of concurrency is a lot wilder than this. Nonetheless, parallelism already allows things that require some form of indirection when modeled on a Turing machine.
Consider the following problem: given computer programs A and B (passed on the tape of a universal Turing machine), execute them both, and return the result of either program; your program must terminate unless both A and B are non-terminating. In a purely sequential world, you can execute A and return the result; or you can execute B and return the result. But if you start by executing A, and it happens to be a non-terminating program while B does terminate, then your execution strategy does not solve the problem. And similarly, if you start by executing B, your execution strategy does not solve the problem because B might not terminate even if A does.
Given that it is undecidable whether A or B terminates, you cannot base your decision of which one to execute first on that. However, there is a very simple way to modify your Turing machine to execute the programs in parallel: put A and B on separate tapes, duplicate your automaton, and execute one step of each program until one of the two terminates. By adding this level of processing, you can solve the parallel execution problem.
Solving this problem only required a slight modification to the model (it is easy to model a dual-tape Turing machine with a single-tape machine). I nonetheless mention it because it is an important example in [lambda calculus](http://en.wikipedia.org/wiki/Lambda calculus), another important model of computation. The operation of reducing (evaluating) two lambda-terms in parallel until one of them reaches a normal form (terminates) is called Plotkin's parallel or. It is known that it is not possible to write a lambda term (a lambda calculus program) that implements parallel or. Hence lambda calculus is said to be “inherently sequential”.
The reason I mention the lambda calculus here is that most programming languages are closer to the lambda calculus than they are to programming machine. So as a programmer, insights from the lambda calculus are often more important than insights from Turing machines. The example of parallel or shows that adding concurrency to a language² can open possibilities that are not available in the original language.
It is possible to add concurrency to a sequential language through essentially the same trick as on Turing machines: execute a small piece of thread A, then a small piece of thread B, and so on. In fact, if you don't do that in your language, the operating system's kernel can typically do it for you. Strictly speaking, this provides concurrent execution of threads, but still using a single processor.
As a theoretical model, this kind of threaded execution suffers the limitation that it is deterministic. Indeed, any system that can be modeled directly on Turing machines is deterministic. When dealing with concurrent systems, it is often important to be able to write non-deterministic programs. Often the exact order in which the multiple threads of computation are interleaved is irrelevant. So two programs are equivalent if they do essentially the same computation, but in a slightly different order. You can make a model of concurrent computation out of a model of sequential computation by looking at sets of possible interleavings instead of single program runs, but that adds a level of indirection that is difficult to manage. Hence most models of concurrency bake nondeterminism into the system. When you do that, you can't run on a Turing machine any more.
¹
In this respect, thought (what happens in our brain) is still magic in the sense that we have no idea how it's done, we don't have a scientific understanding of it. Anything we know how to reproduce (not in the biological sense!) is Turing-computable.
²
Note that here, the language includes everything you can't define by yourself. In this sense, the standard library is part of “the language”.
I'm interested in using OCaml for a project, however I'm not sure about where its parallelization capabilities are anymore. Is there a message passing ability in OCaml? Is OCaml able to efficiently use more than 1 CPU?
Most of what I have read on the subject was written in 2002-2006, and I haven't seen anything more recent.
Thanks!
This 2009 issue of the Caml weekly news ("CWN", a digest of interesting messages from the caml list) shows that:
the official party line on threads and Ocaml hasn't changed. A notable quote:
(...) in general, the whole standard library is not thread-safe. Probably that should be stated in the
documentation for the threads library, but there isn't much point in documenting it per standard library module. -- X. Leroy
(for how Ocaml threads can still be useful, see a remark by the culprit himself in another question on SO)
the most frequently adopted paradigm for parallelism is message-passing, and of note is X. Leroy's OcamlMPI, providing bindings for programming in SPMD style against the MPI standard. The same CWN issue I pointed to above provides references to examples, and numerous other related projects.
another message-passing solution is JoCaml, pioneering new style of concurrent communications known as join calculus. Note that it is binary-compatible with OCaml compilers.
that did not prevent the confection of a runtime whose GC is ok with parallelism, though: see a discussion of OCAML4MC in this other issue of the CWN.
There is also:
Netmulticore - multi-processing sharing ocaml values via mapped shared memory.
CamlP3l - compiler for Caml parallel programs.
OCaml-Java - an OCaml compiler that emits Java bytecode
I haven't followed more recent discussions about Ocaml & parallel programming, though. I'm leaving this CW so that others can update what I mention. It would be great if this question could reach the same level of completeness as the analogous one for Haskell.
At present, the OCaml runtime does not support running across multiple cores in parallel, so a single OCaml process cannot take advantage of multiple cores. This is unlikely to change directly; the direction the OCaml developers are most interested in taking for increased parallelism seems to be allowing multiple OCaml runtimes to run in parallel in a single process; this will allow for very fast message passing, but will not allow multiple threads to run in parallel in a shared-memory configuration. The major hangup is the garbage collector; some years ago, the team experimented with a concurrent GC, but it introduced unacceptable slowdowns in the single-threaded case.
There are a couple of projects, namely Functory and OCamlnet, which provide multicore-happy parallelism by using multiple processes.
In general, the OCaml community tends to favor message passing approaches, which can be done across process boundaries (like OCamlnet does), over single-process shared-memory multithreading. If your program can be split into multiple processes (many can!), then yes, you can efficiently use multiple CPUs.
BSMLlib provides a simplified programming interface for data-parallel programming in OCaml.
Its execution amounts to BSP-style message passing but it is deterministic and even declarative for a subset of OCaml.
The key concept is the 'a par type which corresponds to a vector of values, one per process.
http://traclifo.univ-orleans.fr/BSML/
http://fr.wikipedia.org/wiki/Bulk_Synchronous_Parallel_ML
Gaétan Hains
University Paris-Est
I'm currently writing a large multi threaded C++ program (> 50K LOC).
As such I've been motivated to read up alot on various techniques for handling multi-threaded code. One theory I've found to be quite cool is:
http://en.wikipedia.org/wiki/Communicating_sequential_processes
And it's invented by a slightly famous guy, who's made other non-trivial contributions to concurrent programming.
However, is CSP used in practice? Can anyone point to any large application written in a CSP style?
Thanks!
CSP, as a process calculus, is fundamentally a theoretical thing that enables us to formalize and study some aspects of a parallel program.
If you instead want a theory that enables you to build distributed programs, then you should take a look to parallel structured programming.
Parallel structural programming is the base of the current HPC (high-performance computing) research and provides to you a methodology about how to approach and design parallel programs (essentially, flowcharts of communicating computing nodes) and runtime systems to implements them.
A central idea in parallel structured programming is that of algorithmic skeleton, developed initially by Murray Cole. A skeleton is a thing like a parallel design pattern with a cost model associated and (usually) a run-time system that supports it. A skeleton models, study and supports a class of parallel algorithms that have a certain "shape".
As a notable example, mapreduce (made popular by Google) is just a kind of skeleton that address data parallelism, where a computation can be described by a map phase (apply a function f to all elements that compose the input data), and a reduce phase (take all the transformed items and "combine" them using an associative operator +).
I found the idea of parallel structured programming both theoretical sound and practical useful, so I'll suggest to give a look to it.
A word about multi-threading: since skeletons addresses massive parallelism, usually they are implemented in distributed memory instead of shared. Intel has developed a tool, TBB, which address multi-threading and (partially) follows the parallel structured programming framework. It is a C++ library, so probably you can just start using it in your projects.
Yes and no. The basic idea of CSP is used quite a bit. For example, thread-safe queues in one form or another are frequently used as the primary (often only) communication mechanism to build a pipeline out of individual processes (threads).
Hoare being Hoare, however, there's quite a bit more to his original theory than that. He invented a notation for talking about the processes, defined a specific set of signals that can be sent between the processes, and so on. The notation has since been refined in various ways, quite a bit of work put into proving various aspects, and so on.
Application of that relatively formal model of CSP (as opposed to just the general idea) is much less common. It's been used in a few systems where high reliability was considered extremely important, but few programmers appear interested in learning (yet another) formal design notation.
When I've designed systems like this, I've generally used an approach that's less rigorous, but (at least to me) rather easier to understand: a fairly simple diagram, with boxes representing the processes, and arrows representing the lines of communication. I doubt I could really offer much in the way of a proof about most of the designs (and I'll admit I haven't designed a really huge system this way), but it's worked reasonably well nonetheless.
Take a look at the website for a company called Verum. Their ASD technology is based on CSP and is used by companies like Philips Healthcare, Ericsson and NXP semiconductors to build software for all kinds of high-tech equipment and applications.
So to answer your question: Yes, CSP is used on large software projects in real-life.
Full disclosure: I do freelance work for Verum
Answering a very old question, yet it seems important that one
There is Go where CSPs are a fundamental part of the language. In the FAQ to Go, the authors write:
Concurrency and multi-threaded programming have a reputation for difficulty. We believe this is due partly to complex designs such as pthreads and partly to overemphasis on low-level details such as mutexes, condition variables, and memory barriers. Higher-level interfaces enable much simpler code, even if there are still mutexes and such under the covers.
One of the most successful models for providing high-level linguistic support for concurrency comes from Hoare's Communicating Sequential Processes, or CSP. Occam and Erlang are two well known languages that stem from CSP. Go's concurrency primitives derive from a different part of the family tree whose main contribution is the powerful notion of channels as first class objects. Experience with several earlier languages has shown that the CSP model fits well into a procedural language framework.
Projects implemented in Go are:
Docker
Google's download server
Many more
This style is ubiquitous on Unix where many tools are designed to process from standard in to standard out. I don't have any first hand knowledge of large systems that are build that way, but I've seen many small once-off systems that are
for instance this simple command line uses (at least) 3 processes.
cat list-1 list-2 list-3 | sort | uniq > final.list
This system is only moderately sized, but I wrote a protocol processor that strips away and interprets successive layers of protocol in a message that used a style very similar to this. It was an event driven system using something akin to cooperative threading, but I could've used multithreading fairly easily with a couple of added tweaks.
The program is proprietary (unfortunately) so I can't show off the source code.
In my opinion, this style is useful for some things, but usually best mixed with some other techniques. Often there is a core part of your program that represents a processing bottleneck, and applying various concurrency increasing techniques there is likely to yield the biggest gains.
Microsoft had a technology called ActiveMovie (if I remember correctly) that did sequential processing on audio and video streams. Data got passed from one filter to another to go from input to output format (and source/sink). Maybe that's a practical example??
The Wikipedia article looks to me like a lot of funny symbols used to represent somewhat pedestrian concepts. For very large or extensible programs, the formalism can be very important to check how the (sub)processes are allowed to interact.
For a 50,000 line class program, you're probably better off architecting it as you see fit.
In general, following ideas such as these is a good idea in terms of performance. Persistent threads that process data in stages will tend not to contend, and exploit data locality well. Also, it is easy to throttle the threads to avoid data piling up as a fast stage feeds a slow stage: just block the fast one if its output buffer grows too big.
A little bit off-topic but for my thesis I used a tool framework called TERRA/LUNA which aims for software development for Embedded Control Systems but is used heavily for all sorts of software development at my institute (so only academical use here).
TERRA is a graphical CSP and software architecture editor and LUNA is both the name for a C++ library for CSP based constructs and the plugin you'll find in TERRA to generate C++ code from your CSP models.
It becomes very handy in combination with FDR3 (a CSP refinement checker) to detect any sort of (dead/life/etc) lock or even profiling.
a theoretical question. After reading Armstrongs 'programming erlang' book I was wondering the following:
It will take some time to learn Erlang. Let alone master it. It really is fundamentally different in a lot of respects.
So my question: Is it possible to write 'like erlang' or with some 'erlang like framework', which given that you take care not to create functions with sideffects, you can create scaleable reliable apps as well as in Erlang? Maybe with the same msgs sending, loads of 'mini processes' paradigm.
The advantage would be to not throw all your accumulated C/C++ knowledge over the fence.
Any thoughts about this would be welcome
Yes, it is possible, but...
Probably the best answer for this question is given by Robert Virding’s First Rule:
“Any sufficiently complicated
concurrent program in another language
contains an ad hoc,
informally-specified, bug-ridden, slow
implementation of half of Erlang.”
Very good rule is use the right tool for the task. Erlang excels in concurrency and reliability. C/C++ was not designed with these properties in mind.
If you don't want to throw away your C/C++ knowledge and experience and your project allows this kind of division, good approach is to create a mixed solution. Write concurrent, communication and error handling code in Erlang, then add C/C++ parts, which will do CPU and IO bound stuff.
You clearly can - the Erlang/OTP system is largely written in C (and Erlang). The question is 'why would you want to?'
In 'ye olde days' people used to write their own operating system - but why would you want to?
If you elect to use an operating system your unwritten software has certain properties - it can persist to hard disk, it can speak to a network, it can draw on screens, it can run from the command line, it can be invoked in batch mode, etc, etc...
The Erlang/OTP system is 1.5M lines of code which has been demonstrated to give 99.9999999% uptime in large systems (the UK phone system) - that's 31ms downtime a year.
With Erlang/OTP your unwritten software has high reliability, it can hot-swap itself, your unwritten application can failover when a physical computer dies.
Why would you want to rewrite that functionality?
I would break this into 2 questions
Can you write concurrent, scalable C++ applications
Yes. It's certainly possible to create the low level constructs needed in order to achieve this.
Would you want to write concurrent, scalable, C++ applications
Perhaps. But if I was going for a highly concurrent application, I would choose a language that was either designed to fill that void or easily lent itself to doing so (Erlang, F# and possibly C#).
C++ was not designed to build highly concurrent applications. But it can certainly be tweaked into doing so. The cost might be higher than you expect though once you factor in memory management.
Yes, but you will be doing some extra work.
Regarding side effects, consider how the .net/plinq team is approaching. Plinq won't be able to enforce you hand it stuff with no side effects, but it will assume you do so and play by its rules so we get to use a simpler api. Even if the language doesn't have built-in support for it, it will still simplify things as you can break the operations more easily.
What I can do in one Turing complete language I can do in any other Turing complete language.
So I interpret your question to read, is it as easy to write a reliable and scalable application in C++ as it is in Erlang?
The answer to that is highly subjective. For me it is easier to write it in C++ for the following reasons:
I have already done it in C++ (at least three times).
I don't know Erlang.
I have read a great deal about Stackless Python, which feels to me like a highly concurrent message based cooperative multitasking system in python, but of course python is written on top of C.
Having said that. If you already know both languages, and you have the problem well defined, you can then make the best choice based on all the information you have at hand.
the main 'problem' with C (or C++) for writing reliable and easy to extend programs is that in C you can do anything. so, the first step would be to write a simple framework that restricts just a bit. most good programmers do that anyway.
in this case, the restrictions would be mostly to make it easy to define a 'process' within whatever level of isolation you want. fork() has a reputation of being slow, and threads also need significant time to spawn, so you might want to use a cooperative multitasking, which can be far more efficient, and you could even make it preemptive (i think that's what Erlang does). to get multi-core efficiency, set a pool of threads and make all of them complete to run the tasks.
another important part would be to create an appropriate library of immutable data structures, so that using them (instead of the standard lib) your functions would be (mostly) side-effect-free.
then it's just a matter of setting a good API for message passing and futures... not easy, but at least it doesn't seem like changing the language itself.