Is core.async a replacement to Lamina or does it intend to become a replacement for Lamina?
If not, are there clear situations where one is preferable over the other?
I'm the author of Lamina. I think core.async is a well-made library, with a lot more clarity in its design than Lamina. There are things that I think Lamina is better at, mostly related to introspection, performance, and extensibility.
The big problem I have with core.async is that in addition to a stream abstraction, it brings along an execution model (everything happens on the core.async thread pools), which means that if you use it anywhere, it constrains the design and implementation of everything else in your codebase.
I've seen a number of "async" libraries made that expose streams as core.async channels, which means that you can only use the libraries if you're comfortable using the core.async execution model.
I'm about to release a library that tries to be a "minimal" stream representation that can be used in place of core.async, Lamina, Java blocking queues, etc. called Manifold. A Manifold stream can be coerced to a core.async channel, Lamina channel, and so on, and any of these things can be coerced back into a Manifold stream.
I think the "async" landscape is still quite young, and there are a lot of unexplored problems w.r.t. how well the abstractions scale, how easy they are to debug in production, and so on. The JVM provides a lot of tools for introspection, but since the async mechanisms use a complete different execution model we're basically starting over again from scratch. I wouldn't tell you to use Lamina over core.async, but I would caution that core.async is an application-level abstraction, not a library-level one.
core.async and Lamina are two different projects and they aren't intended to replace each other. Actually, they could play nicely together -if you want to-.
Lamina is a stream oriented approach, while core.async is message oriented.
Which one to use is up to you. In Lamina, the important thing is the callbacks you define for the channel, while in core.async, channels and go blocks are decoupled, and this is more flexible and modular.
Related
There seems to be a myriad of implementations for 'coroutines' or asynchronous logic in clojure, many of the talks by Rich Hickey and other potential authorities on the matter are from almost a decade ago and I'm trying to find out what is the latest and greatest, best practice way to handle this problem.
My favorite abstraction for this type of thing is lua coroutines, but I think these may be a strictly imperative style of doing things, and I'm a little confused as to what the functional way is instead.
In lua though it's really simple and easy with coroutines to:
A) Non-busy wait for X seconds.
B) Non-busy wait for a variable or function to be a specific value, such as true
A can probably be achieved using setTimeout, but B can't really, at least I don't know how. I'm also not sure setTimeout is the best practice for these types of problems?
In a 2013 blog post, Rich Hickey describes the motivations for clojure.core.async. While the JVM has some applications, the primary motive was to give the illusion of threads to the single-threaded Javascript environment.
The "simulated multithreading" provided by clojure.core.async is not as robust as using actual JVM threads (especially when Exceptions/Errors occur), so it is of limited use for JVM Clojure. This will be even more true when Java virtual threads become a reality.
So if you are in ClojureScript, clojure.core.async is much better than nothing (i.e. callback hell). However, even JS is contemplating a multithreading model via WebAssembly, so an alternative to clojure.core.async could exist for ClojureScript in the future.
Is it recommended to extend the functionality of core.async with my own asynchronous functions?
The asynchrony of channels is handled by put! and take! which accept callbacks, but the protocols are nested in the async.impl.protocols namespace. Does impl mean stay out! in this case, or is it okay to implement them?
For example, I could wrap a netty channel or a java socket as a ReadPort and a WritePort.
The intention of the core.async protocols is to serve as implementation hooks for implementing your own buffers, channels, ports, etc. They exist under impl as they are part of the implementation, not the public user API.
The team considers them to be open to change until a non-alpha version of the library is released (I have no timeframe on that). From async's release until now the protocols have not changed, however there is at this moment a breaking change in process specifically to put! and take!.
If you're willing to deal with catching changes for now, feel free to implement as you wish.
Tim B has spent quite a bit of time looking at connecting async channels to the network and it is very challenging to do while retaining the channel semantics. The recommended pattern right now for this is to use dedicated threads that talk to network I/O and communicate "at the edge" with the channels in the application (probably using put! and take!). This pattern does not require implementing the internal protocols.
In the book about Communicating Sequential Processes a lot of time is spent defining events, which have no direction and can involve multiple independent processes.
Only in chapter 4 are channels introduced, which are directed and involve 2 processes.
Yet all implementations of CSP including occam, Go, LuaCSP and clojure.core.async only implement channels.
Even though most practical problems can be solved with (broadcast) channels, I wonder why the book spends so much time on them while nobody uses them.
Well first of all the book happened before the implementations. Therefore your question is better formulated as:
Why does no implementation of CSP put a major focus on events even though the book emphazises them greatly.
Basically, making events a first-class citizen of a language gives it a certain usage specificity that would probably be too narrow for a general purpose programming language.
Additionally, you can easily implement events on channels (and other constructs), in case you are into event driven programming.
In Occam-pi, barriers are an important adjunct to channels. With a barrier, each enrolled process waits on the barrier until they have all done so. At this point they are all released. This is an example of a non-channel form of CSP event.
Occam-pi also has an extended rendezvous using channels. This is a quite different pattern of usage of channels, very similar to the rendezvous in Ada.
Firstly, I am an absolute beginner in programming, so don't make fun of me too much.
The only thing that I have seen signals used for are GUI toolkits, and GUI toolkits all come with their own signaling. So, can Boost:Signals even be used with these GUI toolkits? Would this be a good idea? What other applications do signals have?
Signals is an event messaging implementation, much like Smalltalk/Objective C Messages or Events in various other (e.g. C#) lanugages.
You can use them for a wide variety of tasks, take a look at the Observer Pattern
Why would you use the Observer Pattern?
The benefits are largely organisational, when you work with large applications it's is important to apply patterns of reuse that help maintain development team coherence.
When the implementation of a particular pattern becomes de facto (or close to) it's especially useful because it means that lead up times for new team members are likely to be expedited, not only if they have used the implementation before, but also because the popularity of the implementation will mean that there are widespread resources, and documentation available to speed learning.
From a pure code perspective, ALL patterns appear as bloat, but when you begin to understand that upwards of 60% of the costs involved in software development are in maintenance life-cycle, it is well worth the additional code to gain coherence.
The other benefit is to aid in software reuse, depending on the style of implementation, the Observer Pattern can assist in modularising and decoupling classes from one another. I would suggest that this is also an organisational benefit, in as much that different teams can build components more easily, or simply because components are easier to replace.
Just my two cents, signals are not only used in (or for) GUI toolkits. They are used in contexts where you want to decouple the producer of a datum with the receiver of it (the observer pattern mentioned above, for example). If you mix that idea with threads, you can implement actors easily, an interesting pattern for concurrent tasks (Erlang and Scala use actors, for instance).
One possible use would be in the implementation of a GUI toolkit. You'd basically set up the wiring to get messages (or whatever they happen to be called) from the native system to produce signals. From there, the code for routing and handling signals can be (at least somewhat) portable.
In addition to the Observer pattern that others have mentioned, anytime you find yourself having to write a callback function, so that one class can notify another that something has happened, then you can use Signals and Slots instead. The great advantage over callbacks is that it takes care of lots of the boiler plate code to add and remove the callback function, and deals with automatically disconnecting when either the caller or the callee go out of scope.
Callbacks are really just an instance of the Observer pattern though.
I think Erlang-style concurrency is the answer to exponential growth of core count. You can kind of fake it with other main stream languages. But the solutions always leave me wanting. I am not willing to give up multi-paradigm programming (C++/D) to switch to Erlang's draconian syntax.
What is Erlang-style concurrency:
From one of the language authors(What is Erlang's concurrency model actually ?):
Lightweight concurrency.
Cheap to create threads and cheap to maintain insane numbers.
Asynchronous communication.
Threads only communicate via messages.
Error handling.
Process isolation.
Or from an informed blogger (What is Erlang-Style Concurrency?):
Fast process creation/destruction
Ability to support >> 10 000 concurrent processes with largely unchanged characteristics.
Fast asynchronous message passing.
Copying message-passing semantics (share-nothing concurrency).
Process monitoring.
Selective message reception.
I think D's message passing can accomplish most of these features. The ones I wonder about are ">>10,000 concurrent processes(threads)" and "fast process creation/destruction".
How well does D handle these requirements?
I think that to support them correctly you'd have to use green threads. Can D's message passing features be used with green threads library?
Storage is thread-local by default in D, so nothing is shared between threads unless it is specifically marked as shared. If you mark a variable as shared, you can then use the traditional mutexes and conditions as well as synchronized objects and the like to deal with concurrency. However, the preferred means of communicating between threads is to use the message passing facilities in std.concurrency and let all data stay thread-local, only using shared when you must. All objects passed between threads using std.concurrency must either be passed by value or be immutable, so no sharing occurs there and it is completely thread-safe. However, it can currently be a bit of a pain to get an immutable reference type which isn't an array (idup generally makes it easy for arrays), so it can be a bit annoying to pass anything other than value types or arrays (though hopefully that situation improves soon as compiler and standard library bugs relating to const and immutable get fixed and more code is made const-correct).
Now, while message passing in D will definitely result in cleaner, safer code than what you'd get in languages like C++ or Java, it is built on top of normal, C threads (e.g. Linux uses pthreads), so it does not have the kind of light-weight threads that Erlang does, and so dealing with multiple threads is not going to be as efficient as Erlang.
Of course, I don't see any reason why a more efficient thread system could not be written using D, at which point you might be able to get thread efficiency similar to that of Erlang, and it could presumably use an API similar to that of std.concurrency, but all of D's standard threading stuff is built on top of normal, C threads, so you'd have to do all of that yourself, and depending on how you implemented it and depending on how exactly the thread-local/shared stuff is dealt with by the compiler and druntime, it could be difficult to get the type system to enforce that everything be thread-local with your "green" threads. I'm afraid that I don't know enough about exactly how shared is implemented or how "green" threads work to know for sure.
Regardless, D's message passing system will certainly result in dealing with threads being more pleasant than C++ or even Java, but it's not designed to be streamlined in the same way that Erlang is. D is a general purpose systems language, not a language specifically designed to use threads for everything and thus to use them absolutely as efficiently as possible. A large portion of D's standard facilities are built on top of C, so a lot of its efficiency characteristics will be similar to those of C.
This functionality is frequently used in combination with async I/O to efficiently communicate with external sources of data as well. The vibe.d framework seems to offer both the many-fibers-on-a-few-OS-threads threading model and async I/O libraries (in addition to a whole bunch of web application libraries and project management tools).
As an unrelated side note, it's pretty freaking cool that D is both low-level enough that you could write this framework in it and high-level enough to be a compelling language to write your web applications in on top of the framework. Other popular languages with similar frameworks (node.js, Ruby's EventMachine, coroutines in Python and Go) are unable to compete with D on low-level systems coding. Other popular languages with similar systems programming facilities (C, C++) can't compete on high-level application coding.
I'm new to D, but I gotta say, I like what I see.
From whatever little I know about D: its message passing infrastructure is built on top its threading facilities. If the core threading library is a wrapper on OS threads, there is little chance that concurrency in D will reach the magnitude (>> 10000) of Erlang. Moreover D do not enforce immutability on objects, so it is easy to mess things up. So, Erlang is the best choice for heavy concurrency. Probably you can write the concurrency stuff in Erlang and the rest of the project in D. Still, it is possible to have efficient green threads in C like languages (C++, D etc) - have a look at Protothreads and ZeroMQ. You can implement very efficient messaging frameworks using these, and calling them via a C shim or directly from D.