Trying to figure out how Erlang concurrency works. For testing, I have the following modules:
server.erl:
-module(server).
-export([loop/0]).
loop() ->
receive
{foo, Msg_foo} ->
io:format("~w~n", [Msg_foo]),
loop();
{bar, Msg_bar} ->
io:format("~w~n", [Msg_bar]),
loop();
stop ->
io:format("~s~n", ["End server process"]),
true
end.
process_a.erl
-module(process_a).
-export([go_a/0]).
go_a() ->
receive
{foo, Pid1} ->
Pid1 ! {foo, 'Message foo from process A'},
go_a();
{bar, Pid2} ->
Pid2 ! {bar, 'Message bar from process A'},
go_a()
end.
process_b.erl
-module(process_b).
-export([go_b/0]).
go_b() ->
receive
{foo, Pid1} ->
Pid1 ! {foo, 'Message foo from process B'},
go_b();
{bar, Pid2} ->
Pid2 ! {bar, 'Message bar from process B'},
go_b()
end.
client.erl
-module(client).
-export([start/0]).
-import(server, [loop/0]).
-import(process_a, [go_a/0]).
-import(process_b, [go_b/0]).
go() ->
Server_Pid = spawn(server, loop, []),
Pid_A = spawn(process_a, go_a, []),
Pid_B = spawn(process_b, go_b, []),
Pid_A ! {foo, Server_Pid},
Pid_B ! {bar, Server_Pid},
Pid_A ! {bar, Server_Pid},
Pid_B ! {foo, Server_Pid},
Pid_A ! {foo, Server_Pid},
Pid_B ! {foo, Server_Pid},
Pid_A ! {bar, Server_Pid},
Pid_B ! {bar, Server_Pid}.
start() ->
go().
The client sends messages to process A and process B which in turn send messages to the server. The order of the messages is:
A foo
B bar
A bar
B foo
A foo
B foo
A bar
B bar
but the program output is:
'Message foo from process A'
'Message bar from process A'
'Message foo from process A'
'Message bar from process A'
'Message bar from process B'
'Message foo from process B'
'Message foo from process B'
'Message bar from process B'
The server first processes all messages from process A, then all the messages from process B. My question is, what does determine the message processing order? I thought that it was the order in which the messages were received.
It all depends on process scheduling. After your client code starts the server and procs A and B, those processes are newly created but might not even have been given any time to execute yet (and if they have, they will immediately be suspended in their receives). The client code keeps executing and quickly sends off a bunch of messages to A and B. These are asynchronous operations and the client process will not have to suspend at all before returning from the call to go().
As soon as a suspended process gets a message, it becomes ready to be scheduled for execution, but it can take a fraction of time before this happens. Meanwhile, more messages may keep arriving in their mailboxes, so when A or B actually start running, they are likely to have all four messages from the client already in their mailboxes. In general you can also not be sure which of A and B will start to execute first, even though the scheduling probably is very predictable in a simple case like this.
So in your case, A gets scheduled before B, it starts executing, and in very short time it consumes all its messages. This does not take much work, so A won't even spend a whole time slice. Then it suspends due to its mailbox being empty. Then B gets scheduled and does the same thing.
If there had been many processes, and/or a lot of work, the Erlang VM could have split the processes up across schedulers on different OS threads (running in truly parallel fashion if you have a multicore CPU). But since the example is so simple, these processes are probably handled within a single scheduler, and thus the ordering becomes even more predictable. If both A and B had thousands of messages in their queue, or each message took a lot of computational effort to process, you would see the messages getting interleaved.
(By the way, your import declarations in the client do nothing, since you are using spawn(Module, Fname, Args). If you had written e.g. spawn(fun() -> loop() end) they would be needed.)
Related
Lately I've been trying to understand concurrent servers in Erlang. Consider the following code that makes requests to the server. Depending on the
particular order of execution, different values may get printed by the 3 processes. What are the orders and what is the highest and lowest value of each process?
test() ->
Server = start(),
spawn(fun() ->
incr(Server),
io:format("Child 1 read ~p~n", [read(Server)]) end),
incr(Server),
spawn(fun() ->
incr(Server),
io:format("Child 2 read ~p~n", [read(Server)]) end),
io:format("Parent read ~p~n", [read(Server)]).
The code runs against the server below:
-module(p4).
-export([start/0, init/0, read/1, incr/1, reset/1]).
start() ->
spawn(fun() -> init() end).
init() -> loop(0).
loop(N) ->
receive
{read, Pid} ->
Pid ! {value, self(), N},
loop(N);
{incr, Pid} ->
Pid ! {incr_reply, self()},
loop(N+1);
{reset, Pid} ->
Pid ! {reset_reply, self()},
loop(0)
end.
read(Serv) ->
Serv ! {read, self()},
receive {value, Serv, N} -> N end.
incr(Serv) ->
Serv ! {incr, self()},
receive {incr_reply, Serv} -> ok end.
reset(Serv) ->
Serv ! {reset, self()},
receive {reset_reply, Serv} -> ok end.
Parent: Lowest = 1 Highest = 3
Child1: Lowest = 1 Highest = 3
Child2: Lowest = 1 Highest = 2
I'm not completely sure about the orders, but I guess it could be that:
Child1 can read 1, 2 and 3
Parent can read 1, 2 and 3
Child2 can read 1 and 2
Is this correct for both the lowest, highest values and the orders?
The initial value in the loop is 0. The server's increment operation replies to the caller before performing the increment, but that doesn't matter because no messages are processed between the sending of that reply and the actual increment. Each read message results in a reply containing the effects of all increment messages that arrived before it. Because of guaranteed message ordering from one process to another, any process that increments then reads is guaranteed to read at least its own increment. The server's read operation simply replies with the current loop value. The reset operation is unused.
Child1 increments, then reads. It runs concurrently with Parent initially and then later with Child2 as well, both of which also increment. It can therefore read 1 from just its own increment, 2 from its own increment and that of its parent, or 3 if its read also picks up the increment from Child2.
Child2 also increments, then reads, but it doesn't start until after the Parent has already incremented. The minimum it can read is therefore 2, and since it runs concurrently with Child1, it could alternatively read a 3.
Parent increments, then reads, so the minimum it can read is 1. Its read runs concurrently with Child1 and Child2, so if its read happens before either of their increments, it sees a 1. It could alternatively read a 2 if its read picks up either of the child increments, or a 3 if its read picks up both child increments.
I am building a system where a Producer sends a list of tasks to be queued which will be consumed by a number of Consumers.
Assume I have a list of tasks and they can be categorised into Black, Orange and Yellow. All the Black tasks are sent to Queue_0, Orange to Queue_1 and Yellow to Queue_2. And I will assign a worker to each queue(i.e: Consumer_0 to Queue_0, Consumer_1 to Queue_1 and Consumer_2 to Queue_2). If Black lists get larger, I want to add an extra Consumer(i.e: Consumer_3) to Queue_0 to aid Consumer_0.
I went through RabbitMQ tutorials on Worker Queues and Routing. I thought Routing will solve my problem. I launched three terminals, a producer and two consumers which will receive Black tasks. When the producer sends a few black tasks(Black_Task_1, Black_Task_2), both consumers received the two messages (i.e: Consumer_0 receives Black_Task_1 and Black_Task_2, Consumer_3 also receives Black_Task_1 and Black_Task_2) . I want my consumers to share the task, not do the same task. Example, Consumer_0 does Black_Task_1 while Consumer_3 does Black_Task_2. What configurations can I achieve that?
=============================
Update
This is a sample code taken from RabbitMQ, routing tutorial. I modified a little. Note that this code doesn't sent Black, Orange or Yellow queues. But the concept is there.
emit_log_direct.py
#!/usr/bin/env python
import pika
import sys
connection = pika.BlockingConnection(pika.ConnectionParameters(host='localhost'))
channel = connection.channel()
channel.exchange_declare(exchange='direct_logs',
type='direct')
severity = sys.argv[1] if len(sys.argv) > 1 else 'info'
message = ' '.join(sys.argv[2:]) or 'Hello World!'
channel.basic_publish(exchange='direct_logs',
routing_key=severity,
body=message)
print " [x] Sent %r:%r" % (severity, message)
connection.close()
receive_logs_direct.py
#!/usr/bin/env python
import pika
import sys
import time
connection = pika.BlockingConnection(pika.ConnectionParameters(host='localhost'))
channel = connection.channel()
channel.exchange_declare(exchange='direct_logs',
type='direct')
result = channel.queue_declare(exclusive=True)
queue_name = result.method.queue
severities = sys.argv[1:]
if not severities:
print >> sys.stderr, "Usage: %s [info] [warning] [error]" % \
(sys.argv[0],)
sys.exit(1)
for severity in severities:
channel.queue_bind(exchange='direct_logs',
queue=queue_name,
routing_key=severity)
print ' [*] Waiting for logs. To exit press CTRL+C'
def callback(ch, method, properties, body):
print " [x] %r:%r" % (method.routing_key, body,)
time.sleep(1)
print " [x] Done"
ch.basic_ack(delivery_tag=method.delivery_tag)
channel.basic_qos(prefetch_count=1)
channel.basic_consume(callback,
queue=queue_name)
channel.start_consuming()
Producer
nuttynibbles$ ./4_emit_log_direct.py info "run run info"
[x] Sent 'info':'run run info'
Consumer_0
nuttynibbles$ ./4_receive_logs_direct_customize.py info
[*] Waiting for logs. To exit press CTRL+C
[x] 'info':'run run info'
[x] Done
Consumer_3
nuttynibbles$ ./4_receive_logs_direct_customize.py info
[*] Waiting for logs. To exit press CTRL+C
[x] 'info':'run run info'
[x] Done
I think your basic issue is with this:
If Black lists queue get larger, I want to add an extra Consumer(i.e:
Consumer_3) to Queue_0 to aid Consumer_0.
As soon as you add another consumer to the queue - it will pick up the next available message.
If the first consumer does not acknowledge the message; then multiple workers will be able to work on the same message as it will remain on the queue.
So make sure you are correctly acknowledging the messages:
By default, RabbitMQ will send each message to the next consumer, in
sequence. On average every consumer will get the same number of
messages. This way of distributing messages is called round-robin.
[...]
There aren't any message timeouts; RabbitMQ will redeliver the message
only when the worker connection dies. It's fine even if processing a
message takes a very, very long time.
Depending on the nature of the task, you may be able to split the work between multiple processes by creating a priority queue; which is used by C1 (a consumer) to get additional resources. In this case you'll have to have workers ready and listening on the separate priority queue; thus creating a sub-queue where T1 (a task) is being processed.
However, in other to do this, the initial C1 has to make sure the task is no longer available by acknowledging its receipt.
I think that your problem is that you are creating a new Queue for each consumer. When you call
result = channel.queue_declare(exclusive=True)
queue_name = result.method.queue
in your consumer, this declares a new queue, tells RabbitMQ to generate a unique name for it, and marks it for exclusive use by the channel in the consumer that is calling it. That means that each consumer will have its own queue.
You then bind each new Queue to the exchange using the severity as a routing key. When a message comes into a direct Exchange, RabbitMQ will route a copy of it to every Queue that is bound with a matching routing key. There is no round-robin across the Queues. Each consumer will get a copy of the message, which is what you are observing.
I believe what you want to do is have each consumer use the same name for the queue, specify the name in the queue_declare, and don't make it exclusive. Then all the consumers will be listening to the same queue. The messages will be delivered to one of the consumers, basically in a round-robin fashion.
The producer (the emit_log.py program) doesn't declare or bind the queue - it doesn't have to, but if the binding isn't established before the message is sent, it will be discarded. If you are using a fixed queue, you can have the producer set it up as well, just be sure to use the same parameters (e.g. queue_name) as the consumer.
Does Erlang has equivalents for Future and Promises? Or since future and promises are solving a problem that doesn't exist in Erlang systems (synchronisation orchestrating for example), and hence we don't need them in Erlang.
If I want the semantics of futures and promises in Erlang, they could be emulated via Erlang processes/actors?
You could easily implement a future in Erlang like this:
F = fun() -> fancy_function() end,
% fancy code
Pid = self(),
Other = spawn(fun() -> X = F(), Pid ! {future, self(), X} end).
% more fancy code
Value = receive {future, Other, Val} -> Val end.
Having this functionality in a module and building chains from it should be easy too, but to be honest I never actually missed something like this. You are more flexible if you just freely send messages around.
The RPC module contains the rpc:async_call/4 function that does what you need. It will run a computation anywhere in a cluster (including on node(), the local node), and allow to have the result waited on with rpc:yield/1:
1> MaxTime = rpc:async_call(node(), timer, sleep, [30000]).
<0.48.0>
2> lists:sort([a,c,b]).
[a,b,c]
3> rpc:yield(MaxTime).
... [long wait] ...
ok
You can also poll in non-blocking ways by using rpc:nb_yield/1, or for a limited number of milliseconds with rpc:nb_yield/2:
4> Key2 = rpc:async_call(node(), timer, sleep, [30000]).
<0.52.0>
5> rpc:nb_yield(Key2).
timeout
6> rpc:nb_yield(Key2).
timeout
7> rpc:nb_yield(Key2).
timeout
8> rpc:nb_yield(Key2, 1000).
timeout
9> rpc:nb_yield(Key2, 100000).
... [long wait] ...
{value,ok}
That's all in the standard library and ready to be used.
Say there is a process B, which receives a pid and sends m2 to it. If you spawn A and send it m1, and then send A to B , is A guaranteed to get m1 before m2?
In other words, can this crash?
-module(test).
-compile(export_all).
test() ->
B = spawn_link(fun() -> receive P -> P ! m2 end end),
A = spawn_link(fun() -> receive X -> X=m1 end end),
A ! m1,
B ! A.
Your code cannot crash because all processes are local.
B = spawn_link(fun() -> receive P -> P ! m2 end end), % 1
A = spawn_link(fun() -> receive X -> X=m1 end end), % 2
A ! m1, % 3
B ! A. % 4
When evaluating line 3, both BEAM emulator and HiPE invoke the erl_send built-in function (BIF). Since A is a local process, erl_send (actually do_send) eventually calls erts_send_message which enqueues the message in the mailbox. In SMP mode, the thread actually acquires a lock on the mailbox.
So when evaluating line 4 and sending A to process B, A already has m1 in its mailbox. So m2 can only be enqueued after m1.
Whether this result is particular of the current implementation of Erlang is debatable, even if this is not guaranteed by documentation. Indeed, each process need a mailbox and this mailbox needs to be filled somehow. This is done synchronously on line 3. To do it asynchronously would either require another thread in-between or several mailboxes per process (e.g. one per scheduler to avoid the lock on the mailbox). Yet I do not think this would make sense performance-wise.
If processes A and B were remote but within the same node, the behavior is slightly different but the result would be the same with the current implementation of Erlang. On line 3, message m1 will be enqueued for the remote node and on line 4, message A will be enqueued afterwards. When remote node will dequeue messages, it will first write m1 to A's mailbox before writing A to B's mailbox.
If process A was remote and B was local, the result would still be the same. On line 3, message m1 will be enqueued for the remote node and on line 4, message will be written to B, but then on line 1, message m2 will be enqueued to remote node after m1. So A will get messages in m1, m2 order.
Likewise, if process A was local and B was remote, A will get the message copied to its mailbox on line 3 before anything is sent over the network to B's node.
With the current version of Erlang, the only way for this to crash is to have A and B on distinct remote nodes. In this case, m1 is enqueued to A's node before A is enqueued to B's node. However, delivery of these messages is not synchronous. Delivery to B's node could happen first, for example if many messages are already enqueued for A's node.
The following code (sometimes) triggers the crash by filling queue to A's node with junk messages that slow delivery of m1.
$ erl -sname node_c#localhost
C = spawn_link(fun() ->
A = receive {process_a, APid} -> APid end,
B = receive {process_b, BPid} -> BPid end,
ANode = node(A),
lists:foreach(fun(_) ->
rpc:cast(ANode, erlang, whereis, [user])
end, lists:seq(1, 10000)),
A ! m1,
B ! A
end),
register(process_c, C).
$ erl -sname node_b#localhost
B = spawn_link(fun() -> receive P -> P ! m2 end end),
C = rpc:call(node_c#localhost, erlang, whereis, [process_c]),
C ! {process_b, B}.
$ erl -sname node_a#localhost
A = spawn_link(fun() -> receive X -> X = m1 end, io:format("end of A\n") end),
C = rpc:call(node_c#localhost, erlang, whereis, [process_c]),
C ! {process_a, A}.
If both the two processes are on the same node, it is true that A is guaranteed to get m1 before m2.
But when the two processes are on different nodes, it is not guaranteed.
There is a paper Programming Distributed Erlang Applications: Pitfalls and Recipes about this problem.
Here is the link: http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.116.9929&rep=rep1&type=pdf
Your problem is in 2.2 of this paper, and I think it is really an intereting paper!
I have two processes linked; let's say they're A and B, with A set to trap exits. I want to be able to recover a piece of the B's process data if someone calls exit/2 on it, e.g. exit(B, diediedie).
In B's module, let's call it bmod.erl, I have some code that looks like this:
-module(bmod).
-export([b_start/2]).
b_start(A, X) ->
spawn(fun() -> b_main(A, X) end).
b_main(A, X) ->
try
A ! {self(), doing_stuff},
do_stuff()
catch
exit:_ -> exit({terminated, X})
end,
b_main(A, X).
do_stuff() -> io:format("doing stuff.~n",[]).
And in A's module, let's call it amod.erl, I have some code that looks like this:
-module(amod).
-export([a_start/0]).
a_start() ->
process_flag(trap_exit, true),
link(bmod:b_start(self(), some_stuff_to_do)),
a_main().
a_main() ->
receive
{Pid, doing_stuff} ->
io:format("Process ~p did stuff.~n",[Pid]),
exit(Pid, diediedie),
a_main();
{'EXIT', Pid, {terminated, X}} ->
io:format("Process ~p was terminated, had ~p.~n", [Pid,X]),
fine;
{'EXIT', Pid, _Reason} ->
io:format("Process ~p was terminated, can't find what it had.~n", [Pid]),
woops
end.
(I realize that I should do spawn_link normally but in my original program there is code in between the spawn and the link, and so I modeled this example code this way.)
Now when I run the code, I get this.
2> c(amod).
{ok,amod}
3> c(bmod).
{ok,bmod}
4> amod:a_start().
doing stuff.
Process <0.44.0> did stuff.
doing stuff.
Process <0.44.0> did stuff.
Process <0.44.0> was terminated, can't find what it had.
woops
5>
How do I get b_main() to catch this external exit so it can report its state X?
For b_main() to catch the external exit, it has to trap exit by calling process_flag(trap_exit, true). This will result in a message to the process where it can exit with the state X. The code is as below
b_start(A, X) ->
spawn(fun() -> process_flag(trap_exit, true), b_main(A, X) end).
b_main(A, X) ->
try
A ! {self(), doing_stuff},
do_stuff()
catch
exit:_ ->
io:format("exit inside do_stuff() . ~n"),
exit({terminated, X})
end,
receive
{'EXIT',Pid, Reason} ->
io:format("Process received exit ~p ~p.~n",[Pid, Reason]),
exit({terminated, X})
after 0 ->
ok
end,
b_main(A, X).
The short answer: you should do trap_exit in b_main/2 as well, and receive {'EXIT', ...} messages. It was outlined by #vinod right before my attempt. I, instead, will try to explain some things about what is going on.
If the process is trapping exits and it comes to die, for example when someone called exit(Pid, die) or some linked process ends up itself with exit(die), then it will get the {'EXIT', ...} message in its mailbox instead of dying silently with the same reason. It is the runtime system that issues exit signals to every linked process, and one may trap it instead of dying.
The only exception to this rule is when exit(Pid, kill) call issued, then no matter whether a process is trapping exits or not, it just dies with reason kill.
So, to avoid silent death caused by external exit signal, the process must trap exits. Again, if the process wants to know why someone linked to him just died and take some efforts to recover, that process must trap exits. Every trapped exit signal appears as a message in the process mailbox.
So, there is no effect of your try ... catch exit:_ -> ... statement in the matter of trapping exits.
Generally trap_exit is considered bad practice. There is simple example that shows why:
18> self().
<0.42.0>
19> Pid = spawn_link(fun () -> process_flag(trap_exit, true),
Loop = fun (F) -> receive Any -> io:format("Any: ~p~n", [Any]) end, F(F) end,
Loop(Loop) end).
<0.58.0>
20> exit(Pid, grenade).
Any: {'EXIT',<0.42.0>,grenade}
true
21> exit(Pid, grenade).
Any: {'EXIT',<0.42.0>,grenade}
true
...
As you may see some process is linked, is trapping exits and refuses to exit normally. It is unexpected and, obviously, is potentially dangerous. And it may break a chain of exits issued to a set of linked processes, since links are transitive.
There are bunch of subtle specialties which are laid out wonderfully in this book chapter.