Hyperledger Sawtooth cannot start devmode consensus engine - blockchain

Question

I am trying to start up a Hyperledger Sawtooth network on Ubuntu 16.04. I am following the instructions of https://sawtooth.hyperledger.org/docs/core/releases/latest/app_developers_guide/ubuntu.html.
Starting up the validation service works fine, but starting the devmode consensus engine does not work. The following happened:
mdi#boromir:~$ sudo -u sawtooth devmode-engine-rust -vv --connect tcp://localhost:5050
ERROR | devmode_engine_rust: | ReceiveError: TimeoutError
DEBUG | sawtooth_sdk::messag | Disconnected outbound channel
DEBUG | sawtooth_sdk::messag | Exited stream
DEBUG | zmq:547 | socket dropped
DEBUG | zmq:547 | socket dropped
DEBUG | zmq:454 | context dropped
mdi#boromir:~$
The validation service was running, as follows:
mdi#boromir:~$ sudo -u sawtooth sawtooth-validator -vv
[sudo] password for mdi:
[2019-03-07 16:40:15.601 WARNING (unknown file)] [src/pylogger.rs: 40] Started logger at level DEBUG
[2019-03-07 16:40:15.919 DEBUG ffi] loading library libsawtooth_validator.so
[2019-03-07 16:40:15.926 DEBUG ffi] loading library libsawtooth_validator.so
[2019-03-07 16:40:16.299 INFO path] Skipping path loading from non-existent config file: /etc/sawtooth/path.toml
[2019-03-07 16:40:16.299 INFO validator] Skipping validator config loading from non-existent config file: /etc/sawtooth/validator.toml
[2019-03-07 16:40:16.300 INFO keys] Loading signing key: /etc/sawtooth/keys/validator.priv
[2019-03-07 16:40:16.306 INFO cli] sawtooth-validator (Hyperledger Sawtooth) version 1.1.4
[2019-03-07 16:40:16.307 INFO cli] config [path]: config_dir = "/etc/sawtooth"; config [path]: key_dir = "/etc/sawtooth/keys"; config [path]: data_dir = "/var/lib/sawtooth"; config [path]: log_dir = "/var/log/sawtooth"; config [path]: policy_dir = "/etc/sawtooth/policy"
[2019-03-07 16:40:16.307 WARNING cli] Network key pair is not configured, Network communications between validators will not be authenticated or encrypted.
[2019-03-07 16:40:16.333 DEBUG state_verifier] verifying state in /var/lib/sawtooth/merkle-00.lmdb
[2019-03-07 16:40:16.337 DEBUG state_verifier] block store file is /var/lib/sawtooth/block-00.lmdb
[2019-03-07 16:40:16.338 INFO state_verifier] Skipping state verification: chain head's state root is present
[2019-03-07 16:40:16.339 INFO cli] Starting validator with serial scheduler
[2019-03-07 16:40:16.339 DEBUG core] global state database file is /var/lib/sawtooth/merkle-00.lmdb
[2019-03-07 16:40:16.340 DEBUG core] txn receipt store file is /var/lib/sawtooth/txn_receipts-00.lmdb
[2019-03-07 16:40:16.341 DEBUG core] block store file is /var/lib/sawtooth/block-00.lmdb
[2019-03-07 16:40:16.342 DEBUG threadpool] Creating thread pool executor Component
[2019-03-07 16:40:16.343 DEBUG threadpool] Creating thread pool executor Network
[2019-03-07 16:40:16.343 DEBUG threadpool] Creating thread pool executor Client
[2019-03-07 16:40:16.343 DEBUG threadpool] Creating thread pool executor Signature
[2019-03-07 16:40:16.345 DEBUG threadpool] Creating thread pool executor FutureCallback
[2019-03-07 16:40:16.346 DEBUG threadpool] Creating thread pool executor FutureCallback
[2019-03-07 16:40:16.352 DEBUG threadpool] Creating thread pool executor Executing
[2019-03-07 16:40:16.353 DEBUG threadpool] Creating thread pool executor Consensus
[2019-03-07 16:40:16.353 DEBUG threadpool] Creating thread pool executor FutureCallback
[2019-03-07 16:40:16.358 DEBUG threadpool] Creating thread pool executor Instrumented
[2019-03-07 16:40:16.368 DEBUG selector_events] Using selector: ZMQSelector
[2019-03-07 16:40:16.376 INFO interconnect] Listening on tcp://127.0.0.1:4004
[2019-03-07 16:40:16.377 DEBUG dispatch] Added send_message function for connection ServerThread
[2019-03-07 16:40:16.377 DEBUG dispatch] Added send_last_message function for connection ServerThread
[2019-03-07 16:40:16.382 DEBUG genesis] genesis_batch_file: /var/lib/sawtooth/genesis.batch
[2019-03-07 16:40:16.384 DEBUG genesis] block_chain_id: not yet specified
[2019-03-07 16:40:16.384 INFO genesis] Producing genesis block from /var/lib/sawtooth/genesis.batch
[2019-03-07 16:40:16.385 DEBUG genesis] Adding 1 batches
This output is on time 17:29, so no output has been appended for almost an hour.
I tried to see Sawtooth settings:
mdi#boromir:~$ sawtooth settings list
Error: Unable to connect to "http://localhost:8008": make sure URL is correct
mdi#boromir:~$
And I checked what processes were listening to what ports:
mdi#boromir:~$ netstat -plnt
(Not all processes could be identified, non-owned process info
will not be shown, you would have to be root to see it all.)
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name
tcp 0 0 127.0.0.1:4004 0.0.0.0:* LISTEN -
tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN -
tcp6 0 0 :::22 :::* LISTEN -
mdi#boromir:~$
Does anybody know whether the validator service initiates connection or the consensus engine? What is wrong with my sawtooth settings list command? And does anybody know how to get the consensus engine to work? Thanks.

Answer 1

I found the answers myself. I had another machine with a docker installation of Hyperledger Sawtooth. On that server, the validator log had the line:
[2019-03-08 14:39:02.478 INFO interconnect] Listening on tcp://127.0.0.1:5050
Port 5050 is used for the consensus engine as stated in https://sawtooth.hyperledger.org/docs/core/releases/latest/app_developers_guide/ubuntu.html. This makes clear that the consensus engine initiates the connection to the validator service.
So why didn't the validator service listen to port 5050 on my Ubuntu machine? Because the settings transaction processor did not ever run on the Ubuntu machine. I started this processor according to the command in the Ubuntu tutorial:
sudo -u sawtooth settings-tp -v
Then the validator proceeded and started listening to port 5050. As a consequence, the consensus engine could be started.