Why lein repl blocks STDERR - clojure

Why lein repl blocks STDERR. I run lein repl without project and also lein repl or lein run with a project and having problems printing to STDERR. For example when connecting to NREPL (different terminal tab) and running:
(.println System/err "something")
I can see nothing printed in the launching terminal window - only in the repl. How to get STDERR to print in both as it is used for SYSTEMD logging.
I use newest Leiningen and project is generated wit the app template.

lein repl does no such thing. You can test this yourself:
# stop our individual writes from overlapping
with_lock() { exec {lock_fd}>>./tty_write_lock; flock "$lock_fd"; "$#"; exec {lock_fd}>&-; }
lein repl 3>&2 \
> >(while IFS= read -r line; do with_lock printf 'O:<%q>\n' "$line" >&3; done) \
2> >(while IFS= read -r line; do with_lock printf 'E:[%q]\n' "$line" >&3; done) \
3>&- <<'EOF'
(.println System/err "something on stderr")
(.println System/out "something on stdout")
EOF
...which contains, among other output:
O:<$'user=> (.println System/err "something on stderr")\E[50G\E[8G\E[51G'>
E:[something\ on\ stderr]
O:<nil>
O:<$'user=> (.println System/out "something on stdout")\E[50G\E[8G\E[51G'>
O:<something\ on\ stdout>
O:<nil>
As you can see, something on stdout is inside O:<...>, as created by the process substitution handling stdout; whereas something on stderr is inside E:[...], as created by the process substitution handling stderr.
To look at it a different way:
{ strace -f -e write lein repl 2>&1 | egrep 'write[(][12],'; } <<'EOF'
(.println System/err "something on stderr")
(.println System/out "something on stdout")
EOF
...includes among its output the following syscalls:
[pid 10467] write(2, "something on stderr", 19) = 19
[pid 10467] write(1, "something on stdout", 19) = 19
...as you can see, the stderr one is correctly written to FD 2, whereas the stdout one is correctly written to FD 1.

Related

QThread doesn't provide output from called shellscript

I have a function that executes system calls and writes the output into a QTextEdit.
string SystemCallFactory::runSysCallWithoutButton(unique_ptr<SystemCall> sysCall)
{
cout << "Running " << sysCall->getCommand() << endl;
textEdit->setText("");
textEdit->repaint();
QProcess process;
process.start(QString::fromStdString(sysCall->getCommand()));
process.waitForFinished();
QString output(process.readAllStandardOutput());
textEdit->append(output);
textEdit->repaint();
return output.toStdString();
}
The problem is that this works when getCommand() is a real command, e.g. ls /etc but does not work correctly when it refers to a shellscript. In my calls, the call that does not work looks like
/home/turtle10000/tilematching.sh /some/folder /some/specific.file
The script gets executed and does what it's supposed to do, but readAllStandardOutput() returns an empty string. When I run it in a terminal, it shows the expected output.
This is the shellscript:
#!/bin/sh -v
WORKSPACE=$1
SVPFILE=$2
cd $WORKSPACE
ls -1 *.all > datalist.0.mb-1
mbset -I datalist.0.mb-1 -PSVPMODE:1 -PSVPFILE:$SVPFILE
mbprocess -I datalist.0.mb-1
ls -1 *p.mb58 > datalist.1.mb-1
mbset -I datalist.1.mb-1 -PSONAROFFSETX:-0.079 -PSONAROFFSETY:0.196 -PSONAROFFSETZ:0.048 -PVRUOFFSETX:-0.4473 -PVRUOFFSETY:0.000 -PVRUOFFSETZ:-0.3395 -PROLLBIAS:0.1 -PPITCHBIAS:1.32
mbprocess -I datalist.1.mb-1
ls -1 *pp.mb58 > datalist.2.mb-1
mblist -I datalist.2.mb-1 -MA -O^X^Y-z -JU > output.xyz
#remove broken lines (error in input files)
sed -ie '/ /d' output.xyz
Edit: as Botje guessed, some of output appears in stderr instead of stdout.
As Botje suggested, I checked if the output went to stderr and it did. For whatever reason, the output of mbset and mbprocess, tools from the mb-system package, goes to stderr.
In this case, I changed the line
QString output(process.readAllStandardOutput());
to
QString output(process.readAllStandardError());
Another option would probably be to channel the stderr output to stdout in the script.

Program (nload) runs as a daemon when executed in shell but not in startup/automation script

I would like to run nload (a network throughput monitor) as a daemon on startup (or just automate in general). I can successfully run it as a daemon from the command line by typing this:
nload eth0 >& /dev/null &
Just some background: I modified the nload source code (written in C++) slightly to write to a file in addition to outputting to the screen. I would like to read the throughput values from the file that nload writes to. The reason I am outputting to /dev/null is so that I don't need to worry about the stdout output.
The weird thing is that, when I run it manually it runs just fine as a dameon and I am able to read throughput values from the file. But every attempt at automation has failed. I have tried init.d, rc.local, cron but no luck. The script I wrote to run this in automation is:
#!/bin/bash
echo "starting nload"
/usr/bin/nload eth0 >& /dev/null &
if [ $? -eq 0 ]; then
echo started nload
else
echo failed to start nload
fi
I can confirm that when automated, the script does run, since I tried logging the output. It even logs "started nload", but when I look at the list of processes running nload is not one of them. I can also confirm that when the script is run manually from the shell, nload starts up just fine as a daemon.
Does anyone know what could be preventing this program from running when run via an automated script?
looks like nload is crashing if it's not run from terminal.
viroos#null-linux:~$ cat /etc/rc.local
#!/bin/sh -e
#
# rc.local
#
# This script is executed at the end of each multiuser runlevel.
strace -o /tmp/nload.trace /usr/bin/nload
exit 0
looks like HOME env var is missing:
viroos#null-linux:~$ cat /tmp/nload.trace
brk(0x1f83000) = 0x1f83000
write(2, "Could not retrieve home director"..., 34) = 34
write(2, "\n", 1) = 1
exit_group(1) = ?
+++ exited with 1 +++
lets fix this:
#!/bin/sh -e
#
# rc.local
#
# This script is executed at the end of each multiuser runlevel.
export HOME=/tmp
strace -o /tmp/nload.trace /usr/bin/nload
exit 0
we have another problem:
viroos#null-linux:~$ cat /tmp/nload.trace
read(3, "\32\1\36\0\7\0\1\0\202\0\10\0unknown|unknown term"..., 4096) = 320
read(3, "", 4096) = 0
close(3) = 0
munmap(0x7f23e62c9000, 4096) = 0
ioctl(2, SNDCTL_TMR_TIMEBASE or SNDRV_TIMER_IOCTL_NEXT_DEVICE or TCGETS, 0x7ffedd149010) = -1 ENOTTY (Inappropriate ioctl for device)
ioctl(2, SNDCTL_TMR_TIMEBASE or SNDRV_TIMER_IOCTL_NEXT_DEVICE or TCGETS, 0x7ffedd148fb0) = -1 ENOTTY (Inappropriate ioctl for device)
write(2, "Error opening terminal: unknown."..., 33) = 33
exit_group(1) = ?
+++ exited with 1 +++
I saw you mentioned that you modified nload code but my guess is you haven't removed handling missing termin. You can try further editing nload code or use screen in detached mode:
viroos#null-linux:~$ cat /etc/rc.local
#!/bin/sh -e
#
# rc.local
#
# This script is executed at the end of each multiuser runlevel.
export HOME=/tmp
screen -S nload -dm /usr/bin/nload
exit 0

how to save the stack trace of a linux demon/service into an external file

I have a problem with printing the stack trace (back-trace) of a demon which is created for C++ application. service is based on /etc/init.d/ init script based architecture and I have seen several services followed the same to deliver applications as services in Linux. in this case when stopping my service by service name stop stack trace of the program is printing on the console. This backtrace contains the uncleaned memory map of the program.
I just need to avoid printing this on the terminal and save it into a separate file till I fix the memory issues of the program.
EDITED:
Can this do by adding more commands to the init script ?
Is there any compilation option to avoid printing the program
stack-trace after stopping the program ?
Can a block of c/c++ source code use to handle this ?
Below is my /etc/init.d/init script.
# Source function library.
. /etc/rc.d/init.d/functions
cscored=${CSCORED-/opt/application-name/bin/cscore &}
prog=cscored
pidfile=${PIDFILE-/var/run/cscored/cscored.pid}
lockfile=${LOCKFILE-/var/lock/subsys/cscored}
RETVAL=0
STOP_TIMEOUT=${STOP_TIMEOUT-10}
start() {
echo -n $"Starting $prog: "
daemon $cscored
RETVAL=$?
echo
[ $RETVAL = 0 ] && touch ${lockfile}
return $RETVAL
}
stop() {
echo -n $"Stopping $prog: "
killproc $cscored
RETVAL=$?
echo
[ $RETVAL = 0 ] && rm -f ${lockfile} ${pidfile}
}
reload() {
echo -n $"Reloading $prog: "
if ! $cscored -t >&/dev/null; then
RETVAL=6
echo $"not reloading due to configuration syntax error"
failure $"not reloading $cscored due to configuration syntax error"
else
# Force LSB behaviour from killproc
LSB=1 killproc -p ${pidfile} $cscored -HUP
RETVAL=$?
if [ $RETVAL -eq 7 ]; then
failure $"cscored shutdown"
fi
fi
echo
}
# See how we were called.
case "$1" in
start)
start
;;
stop)
stop
;;
status)
status $cscored
RETVAL=$?
;;
restart)
stop
start
;;
condrestart|try-restart)
if status $cscored >&/dev/null; then
stop
start
fi
;;
force-reload|reload)
reload
;;
*)
echo $"Usage: $prog {start|stop|restart|condrestart|try-restart|force-reload|reload|status|fullstatus|graceful|help|configtest}"
RETVAL=2
esac
exit $RETVAL
You can redirect the output of the console to another file either using a pipe or dup().

How can I reliably launch multiple DJango FCGI servers at startup?

I currently use the following script to launch my DJango FCGI servers:
#!/bin/bash
MYAPP=$1
PIDFILE=/var/run/${MYAPP}_fcgi.pid
SOCKET=/var/django/${MYAPP}/socket.sock
MANAGESCRIPT=/var/django/${MYAPP}/manage.py
# Maximum requests for a child to service before expiring
#MAXREQ=
# Spawning method - prefork or threaded
#METHOD=
# Maximum number of children to have idle
MAXSPARE=2
# Minimum number of children to have idle
MINSPARE=1
# Maximum number of children to spawn
MAXCHILDREN=3
cd "`dirname $0`"
function failure () {
STATUS=$?;
echo; echo "Failed $1 (exit code ${STATUS}).";
exit ${STATUS};
}
function start_server () {
$MANAGESCRIPT runfcgi socket=$SOCKET pidfile=$PIDFILE \
${MAXREQ:+maxrequests=$MAXREQ} \
${METHOD:+method=$METHOD} \
${MAXSPARE:+maxspare=$MAXSPARE} \
${MINSPARE:+minspare=$MINSPARE} \
${MAXCHILDREN:+maxchildren=$MAXCHILDREN} \
${DAEMONISE:+damonize=True}
touch $SOCKET
chown www-data:www-data $SOCKET
chmod 755 $SOCKET
}
function stop_server () {
if [ -f "$PIDFILE" ]
then
kill `cat $PIDFILE` || failure "Server was not running."
rm $PIDFILE
fi
}
DAEMONISE=$3
case "$2" in
start)
echo -n "Starting fcgi: "
[ -e $PIDFILE ] && { echo "PID file exsts."; exit; }
start_server || failure "starting fcgi"
echo "Done."
;;
stop)
echo -n "Stopping fcgi: "
[ -e $PIDFILE ] || { echo "No PID file found."; exit; }
stop_server
echo "Done."
;;
restart)
echo -n "Restarting fcgi: "
[ -e $PIDFILE ] || { echo -n "No PID file found..."; }
stop_server
start_server || failure "restarting fcgi"
echo "Done."
;;
*)
echo "Usage: $0 {start|stop|restart} [--daemonise]"
;;
esac
exit 0
Which I manually call like this:
/var/django/server.sh mysite start
This works fine but when my hosting company reboots our server it leaves me two issues:
I don't have an automated way to launch multiple sites.
I end up with a mysite_fcgi.pid file existing but no associated process.
So I have two questions:
How can I launch a list of sites (stored in a plain text file) automatically on startup? i.e. call /var/django/server.sh mysite1 start then /var/django/server.sh myothersite start?
How can I get rid of the .pid file if the process doesn't exist and attempt to start the server as normal?
Create an init script and assign it to the appropriate runlevel.
You need to implement this in your startup/init script (that you would write in step 1)
Or, use a process manager like supervisord which takes care of all your concerns.
Here is a configuration example for fcgi from supervisord.
[fcgi-program:fcgiprogramname]
command=/usr/bin/example.fcgi
socket=unix:///var/run/supervisor/%(program_name)s.sock
process_name=%(program_name)s_%(process_num)02d
numprocs=5
priority=999
autostart=true
autorestart=unexpected
startsecs=1
startretries=3
exitcodes=0,2
stopsignal=QUIT
stopwaitsecs=10
user=chrism
redirect_stderr=true
stdout_logfile=/a/path
stdout_logfile_maxbytes=1MB
stdout_logfile_backups=10
stderr_logfile=/a/path
stderr_logfile_maxbytes=1MB
stderr_logfile_backups
environment=A=1,B=2
How can I launch a list of sites (stored in a plain text file) automatically on startup?
In general, your OS provides a file where you can hook your commands at startup. For example, arch linux uses rc.local, gentoo either /etc/local.start either /etc/local.d/*.start, debian requires you to make an init script - which is basically a script that takes "start" or "stop" as argument and lives in /etc/init.d or /etc/rc.d depending on the distribution ...
You can use some bash code as such.
for site in $(</path/to/text/file); do
/var/django/server.sh $site start
done
How can I get rid of the .pid file if the process doesn't exist and attempt to start the server as normal?
if [[ -f $PIDFILE ]]; then # if pidfile exists
if [[ ! -d /proc/$(<$PIDFILE)/ ]]; then # if it contains a non running proc
unlink $PIDFILE # delete the pidfile
fi
fi

sos Job scheduler

i am using sos job scheduler which support many language.i accept the shell script to write jobs but i am not a shell script writer.i want to implement a following points in job scheduler:
execute a shell script A. script A return "success" if time is between 6:00AM and 3PM.else it return "fail".
on "success" execute a shell script C or on "Fail" it execute shell script B.
Script B and Script C send email with“Success” or “Failure” in subject line.
please help me to sortout the above discuss problem.
Thanks
There are two command line utilities that are helpful in this case:
date: Displays the current time/date in a specified format.
mail: Sends e-mail from the command line.
Since we only need the full hour for our logic I use the date format "+%H" (hour from 0-23). This gives the following script basis:
#!/bin/sh
hour=$(date +%H)
if [ $hour -gt 6 -a $hour -lt 15 ]; then
echo "message body" | mail -s Success <your e-mail address>
else
echo "message body" | mail -s Failure <your e-mail address>
fi
#!/bin/bash
hour=$(date +%H)
recipient="root"
case "$hour" in
[6-9]|1[0-5])
subject="success"
body="message"
;;
*)
subject="failure"
body="message"
;;
esac
echo $body | mailx -s "$subject" "$recipient"