When looking at various daemon scripts in /etc/init.d/, I can't seem to understand the purpose of the 'lockfile' variable. It seems like the 'lockfile' variable is not being checked before starting the daemon.
For example, some code from /etc/init.d/ntpd:
prog=ntpd
lockfile=/var/lock/subsys/$prog
start() {
[ "$EUID" != "0" ] && exit 4
[ "$NETWORKING" = "no" ] && exit 1
[ -x /usr/sbin/ntpd ] || exit 5
[ -f /etc/sysconfig/ntpd ] || exit 6
. /etc/sysconfig/ntpd
# Start daemons.
echo -n $"Starting $prog: "
daemon $prog $OPTIONS
RETVAL=$?
echo
[ $RETVAL -eq 0 ] && touch $lockfile
return $RETVAL
}
What is the 'lockfile' variable doing?
Also, when writing my own daemon in C++ (such as following the example at the bottom of http://www.itp.uzh.ch/~dpotter/howto/daemonize), do I put the compiled binary directly in /etc/init.d/ or do I put a script there that calls the binary. (i.e. replacing the 'daemon $prog' in the code above with a call to my binary?)
The whole thing is a very fragile and misguided attempt to keep track of whether a given daemon is running in order to know whether/how to shut it down later. Using pids does not help, because a pid is meaningless to any process except the direct parent of a process; any other use has unsolvable and dangerous race conditions (i.e. you could end up killing another unrelated process). Unfortunately, this kind of ill-designed (or rather undesigned) hackery is standard practice on most unix systems...
There are a couple approaches to solving the problem correctly. One is the systemd approach, but systemd is disliked among some circles for being "bloated" and for making it difficult to use a remote /usr mounted after initial boot. In any case, solutions will involve either:
Use of a master process that spawns all daemons as direct children (i.e. inhibiting "daemonizing" within the individual daemons) and which thereby can use their pids to watch for them exiting, keep track of their status, and kill them as desired.
Arranging for every daemon to inherit an otherwise-useless file descriptor, which it will keep open and atomically close only as part of process termination. Pipes (anonymous or named fifos), sockets, or even ordinary files are all possibilities, but file types which give EOF as soon as the "other end" is closed are the most suitable, since it's possible to block waiting for this status. With ordinary files, the link count (from stat) could be used but there's no way to wait on it without repeated polling .
In any case, the lockfile/pidfile approach is ugly, error-prone, and hardly any better than lazy approaches like simply killall foobard (which of course are also wrong).
What is the 'lockfile' variable doing?
It could be nothing or it could be eg. injected into $OPTIONS by this line
. /etc/sysconfig/ntpd
The daemon takes the option -p pidfile where $lockfile could go. The daemon writes its $PID in this file.
do I put the compiled binary directly in /etc/init.d/ or do I put a script there that calls the binary
The latter. There should be no binaries in /etc, and its customary to edit /etc/init.d scripts for configuration changes. Binaries should go to /(s)bin or /usr/(s)bin.
The rc scripts keep track of whether or not it is running and don't bother stopping what is not running.
Related
I am looking for information on looping in Informatica. Specifically, I need to check if a source table has been loaded, if it has, move to next step, if not wait X minutes and check the status table again. I would prefer direction to a place I can learn this on my own, but I need to confirm this is even possible as I have not found anything on my google searches.
You can use a simple shell script to do this wait and watch capability.
#/bin/sh
# call it as script_name.sh
# it will wait for 10 min and check again for data, in total it will wait for 2hours. change them if you want to
# Source is assumed as oracle. change it as per your source.
interval=600
loop_count=10
counter=0
while true
do
$counter=`expr $counter + 1 `
db_value=`sqlplus -s user/pass#local_SID <<EOF
set heading off
set feedback off
SELECT count(*) FROM my_source_table;
exit
EOF`;
if [ $db_value -gt 0 ]; then
echo "Data Found."
exit 0
else
if [ $counter -eq $loop_count ]
then
echo "No data found in source after 2hours"
exit 1
else
sleep $interval
fi
fi
done
And add this shell script(in a CMD task) to the beginning of the workflow.
Then use informatica link condition as if status= 0, proceed else email that wait time is over.
You can refer to the pic below. This will send a mail if wait time is over and still data is not there in source.
In general, looping is not supported in Informatica PowerCenter.
One way is to use scripts, as discussed by Koushik.
Another way to do that is to have a Continuously Running Workflow with a timer. This is configurable on Scheduler tab of your workflow:
Such configuration makes the workflow start again right after it succeeds. Over and over again.
Workflow would look like:
Start -> s_check_source -> Decision -> timer
|-> s_do_other_stuff -> timer
This way it will check source. Then if it has not been loaded trigger the timer. Thet it will succeed and get triggered again.
If source turns out to be loaded, it will trigger the other session, complete and probably you'd need another timer here to wait till next day. Or basically till whenever you'd like the workflow to be triggered again.
There are two shell scripts:
#shell_script_1
nc -l -p 2234
#shell_script_2
echo "hello" | nc -p 1234 localhost 2234 -w0
From inside the C++ program, I want to run shell script no.1 first, and then run shell script no.2. What I have now is something like this:
#include <string>
#include <thread>
#include <cstdlib>
#include <unistd.h>
int main()
{
std::string sh_1 = "./shell_script_1";
std::string sh_2 = "./shell_script_2";
std::thread t1( &system, sh_1.c_str() );
usleep( 5000000 ); //wait for 5 seconds
std::thread t2( &system, sh_2.c_str() );
t1.join();
t2.join();
}
When I run the program above, shell_script_1 runs before shell_script_2, as expected. However, is a 5-second wait enough to make sure that the two shell scripts start in order? Is there anyway I can enforce the order other than set a timer and cross my finger? Thanks.
It is not enough to "start" the first script before the second. You want the first script to actually be listening on the port you've specified. To make that happen, you need to check periodically. This will be platform dependant, but on Linux you could check /proc/PID of the first child to know what file descriptors it has open, and/or run nc -z to check if the port is listening.
A simpler approach would be to automatically retry the second script a few times if it fails to connect and the first thread is still running.
A more sophisticated approach would be make your C++ program bind two ports and listen on both, and change your first script to connect instead of listen. This way both scripts would act as clients, and your C++ launcher would act as the server (even if all it does is pass the data between the two children), giving you more control and avoiding a race.
I have a program written by someone else that uses OpenMP. I am running it on a cluster that uses Slurm as its job manager. Despite setting OMP_NUM_THREADS=72 and properly requesting 72 cores for the job, the job is only using four cores.
I have already used scontrol show job <job_id> --details to verify that there are 72 cores assigned to the job. I have also remoted into the node that the job is running on and used htop to inspect it. It was running 72 threads, all on four cores. It is worth noting that this is on an SMT4 power9 cpu, meaning that each physical core executes 4 simultaneous threads. Ultimately, it looks like openMP is putting all threads on one physical core. This is further complicated by the fact that this is an IBM system. I can't seem to find any useful documentation on more fine control of the openMP environment. Everything I find is for Intel.
I have also tried using taskset to manually change the affinity. This worked as intended and moved one of the threads to an unused core. The program continued to work as intended after this.
I could theoretically write a script to find all of the threads and call taskset to assign them to cores in a logical way, but I am afraid to do this. It seems like a bad idea to me. It would also take a while.
I guess my main question would be, is this a Slurm problem, an openMP problem, an IBM problem or a user error? Is there some environment variable I don't know about that I need to set? Will it break Slurm if I manually call taskset using a script? I would use scontrol to figure out which cpus are assigned to the job if I did that. I don't want to anger the people who run the cluster by messing things up though.
Here is the submission script. I can't include any of the actual running code due to license issues though. I'm hoping this will just be a simple matter of fixing an environment variable. The MPI_OPTIONS variables were recommended by the guy who administers the system. If by some chance someone here has worked with the ENKI cluster before, that's where this is running.
wrk_path=${PWD}
cat >slurm.sh <<!
#!/bin/bash
#SBATCH --partition=general
#SBATCH --time 2:00:00
#SBATCH -o log
#SBATCH --nodes=1
#SBATCH --ntasks=1
#SBATCH --ntasks-per-node=1
#SBATCH --ntasks-per-socket=1
#SBATCH --cpus-per-task=72
#SBATCH -c 72
#SBATCH -J Cu-A-E
#SBATCH -D $wrk_path
cd $wrk_path
module load openmpi/3.1.3/2019
module load pgi/2019
export OMP_NUM_THREADS=72
MPI_OPTIONS="--mca btl_openib_if_include mlx5_0"
MPI_OPTIONS="$MPI_OPTIONS --bind-to socket --map-by socket --report-bindings"
time mpirun $MPI_OPTIONS ~/bin/pgmc-enki > out.dat
!
sbatch slurm.sh
Edit: Fix resulted in a 7x speedup when using 72 cores, vs. just running on 4 cores. Considering the nature of the calculations being run, this is pretty good.
Edit 2: Fix resulted in a 17x speedup when using 160 vs. just running on 4 cores.
This might not work for everyone, but I have a really hacky solution. I wrote a python script that uses psutil to find all threads that are children of the running process and set their affinity manually. This script uses scontrol to figure out which cpus are assigned to the job and uses taskset to force the threads to distribute across those cpus.
So far the process is running a lot faster. I'm sure that forcing CPU affinity isn't the best way to do it, but its a lot better than not using the available resources at all.
Here is the basic idea behind the code. The program I am running is called pgmc, hence the variable names. You will need to create an anaconda environment with psutil installed if you are running on a system like mine.
import psutil
import subprocess
import os
import sys
import time
# Gets the id for the current job.
def get_job_id():
return os.environ["SLURM_JOB_ID"]
# Returns a list of processors assigned to the job and the total number of cpus
# assigned to the job.
def get_proc_info():
run_str = 'scontrol show job %s --details'%get_job_id()
stdout = subprocess.getoutput(run_str)
id_spec = None
num_cpus = None
chunks = stdout.split(' ')
for chunk in chunks:
if chunk.lower().startswith("cpu_ids"):
id_spec = chunk.split('=')[1]
start, stop = id_spec.split('-')
id_spec = list(range(int(start), int(stop) + 1))
if chunk.lower().startswith('numcpus'):
num_cpus = int(chunk.split('=')[1])
if id_spec is not None and num_cpus is not None:
return id_spec, num_cpus
raise Exception("Couldn't find information about the allocated cpus.")
if __name__ == '__main__':
# Before we do anything, make sure that we can get the list of cpus
# assigned to the job. Once we have that, run the command line supplied.
cpus, cpu_count = get_proc_info()
if len(cpus) != cpu_count:
raise Exception("CPU list didn't match CPU count.")
# If we successefully got to here, run the command line.
program_name = ' '.join(sys.argv[1:])
pgmc = subprocess.Popen(sys.argv[1:])
time.sleep(10)
pid = [proc for proc in psutil.process_iter() if proc.name() == "your program name here"][0].pid
# Now that we have the pid of the pgmc process, we need to get all
# child threads of the process.
pgmc_proc = psutil.Process(pid)
pgmc_threads = list(pgmc_proc.threads())
# Now that we have a list of threads, we loop over available cores and
# assign threads to them. Once this is done, we wait for the process
# to complete.
while len(pgmc_threads) != 0:
for core_id in cpus:
if len(pgmc_threads) != 0:
thread_id = pgmc_threads[-1].id
pgmc_threads.remove(pgmc_threads[-1])
taskset_string = 'taskset -cp %i %i'%(core_id, thread_id)
print(taskset_string)
subprocess.getoutput(taskset_string)
else:
break
# All of the threads should now be assigned to a core.
# Wait for the process to exit.
pgmc.wait()
print("program terminated, exiting . . . ")
Here is the submission script used.
wrk_path=${PWD}
cat >slurm.sh <<!
#!/bin/bash
#SBATCH --partition=general
#SBATCH --time 2:00:00
#SBATCH -o log
#SBATCH --nodes=1
#SBATCH --ntasks=1
#SBATCH --ntasks-per-node=1
#SBATCH --ntasks-per-socket=1
#SBATCH --cpus-per-task=72
#SBATCH -c 72
#SBATCH -J Cu-A-E
#SBATCH -D $wrk_path
cd $wrk_path
module purge
module load openmpi/3.1.3/2019
module load pgi/2019
module load anaconda3
# This is the anaconda environment I created with psutil installed.
conda activate psutil-node
export OMP_NUM_THREADS=72
# The two MPI_OPTIONS lines are specific to this cluster if I'm not mistaken.
# You probably won't need them.
MPI_OPTIONS="--mca btl_openib_if_include mlx5_0"
MPI_OPTIONS="$MPI_OPTIONS --bind-to socket --map-by socket --report-bindings"
time python3 affinity_set.py mpirun $MPI_OPTIONS ~/bin/pgmc-enki > out.dat
!
sbatch slurm.sh
My main reason for including the submission script is to demonstrate how the python script is used. More specifically, you call it, with your real job as an argument.
I need to detect when a computer is idle for a certain time period. My definition of idleness is:
No users logged in, either by remote methods or on the local machine
X server inactivity, with no movement of mouse or key presses
TTY keyboard inactivity (hopefully)
Since the majority of distros have now moved to logind, I should be able to use its DBUS interface to find out if users are logged in, and also to monitor logins/logouts. I have used xautolock to detect X idleness before, and I could continue using that, but xscreensaver is also available. Preferably however I want to move away from any specific dependencies like the screensaver due to different desktop environments using different components.
Ideally, I would also be able to base idleness on TTY keyboard inactivity, however this isn't my biggest concern. According to this answer, I should be able to directly query the /dev/input/* interfaces, however I have no clue how to go about this.
My previous attempts at making such a monitor have used Bash, due to the ease of changing a plain text script file, howver I am happy using C++ in case more advanced methods are required to accomplish this.
From a purely shell standpoint (since you tagged this bash), you can get really close to what you want.
#!/bin/sh
users_are_logged_in() {
who |grep -q .
return $?
}
x_is_blanked() {
local DISPLAY=:0
if xscreensaver-command -time |grep -q 'screen blanked'; then
return 0 # we found a blanked xscreensaver: return true
fi
# no blanked xscreensaver. Look for DPMS modes
xset -q |awk '
/DPMS is Enabled/ { dpms = 1 } # DPMS is enabled
/Monitor is On$/ { monitor = 1 } # The monitor is on
END { if(dpms && !monitor) { exit 0 } else { exit 1 } }'
return $? # true when DPMS is enabled and the monitor is not on
}
nobody_here() {
! users_are_logged_in && x_is_blanked
return $?
}
if nobody_here; then
sleep 2m
if nobody_here; then
# ...
fi
fi
This assumes that a user can log in in two minutes and that otherwise, there is no TTY keyboard activity.
You should verify that the who |grep works on your system (i.e. no headers). I had originally grepped for / but then it won't work on FreeBSD. If who has headers, maybe try [ $(who |grep -c .) -gt 1 ] which will tell you that the number of lines that who outputs is more than one.
I share your worry about the screensaver part; xscreensaver likely isn't running in the login manager (any other form of X would involve a user logged in, which who would detect), e.g. GDM uses gnome-screensaver, whose syntax would be slightly different. The DPMS part may be good enough, giving a far larger buffer for graphical logins than the two minutes for console login.
Using return $? in the last line of a function is redundant. I used it to clarify that we're actually using the return value from the previous line. nobody_here short circuits, so if no users are logged in, there is no need to run the more expensive check for the status of X.
Side note: Be careful about using the term "idle" as it more typically refers to resource (hardware, that is) consumption (e.g. CPU load). See the uptime command for load averages for the most common way of determining system (resource) idleness. (This is why I named my function nobody_here instead of e.g. is_idle)
I want to write a script for gdb, which will save backtrace (stack) of process every 10 ms. How can I do this?
It can be smth like call graph profiling for 'penniless' (for people, who can't use any sort of advanced profiler).
Yes, there are a lot of advanced profilers. For popular CPUs and for popular OSes. Shark is very impressive and easy to use, but I want to get a basic functionality with such script, working with gdb.
Can you get lsstack? Perhaps you could run that from a script outside your app. Why 10ms? Percentages will be about the same at 100ms or more. If the app is too fast, you could artificially slow it down with an outer loop, and that wouldn't change the percentages either. For that matter, you could just use Ctrl-C to get the samples manually under gdb, if the app runs long enough and if your goal is to find out where the performance problems are.
(1) Manual. Execute the following in a shell. Keep pressing Ctrl+C repeatedly on shell prompt.
gdb -x print_callstack.gdb -p pid
or, (2) send signals to pid repeatedly same number of times on another shell as in below loop
let count=0; \
while [ $count -le 100 ]; do \
kill -INT pid ; sleep 0.10; \
let $count=$count+1; \
done
The source of print_callstack.gdb from (1) is as below:
set pagination 0
set $count = 0
while $count < 100
backtrace
continue
set $count = $count + 1
end
detach
quit
man page of pstack https://linux.die.net/man/1/pstack
cat > gdb.run
set pagination 0
backtrace
continue
backtrace
continue
... as many more backtrace + continue's as needed
backtrace
continue
detach
quit
Of course, omit the duplicate newlines, how do you do single newlines in this forum software? :(
gdb -x gdb.run -p $pid
Then just use do
kill -INT $pid ; sleep 0.01
in a loop in another script.
kill -INT is what the OS does when you hit ctrl-C. Exercise for the reader: make the gdb script use a loop with $n iterations.