Is there a way to know, at a real time, what threads are opened and what application opened them?
You can look in /proc/<PID>/task/ (where <PID> is a process-ID) which will have a number of subdirectories, each with the name equal to the thread-ID of one of the threads in that task.
Note that this is only sort-of real-time though -- unless you were to "freeze" the entire system for the duration, the information you get can always be stale, because a process may create or destroy threads even as you're looking at the data.
In modern Linuxes, threads are very much like processes. Each thread has an LWP ("light-weight process") identifier, which is internally implemented as PID. However, if such "light-weight process" (i.e. thread) is queried for a PID, the system yields the PID of the process that spawned the thread (instead of LWP). Note also, that if the process has only one thread, it's LWP will be equal to its PID.
ps is capable to processing threads with -L modifier, as ereOn described in his answer. But I should note that ps is not just for manual invocation. It has capabilities to print output in such a way that's easy to parse by another program.
The following command will print LWPs (-o lwp=) and PIDs (-o pid=) of all (-A) threads (-L) in the system. Each string represents one thread, second column being the process that spawned it:
$ ps -A -L -o lwp= -o pid=
...
27747 27747
27749 27749
27750 27750
27751 27750
27752 27750
27755 27750
27756 27750
27772 27772
27858 27858
30457 30457
30886 30886
Quite easy to parse with C or C++, isn't it? To actually read this from your program, you can use popen or one of its C++ equivalents.
Note that using ps is not only easier than reading /proc. It's also much safer than manually parsing /proc filesystem. ps is a POSIX command*, it's guaranteed to work; it does use /proc under Linux, but that's internal details. When underlying infrastructure changes, ps will be rewritten, and will keep working, while your code, if written based on /proc, will break.
*To be honest, POSIX does not specify -L switch. But in any Linux, which has GNU toolchain, it will be available.
I guess ps -L should do the trick.
Here is psdocumentation.
Not sure if you ask how to do that programmaticaly, but in this case, since ps is open-source, so you can probably take a look at the sources.
Related
I am trying to run multiple command in ubuntu using c++ code at the same time.
I used system() call to run multiple command but the problem with system() call is it invoke only one command at a time and rest commands are in waiting.
below I wrote my sample code, may this help you to get what I am trying to do.
major thing is I want to run all these command at a time not one by one. Please help me.
Thanks in advance.
main()
{
string command[3];
command[0]= "ls -l";
command[1]="ls";
command[2]="cat main.cpp";
for(int i=0;i<3;i++){
system(command[i].c_str());
}
}
You should read Advanced Linux Programming (a bit old, but freely available). You probably want (in the traditional way, like most shells do):
perhaps catch SIGCHLD (set the signal handler before fork, see signal(7) & signal-safety(7)...)
call fork(2) to create a new process. Be sure to check all three cases (failure with a negative returned pid_t, child with a 0 pid_t, parent with a positive pid_t). If you want to communicate with that process, use pipe(2) (read about pipe(7)...) before the fork.
in the child process, close some useless file descriptors, then run some exec function (or the underlying execve(2)) to run the needed program (e.g. /bin/ls)
call (in the parent, perhaps after having got a SIGCHLD) wait(2) or waitpid(2) or related functions.
This is very usual. Several chapters of Advanced Linux Programming are explaining it better.
There is no need to use threads in your case.
However, notice that the role of ls and cat could be accomplished with various system calls (listed in syscalls(2)...), notably read(2) & stat(2). You might not even need to run other processes. See also opendir(3) & readdir(3)
Perhaps (notably if you communicate with several processes thru several pipe(7)-s) you might want to have some event loop using poll(2) (or the older select(2)). Some libraries provide an event loop (notably all GUI widget libraries).
You have a few options (as always):
Use threads (C++ standard library implementation is good) to spawn multiple threads which each perform a system call then terminate. join on the thread list to wait for them all to terminate.
Use the *NIX fork command to spawn a new process, then within each child process use exec to execute the desired command (see here for an example of "getting the right string to the right child"). Parent process can use waitpid to determine when all children have finished running, in order to move on with the program.
Append "&" to each of your commands, which'll tell the shell to run each one in the background (specifically, system will start the process in the background then return, without waiting for the result). Not tried this, don't know if it'll work. You can't then wait for the call to terminate though (thanks PSkocik).
Just pointing out - if you run those 3 specific commands at the same time, you're unlikely to be able to read the output as they'll all print text to the terminal at the same time.
If you do require reading the output from within the program (though not mentioned in your question), this is relevant (although it doesn't use system).
I tried looking for a thread on this subject, however couldn't find one. So posting this question.
Assume, I have created couple of threads in C++ in Linux and the code is running.
Now I would like to monitor the process and the threads of the process using a shell script and do some additional processing.
Also I would need the amount of CPU and Memory being used by each thread. I know that a thread is associated to a process, however my requirement is to identify the resources utilized by this thread.
I couldn't find the exact way to identify the threads associated to a process. I tried using PS however I couldn't find any clues. Running RHEL.
From a man page of ps:
To get info about threads:
ps -eLf
ps axms
The manpage for ps might give you more clues.
In particular, it should tell you that -L shows threads, and -o %cpu,%mem will display the amount of CPU and memory being used.
Note that memory is associated with the process, not with any thread, so there is no concept of "memory used by each thread".
Apart from using Linux commands you can use Generic Memory Manager library. Here it defined ThreadingModel class.
How does GDB achieves the feat of attaching itself to a running procesS?
I need a similar capability, where i can issue CLI commands to a running process. For example, i can query the process internal state such as show total_messages_processed? How can i build support for issuing commands to a running process under linux?
Is there a library that can provide CLI communication abilities to a running process and can be extended for custom commands?
The process itself is written in c++
GDB doesn't use the CLI to communicate with its debugee; it uses the ptrace system call / API.
CLI means "command-line interface". The simplest form of communication between processes is stdin / stdout. This is achieved through pipes. For example:
ps -ef | grep 'httpd'
The standard output of ps (which will be a process listing) is connected to the standard input of grep, who will process that process listing output line-by-line.
Are you writing both programs, or you want to communicate with an already-existing process? I have no idea what "show total_messages_processed" means without context.
If you simply want the program to communicate some status, a good approach is that which dd takes: Sending the process the SIGUSR1 signal causes it to dump out its current stats to stderr and continue processing:
$ dd if=/dev/zero of=/dev/null&
[1] 19716
$ pid=$!
$ kill -usr1 $pid
$ 10838746+0 records in
10838746+0 records out
5549437952 bytes (5.5 GB) copied, 9.8995 s, 561 MB/s
Did you consider using AF_UNIX sockets in your process? or D-bus? or make it an HTTP server (e.g. using libonion or libmicrohttpd), perhaps for SOAP, or RCP/XDR
Read some books on Advanced Linux Programming, or Advanced Unix Programming; you surely want to use (perhaps indirectly) some multiplexing syscall like poll(2) perhaps above some event libary like libev. Maybe you want to dedicate a thread for that.
We cannot tell more without knowing what kind of process are you thinking of. You may have to redesign some part of it. If the process is some traditional compute-intensive thing it is not the same as a SMTP server process. In particular, if you have some event loop in the process, use & extend it for monitoring purposes. If you don't have any event loop (e.g. in a traditional number crunching "batch" application) you may need to add one.
In this case I'd suggest 'fork', which splits the currently running process into two. The parent process would read stdin, process the commands and be able to handle all memory that is shared between the two processes. One could theoretically even skip advanced forms of interprocess communication: locks, mutexes, semaphores, signals, sockets or pipes -- but be prepared that the child process has not necessarily written it's state to memory but keeps it in registers.
At fork Operating System makes a copy of the process local variables, after which each process have their own internal state -- thus the easiest method for passing data would be to allocate "shared memory".
One can also write a signal handler to the child process, that goes to sleep/wait state and exits only on another signal -- in that way one can have more time to inspect the child processes internal state. The main rationale for this kind of approach is that one doesn't have to make the process under debugging aware of being debugged: the parent and child processes share the same code base and it's enough for the parent process to implement necessary output methods (formatting to screen?) and serializing the data etc.
I'm looking at the code for a c++ program which pipes the contents of a file to more. I don't quite understand it, so I was wondering if someone could write pseudocode for a c++ program that pipes something to something else? Why is it necessary to use fork?
create pipe
fork process
if child:
connect pipe to stdin
exec more
write to pipe
You need fork() so that you can replace stdin of the child before calling, and so that you don't wait for the process before continuing.
You will find your answer precisely here
Why is it necessary to use fork?
When you run a pipeline from the shell, eg.
$ ls | more
what happens? The shell runs two processes (one for ls, one for more). Additionally, the output (STDOUT) of ls is connected to the input (STDIN) of more, by a pipe.
Note that ls and more don't need to know anything about pipes, they just write to (and read from) their STDOUT (and STDIN) respectively. Further, because they're likely to do normal blocking reads and writes, it's essential that they can run concurrently. Otherwise ls could just fill the pipe buffer and block forever before more gets a chance to consume anything.
... pipes something to something else ...
Note also that aside from the concurrency argument, if your something else is another program (like more), it must run in another process. You create this process using fork. If you just run more in the current process (using exec), it would replace your program.
In general, you can use a pipe without fork, but you'll just be communicating within your own process. This means you're either doing non-blocking operations (perhaps in a synchronous co-routine setup), or using multiple threads.
What is the best way on Linux platform for the process (C++ application) to check its instance is not already running?
The standard way to do this is to create a pidfile somewhere, typically containing the pid of your program.
You don't need to put the pid in there, you could just put an exclusive lock on it. If you open it for reading/writing and flock it with LOCK_EX | LOCK_NB, it will fail if the file is already locked. This is race-condition free, and the lock will be automatically released if the program crashes.
Normally you'd want to do it per-user, so the user's home directory is a good place to put the file.
If it's a daemon, somewhere like /var/run is better.
You can use files and file locks to accomplish this, but, beware it isn't perfect and don't copy the infamous Firefox bug where it refuses to start sometimes even if it isn't already running.
The basic logic of it is:
Invariant:
File xxxxx will exist if and only if the program is running, and the
contents of the file will contain the PID of that program.
On startup:
If file xxxxx exists:
If there is a process with the PID contained in the file:
Assume there is some instance of the program, and exit
Else:
Assume that the program terminated abnormally, and
overwrite file xxxx with the PID of this program
Else:
Create file xxxx, and save the current PID to that file.
On termination (typically registered via atexit):
Delete file xxxxx
In addition to the logic above, you should also use a second file that you lock in order to synchronize access to the PID file (i.e. to act as a mutex to make it safe in terms of process-level concurrency).
A related alternative to Michael's solution is to create a directory in a known location (probably under /var/run or /tmp) and use the success/failure of the system call as the mechanism for ensuring mutual exclusion. This is the same mutual-exclusion trick CVS has used for years as directory creation is atomic on most (maybe all) commodity OSes. A PID file is still useful in the case where the directory + PID creating process dies unexpectedly and fails to clean up. Additionally, when checking to see if the existing directory + PID is valid, I'd suggest explicitly checking the /proc/<PID>/exe symlink to verify that it points to your executable rather than just assuming the PID hasn't been recycled.
For a desktop app, it is probably more feasible to check whether an instance is started for current user, so that two users can have their own instances running.
You could use either some libraries (libunique (GTK+) or QtSingleApplication (Qt)), or do it yourself. In addition to pid-file mentioned earlier, you can open a FIFO or UNIX-domain socket somewhere in user's home directory. This way, you could communicate with running instance, eg. raise window of running instance or tell running instance to open new file/URI/whatever.
You could use a POSIX named semaphore to do this. It is much safer than using a file lock.