Running an executable from a C++ program in the same process - c++

Is that possible? I'd like an easy access to the executable's memory to edit it. Alternately, when I'm not the administrator, is it possible to edit the executable's memory from another process? I've tried the ptrace library and it fails if I'm not the administrator. I'm on Linux

I'm not entirely sure what you are asking, but this is possible with shared memory.
See here: http://www.kernel.org/doc/man-pages/online/pages/man7/shm_overview.7.html

This is what a debugger does. You could look at the code of an open source debugger, e.g. gdb, to see how it works.

The answer:
Yes - it works: you don't have to be administrator / root, but of course you need the rights to access the process' memory, i.e. same user.
No - it is not easy
The possibility to write to /proc/pid/mem was added some time ago to the Linux kernel. Therefore it depends on the kernel you are using. The small programs were checked with kernel 3.2 where this works and 2.6.32 where it fails.
The solution consists of two programs:
A 'server' which is started, allocates some memory, writes some pattern into this memory and outputs every three seconds the memory contents which is placed after the pattern is printed.
A 'client' which connects via the /proc/pid/maps and /proc/pid/mem to the server, searches for the pattern and writes some other string into the server's memory.
The implementation uses heap - but as long as the permissions allow - it is also possible to change other portions of the other process' memory.
This is implemented in C, because it is very 'low level' - but it should work in C++. It is a proof of concept - no production code - e.g. there are some error checks missing and it has some fixed size buffers.
memholder.c
/*
* Alloc memory - write in some pattern and print out the some bytes
* after the pattern.
*
* Compile: gcc -Wall -Werror memholder.c -o memholder.o
*/
#include <sys/types.h>
#include <string.h>
#include <stdlib.h>
#include <stdio.h>
#include <unistd.h>
int main() {
char * m = (char*) malloc(2048);
memset(m, '\xAA', 1024);
strcpy(m + 1024, "Some local data.");
printf("PID: %d\n", getpid());
while(1) {
printf("%s\n", m + 1024);
sleep(3);
}
return 0;
}
memwriter.c
/*
* Searches for a pattern in the given PIDs memory
* and changes some bytes after them.
*
* Compile: gcc -Wall -std=c99 -Werror memwriter.c -o memwriter
*/
#include <sys/types.h>
#include <stdio.h>
#include <stdlib.h>
#include <sys/stat.h>
#include <fcntl.h>
#include <unistd.h>
#include <string.h>
#include <sys/mman.h>
#include <sys/ptrace.h>
#include <sys/wait.h>
int open_proc_file(pid_t other_pid, char const * const sn,
int flags) {
char fname[1024];
snprintf(fname, 1023, "/proc/%d/%s", other_pid, sn);
// Open file for reading and writing
int const fd = open(fname, flags );
if(fd==-1) {
perror("Open file");
exit(1);
}
return fd;
}
void get_heap(int fd_maps, size_t * heap_start, size_t * heap_end) {
char buf[65536];
ssize_t const r = read(fd_maps, buf, 65535);
if(r==-1) {
perror("Reading maps file");
exit(1);
}
buf[r] = '\0';
char * const heap = strstr(buf, "[heap]");
if(heap==NULL) {
printf("[heap] not found in maps file");
exit(1);
}
// Look backward to the latest newline
char const * hl_start;
for(hl_start = heap; hl_start > buf && *hl_start != '\n';
--hl_start) {}
// skip \n
++hl_start;
// Convert to beginnig and end address
char * lhe;
*heap_start = strtol(hl_start, &lhe, 16);
++lhe;
*heap_end = strtol(lhe, &lhe, 16);
}
int main(int argc, char *argv[]) {
if(argc!=2) {
printf("Usage: memwriter <pid>\n");
return 1;
}
pid_t const other_pid = atoi(argv[1]);
int fd_mem = open_proc_file(other_pid, "mem", O_RDWR);
int fd_maps = open_proc_file(other_pid, "maps", O_RDONLY);
size_t other_mem_start;
size_t other_mem_end;
get_heap(fd_maps, &other_mem_start, &other_mem_end);
ptrace(PTRACE_ATTACH, other_pid, NULL, NULL);
waitpid(other_pid, NULL, 0);
if( lseek(fd_mem, other_mem_start, SEEK_SET) == -1 ) {
perror("lseek");
return 1;
}
char buf[512];
do {
ssize_t const r = read(fd_mem, buf, 512);
if(r!=512) {
perror("read?");
break;
}
// Check for pattern
int pat_found = 1;
for(int i = 0; i < 512; ++i) {
if( buf[i] != '\xAA' )
pat_found = 0;
break;
}
if( ! pat_found ) continue;
// Write about one k of strings
char const * const wbuf = "REMOTE DATA - ";
for(int i = 0; i < 70; ++i) {
ssize_t const w = write(fd_mem, wbuf, strlen(wbuf));
if( w == -1) {
perror("Write");
return 1;
}
}
// Append a \0
write(fd_mem, "\0", 1);
break;
} while(1);
ptrace(PTRACE_DETACH, other_pid, NULL, NULL);
close(fd_mem);
close(fd_maps);
return 0;
}
Example output
$ ./memholder
PID: 2621
Some local data.
Some local data.
MOTE DATA - REMOTE DA...
Other interpretation
There is also another interpretation of your question (when reading the headline and not the question), that you want to replace the 'executable' from one process with another one. That can be easily handled by exec() (and friends):
From man exec:
The exec() family of functions replaces the current process image with a new process image.

In Windows, the methods used for this are named ReadProcessMemory / WriteProcessMemory, you will, however, need administrative rights for this. The same is for linux, as I've said in my comment, no sane system would allow user process to modify non-owned memory.
For linux, the only function is ptrace. You will need to be administrator.
http://cboard.cprogramming.com/cplusplus-programming/92093-readprocessmemory-writeprocessmemory-linux-equivalent.html contains more detailed discussion.
Can you imagine the consequences of allowing process to modify other process memory, without being administrator?

Related

wait() hangs when CLONE_THREAD

I am tracing some processes and their children using ptrace. I am trying to print specific system call (using Seccomp filter that notifies ptrace, see this blogpost).
In most cases my code (see below) is working fine. However, when I am tracing a java program (from the default-jre package), the latter clones using the CLONE_THREAD flag. And for some reason, my tracer hangs (I believe) because I can't receive signals from the cloned process. I think the reason is that (according to this discussion) the child process in fact becomes a child of the original process' parent, instead of becoming the original process' child.
I reproduced this issue by using a simple program that simply calls clone() with flags and perform actions. When I used the when I use CLONE_THREAD | CLONE_SIGHAND | CLONE_VM flags (as clone() documentation specifies they should come together since Linux 2.6.0), at least I am able to trace everything correctly until one of the two thread finishes.
I would like to trace both thread independently. Is it possible?
More importantly, I need to trace a Java program, and I cannot change it. Here a strace of the Java program clone call:
[...]
4665 clone(child_stack=0x7fb166e95fb0, flags=CLONE_VM|CLONE_FS|CLONE_FILES|CLONE_SIGHAND|CLONE_THREAD|CLONE_SYSVSEM|CLONE_SETTLS|CLONE_PARENT_SETTID|CLONE_CHILD_CLEARTID, parent_tid=[4666], tls=0x7fb166e96700, child_tidptr=0x7fb166e969d0) = 4666
[...]
So Java seems to respect the rules. I wanted to experiment to understand: I ruled out any flags unrelated to thread (i.e., `CLONE_FS | CLONE_FILES | CLONE_SYSVSEM).
Here are the results of running my test program with different combination of flags (I know, I am really desperate):
CLONE_VM|CLONE_SIGHAND|CLONE_THREAD|CLONE_SETTLS: only gets trace from parent
CLONE_VM|CLONE_SIGHAND|CLONE_THREAD|CLONE_PARENT_SETTID: inconsistent; gets trace from both until the parent finishes
CLONE_VM|CLONE_SIGHAND|CLONE_THREAD|CLONE_CHILD_CLEARTID: inconsistent; gets trace from both until the child finishes
CLONE_VM|CLONE_SIGHAND|CLONE_THREAD|CLONE_SETTLS|CLONE_PARENT_SETTID: only gets trace from parent
CLONE_VM|CLONE_SIGHAND|CLONE_THREAD|CLONE_SETTLS|CLONE_CHILD_CLEARTID: only gets trace from parent
CLONE_VM|CLONE_SIGHAND|CLONE_THREAD|CLONE_PARENT_SETTID|CLONE_SETTLS: only gets trace from parent
CLONE_VM|CLONE_SIGHAND|CLONE_THREAD|CLONE_PARENT_SETTID|CLONE_CHILD_CLEARTID: inconsistent; gets trace from both until the child finishes
CLONE_VM|CLONE_SIGHAND|CLONE_THREAD|CLONE_CHILD_CLEARTID|CLONE_SETTLS: only gets trace from parent
CLONE_VM|CLONE_SIGHAND|CLONE_THREAD|CLONE_CHILD_CLEARTID|CLONE_PARENT_SETTID: inconsistent; gets trace from both until the child finishes
CLONE_VM|CLONE_SIGHAND|CLONE_THREAD|CLONE_SETTLS|CLONE_PARENT_SETTID|CLONE_CHILD_CLEARTID:
only gets trace from parent
So at least I get the same behaviour from my program and the Java program: it does not work.
How can I make it work? For instance, how does strace successfully traces any kind of clone? I tried to dig into its code but I can't find how they are doing it.
Any help might appreciated!
Best regards,
The tracer code (compile with g++ tracer.cpp -o tracer -g -lseccomp -lexplain):
#include <stdio.h>
#include <string.h>
#include <stdlib.h>
#include <errno.h>
#include <stddef.h>
#include <sys/ptrace.h>
#include <sys/reg.h>
#include <signal.h>
#include <sys/types.h>
#include <sys/wait.h>
#include <unistd.h>
#include <sys/user.h>
#include <sys/prctl.h>
#include <fcntl.h>
#include <linux/limits.h>
#include <linux/filter.h>
#include <linux/seccomp.h>
#include <linux/unistd.h>
#include <libexplain/waitpid.h>
#include <tuple>
#include <vector>
#define DEFAULT_SIZE 1000
#define MAX_SIZE 1000
int process_signals();
int inspect(pid_t);
void read_string_into_buff(const pid_t, unsigned long long, char *, unsigned int);
int main(int argc, char **argv){
pid_t pid;
int status;
if (argc < 2) {
fprintf(stderr, "Usage: %s <prog> <arg1> ... <argN>\n", argv[0]);
return 1;
}
if ((pid = fork()) == 0) {
/* If execve syscall, trace */
struct sock_filter filter[] = {
BPF_STMT(BPF_LD+BPF_W+BPF_ABS, offsetof(struct seccomp_data, nr)),
BPF_JUMP(BPF_JMP+BPF_JEQ+BPF_K, __NR_getpid, 0, 1),
BPF_STMT(BPF_RET+BPF_K, SECCOMP_RET_TRACE),
BPF_STMT(BPF_RET+BPF_K, SECCOMP_RET_ALLOW),
};
struct sock_fprog prog = {
.len = (unsigned short) (sizeof(filter)/sizeof(filter[0])),
.filter = filter,
};
ptrace(PTRACE_TRACEME, 0, 0, 0);
if (prctl(PR_SET_NO_NEW_PRIVS, 1, 0, 0, 0) == -1) {
perror("prctl(PR_SET_NO_NEW_PRIVS)");
return 1;
}
if (prctl(PR_SET_SECCOMP, SECCOMP_MODE_FILTER, &prog) == -1) {
perror("when setting seccomp filter");
return 1;
}
kill(getpid(), SIGSTOP);
return execvp(argv[1], argv + 1);
} else {
waitpid(pid, &status, 0);
ptrace(PTRACE_SETOPTIONS, pid, 0, PTRACE_O_TRACESECCOMP | PTRACE_O_TRACEFORK | PTRACE_O_TRACECLONE | PTRACE_O_TRACEVFORK );
ptrace(PTRACE_CONT, pid, 0, 0);
process_signals();
return 0;
}
}
int process_signals(){
int status;
while (1){
pid_t child_pid;
// When child status changes
if ((child_pid = waitpid(-1, &status, 0)) < 0){
fprintf(stderr, "%s\n", explain_waitpid(child_pid, &status, 0));
exit(EXIT_FAILURE);
}
//printf("Sigtrap received\n");
// Checking if it is thanks to seccomp
if (status >> 8 == (SIGTRAP | (PTRACE_EVENT_SECCOMP << 8))){
// Perform argument inspection with ptrace
int syscall = inspect(child_pid);
}
// Resume no matter what
ptrace(PTRACE_CONT, child_pid, 0, 0);
}
}
int inspect(pid_t pid){
printf("From PID: %d\n", pid);
struct user_regs_struct regs;
ptrace(PTRACE_GETREGS, pid, 0, &regs);
// Get syscall number
int syscall = regs.orig_rax;
printf("------\nCaught syscall: %d\n", syscall);
if (syscall == __NR_getpid){
printf("Getpid detected\n");
}
return syscall;
}
void read_string_into_buff(const pid_t pid, unsigned long long addr, char * buff, unsigned int max_len){
/* Are we aligned on the "start" front? */
unsigned int offset=((unsigned long)addr)%sizeof(long);
addr-=offset;
unsigned int i=0;
int done=0;
int word_offset=0;
while( !done ) {
unsigned long word=ptrace( PTRACE_PEEKDATA, pid, addr+(word_offset++)*sizeof(long), 0 );
// While loop to stop at the first '\0' char indicating end of string
while( !done && offset<sizeof(long) && i<max_len ) {
buff[i]=((char *)&word)[offset]; /* Endianity neutral copy */
done=buff[i]=='\0';
++i;
++offset;
}
offset=0;
done=done || i>=max_len;
}
}
The sample program (compile with gcc sample.c -o sample):
#define _GNU_SOURCE
#include <stdio.h>
#include <sched.h>
#include <stdlib.h>
#include <sys/wait.h>
#include <unistd.h>
#include <signal.h>
#define FLAGS CLONE_VM|CLONE_SIGHAND|CLONE_THREAD|CLONE_SETTLS|CLONE_PARENT_SETTID|CLONE_CHILD_CLEARTID
int fn(void *arg)
{
printf("\nINFO: This code is running under child process.\n");
int i = 0;
int n = atoi(arg);
for ( i = 1 ; i <= 10 ; i++ )
printf("[%d] %d * %d = %d\n", getpid(), n, i, (n*i));
printf("\n");
return 0;
}
void main(int argc, char *argv[])
{
printf("[%d] Hello, World!\n", getpid());
void *pchild_stack = malloc(1024 * 1024);
if ( pchild_stack == NULL ) {
printf("ERROR: Unable to allocate memory.\n");
exit(EXIT_FAILURE);
}
int pid = clone(fn, pchild_stack + (1024 * 1024), FLAGS, argv[1]);
if ( pid < 0 ) {
printf("ERROR: Unable to create the child process.\n");
exit(EXIT_FAILURE);
}
fn(argv[1]);
wait(NULL);
free(pchild_stack);
printf("INFO: Child process terminated.\n");
}
You can test what you want by running ./tracer ./sample. You can also test the original test case ./tracer java and observe that both the tracer and java hangs.
ANSWER:
As pointed it out in the comment, I had issues in that example that were preventing me from handling signals from the child.
In my original code (not listed here because too complex), I was only attaching ptrace AFTER the processes started... and I was only attaching to PID listed by pstree. My mistake was that I omitted the threads (and java is one program that does create threads), explaining why I had issue tracing java only.
I modified the code to attach to all the children process and thread (ps -L -g <Main_PID> -o tid=) and everything works again.
Your sample program has a bug: it may free the second thread’s stack before that thread exits, causing a SEGV. And your tracer just doesn’t handle signals well.
If the traced program gets a signal, your tracer intercepts it, not passing it down to the program. When it continues the program, it continues from the very same operation that caused SEGV, so it gets SEGV again. Ad infinitum. Both the tracer and the tracee appear to hang but in fact, they are in an infinite loop.
Rewriting the continuation like the following seems to work:
if (status >> 8 == (SIGTRAP | (PTRACE_EVENT_SECCOMP << 8))){
// Perform argument inspection with ptrace
int syscall = inspect(child_pid);
ptrace(PTRACE_CONT, child_pid, 0, 0);
} else if (WIFSTOPPED(status)) {
ptrace(PTRACE_CONT, child_pid, 0, WSTOPSIG(status));
} else {
ptrace(PTRACE_CONT, child_pid, 0, 0);
}
Not sure of Java but it seems to get SEGVs in regular operation...

FIFO: One process never reads from pipe

I'm following THIS TutorialsPoint guide to Linux Piping, and I specifically need to use FIFOs.
However, the code doesn't work at all for the server side.
The server file either hangs indefinitely or it reads nothing, while the client instead writes on the FIFO and immediately reads it has just written.
Here's the full code for both files in case you don't want to go through TutorialsPoint:
fifoserver_twoway.cpp
#include <stdio.h>
#include <sys/stat.h>
#include <sys/types.h>
#include <fcntl.h>
#include <unistd.h>
#include <string.h>
#define FIFO_FILE "/tmp/fifo_twoway"
void reverse_string(char *);
int main() {
int fd;
char readbuf[80];
char end[10];
int to_end;
int read_bytes;
/* Create the FIFO if it does not exist */
mkfifo(FIFO_FILE, S_IFIFO|0640);
strcpy(end, "end");
fd = open(FIFO_FILE, O_RDWR);
while(1) {
read_bytes = read(fd, readbuf, sizeof(readbuf));
readbuf[read_bytes] = '\0';
printf("FIFOSERVER: Received string: \"%s\" and length is %d\n", readbuf, (int)strlen(readbuf));
to_end = strcmp(readbuf, end);
if (to_end == 0) {
close(fd);
break;
}
reverse_string(readbuf);
printf("FIFOSERVER: Sending Reversed String: \"%s\" and length is %d\n", readbuf, (int) strlen(readbuf));
write(fd, readbuf, strlen(readbuf));
/*
sleep - This is to make sure other process reads this, otherwise this
process would retrieve the message
*/
sleep(2);
}
return 0;
}
void reverse_string(char *str) {
int last, limit, first;
char temp;
last = strlen(str) - 1;
limit = last/2;
first = 0;
while (first < last) {
temp = str[first];
str[first] = str[last];
str[last] = temp;
first++;
last--;
}
return;
}
fifoclient_twoway.cpp
#include <stdio.h>
#include <sys/stat.h>
#include <sys/types.h>
#include <fcntl.h>
#include <unistd.h>
#include <string.h>
#define FIFO_FILE "/tmp/fifo_twoway"
int main() {
int fd;
int end_process;
int stringlen;
int read_bytes;
char readbuf[80];
char end_str[5];
printf("FIFO_CLIENT: Send messages, infinitely, to end enter \"end\"\n");
fd = open(FIFO_FILE, O_CREAT|O_RDWR);
strcpy(end_str, "end");
while (1) {
printf("Enter string: ");
fgets(readbuf, sizeof(readbuf), stdin);
stringlen = strlen(readbuf);
readbuf[stringlen - 1] = '\0';
end_process = strcmp(readbuf, end_str);
//printf("end_process is %d\n", end_process);
if (end_process != 0) {
write(fd, readbuf, strlen(readbuf));
printf("FIFOCLIENT: Sent string: \"%s\" and string length is %d\n", readbuf, (int)strlen(readbuf));
read_bytes = read(fd, readbuf, sizeof(readbuf));
readbuf[read_bytes] = '\0';
printf("FIFOCLIENT: Received string: \"%s\" and length is %d\n", readbuf, (int)strlen(readbuf));
} else {
write(fd, readbuf, strlen(readbuf));
printf("FIFOCLIENT: Sent string: \"%s\" and string length is %d\n", readbuf, (int)strlen(readbuf));
close(fd);
break;
}
}
return 0;
}
When I run both processes, this is what I get:
./fifoserver_twoway
FIFOSERVER: Received string: "" and length is 0
FIFOSERVER: Sending Reversed String: "" and length is 0
FIFOSERVER: Received string: "" and length is 0
FIFOSERVER: Sending Reversed String: "" and length is 0
./fifoclient_twoway
FIFOCLIENT: Sent string: "ciao" and string length is 4
FIFOCLIENT: Received string: "ciao" and length is 4
Enter string: why won't you reverse?
FIFOCLIENT: Sent string: "why won't you reverse?" and string length is 29
FIFOCLIENT: Received string: "why won't you reverse?" and length is 29
It's also worth noting that before starting to write this question, the server behaviour was completely different: instead of receiving nothing and printing like you see here, it would hang indefinitely after the "read" (and I haven't changed the code one bit, except for changing the FIFO_FILE path)
You let the server sleep after writing – but not the client. That way, the client still might read its own output back before the server can fetch it. So at very least you should add a sleep after both writes, letting the server sleep a bit longer to make sure the client wakes up first to read the servers output.
Accessing the same end of unnamed pipes (created via pipe functions) concurrently is undefined behaviour. While not sure for named pipes, I'd assume pretty much the same there as well. Synchronising concurrent access to such ends via simple delays (sleep, usleep) might perhaps do the trick, but it is a pretty unsafe method.
I'd rather recommend two separate pipes instead (as Tony Tannous proposed already), one for each direction (open the respective ends RDONLY or WRONLY as needed), then you get full duplex communication instead of half duplex and you don't need further synchronisation either (delays in most simple variant):
// server
int fd_cs = open(FIFO_FILE_CS, O_RDONLY);
int fd_sc = open(FIFO_FILE_SC, O_WRONLY);
read(fd_cs, ...);
write(fd_sc, ...);
// client
int fd_cs = open(FIFO_FILE_CS, O_WRONLY);
int fd_sc = open(FIFO_FILE_SC, O_RDONLY);
write(fd_cs, ...);
read(fd_sc, ...);

Have a very long buffer but only use the last 1GB bytes of data.

Need to write an application in C/C++ on Linux that receives a stream of bytes from a socket and process them. The total bytes could be close to 1TB. If I have unlimited amount memory, I will just put it all in the memory, so my application can easily process data. It's much easy to do many things on flat memory space, such as memmem(), memcmp() ... On a circular buffer, the application has to be extra smart to be aware of the circular buffer.
I have about 8G of memory, but luckily due to locality, my application never needs to go back by more than 1GB from the latest data it received. Is there a way to have a 1TB buffer, with only the latest 1GB data mapped to physical memory? If so, how to do it?
Any ideas? Thanks.
Here's an example. It sets up a full terabyte mapping, but initially inaccessible (PROT_NONE). You, the programmer, maintain a window that can only extend and move upwards in memory. The example program uses a one and a half gigabyte window, advancing it in steps of 1,023,739,137 bytes (the mapping_use() makes sure the available pages cover at least the desired region), and does actually modify every page in every window, just to be sure.
#define _GNU_SOURCE
#define _POSIX_C_SOURCE 200809L
#include <stdlib.h>
#include <unistd.h>
#include <sys/mman.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>
#include <errno.h>
#include <string.h>
#include <stdio.h>
typedef struct mapping mapping;
struct mapping {
unsigned char *head; /* Start of currently accessible region */
unsigned char *tail; /* End of currently accessible region */
unsigned char *ends; /* End of region */
size_t page; /* Page size of this mapping */
};
/* Discard mapping.
*/
void mapping_free(mapping *const m)
{
if (m && m->ends > m->head) {
munmap(m->head, (size_t)(m->ends - m->head));
m->head = NULL;
m->tail = NULL;
m->ends = NULL;
m->page = 0;
}
}
/* Move the accessible part up in memory, to [from..to).
*/
int mapping_use(mapping *const m, void *const from, void *const to)
{
if (m && m->ends > m->head) {
unsigned char *const head = ((unsigned char *)from <= m->head) ? m->head :
((unsigned char *)from >= m->ends) ? m->ends :
m->head + m->page * (size_t)(((size_t)((unsigned char *)from - m->head)) / m->page);
unsigned char *const tail = ((unsigned char *)to <= head) ? head :
((unsigned char *)to >= m->ends) ? m->ends :
m->head + m->page * (size_t)(((size_t)((unsigned char *)to - m->head) + m->page - 1) / m->page);
if (head > m->head) {
munmap(m->head, (size_t)(head - m->head));
m->head = head;
}
if (tail > m->tail) {
#ifdef USE_MPROTECT
mprotect(m->tail, (size_t)(tail - m->tail), PROT_READ | PROT_WRITE);
#else
void *result;
do {
result = mmap(m->tail, (size_t)(tail - m->tail), PROT_READ | PROT_WRITE, MAP_ANONYMOUS | MAP_FIXED | MAP_PRIVATE | MAP_NORESERVE, -1, (off_t)0);
} while (result == MAP_FAILED && errno == EINTR);
if (result == MAP_FAILED)
return errno = ENOMEM;
#endif
m->tail = tail;
}
return 0;
}
return errno = EINVAL;
}
/* Initialize a mapping.
*/
int mapping_create(mapping *const m, const size_t size)
{
void *base;
size_t page, truesize;
if (!m || size < (size_t)1)
return errno = EINVAL;
m->head = NULL;
m->tail = NULL;
m->ends = NULL;
m->page = 0;
/* Obtain default page size. */
{
long value = sysconf(_SC_PAGESIZE);
page = (size_t)value;
if (value < 1L || (long)page != value)
return errno = ENOTSUP;
}
/* Round size up to next multiple of page. */
if (size % page)
truesize = size + page - (size % page);
else
truesize = size;
/* Create mapping. */
do {
errno = ENOTSUP;
base = mmap(NULL, truesize, PROT_NONE, MAP_ANONYMOUS | MAP_PRIVATE | MAP_NORESERVE, -1, (off_t)0);
} while (base == MAP_FAILED && errno == EINTR);
if (base == MAP_FAILED)
return errno;
/* Success. */
m->head = base;
m->tail = base;
m->ends = (unsigned char *)base + truesize;
m->page = page;
errno = 0;
return 0;
}
static void memtouch(void *const ptr, const size_t size)
{
if (ptr && size > 0) {
unsigned char *mem = (unsigned char *)ptr;
const size_t step = 2048;
size_t n = size / (size_t)step - 1;
mem[0]++;
mem[size-1]++;
while (n-->0) {
mem += step;
mem[0]++;
}
}
}
int main(void)
{
const size_t size = (size_t)1024 * (size_t)1024 * (size_t)1024 * (size_t)1024;
const size_t need = (size_t)1500000000UL;
const size_t step = (size_t)1023739137UL;
unsigned char *base;
mapping map;
size_t i;
if (mapping_create(&map, size)) {
fprintf(stderr, "Cannot create a %zu-byte mapping: %m.\n", size);
return EXIT_FAILURE;
}
printf("Have a %zu-byte mapping at %p to %p.\n", size, (void *)map.head, (void *)map.ends);
fflush(stdout);
base = map.head;
for (i = 0; i <= size - need; i += step) {
printf("Requesting %p to %p .. ", (void *)(base + i), (void *)(base + i + need));
fflush(stdout);
if (mapping_use(&map, base + i, base + i + need)) {
printf("Failed (%m).\n");
fflush(stdout);
return EXIT_FAILURE;
}
printf("received %p to %p.\n", (void *)map.head, (void *)map.tail);
fflush(stdout);
memtouch(base + i, need);
}
mapping_free(&map);
return EXIT_SUCCESS;
}
The approach is twofold. First, an inaccessible (PROT_NONE) mapping is created to reserve the necessary virtual contiguous address space. If we omit this step, it would make it possible for a malloc() call or similar to acquire pages within this range, which would defeat the entire purpose; a single terabyte-long mapping.
Second, when the accessible window extends into the region, either mprotect() (if USE_MPROTECT is defined), or mmap() is used to make the required pages accessible. Pages no longer needed are completely unmapped.
Compile and run using
gcc -Wall -Wextra -std=c99 example.c -o example
time ./example
or, to use mmap() only once and mprotect() to move the window,
gcc -DUSE_MPROTECT=1 -Wall -Wextra -std=c99 example.c -o example
time ./example
Note that you probably don't want to run the test if you don't have at least 4GB of physical RAM.
On this particular machine (i5-4200U laptop with 4GB of RAM, 3.13.0-62-generic kernel on Ubuntu x86_64), quick testing didn't show any kind of performance difference between mprotect() and mmap(), in execution speed or resident set size.
If anyone bothers to compile and run the above, and finds that one of them has a repeatable benefit/drawback (resident set size or time used), I'd very much like to know about it. Please also define your kernel and CPU used.
I'm not sure which details I should expand on, since this is pretty straightforward, really, and the Linux man pages project man 2 mmap and man 2 mprotect pages are quite descriptive. If you have any questions on this approach or program, I'd be happy to try and elaborate.

inotify notifies of a new file wrongly multiple times

Using inotify to monitor a directory for any new file created in the directory by adding a watch on the directory by
fd = inotify_init();
wd = inotify_add_watch(fd, "filename_with_path", IN_CLOSE_WRITE);
inotify_add_watch(fd, directory_name, IN_CLOSE_WRITE);
const int event_size = sizeof(struct inotify_event);
const int buf_len = 1024 * (event_size + FILENAME_MAX);
while(true) {
char buf[buf_len];
int no_of_events, count = 0;
no_of_events = read(fd, buf, buf_len);
while(count < no_of_events) {
struct inotify_event *event = (struct inotify_event *) &buf[count];
if (event->len) {
if (event->mask & IN_CLOSE_WRITE) {
if (!(event->mask & IN_ISDIR)) {
//It's here multiple times
}
}
}
count += event_size + event->len;
}
When I scp a file to the directory, this loops infinitely. What is the problem with this code ? It shows the same event name and event mask too. So , it shows that the event for the same, infinite times.
There are no break statements. If I find an event, I just print it and carry on waiting for another event on read(), which should be a blocking call. Instead, it starts looping infinitely. This means, read doesn't block it but returns the same value for one file infinitely.
This entire operation runs on a separate boost::thread.
EDIT:
Sorry all. The error I was getting was not because of the inotify but because of sqlite which was tricky to detect at first. I think I jumped the gun here. With further investigation, I did find that the inotify works perfectly well. But the error actually came from the sqlite command : ATTACH
That command was not a ready-only command as it was supposed to. It was writing some meta data to the file. So inotify gets notification again and again. Since they were happening so fast, it screwed up the application.I finally had to breakup the code to understand why.
Thanks everyone.
I don't see anything wrong with your code...I'm running basically the same thing and it's working fine. I'm wondering if there's a problem with the test, or some part of the code that's omitted. If you don't mind, let's see if we can remove any ambiguity.
Can you try this out (I know it's almost the same thing, but just humor me) and let me know the results of the exact test?
1) Put the following code into test.c
#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>
#include <string.h>
#include <stdio.h>
#include <errno.h>
#include <sys/inotify.h>
int main (int argc, char *argv[])
{
char target[FILENAME_MAX];
int result;
int fd;
int wd; /* watch descriptor */
const int event_size = sizeof(struct inotify_event);
const int buf_len = 1024 * (event_size + FILENAME_MAX);
strcpy (target, ".");
fd = inotify_init();
if (fd < 0) {
printf ("Error: %s\n", strerror(errno));
return 1;
}
wd = inotify_add_watch (fd, target, IN_CLOSE_WRITE);
if (wd < 0) {
printf ("Error: %s\n", strerror(errno));
return 1;
}
while (1) {
char buff[buf_len];
int no_of_events, count = 0;
no_of_events = read (fd, buff, buf_len);
while (count < no_of_events) {
struct inotify_event *event = (struct inotify_event *)&buff[count];
if (event->len){
if (event->mask & IN_CLOSE_WRITE)
if(!(event->mask & IN_ISDIR)){
printf("%s opened for writing was closed\n", target);
fflush(stdout);
}
}
count += event_size + event->len;
}
}
return 0;
}
2) Compile it with gcc:
gcc test.c
3) kick it off in one window:
./a.out
4) in a second window from the same directory try this:
echo "hi" > blah.txt
Let me know if that works correctly to show output every time the file is written to and does not loop as your code does. If so, there's something important your omiting from your code. If not, then there's some difference in the systems.
Sorry for putting this in the "answer" section, but too much for a comment.
My guess is that read is returning -1 and since you dont ever try to fix the error, you get another error on the next call to read which also returns -1.

linux - get pid of process

How can I get the PID of a service called abc using C++ on Linux without using a system call? I would appreciate any examples that you care to offer.
Since use of sysctl has been discouraged for ages now, the recommended way of doing this is by examining each of the process entries in /proc and reading the comm file in each folder. If, for your example, the contents of that file are abc\n, that's the process you're looking for.
I don't really speak C++, but here's a possible solution in POSIX C89:
#include <glob.h>
#include <stdlib.h>
#include <string.h>
#include <sys/types.h>
pid_t find_pid(const char *process_name)
{
pid_t pid = -1;
glob_t pglob;
char *procname, *readbuf;
int buflen = strlen(process_name) + 2;
unsigned i;
/* Get a list of all comm files. man 5 proc */
if (glob("/proc/*/comm", 0, NULL, &pglob) != 0)
return pid;
/* The comm files include trailing newlines, so... */
procname = malloc(buflen);
strcpy(procname, process_name);
procname[buflen - 2] = '\n';
procname[buflen - 1] = 0;
/* readbuff will hold the contents of the comm files. */
readbuf = malloc(buflen);
for (i = 0; i < pglob.gl_pathc; ++i) {
FILE *comm;
char *ret;
/* Read the contents of the file. */
if ((comm = fopen(pglob.gl_pathv[i], "r")) == NULL)
continue;
ret = fgets(readbuf, buflen, comm);
fclose(comm);
if (ret == NULL)
continue;
/*
If comm matches our process name, extract the process ID from the
path, convert it to a pid_t, and return it.
*/
if (strcmp(readbuf, procname) == 0) {
pid = (pid_t)atoi(pglob.gl_pathv[i] + strlen("/proc/"));
break;
}
}
/* Clean up. */
free(procname);
free(readbuf);
globfree(&pglob);
return pid;
}
Caveat: if there are multiple running processes with the name you're looking for, this code will only return one. If you're going to change that, be aware that with the naive glob written, you'll also examine /proc/self/comm, which could potentially lead to a duplicate entry.
If there are multiple processes with the same name, there isn't really a way to ensure you got the right one. Many daemons have the ability to save their pids to a file for this reason; check your documentation.
Google has this covered :)
http://programming-in-linux.blogspot.com/2008/03/get-process-id-by-name-in-c.html
Although it does use sysctl, which is a system call!
It's C but should work just as well in C++