Related
Context: I'm using memory mapped file in my code created using ACE_Mem_Map. It is observed that the memory mapped file is consuming more disk space than expected.
Scenario:
I have a structure containing a char array of 15KB. I have created a memory map file for array of this struct with file size ~2GB.
If I try to access few bytes of the char array(say 256), then, file size consumed is shown as 521 MB but actual disk usage shown by filesystem(using df -h) is more than 3GB.
If I access all bytes of the memory, then both file size and disk usage is shown as 2 GB.
Environment:
OS: Oracle Linux 7.3
Kernel version: 3.10.0/4.1.12
Code:
#include<ace/Mem_Map.h>
#include <stdio.h>
#define TEST_BUFF_SIZE 15*1024
typedef struct _test_struct_ {
char test[TEST_BUFF_SIZE];
_test_struct_() {
reset();
}
void reset() {
/* Issue replicating */
memset(test, '\0', 256);
/* Issue not replicating */
memset(test, '\0', TEST_BUFF_SIZE);
}
}TestStruct_t;
int main(int argc, char *argv[]) {
if(3 != argc) {
printf("Usage: %s <num of blocks> <filename>\n",
argv[0]);
return -1;
}
ACE_Mem_Map map_buf_;
size_t num_of_blocks = strtoull(argv[1], NULL, 10);
size_t MAX_SIZE = num_of_blocks*sizeof(TestStruct_t);
char* mmap_file_name = argv[2];
printf("num_of_blocks[%llu], sizeof(TestStruct_t)[%llu], MAX_SIZE[%llu], mmap_file_name[%s]\n",
num_of_blocks,
sizeof(TestStruct_t),
MAX_SIZE,
mmap_file_name);
TestStruct_t *base_addr_;
ACE_HANDLE fp_ = ACE_OS::open(mmap_file_name,O_RDWR|O_CREAT,
ACE_DEFAULT_OPEN_PERMS,0);
if (fp_ == ACE_INVALID_HANDLE)
{
printf("Error opening file\n");
return -1;
}
map_buf_.map(fp_,MAX_SIZE,PROT_WRITE,MAP_SHARED);
base_addr_ = (TestStruct_t*)map_buf_.addr();
if (base_addr_ == MAP_FAILED)
{
printf("Map init failure\n");
ACE_OS::close(fp_);
return -1;
}
printf("map_buf_ size[%llu]\n",
map_buf_.size());
for(size_t i = 0; i < num_of_blocks; i++) {
base_addr_[i].reset();
}
return 0;
}
Can anyone explain why is scenario 1 happening??
Note: In scenario 1, if I make a copy of generated mmap file and then delete that copy, then the additional 2.5GB disk space gets freed. Don't know the reason
I 'upgraded' your program to nearly C and minus whatever ACE is and got this:
$ ./a.out 32 fred
num_of_blocks[32], sizeof(TestStruct_t)[15360], MAX_SIZE[491520], mmap_file_name[fred]
Bus error: 10
Which is pretty much expected. Mmap does not extend the size of the mapped file, so it generates an address error when you try to reference an unfilled part.
So, the answer is that whatever ACE.map does, it likely invokes something like ftruncate(2) to extend the file to the size you give as a parameter. #John Bollinger hints at this by asking how are you measuring that: ls or du. You should use the latter.
Anyway, almost C version:
#include <sys/mman.h>
#include <sys/types.h>
#include <stdlib.h>
#include <string.h>
#include <unistd.h>
#include <fcntl.h>
#include <stdio.h>
#define TEST_BUFF_SIZE 15*1024
typedef struct _test_struct_ {
char test[TEST_BUFF_SIZE];
_test_struct_() {
reset();
}
void reset() {
/* Issue replicating */
memset(test, '\0', 256);
/* Issue not replicating */
memset(test, '\0', TEST_BUFF_SIZE);
}
}TestStruct_t;
int main(int argc, char *argv[]) {
if(argc < 3) {
printf("Usage: %s <num of blocks> <filename>\n",
argv[0]);
return 1;
}
void *buf;
size_t num_of_blocks = strtoull(argv[1], NULL, 10);
size_t MAX_SIZE = num_of_blocks*sizeof(TestStruct_t);
char* mmap_file_name = argv[2];
printf("num_of_blocks[%zu], sizeof(TestStruct_t)[%zu], MAX_SIZE[%zu], mmap_file_name[%s]\n",
num_of_blocks,
sizeof(TestStruct_t),
MAX_SIZE,
mmap_file_name);
int fp = open(mmap_file_name,O_RDWR|O_CREAT,0666);
if (fp == -1)
{
perror("Error opening file");
return 1;
}
/*SOMETHING CLEVER*/
switch (argc) {
case 3:
break;
case 4:
if (ftruncate(fp, MAX_SIZE) != 0) {
perror("ftruncate");
return 1;
}
break;
case 5:
if (lseek(fp, MAX_SIZE-1, SEEK_SET) != MAX_SIZE-1 ||
write(fp, "", 1) != 1) {
perror("seek,write");
return 1;
}
}
void *b = mmap(0, MAX_SIZE, PROT_READ|PROT_WRITE, MAP_SHARED, fp, 0);
if (b == MAP_FAILED)
{
perror("Map init failure");
return 1;
}
TestStruct_t *base_addr = (TestStruct_t *)b;
for(size_t i = 0; i < num_of_blocks; i++) {
base_addr[i].reset();
}
return 0;
}
The SOMETHING CLEVER bit allows you to either work with an empty file (argc == 3), grow it with ftruncate (argc == 4), or grow it with lseek && write (argc == 5).
On UNIX-y systems, ftruncate may or may not reserve space for your file; a lengthened file without reserved space is called sparce. Almost universally, the lseek && write will create a sparse file, unless your system doesn't support that.
The sparce file will allocate actual disk blocks as you write to it, however, if it fails, it will deliver a signal whereas the pre-allocated one will not.
Your loop at the bottom walks the whole extent, so the file will always be grown; reduce that loop and you can see if the options make a difference on your system.
I am trying to create a minimal code to use pipe/fork/execlp.
So far so good, I am using execlp with bash -c, so if I do.
echo asd |./a.out cat
> asd
So it is working as expected.
But if I try to use anything that needs a TTY, it does not work.
Like ./a.out vim, I get "Vim: Warning: Input is not from a terminal"
And the vim that was open does not works as expected.
I tried to find on the internet an example on how to open a TTY, the only one that I found was:
http://www.danlj.org/lad/src/minopen.c
My Code, so far is:
#include <iostream>
#include <cstdio>
#include <string.h>
#include <cstdlib>
#include <unistd.h>
#include <sys/wait.h>
typedef struct pCon{
int fout[2];
int fin[2];
int fd[2];
int pid1, pid2;
} connectionManager;
std::string command = "";
/*
* Implementation
*/
void childFork(connectionManager *cm);
int main(int argc, char *argv[]) {
int size;
if(argc < 2) exit(1);
else command = argv[1];
connectionManager *cm = new connectionManager;
pipe(cm->fd);
if((cm->pid1 = fork()) == -1)exit(1);
if (cm->pid1 == 0)
{
const unsigned int RCVBUFSIZE = 2000;
char echoString[RCVBUFSIZE];
while((size = read(fileno(stdin),echoString,RCVBUFSIZE)) > 0)
write(cm->fd[1], echoString, size);
close(cm->fd[1]);
}
else
childFork(cm);
return 0;
}
void childFork(connectionManager *cm){
char *buffer = new char[2000];
int size;
close(cm->fd[1]);
dup2(cm->fd[0], 0);
close(cm->fd[0]);
pipe(cm->fout);
if((cm->pid2 = fork()) == -1)exit(1);
if (cm->pid2 == 0)
{
close(cm->fout[0]);
int returnCode = execlp("bash", "bash", "-c", command.c_str(), NULL);
if(returnCode!=0)
std::cerr << "Error starting the bash program" << std::endl;
}
else
{
close(cm->fout[1]);
while((size = read(cm->fout[0], buffer, 2000 )) > 0 )
write(fileno(stdout), buffer, size);
}
}
I tried to keep the minimal necessary code to make it work.
Is there any way to implement TTY on this code, I know that does not seems to be such trivial task.
Can someone help me with that?
I also tried to open the tty and dup it, but no luck so far.
Try to use pseudo terminal. You can use opentty. For your purpose you can use forkpty which combines pty with fork. I've created a small example for you. About the same as your program, just it works. I've kept it simple, so I don't handle the terminal control characters.
#include <pty.h>
#include <unistd.h>
#include <fcntl.h>
#include <termios.h>
#include <sys/select.h>
int main(int argc, char *argv[])
{
if (argc<1) return 1;
int master;
pid_t pid = forkpty(&master, NULL, NULL, NULL); // opentty + login_tty + fork
if (pid < 0) {
return 1; // fork with pseudo terminal failed
}
else if (pid == 0) { // child
char *args[] = { argv[1], argv[2], NULL }; // prg + 1 argument
execvp(argv[1], args); // run the program given in first param
}
else { // parent
struct termios tios;
tcgetattr(master, &tios);
tios.c_lflag &= ~(ECHO | ECHONL);
tcsetattr(master, TCSAFLUSH, &tios);
while(1) {
fd_set read_fd, write_fd, err_fd;
FD_ZERO(&read_fd);
FD_ZERO(&write_fd);
FD_ZERO(&err_fd);
FD_SET(master, &read_fd);
FD_SET(STDIN_FILENO, &read_fd);
select(master+1, &read_fd, &write_fd, &err_fd, NULL);
if (FD_ISSET(master, &read_fd))
{
char ch;
int c;
if (c=read(master, &ch, 1) != -1) // read from program
write(STDOUT_FILENO, &ch, c); // write to tty
else
break; // exit when end of communication channel with program
}
if (FD_ISSET(STDIN_FILENO, &read_fd))
{
char ch;
int c=read(STDIN_FILENO, &ch, 1); // read from tty
write(master, &ch, c); // write to program
}
}
}
return 0;
}
For compiling use -lutil .
While running a new tty device appears in /dev/pts .
vim accepts it as a terminal.
I want to try "userspace" driver(it isn't my code) on Ubuntu. And I have this code:
#include <stdio.h>
#include <string.h>
#include <usb.h>
#include <usbpp.h>
#define DEBUG_LEVEL 0
struct usb_bus *busses,*bus,*dbus;
int main(int argc, char *argv[])
{
struct usb_device *dev;
char *buf;
int n,x,r, ret;
char *ibus, *idev;
char string[64];
int found = 0;
usb_dev_handle *udev;
usb_set_debug(DEBUG_LEVEL);
usb_init(); //initilize the usb library
usb_find_busses();
usb_find_devices();
busses=usb_get_busses();
//use the command line arguments for Bus and Device
if (argc >= 3) {
ibus = argv[1];
idev = argv[2];
}
else { printf("Usage: usbtest <Bus> <Device> [<numBytes>])\n"); exit(1); }
printf("Bus = %s Device = %s\n", ibus, idev);
for (bus=busses; bus && (found == 0); bus=bus->next) { // busses loop
if (strcmp(bus->dirname, ibus) == 0) {
for (dev=bus->devices; dev; dev=dev->next) { // devices loop
if (strcmp(dev->filename, idev) == 0) { dbus=bus; found=1; break; }
}
}
}
if (found == 0) { printf("Unable to find the required device !\nexiting\n"); exit(1); }
printf("Found device\n");
printf("Now we are dealing with device from vendor ID : %d (%x) \n",dev->descriptor.idVendor,dev->descriptor.idVendor);
printf("Trying to open the device...\n");
if (udev=usb_open(dev)) printf("Device opened successfully.\n");
else { printf("Operation failed :-("); exit(1);}
buf=(char*)calloc(1,100);
if (usb_get_driver_np(udev,0,buf,100)) printf("Could not read the driver name :-( %s\n",buf);
else printf("Kernel Using Driver :\n");
// detach the driver from the kernel , seems to be just like rmmod?
if (usb_detach_kernel_driver_np(udev,0)) printf("Error detaching the device :-(\n");
else printf("Device detached successfully from the kernel.\n");
// reserving the device interface for our application, if another driver/software
//is using the device , it will return 'interface busy'
if (r=usb_claim_interface(udev,0)) printf("Interface Claimed !!\n");
printf("Interface Claim Status : %d\n",r);
printf("Device Protocol : %d\n",dev->descriptor.bDeviceProtocol);
printf("Report Length : %d\n",dev->descriptor.bLength);
printf("Decriptor Type : %d\n",dev->descriptor.bDescriptorType);
printf("End Points : %d\n",dev->config->interface->altsetting->bNumEndpoints);
printf("Interface Class : %d\n",dev->config->interface->altsetting->bInterfaceClass);
printf("Protocol : %d\n",dev->config->interface->altsetting->bInterfaceProtocol);
printf("Interface Number: %d\n",dev->config->interface->altsetting->bInterfaceNumber);
printf("Device Filename : %s\n",dev->filename);
printf("Bus Dir Name : %s\n",dbus->dirname);
usb_get_string_simple(udev,dev->descriptor.iManufacturer,string,sizeof(string));
printf("Device Manfucaturer : %s\n",string);
usb_get_string_simple(udev,dev->descriptor.iProduct,string,sizeof(string));
printf("Product Name : %s\n",string);
usb_get_string_simple(udev,dev->descriptor.iSerialNumber,string,sizeof(string));
printf("Device Serial Number: %s\n",string);
printf("End point addresses : 0x%x\n",dev->config->interface->altsetting->endpoint->bEndpointAddress);
int numBytes = 4;
int errCount = 0;
unsigned char endPoint = 0x81;
int i;
int tmp1 = 5;
int tmp2 = 5;
// reset usb device if command line numBytes = 0 and exit
if (argc == 4 && atoi(argv[3]) == 0) { usb_reset(udev); exit(0); }
// use command line value for numBytes if supplied
if (argc == 4) numBytes = atoi(argv[3]);
if (numBytes > 64) numBytes = 64;
while (1) {
for (x=0; x<numBytes; x++) string[x]=0;
// read numBytes bytes using interrupt_read,
r = usb_interrupt_read(udev, endPoint, string, numBytes, 0);
system("clear");
tmp1+=string[1];
tmp2+=string[2];
if (tmp1>=0 && tmp2>=0 && tmp1<=150 && tmp2<=50){
for (i=0;i<tmp2;i++) printf("\n");
for (i=0;i<tmp1;i++) printf(".");
if (string[0]==0) printf ("X");
else if (string[0]==1) printf("L");
else if(string[0]==2) printf("R");
else if(string[0]==3) printf("LR");
else if(string[0]==4) printf("M");
}
else {
if(tmp1<0) tmp1=0;
else if (tmp1>150) tmp1=150;
if (tmp2<0) tmp2=0;
else if (tmp2>50) tmp2=50;
}
if (string[3]==1) printf ("\nSCROLL UP");
if (string[3]==-1) printf ("\nSCROLL DOWN");
printf ("\n(%d, %d)",tmp1,tmp2);
if ( r < 0 ) errCount++;
if (errCount >= 100) break;
usb_clear_halt(udev,endPoint);
}
printf("Closing Device.\n");
usb_release_interface(udev,0);
usb_close(udev);
return EXIT_SUCCESS;
}
In this code are using API from libusb-0.1 - but i don't know excact version of it. Probably - it is libusb-0.1.12 (it have both headers "usb.h" and "usbpp.h", which are using in this code). I have download this version on ubuntu, install it, using
./configure && make && make install && sudo ldconfig
And after that I tried to do
gcc -c mouse.c -w
to compile this code. Bit it doesnt compile. It has different errors in "usbpp.h" (in standart includes like "string" or "list") or smth else..
Can anyone tell me, what i'n doing fundamentally wrong? And how can i try this code?
PS - i try do test this code on ubuntu 9.04 and ubuntu 12>, both as virtual machines. It's my task in university.
usbpp.h is a c++ header file but you run gcc which is a c compiler. Run g++ instead.
I found one another way to solve this problem:
gcc -o mouse ./mouse.cpp -lstdc++ -lusb
I need a C/C++ API that allows me to list the running processes on a Linux system, and list the files each process has open.
I do not want to end up reading the /proc/ file system directly.
Can anyone think of a way to do this?
http://procps.sourceforge.net/
http://procps.cvs.sourceforge.net/viewvc/procps/procps/proc/readproc.c?view=markup
Is the source of ps and other process tools. They do indeed use proc (indicating it is probably the conventional and best way). Their source is quite readable. The file
/procps-3.2.8/proc/readproc.c
May be useful. Also a useful suggestion as posted by ephemient is linking to the API provided by libproc, which should be available in your repo (or already installed I would say) but you will need the "-dev" variation for the headers and what-not.
Good Luck
If you do not want to read from '/proc. Then you can consider writing a Kernel module which will implement your own system call. And your system call should be written so that it can obtain the list of current processes, such as:
/* ProcessList.c
Robert Love Chapter 3
*/
#include < linux/kernel.h >
#include < linux/sched.h >
#include < linux/module.h >
int init_module(void) {
struct task_struct *task;
for_each_process(task) {
printk("%s [%d]\n",task->comm , task->pid);
}
return 0;
}
void cleanup_module(void) {
printk(KERN_INFO "Cleaning Up.\n");
}
The code above is taken from my article here at http://linuxgazette.net/133/saha.html.Once you have your own system call, you can call it from your user space program.
Here you go (C/C++):
You could have found it here:
http://ubuntuforums.org/showthread.php?t=657097
Essentially, what it does is loop through all numeric folders in /proc/<pid>, and then it does a readlink on /proc/<pid>/exe, or if you want the command-line-arguments cat /proc/<pid>/cmdline
The file-descriptors open by the process are in /proc/<pid>/fd/<descriptor>, and you get the file name by doing a readlink on each symlink, e.g. readlink /proc/<pid>/fd/<descriptor>. fd can be a device, such as /dev/null, a socket, or a file, and potentially more.
#include <unistd.h>
ssize_t readlink(const char *path, char *buf, size_t bufsiz);
On success, readlink() returns the number of bytes placed in buf.
On error, -1 is returned and errno is set to indicate the error.
This is, by the way, the same that readproc.c does (or at least did).
Of course, hopefully they did it without buffer overflow possiblity.
#ifndef __cplusplus
#define _GNU_SOURCE
#endif
#include <unistd.h>
#include <dirent.h>
#include <sys/types.h> // for opendir(), readdir(), closedir()
#include <sys/stat.h> // for stat()
#ifdef __cplusplus
#include <iostream>
#include <cstdlib>
#include <cstring>
#include <cstdarg>
#else
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <stdarg.h>
#endif
#define PROC_DIRECTORY "/proc/"
#define CASE_SENSITIVE 1
#define CASE_INSENSITIVE 0
#define EXACT_MATCH 1
#define INEXACT_MATCH 0
int IsNumeric(const char* ccharptr_CharacterList)
{
for ( ; *ccharptr_CharacterList; ccharptr_CharacterList++)
if (*ccharptr_CharacterList < '0' || *ccharptr_CharacterList > '9')
return 0; // false
return 1; // true
}
int strcmp_Wrapper(const char *s1, const char *s2, int intCaseSensitive)
{
if (intCaseSensitive)
return !strcmp(s1, s2);
else
return !strcasecmp(s1, s2);
}
int strstr_Wrapper(const char* haystack, const char* needle, int intCaseSensitive)
{
if (intCaseSensitive)
return (int) strstr(haystack, needle);
else
return (int) strcasestr(haystack, needle);
}
#ifdef __cplusplus
pid_t GetPIDbyName(const char* cchrptr_ProcessName, int intCaseSensitiveness, int intExactMatch)
#else
pid_t GetPIDbyName_implements(const char* cchrptr_ProcessName, int intCaseSensitiveness, int intExactMatch)
#endif
{
char chrarry_CommandLinePath[100] ;
char chrarry_NameOfProcess[300] ;
char* chrptr_StringToCompare = NULL ;
pid_t pid_ProcessIdentifier = (pid_t) -1 ;
struct dirent* de_DirEntity = NULL ;
DIR* dir_proc = NULL ;
int (*CompareFunction) (const char*, const char*, int) ;
if (intExactMatch)
CompareFunction = &strcmp_Wrapper;
else
CompareFunction = &strstr_Wrapper;
dir_proc = opendir(PROC_DIRECTORY) ;
if (dir_proc == NULL)
{
perror("Couldn't open the " PROC_DIRECTORY " directory") ;
return (pid_t) -2 ;
}
// Loop while not NULL
while ( (de_DirEntity = readdir(dir_proc)) )
{
if (de_DirEntity->d_type == DT_DIR)
{
if (IsNumeric(de_DirEntity->d_name))
{
strcpy(chrarry_CommandLinePath, PROC_DIRECTORY) ;
strcat(chrarry_CommandLinePath, de_DirEntity->d_name) ;
strcat(chrarry_CommandLinePath, "/cmdline") ;
FILE* fd_CmdLineFile = fopen (chrarry_CommandLinePath, "rt") ; // open the file for reading text
if (fd_CmdLineFile)
{
fscanf(fd_CmdLineFile, "%s", chrarry_NameOfProcess) ; // read from /proc/<NR>/cmdline
fclose(fd_CmdLineFile); // close the file prior to exiting the routine
if (strrchr(chrarry_NameOfProcess, '/'))
chrptr_StringToCompare = strrchr(chrarry_NameOfProcess, '/') +1 ;
else
chrptr_StringToCompare = chrarry_NameOfProcess ;
//printf("Process name: %s\n", chrarry_NameOfProcess);
//printf("Pure Process name: %s\n", chrptr_StringToCompare );
if ( CompareFunction(chrptr_StringToCompare, cchrptr_ProcessName, intCaseSensitiveness) )
{
pid_ProcessIdentifier = (pid_t) atoi(de_DirEntity->d_name) ;
closedir(dir_proc) ;
return pid_ProcessIdentifier ;
}
}
}
}
}
closedir(dir_proc) ;
return pid_ProcessIdentifier ;
}
#ifdef __cplusplus
pid_t GetPIDbyName(const char* cchrptr_ProcessName)
{
return GetPIDbyName(cchrptr_ProcessName, CASE_INSENSITIVE, EXACT_MATCH) ;
}
#else
// C cannot overload functions - fixed
pid_t GetPIDbyName_Wrapper(const char* cchrptr_ProcessName, ... )
{
int intTempArgument ;
int intInputArguments[2] ;
// intInputArguments[0] = 0 ;
// intInputArguments[1] = 0 ;
memset(intInputArguments, 0, sizeof(intInputArguments) ) ;
int intInputIndex ;
va_list argptr;
va_start( argptr, cchrptr_ProcessName );
for (intInputIndex = 0; (intTempArgument = va_arg( argptr, int )) != 15; ++intInputIndex)
{
intInputArguments[intInputIndex] = intTempArgument ;
}
va_end( argptr );
return GetPIDbyName_implements(cchrptr_ProcessName, intInputArguments[0], intInputArguments[1]);
}
#define GetPIDbyName(ProcessName,...) GetPIDbyName_Wrapper(ProcessName, ##__VA_ARGS__, (int) 15)
#endif
int main()
{
pid_t pid = GetPIDbyName("bash") ; // If -1 = not found, if -2 = proc fs access error
printf("PID %d\n", pid);
return EXIT_SUCCESS ;
}
If you don't do it, then I guess whatever API you will use will end up reading the /proc filesystem. Here are some examples of program doing this:
qps
htop
procps
But unfortunately, that does not constitute an API.
PS and every other tool(EXCEPT for Kernel Modules) read from /proc. /proc is a special filesystem created on the fly by the kernel so that user mode processes can read data that will otherwise only be available for the kernel.
The recommended way is therefore, reading from /proc.
You can quickly intuitively look at the /proc filesystem to see how its structured.
For every process there is a /proc/pid where pid is the process id number. Inside this folder there are several files which include different data about the current process.
If you run
strace ps -aux
you will see how the program ps reads this data from /proc.
The only way to do this without reading /proc would be to call "ps aux", go through every line, read the second column (the PID) and call lsof -p [PID] with it.
...I'd suggest reading /proc ;)
There's a library libprocps from the procps-ng project. On Ubuntu 13.04, if you do strace ps, then you can see that ps uses libprocps.
Reading proc is not too bad. I can't show you in C++, but the following D code should point you in the right direction:
import std.stdio;
import std.string;
import std.file;
import std.regexp;
import std.c.linux.linux;
alias std.string.split explode;
string srex = "^/proc/[0-9]+$";
string trex = "State:[ \t][SR]";
RegExp rex;
RegExp rext;
string[] scanPidDirs(string target)
{
string[] result;
bool callback(DirEntry* de)
{
if (de.isdir)
{
if (rex.find(de.name) >= 0)
{
string[] a = explode(de.name, "/");
string pid = a[a.length-1];
string x = cast(string) std.file.read(de.name ~ "/status");
int n = rext.find(x);
if (n >= 0)
{
x = cast(string) std.file.read(de.name ~ "/cmdline");
// This is null terminated
if (x.length) x.length = x.length-1;
a = explode(x, "/");
if (a.length)
x = a[a.length-1];
else
x = "";
if (x == target)
{
result ~= pid ~ "/" ~x;
}
}
}
}
return true;
}
listdir("/proc", &callback);
return result.dup;
}
void main(string[] args)
{
rex= new RegExp(srex);
rext= new RegExp(trex);
string[] a = scanPidDirs(args[1]);
if (!a.length)
{
writefln("Not found");
return;
}
writefln("%d matching processes", a.length);
foreach (s; a)
{
string[] p = explode(s, "/");
int pid = atoi(p[0]);
writef("Stop %s (%d)? ", s, pid);
string r = readln();
if (r == "Y\n" || r == "y\n")
kill(pid, SIGUSR1);
}
}
Easy way to fin pid of any process by name
pid_t GetPIDbyName(char* ps_name)
{
FILE *fp;
char *cmd=(char*)calloc(1,200);
sprintf(cmd,"pidof %s",ps_name);
fp=popen(cmd,"r");
fread(cmd,1,200,fp);
fclose(fp);
return atoi(cmd);
}
I want an easy way to create multiple directories in C++/Linux.
For example I want to save a file lola.file in the directory:
/tmp/a/b/c
but if the directories are not there I want them to be created automagically. A working example would be perfect.
Easy with Boost.Filesystem: create_directories
#include <boost/filesystem.hpp>
//...
boost::filesystem::create_directories("/tmp/a/b/c");
Returns: true if a new directory was created, otherwise false.
With C++17 or later, there's the standard header <filesystem> with
function
std::filesystem::create_directories
which should be used in modern C++ programs.
The C++ standard functions do not have the POSIX-specific explicit
permissions (mode) argument, though.
However, here's a C function that can be compiled with C++ compilers.
/*
#(#)File: mkpath.c
#(#)Purpose: Create all directories in path
#(#)Author: J Leffler
#(#)Copyright: (C) JLSS 1990-2020
#(#)Derivation: mkpath.c 1.16 2020/06/19 15:08:10
*/
/*TABSTOP=4*/
#include "posixver.h"
#include "mkpath.h"
#include "emalloc.h"
#include <errno.h>
#include <string.h>
/* "sysstat.h" == <sys/stat.h> with fixup for (old) Windows - inc mode_t */
#include "sysstat.h"
typedef struct stat Stat;
static int do_mkdir(const char *path, mode_t mode)
{
Stat st;
int status = 0;
if (stat(path, &st) != 0)
{
/* Directory does not exist. EEXIST for race condition */
if (mkdir(path, mode) != 0 && errno != EEXIST)
status = -1;
}
else if (!S_ISDIR(st.st_mode))
{
errno = ENOTDIR;
status = -1;
}
return(status);
}
/**
** mkpath - ensure all directories in path exist
** Algorithm takes the pessimistic view and works top-down to ensure
** each directory in path exists, rather than optimistically creating
** the last element and working backwards.
*/
int mkpath(const char *path, mode_t mode)
{
char *pp;
char *sp;
int status;
char *copypath = STRDUP(path);
status = 0;
pp = copypath;
while (status == 0 && (sp = strchr(pp, '/')) != 0)
{
if (sp != pp)
{
/* Neither root nor double slash in path */
*sp = '\0';
status = do_mkdir(copypath, mode);
*sp = '/';
}
pp = sp + 1;
}
if (status == 0)
status = do_mkdir(path, mode);
FREE(copypath);
return (status);
}
#ifdef TEST
#include <stdio.h>
#include <unistd.h>
/*
** Stress test with parallel running of mkpath() function.
** Before the EEXIST test, code would fail.
** With the EEXIST test, code does not fail.
**
** Test shell script
** PREFIX=mkpath.$$
** NAME=./$PREFIX/sa/32/ad/13/23/13/12/13/sd/ds/ww/qq/ss/dd/zz/xx/dd/rr/ff/ff/ss/ss/ss/ss/ss/ss/ss/ss
** : ${MKPATH:=mkpath}
** ./$MKPATH $NAME &
** [...repeat a dozen times or so...]
** ./$MKPATH $NAME &
** wait
** rm -fr ./$PREFIX/
*/
int main(int argc, char **argv)
{
int i;
for (i = 1; i < argc; i++)
{
for (int j = 0; j < 20; j++)
{
if (fork() == 0)
{
int rc = mkpath(argv[i], 0777);
if (rc != 0)
fprintf(stderr, "%d: failed to create (%d: %s): %s\n",
(int)getpid(), errno, strerror(errno), argv[i]);
exit(rc == 0 ? EXIT_SUCCESS : EXIT_FAILURE);
}
}
int status;
int fail = 0;
while (wait(&status) != -1)
{
if (WEXITSTATUS(status) != 0)
fail = 1;
}
if (fail == 0)
printf("created: %s\n", argv[i]);
}
return(0);
}
#endif /* TEST */
The macros STRDUP() and FREE() are error-checking versions of
strdup() and free(), declared in emalloc.h (and implemented in
emalloc.c and estrdup.c).
The "sysstat.h" header deals with broken versions of <sys/stat.h>
and can be replaced by <sys/stat.h> on modern Unix systems (but there
were many issues back in 1990).
And "mkpath.h" declares mkpath().
The change between v1.12 (original version of the answer) and v1.13
(amended version of the answer) was the test for EEXIST in
do_mkdir().
This was pointed out as necessary by
Switch — thank
you, Switch.
The test code has been upgraded and reproduced the problem on a MacBook
Pro (2.3GHz Intel Core i7, running Mac OS X 10.7.4), and suggests that
the problem is fixed in the revision (but testing can only show the
presence of bugs, never their absence).
The code shown is now v1.16; there have been cosmetic or administrative
changes made since v1.13 (such as use mkpath.h instead of jlss.h and
include <unistd.h> unconditionally in the test code only).
It's reasonable to argue that "sysstat.h" should be replaced by
<sys/stat.h> unless you have an unusually recalcitrant system.
(You are hereby given permission to use this code for any purpose with attribution.)
This code is available in my SOQ
(Stack Overflow Questions) repository on GitHub as files mkpath.c and
mkpath.h (etc.) in the
src/so-0067-5039
sub-directory.
system("mkdir -p /tmp/a/b/c")
is the shortest way I can think of (in terms of the length of code, not necessarily execution time).
It's not cross-platform but will work under Linux.
Here is my example of code (it works for both Windows and Linux):
#include <iostream>
#include <string>
#include <sys/stat.h> // stat
#include <errno.h> // errno, ENOENT, EEXIST
#if defined(_WIN32)
#include <direct.h> // _mkdir
#endif
bool isDirExist(const std::string& path)
{
#if defined(_WIN32)
struct _stat info;
if (_stat(path.c_str(), &info) != 0)
{
return false;
}
return (info.st_mode & _S_IFDIR) != 0;
#else
struct stat info;
if (stat(path.c_str(), &info) != 0)
{
return false;
}
return (info.st_mode & S_IFDIR) != 0;
#endif
}
bool makePath(const std::string& path)
{
#if defined(_WIN32)
int ret = _mkdir(path.c_str());
#else
mode_t mode = 0755;
int ret = mkdir(path.c_str(), mode);
#endif
if (ret == 0)
return true;
switch (errno)
{
case ENOENT:
// parent didn't exist, try to create it
{
int pos = path.find_last_of('/');
if (pos == std::string::npos)
#if defined(_WIN32)
pos = path.find_last_of('\\');
if (pos == std::string::npos)
#endif
return false;
if (!makePath( path.substr(0, pos) ))
return false;
}
// now, try to create again
#if defined(_WIN32)
return 0 == _mkdir(path.c_str());
#else
return 0 == mkdir(path.c_str(), mode);
#endif
case EEXIST:
// done!
return isDirExist(path);
default:
return false;
}
}
int main(int argc, char* ARGV[])
{
for (int i=1; i<argc; i++)
{
std::cout << "creating " << ARGV[i] << " ... " << (makePath(ARGV[i]) ? "OK" : "failed") << std::endl;
}
return 0;
}
Usage:
$ makePath 1/2 folderA/folderB/folderC
creating 1/2 ... OK
creating folderA/folderB/folderC ... OK
#include <sys/types.h>
#include <sys/stat.h>
int status;
...
status = mkdir("/tmp/a/b/c", S_IRWXU | S_IRWXG | S_IROTH | S_IXOTH);
From here. You may have to do separate mkdirs for /tmp, /tmp/a, /tmp/a/b/ and then /tmp/a/b/c because there isn't an equivalent of the -p flag in the C api. Be sure and ignore the EEXISTS errno while you're doing the upper level ones.
It should be noted that starting from C++17 filesystem interface is part of the standard library. This means that one can have following to create directories:
#include <filesystem>
std::filesystem::create_directories("/a/b/c/d")
More info here: https://en.cppreference.com/w/cpp/filesystem/create_directory
Additionally, with gcc, one needs to "-std=c++17" to CFLAGS. And "-lstdc++fs" to LDLIBS. The latter potentially is not going to be required in the future.
This is similar to the previous but works forward through the string instead of recursively backwards. Leaves errno with the right value for last failure. If there's a leading slash, there's an extra time through the loop which could have been avoided via one find_first_of() outside the loop or by detecting the leading / and setting pre to 1. The efficiency is the same whether we get set up by a first loop or a pre loop call, and the complexity would be (slightly) higher when using the pre-loop call.
#include <iostream>
#include <string>
#include <sys/stat.h>
int
mkpath(std::string s,mode_t mode)
{
size_t pos=0;
std::string dir;
int mdret;
if(s[s.size()-1]!='/'){
// force trailing / so we can handle everything in loop
s+='/';
}
while((pos=s.find_first_of('/',pos))!=std::string::npos){
dir=s.substr(0,pos++);
if(dir.size()==0) continue; // if leading / first time is 0 length
if((mdret=mkdir(dir.c_str(),mode)) && errno!=EEXIST){
return mdret;
}
}
return mdret;
}
int main()
{
int mkdirretval;
mkdirretval=mkpath("./foo/bar",0755);
std::cout << mkdirretval << '\n';
}
You said "C++" but everyone here seems to be thinking "Bash shell."
Check out the source code to gnu mkdir; then you can see how to implement the shell commands in C++.
bool mkpath( std::string path )
{
bool bSuccess = false;
int nRC = ::mkdir( path.c_str(), 0775 );
if( nRC == -1 )
{
switch( errno )
{
case ENOENT:
//parent didn't exist, try to create it
if( mkpath( path.substr(0, path.find_last_of('/')) ) )
//Now, try to create again.
bSuccess = 0 == ::mkdir( path.c_str(), 0775 );
else
bSuccess = false;
break;
case EEXIST:
//Done!
bSuccess = true;
break;
default:
bSuccess = false;
break;
}
}
else
bSuccess = true;
return bSuccess;
}
So I need mkdirp() today, and found the solutions on this page overly complicated.
Hence I wrote a fairly short snippet, that easily be copied in for others who
stumble upon this thread an wonder why we need so many lines of code.
mkdirp.h
#ifndef MKDIRP_H
#define MKDIRP_H
#include <sys/stat.h>
#define DEFAULT_MODE S_IRWXU | S_IRGRP | S_IXGRP | S_IROTH | S_IXOTH
/** Utility function to create directory tree */
bool mkdirp(const char* path, mode_t mode = DEFAULT_MODE);
#endif // MKDIRP_H
mkdirp.cpp
#include <errno.h>
bool mkdirp(const char* path, mode_t mode) {
// const cast for hack
char* p = const_cast<char*>(path);
// Do mkdir for each slash until end of string or error
while (*p != '\0') {
// Skip first character
p++;
// Find first slash or end
while(*p != '\0' && *p != '/') p++;
// Remember value from p
char v = *p;
// Write end of string at p
*p = '\0';
// Create folder from path to '\0' inserted at p
if(mkdir(path, mode) == -1 && errno != EEXIST) {
*p = v;
return false;
}
// Restore path to it's former glory
*p = v;
}
return true;
}
If you don't like const casting and temporarily modifying the string, just do a strdup() and free() it afterwards.
Since this post is ranking high in Google for "Create Directory Tree", I am going to post an answer that will work for Windows — this will work using Win32 API compiled for UNICODE or MBCS. This is ported from Mark's code above.
Since this is Windows we are working with, directory separators are BACK-slashes, not forward slashes. If you would rather have forward slashes, change '\\' to '/'
It will work with:
c:\foo\bar\hello\world
and
c:\foo\bar\hellp\world\
(i.e.: does not need trailing slash, so you don't have to check for it.)
Before saying "Just use SHCreateDirectoryEx() in Windows", note that SHCreateDirectoryEx() is deprecated and could be removed at any time from future versions of Windows.
bool CreateDirectoryTree(LPCTSTR szPathTree, LPSECURITY_ATTRIBUTES lpSecurityAttributes = NULL){
bool bSuccess = false;
const BOOL bCD = CreateDirectory(szPathTree, lpSecurityAttributes);
DWORD dwLastError = 0;
if(!bCD){
dwLastError = GetLastError();
}else{
return true;
}
switch(dwLastError){
case ERROR_ALREADY_EXISTS:
bSuccess = true;
break;
case ERROR_PATH_NOT_FOUND:
{
TCHAR szPrev[MAX_PATH] = {0};
LPCTSTR szLast = _tcsrchr(szPathTree,'\\');
_tcsnccpy(szPrev,szPathTree,(int)(szLast-szPathTree));
if(CreateDirectoryTree(szPrev,lpSecurityAttributes)){
bSuccess = CreateDirectory(szPathTree,lpSecurityAttributes)!=0;
if(!bSuccess){
bSuccess = (GetLastError()==ERROR_ALREADY_EXISTS);
}
}else{
bSuccess = false;
}
}
break;
default:
bSuccess = false;
break;
}
return bSuccess;
}
I know it's an old question but it shows up high on google search results and the answers provided here are not really in C++ or are a bit too complicated.
Please note that in my example createDirTree() is very simple because all the heavy lifting (error checking, path validation) needs to be done by createDir() anyway. Also createDir() should return true if directory already exists or the whole thing won't work.
Here's how I would do that in C++:
#include <iostream>
#include <string>
bool createDir(const std::string dir)
{
std::cout << "Make sure dir is a valid path, it does not exist and create it: "
<< dir << std::endl;
return true;
}
bool createDirTree(const std::string full_path)
{
size_t pos = 0;
bool ret_val = true;
while(ret_val == true && pos != std::string::npos)
{
pos = full_path.find('/', pos + 1);
ret_val = createDir(full_path.substr(0, pos));
}
return ret_val;
}
int main()
{
createDirTree("/tmp/a/b/c");
return 0;
}
Of course createDir() function will be system-specific and there are already enough examples in other answers how to write it for linux, so I decided to skip it.
So many approaches has been described here but most of them need hard coding of your path into your code.
There is an easy solution for that problem, using QDir and QFileInfo, two classes of Qt framework. Since your already in Linux environment it should be easy to use Qt.
QString qStringFileName("path/to/the/file/that/dont/exist.txt");
QDir dir = QFileInfo(qStringFileName).dir();
if(!dir.exists()) {
dir.mkpath(dir.path());
}
Make sure you have write access to that Path.
If dir does not exist, create it:
boost::filesystem::create_directories(boost::filesystem::path(output_file).parent_path().string().c_str());
Here's C/C++ recursive function that makes use of dirname() to traverse bottom-up the directory tree. It will stop as soon as it finds an existing ancestor.
#include <libgen.h>
#include <string.h>
int create_dir_tree_recursive(const char *path, const mode_t mode)
{
if (strcmp(path, "/") == 0) // No need of checking if we are at root.
return 0;
// Check whether this dir exists or not.
struct stat st;
if (stat(path, &st) != 0 || !S_ISDIR(st.st_mode))
{
// Check and create parent dir tree first.
char *path2 = strdup(path);
char *parent_dir_path = dirname(path2);
if (create_dir_tree_recursive(parent_dir_path, mode) == -1)
return -1;
// Create this dir.
if (mkdir(path, mode) == -1)
return -1;
}
return 0;
}
mkdir -p /dir/to/the/file
touch /dir/to/the/file/thefile.ending
If you don't have C++17 yet and look for a platform agnostic solution, use ghc::filesystem. The header-ony code is compatible to C++17 (in fact a backport) and easy to migrate later on.
The others got you the right answer, but I thought I'd demonstrate another neat thing you can do:
mkdir -p /tmp/a/{b,c}/d
Will create the following paths:
/tmp/a/b/d
/tmp/a/c/d
The braces allow you to create multiple directories at once on the same level of the hierarchy, whereas the -p option means "create parent directories as needed".