Basically I want to read all data from stdout when running a program with popen. However, the program I run will not return the last output line into my buffer and I do not understand why (my understanding: I would get any output sent to stdout {unbuffered?} from popen).
// $Id: popen.cpp 126 2011-04-25 18:48:02Z wus $
#include <iostream>
#include <stdio.h>
using namespace std;
/**
* run debians apt-get and check output
*/
int main(int argc, char **argv, char **envp) {
FILE *fp;
char buffer[9];
// select a package which is not installed and has uninstalled dependencies,
// apt-get will ask if it should installed «Y» or nor «n».
char command[255] = "apt-get install python-wxtools";
cout << command << endl;
// Execute command, open /dev/stdout for reading
fp = popen(command, "r");
// read output character by character
while (fread(buffer, 1, 1, fp) != EOF) {
cout << buffer;
}
// close
pclose(fp);
}
native output looks like this:
$ sudo apt-get install python-wxtools
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following extra packages will be installed:
python-wxgtk2.8 python-wxversion
Suggested packages:
wx2.8-doc wx2.8-examples ruby wish tk8.5 tcsh csh octave3.0 mksh pdksh
python-xml editra
The following NEW packages will be installed:
python-wxgtk2.8 python-wxtools python-wxversion
0 upgraded, 3 newly installed, 0 to remove and 8 not upgraded.
Need to get 5,942kB of archives.
After this operation, 25.0MB of additional disk space will be used.
Do you want to continue [Y/n]?
NOTE: no newline at the end of the last line.
when I use my own program, the last line is missing (output)
$ sudo ./test/popen
g++ -Wall -o test/popen test/popen.cpp
test/popen.cpp: In function ‘int main(int, char**, char**)’:
test/popen.cpp:22: warning: comparison between signed and unsigned integer expressions
apt-get install python-wxtools
Reading package lists...
Building dependency tree...
Reading state information...
The following extra packages will be installed:
python-wxgtk2.8 python-wxversion
Suggested packages:
wx2.8-doc wx2.8-examples ruby wish tk8.5 tcsh csh octave3.0 mksh pdksh
python-xml editra
The following NEW packages will be installed:
python-wxgtk2.8 python-wxtools python-wxversion
0 upgraded, 3 newly installed, 0 to remove and 8 not upgraded.
Need to get 5,942kB of archives.
After this operation, 25.0MB of additional disk space will be used.
NOTE: newline at the end of output
fread does not return EOF when it reaches end of file. Rather, it returns 0. But I suspect the problem is that apt-get is detecting that its input is not a tty and is thus not printing the prompt at all.
The problem isn't that you aren't reading the last line, the problem is that you aren't displaying it due to cout being line buffered.
cout << buffer << flush;
should work.
Related
I'm trying to determine the mime-type for several types of files using libmagic and the following bit of code:
auto handle = ::magic_open(MAGIC_MIME_TYPE);
::magic_load(handle, NULL);
// Both of these fail with the same error
// file_path being a const char* with the path to the file.
auto type2 = ::magic_file(handle, file_path);
// svg_content being an std::vector<char> with the contents of the file.
//auto type2 = ::magic_buffer(handle, svg_content.data(), svg_content.size());
if(!type2)
{
std::cout << magic_error(handle) << std::endl;
}
::magic_close(handle);
But for any file or buffer I try I receive regex error, either being or similar to:
46: regex error 17 for `(dryad-bibo/v)[0-9].[0-9]', (match failed)
For example with this .svg file:
<svg xmlns="http://www.w3.org/2000/svg" id="flag-icon-css-no" viewBox="0 0 640 480">
<path fill="#ed2939" d="M0 0h640v480H0z"/>
<path fill="#fff" d="M180 0h120v480H180z"/>
<path fill="#fff" d="M0 180h640v120H0z"/>
<path fill="#002664" d="M210 0h60v480h-60z"/>
<path fill="#002664" d="M0 210h640v60H0z"/>
</svg>
What I've tried so far:
libmagic 5.35
libmagic 5.39
libmagic 5.40
libmagic from opensource.apple
setting LC_TYPE and LANG to "C"
I'm linking against a version of libmagic that was built locally, could there be anything I have missed while building? Are any of the calls incorrect or is there something I'm missing?
I get similar errors when trying to run the related file binary that was compiled locally. Whereas when I use the file command that is available by default I do get image/svg+xml as output.
Edit
To build libmagic (for macOS and Ubuntu), I followed these steps:
Downloaded relevant release from Github
autoreconf --install
./configure
make
make install
Update
It looks like the regex at the bottom of this file is causing issues (at least for the svg):
https://github.com/file/file/blob/b56b58d499dbe58f2bed28e6b3c297fe7add992e/magic/Magdir/dataone
Update 2
Something strange going on; On the system where I've got it working, magic_version() reports 540, as expected. But on the systems where it fails with this error, magic_version() reports 538.
This makes little sense to me as I can't find that version on the system itself anywhere and when I run ./file --version in the build library, it reports file-5.40.
Very dissatisfying answer, but it was linking against GoogleTest causing this error somehow, not even running any tests, just linking against it.
I switched to using Catch2 instead and the issue was resolved.
Tested on Ubuntu 20.04.:
Clone the repo
git clone git#github.com:file/file.git
cd file/
Try this in fresh clone of the repo:
autoreconf -f -i
./configure --disable-silent-rules
make -j4
make -C tests check
And see whether there are any errors reported. After installation with make install, grab some valid xml file named as "test.xml" and put it to some folder together with this main.c:
#include <stdio.h>
#include <magic.h>
int main(void)
{
char *actual_file = "test.xml";
const char *magic_full;
magic_t magic_cookie;
/* MAGIC_MIME tells magic to return a mime of the file,
but you can specify different things */
magic_cookie = magic_open(MAGIC_MIME);
if (magic_cookie == NULL) {
printf("unable to initialize magic library\n");
return 1;
}
printf("Loading default magic database\n");
if (magic_load(magic_cookie, NULL) != 0) {
printf("cannot load magic database - %s\n", magic_error(magic_cookie));
magic_close(magic_cookie);
return 1;
}
magic_full = magic_file(magic_cookie, actual_file);
printf("%s\n", magic_full);
magic_close(magic_cookie);
return 0;
}
(Courtesy to vivithemage.)
Compile and try:
$ gcc main.c -lmagic
$ ./a.out
Loading default magic database
text/xml; charset=us-ascii
If it does not work on your system, report bug on project's bugtracker with specification of your OS and architecture. You can try to hotfix your problem by removing offending record from the file you have found in your update.
I've followed every single guide I could possibly find but to be completely honest I have no idea what some installation 'steps' even mean.
I tried installing Cygwin (and MYSY) and running the commands that the guides told me to, but the terminal either doesn't do anything or it gives me the error: 'no such file or directory'. (I changed the folder directory to where my files where)
Also I'm not entirely sure I installed everything correctly because I should've checked some add-ons during the installation right? I did as in the guide but still I maybe missed something...
Could someone please explain to me step by step how to install it saying explicitly all that has to be done, (I'm running windows 7) considering It's the first time I do such thing and I have no idea what ./configure , make and all the other commands even mean...
The following is a simple step by step using only cygwin tools.
To compile C++ we need the g++ compiler; to locate the correct package to be installed the cygwin tool is cygcheck (that is installed by default), with the -p switch it interrogates the database at https://cygwin.com/packages/:
$ cygcheck -p bin/g++
Found 3 matches for bin/g++
gcc-g++-7.3.0-1 - gcc-g++: GNU Compiler Collection (C++)
gcc-g++-7.3.0-2 - gcc-g++: GNU Compiler Collection (C++)
gcc-g++-7.3.0-3 - gcc-g++: GNU Compiler Collection (C++)
so we need the gcc-g++ package.
To install it, we run the cygwin setup, select the Full view, search the gcc-g to filter the thousands of packages and click on skip at the gcc-g++ row
after complety the installation, to verify we have it correctly installed:
$ cygcheck -c gcc-g++
Cygwin Package Information
Package Version Status
gcc-g++ 7.3.0-3 OK
Installing gcc-g++ will pull also the installation of the C compiler package gcc-core.
To compile a gmp program we need the proper header and shared library; that are usually included in a "*-devel" package:
$ cygcheck -p include/gmpxx.h
Found 9 matches for include/gmpxx.h
libgmp-devel-6.1.0-3p1 - libgmp-devel: Library for arbitrary precision arithmetic (development) (installed binaries and support files)
libgmp-devel-6.1.1-1 - libgmp-devel: Library for arbitrary precision arithmetic (development) (installed binaries and support files)
libgmp-devel-6.1.2-1 - libgmp-devel: Library for arbitrary precision arithmetic (development)
mingw64-i686-gmp-6.0.0a-2 - mingw64-i686-gmp: Multiple Precision arithmetic library for Win32 toolchain (installed binaries and support files)
...
All the packages with mingw64 are for cross compiling and we can ignore, so it is libgmp-devel. Verifying that is properly installed
$ cygcheck -c libgmp-devel
Cygwin Package Information
Package Version Status
libgmp-devel 6.1.2-1 OK
And the package content is the header files and the import libraries
$ cygcheck -l libgmp-devel
/usr/include/gmp.h
/usr/include/gmpxx.h
/usr/lib/libgmp.dll.a
/usr/lib/libgmpxx.dll.a
Now we can program one example, I am taking it from
https://gmplib.org/manual/C_002b_002b-Interface-General.html
and written in a file called mpz_add.cpp
You can use whatever editor you want. The important is that the file follows
the Unix line termination standard LF and not the Windows CR+LF (see note below if not)
$ file mpz_add.cpp
mpz_add.cpp: C++ source, ASCII text
$ cat mpz_add.cpp
#include <gmpxx.h>
#include <iostream>
using namespace std;
int main (void)
{
mpz_class a, b, c;
a = 1234;
b = "-5678";
c = a+b;
cout << "sum is " << c << "\n";
cout << "absolute value is " << abs(c) << "\n";
return 0;
}
To compile our example and test it:
$ g++ mpz_add.cpp -lgmpxx -lgmp -o mpz_add
$ ./mpz_add
sum is -4444
absolute value is 4444
We can also verify which library are linked in the program mpz_add, I added some extra comment:
$ cygcheck ./mpz_add
D:\cyg_pub\tmp\gmp\mpz_add.exe
D:\cygwin64\bin\cygwin1.dll <= cygwin library
C:\WINDOWS\system32\KERNEL32.dll <= windows system library
C:\WINDOWS\system32\ntdll.dll ...
C:\WINDOWS\system32\KERNELBASE.dll ...
D:\cygwin64\bin\cyggmp-10.dll <= GMP C library
D:\cygwin64\bin\cyggmpxx-4.dll <= GMP C++ library
D:\cygwin64\bin\cyggcc_s-seh-1.dll <= C library
D:\cygwin64\bin\cygstdc++-6.dll <= C++ library
If the file has the wrong line termination, the best tool is d2u (Dos to Unix)
$ cygcheck -p bin/d2u
Found 6 matches for bin/d2u
...
dos2unix-7.4.0-1 - dos2unix: Line Break Conversion
$ file mpz_add.cpp
mpz_add.cpp: C++ source, ASCII text, with CRLF line terminators
$ d2u mpz_add.cpp
dos2unix: converting file mpz_add.cpp to Unix format...
$ file mpz_add.cpp
mpz_add.cpp: C++ source, ASCII text
As you added also the tag makefile and autotools, the first is in the package make:
$ cygcheck -p bin/make.exe
Found 6 matches for bin/make.exe
..
make-4.2.1-2 - make: The GNU version of the 'make' utility
The second is more complex and you need the packages autoconf automake and libtool,
I am trying to fetch from my remote repository on BitBucket. git_remote_fetch returns error with message:
there is no TLS stream available
Some ideas how to fix that?
Here is my code :
if(error = git_remote_fetch(remote, NULL,NULL,"fetch")!=0) {
const git_error *lastError = giterr_last();
cerr << "problem with fetch, error message : '" << lastError->message <<"'"<< endl;
}
I got the same error when trying to use Python and pygit2 (which in turn depends on libgit2) to pull from a git repository. I am using a self-compiled version of libgit2 (0.25.0).
Recompiling libgit2 with the following flag solved the issue: cmake -DCURL=OFF as described in issue #3786[1] of the libgit2 project.
Full process of compiling as per pygit2 documentation[2] with added flag:
$ wget https://github.com/libgit2/libgit2/archive/v0.25.0.tar.gz
$ tar xzf v0.25.0.tar.gz
$ cd libgit2-0.25.0/
$ cmake -DCURL=OFF .
$ make
$ sudo make install
Make sure to delete the build directory of libgit2 if you compiled it once earlier because otherwise the flag will have no effect.
[1] https://github.com/libgit2/libgit2/issues/3786
[2] http://www.pygit2.org/install.html#quick-install
If you use the built-in version of libgit of your OS I suggest trying to use a compiled version.
#include <iostream>
using namespace std;
int main(){
int intake;
cout<<"Enter the maximum number of intake in this session of 2015, September: ";
cin>>intake;
for(int i=20150900; i<intake+20150900; i++){
cout<<"Enter the total percentage of Student ID numbered #"<<i<<": ";
}
return 0;
}
This is my code and when i compile it on code::blocks it gives the following error. I am new to linux OS so i dont know much about how it all works. Thankx for your help! :)
Error Message:
g++ -c /home/subbs/Desktop/entrance_exam/main.cpp -o
/home/subbs/Desktop/entrance_exam/main.o /bin/sh: 1: g++: not found
You need to install the compiler. Run:
sudo apt-get install build-essential
And then, try again to compile.
UPDATE:
If when you're trying to run the program, you get an error message like this:
Process terminated with status 255
It's because code blocks tries to run the program with xterm by default, and it may be the case that you don' have it installed. To solve it you have to go to:
Settings > Environment
And in the drop down menu next to Terminal to launch console programs choose your terminal. In the case of Linux Mint, it was gnome-terminal --disable-factory -t $TITLE -x
- source
Open the terminal and enter sudo apt-get install g++. This will take care of install g++ and everything else you will need. It is real easy.
This might sound like a stupid question, I am new to C++.
On debian I try to invoke apt-get install with popen. I need to parse the output of this programm. Unfortunately I am not able to read the full output of apt-get.
At some point apt-get might ask for user input (asking if dependencies should be installed). My programm is not able to output this line.
Whis is the last line (see examples below, the line missing in my program is: "Do you want to continue [Y/n]?") not outputted?
When I run the apt-get command manually, the console output looks like this:
$ sudo apt-get install python-wxtools
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following extra packages will be installed:
python-wxgtk2.8 python-wxversion
Suggested packages:
wx2.8-doc wx2.8-examples ruby wish tk8.5 tcsh csh octave3.0 mksh pdksh
python-xml editra
The following NEW packages will be installed:
python-wxgtk2.8 python-wxtools python-wxversion
0 upgraded, 3 newly installed, 0 to remove and 8 not upgraded.
Need to get 5,942kB of archives.
After this operation, 25.0MB of additional disk space will be used.
Do you want to continue [Y/n]?
NOTE: no newline at the end of the last line.
when I use my own program, the last line is missing (output)
$ sudo ./test/popen
g++ -Wall -o test/popen test/popen.cpp
test/popen.cpp: In function ‘int main(int, char**, char**)’:
test/popen.cpp:22: warning: comparison between signed and unsigned integer expressions
apt-get install python-wxtools
Reading package lists...
Building dependency tree...
Reading state information...
The following extra packages will be installed:
python-wxgtk2.8 python-wxversion
Suggested packages:
wx2.8-doc wx2.8-examples ruby wish tk8.5 tcsh csh octave3.0 mksh pdksh
python-xml editra
The following NEW packages will be installed:
python-wxgtk2.8 python-wxtools python-wxversion
0 upgraded, 3 newly installed, 0 to remove and 8 not upgraded.
Need to get 5,942kB of archives.
After this operation, 25.0MB of additional disk space will be used.
NOTE: newline at the end of output
My reference implementation of popen looks like this in c++:
// $Id: popen.cpp 126 2011-04-25 18:48:02Z wus $
#include <iostream>
#include <stdio.h>
using namespace std;
/**
* run debians apt-get and check output
*/
int main(int argc, char **argv, char **envp) {
FILE *fp;
char buffer[9];
// must use a package which asks for dependencies
char command[255] = "apt-get install python-wxtools";
cout << command << endl;
// Execute command, open /dev/stdout for reading
fp = popen(command, "r");
// read output character by character
while (fread(buffer, 1, 1, fp) != EOF) {
cout << buffer;
}
// close
pclose(fp);
}
I try this to do on a Linux System
$ uname -a
Linux shell1 2.6.35-28-server #50-Ubuntu SMP Fri Mar 18 18:59:25 UTC 2011 x86_64 GNU/Linux
$ gcc --version
gcc (Ubuntu/Linaro 4.4.4-14ubuntu5) 4.4.5
Copyright (C) 2010 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
Try adding the -y option to apt-get. This will make it assume the user says Yes to everything, which should make it "just work." I think it's failing now because it wants to prompt the user but it realizes there is no keyboard input available (possibly by calling isatty(3)) so it gives up.
The problem was not actually input buffering but output buffering of cout. Manual flushes solved the problem:
while (fread(buffer, 1, 1, fp) != EOF) {
cout << buffer << flush;
}