How do command line tools change their output after outputting it? - c++

I've noticed that a lot of command line tools, wget for example, will show progress as a number or progress bar that advances as a process is completed. While the question isn't really language-specific, out of the languages I use most often for command line tools (C++, Node.js, Haskell) I haven't seen a way to do this.
Here's an example, three snapshots of a single line of Terminal as wget downloads a file:
Along with other information, wget shows a progress bar (<=>) that advances as it downloads a file. The amount of data downloaded so far (6363, 179561, 316053) and the current download speed (10.7KB/s, 65.8KB/s, 63.0KB/s) update as well. How is this done?
Ideally, please include a code sample from one or more of the three languages mentioned above.

Just print a CR (without a newline) to overwrite a line. Here is an example program in perl:
#!/usr/bin/env perl
$| = 1;
for (1..10) {
print "the count is: $_\r";
sleep(1)
}
I've also disabled output buffering ($| = 1) so that the print command sends its output to the console immediately instead of buffering it.
Haskell example:
import System.IO
import Control.Monad
import Control.Concurrent
main = do
hSetBuffering stdout NoBuffering
forM_ [1..10] $ \i -> do
putStr $ "the count is: " ++ show i ++ "\r"
threadDelay 1000000

Looking at GNU wget repo on GitHub -- progress.c
It seems they do it the same way i.e. print a \r and then overwrite.
/* Print the contents of the buffer as a one-line ASCII "image" so
that it can be overwritten next time. */
static void
display_image (char *buf)
{
bool old = log_set_save_context (false);
logputs (LOG_VERBOSE, "\r");
logputs (LOG_VERBOSE, buf);
log_set_save_context (old);
}

I can only speak about node.js, but the built-in readline module has some very basic screen handling functionality built-in. For example:
var readline = require('readline');
var c = 0;
var intvl = setInterval(function() {
// Clear entirety of current line
readline.clearLine(process.stdout, 0);
readline.cursorTo(process.stdout, 0);
process.stdout.write('Progress: ' + (++c) + '%');
if (c === 100)
clearInterval(intvl);
}, 500);
There are also third party modules if you want to get fancier, such as multimeter/meterbox and blessed/blessed-contrib.
Generally speaking though, some programs use ncurses, while others simply just manually output the ANSI escape codes to clear and redraw the current line.

They probably use the fancy ncurses library but on my Linux for my personal command-line tools I simply send '\r' to move the cursor back to the start of the line to overwrite it with new progress information.
#include <thread>
#include <chrono>
#include <iostream>
int main()
{
for(auto i = 0; i < 100; ++i)
{
std::cout << "\rprogress: " << i << "% " << std::flush;
std::this_thread::sleep_for(std::chrono::milliseconds(100));
}
std::cout << "\rprogress: DONE " << std::flush;
}

Related

How do I address the screen on a linux terminal?

I am teaching myself C++ and currently run the latest Fedora.
On a Windows command prompt you can address the screen location.
I am led to the believe that a Linux command terminal works, effectively, like characters printed to a piece of paper. i.e. You can't go "up" from where you are.
However, when you install in Linux, there is a progress bar (in text) which doesn't appear to print a new line each time with the increase of progress.
Also, take for example raspbi-config in which you can tab/arrow up and down a "menu" and select "OK" and "Cancel" etc.
How does one achieve this?
Can this then be used to do simple text graphics applications in Linux like one can do under Windows?
Any help would be appreciated.
Uberlinc.
Coding in C++.
General non-gui apps which simply "cout" to the stdout.
Only found one or two examples which show a progress bar that prints out to the same line, but are not clear to me.
On a basic level, there are two things that enable these kinds of things on a Linux (or other POSIX) terminal: ASCII control characters and ANSI escape codes.
For a simple progress bar, it's enough to know how wide the screen is and to have a way to get back to the beginning of the current terminal line. This can be one by reading the environment variable $COLUMNS with getenv and by printing the ASCII control character \r. A very bare-bones and unbeautified example is
#include <iostream>
#include <sstream>
#include <string>
#include <unistd.h>
int main() {
// Default screen width
int width = 80;
// Get screenwidth from environment
char const *envwidth = getenv("COLUMNS");
if(envwidth != nullptr) {
std::istringstream(envwidth) >> width;
}
// save some space for numeric display
width -= 10;
for(int i = 1; i < width; ++i) {
std::cout
<< '\r' // go back to start of line
<< std::string(i, '.') // progress bar
<< std::string(width - i, ' ') // padding
<< "| " << i << '/' << width // numeric display
<< std::flush; // make the progress bar instantly visible
// pretend to work
usleep(100000);
}
}
For more involved terminal "graphics,", there are ANSI escape codes. These allow you to set foreground and background color, move around on the screen at will, and a few other things, by printing special character sequences to stdout. For example,
std::cout << "\x1b[10;20HHello, world." << std::flush;
will print Hello, world. at position 10, 20 on the screen.
There is also the ncurses library that provides more high-level terminal UI functionality (windows, dialogs, that sort of thing). They use ANSI escape codes under the hood, but it's normally nicer to use that than to roll your own UI framework.

In C/C++, how do you edit a certain 'coordinate' in stdout?

I've been using Vim a lot lately, and I was wondering how the program manages to change the characters at certain positions in the terminal. For example, when using :rc, it replaces the character under the cursor with c.
I have also seen similar things done with Homebrew, which prints a progress bar to the screen and updates it when necessary.
How is this done in C/C++?
There is no standard way of doing this in C++.
It is done with OS dependent lbiraries, such as curses and similar libraries (ncurses) in the Unix/Linux world. Some of these libraries have been ported on across platforms (example: PDCurses)
For very simple things such as a progress bar or a counter, and as long as you remain on a single line there is the trick of using "\r" (carriage return) in the output, to place the cursor back at the begin of the current line. Example:
for (int i = 0; i < 100; i++) {
cout << "\rProgress: " << setw(3) << i;
this_thread::sleep_for(chrono::milliseconds(100));
}
Certainly, using ncurses or similar library is a good answer. An alternative may be to use ANSI Escape Codes to control the cursor in some terminal emulators (but not Windows command shell). For example, this code prints a line in multiple colors and then moves the cursor to 2,2 (coordinates are 1-based with 1,1 being the upper left corner) and prints the word "red" in the color red.
#include <iostream>
#include <string>
const std::string CSI{"\x1b["};
const std::string BLUE{CSI + "34m"};
const std::string RED{CSI + "31m"};
const std::string RESET{CSI + "0m"};
std::ostream &curpos(int row, int col)
{
return std::cout << CSI << row << ';' << col << 'H';
}
int main()
{
std::cout << "This is " << BLUE << "blue" << RESET << " and white.\n";
curpos(2,2);
std::cout << RED << "red" << RESET << '\n';
}
As mentioned that's not a matter of any C/C++ standard operations provided with stdout or cout (besides writing the necessary control characters to the screen).
Controlling the screen cursor of an ASCII terminal totally depends on implementation of the particular terminal program used, and besides a very narrow set of control characters, there's no standard established.
There are libraries like ncurses for a broader variety of linux terminal implementations, or PDcurses for a windows CMD shell.
I'm not sure to understand you completely but with creating an array of 100 elements of type char you can modify any position of the array and loop it with a std:cout to mostrate it on the console.
Perhaps could be better define the array of 50 chars to resuce the size of the printed result.
For example, if you have to print a progessbar in the 1% process, you should print:
Char progressbar[100] = {'X','','','','','','','','',........}

How to handle the output of terminal command execution from C++, if command doesn't just printing the result, but enters some interactive mode?

This question was inspired by that one.
I've understood how to execute utils from C or C++ code. But how can we get result from some command, that doesn't just printing the result, but enters some interactive mode and works until we press Ctrl+zor something like this? The example of such a command is top.
Usually you don't. You launch the command with options that makes them non interactive.
Technically you could get information from an interactive terminal interface, but you will have a hard time doing it, because in order for the interface to be human like, terminal capabilities are often used (termcaps, ncurses..) which basically works by outputting special characters, so you 'll have to dodge these characters by knowing what is expected when, so except if the interface is quite simple and static (actually even in this case) it's gonna be a pain.
Some applications (such as "dialog") can work interactively and write their final result to a different output stream. In the case of dialog, that is done using stderr (by default). If you have control over the application, you can provide for doing something like that, for passing back information to the calling application.
You simply can access stdin, stdout and stderr. Before you fork create the pipes as needed and fork and after that call execv or any other variant.
An example can be found here:
https://jineshkj.wordpress.com/2006/12/22/how-to-capture-stdin-stdout-and-stderr-of-child-program/
There is also a common used library to capture the output of a child program and react with some new actions on found items. This library was first written for tcl but can also be used for c and c++.
http://expect.sourceforge.net/
With some glue code around the expect lib your source can look like this:
int main()
{
Tcl_Interp *interp = Tcl_CreateInterp();
Expect_Init(interp);
// read from file
int lfd = open( "test.txt", O_RDONLY );
int fd = exp_spawnfd(lfd);
// or read from application
int fd ? exp_spawn("top","top", (char*)0)));
bool cont= true;
Expections set1 =
{
{ exp_glob, "YYY", []( Expection& exp)->void { cout << "found: YYY" << endl; }},
{ exp_regexp, "XXX ([0-9]+)", []( Expection& exp)->void { cout << "found: XXX" << exp.GetResult(1) << endl;}},
{ exp_glob, "END", [&cont]( Expection& exp)->void { cout << "EOF" << endl; cont = false; }}
};
int cnt = 0;
do
{
cout << "Run " << cnt << endl;
cnt++;
Expect ( fd, set1, 2 );
} while ( cont );
cout << "Finish" << endl;
return 0;
}

Embedding Python 3.3 in a C++ program while only able to read one line at a time from input

I am currently working on adding embedded Python support (and yes, extending is not an option) to a large program as part of my summer internship. Ideally, I can keep the Python support within a single .DLL, which currently contains the program's in-house scripting language and is the easiest place to integrate said language and Python.
However, due to the program's API, I only have a single input function to use. This function's return value is a single line from the current input, which could be the console or a file. The input interface cannot (within the .DLL) be converted into a stream object, buffer, or FILE pointer.
My current test code (written outside of the program, using std::string, istream, and getline to ape the restriction) is
// start python
Py_Initialize();
try
{
cout << "Python " << Py_GetVersion() << endl;
string block;
bool in_block = false;
while ( !cin.eof() )
{
string str;
cout << (in_block ? "... " : ">>> "); // prompt string
getline(cin,str);
if ( in_block ) // if in an indented block
{
if ( str.front() != ' ' && str.front() != '\t' ) // if ending the indented block
{
PyRun_SimpleString(block.c_str()); // run Python code contained in block string
block.clear(); // clear string for next block
in_block = false; // no longer in block
}
else // a component of an indented block
{
block += (str+'\n'); // append input to block string
continue; // do not pass block exit code, do not collect two hundred dollars
}
}
// either not in an indented block, or having just come out of one
if ( str.back() == ':' ) // if colon, start new indented block
{
block = (str+'\n');
in_block = true;
continue;
}
else { PyRun_SimpleString(str.c_str()); } // otherwise, run block-free code
}
}
catch ( error_already_set e ) { PyErr_Print(); }
// close python
Py_Finalize();
// done
return 0;
I have not encountered serious problems with this hack, but it strikes me as woefully inelegant. Can anyone think of a better way to do this?
I have already cleared the boost.python libraries with my boss, if that offers some useful trick that eluded me.
EDIT: I should probably mention the program itself, while not my meagre testbed, must run on MS Windows.
What you've written is going to look superficially similar to the stock interpreter, but it won't follow the same indent/dedent and continuation rules for any but the most trivial cases.
The easiest way to embed an interactive interpreter shell is to embed a bare interpreter that runs an interactive interpreter written in pure Python, usually via the code module.
For that to work, you will have to hook up your line-by-line reader to the embedded stdin file object, but it seems like for any realistic use you're going to want that anyway. (Otherwise, what's going to happen if, e.g., the user types input() at the shell?)
An alternative is to set up a pty and run the stock interactive interpreter against that (possibly as a child process), feeding the pty's input pipe from your line reader.

Python embedded in c . Is calling PyRun_SimpleString synchronous?

The application's purpose is to translate lemmas of words present in the sentence from Russian to English. I'm doing it with help of sdict formatted vocabulary, which is queried by python script which is called by c++ program.
My purpose is to get the following output :
Выставка/exhibition::1 конгресс/congress::2 организаторами/organizer::3 которой/ which::4 являются/appear::5 РАО/NONE::6 ЕЭС/NONE::7 России/NONE::8 EESR/NONE::9 нефтяная/oil::10 компания/company::11 ЮКОС/NONE::12 YUKOS/NONE::13 и/and::14 администрация/administration::15 Томской/NONE::16 области/region::17 продлится/last::18 четыре/four::19 дня/day::20
The following code succeeded for the sentence, however for the second sentence and so on I get a wrong output:
Егор/NONE::1 Гайдар/NONE::2 возглавлял/NONE::3 первое/head::4 российское/first::5 правительство/NONE::6 которое/government::7 называли/which::8 правительством/call::9 камикадзе/government::10
Note: NONE is used for words lacking translation.
I'm running the following C++ code excerpt which actually calls PyRun_SimpleString:
for (unsigned int i = 0; i < theSentenceRows->size(); i++){
stringstream ss;
ss << (i + 1);
parsedFormattedOutput << theSentenceRows->at(i)[FORMINDEX] << "/";
getline(lemmaOutFileForTranslation, lemma);
PyObject *main_module, *main_dict;
PyObject *toTranslate_obj, *translation, *emptyString;
/* Setup the __main__ module for us to use */
main_module = PyImport_ImportModule("__main__");
main_dict = PyModule_GetDict(main_module);
/* Inject a variable into __main__, in this case toTranslate */
toTranslate_obj = PyString_FromString(lemma.c_str());
PyDict_SetItemString(main_dict, "start_word", toTranslate_obj);
/* Run the code snippet above in the current environment */
PyRun_SimpleString(pycode);
**usleep(2);**
translation = PyDict_GetItemString(main_dict, "translation");
Py_XDECREF(toTranslate_obj);
/* writing results */
parsedFormattedOutput << PyString_AsString(translation) << "::" << ss.str() << " ";
Where pycode is defined as:
const char *pycode =
"import sys\n"
"import re\n"
"import sdictviewer.formats.dct.sdict as sdict\n"
"import sdictviewer.dictutil\n"
"dictionary = sdict.SDictionary( 'rus_eng_full2.dct' )\n"
"dictionary.load()\n"
"translation = \"*NONE*\"\n"
"p = re.compile('( )([a-z]+)(.*?)( )')\n"
"for item in dictionary.get_word_list_iter(start_word):\n"
" try:\n"
" if start_word == str(item):\n"
" instance, definition = item.read_articles()[0]\n"
" translation = p.findall(definition)[0][1]\n"
" except:\n"
" continue\n";
I've noticed some delay in the second sentence's output, so I added the usleep(2); to C++ while thinking that it happens because calling PyRun_SimpleString is not synchronous. It didn't help, however and I'm not sure that this is the reason. The delay bug happens for sentences that follow and increases.
So, is the call to PyRun_SimpleString synchronous? Maybe, sharing of variable values between C++ and Python is not right?
Thank you in advance.
According to the docs, it is synchronous.
I would advise you to test the python code seperately from the C++ code, that would make debugging it much easier. One way of doing that is pasting the code in the interactive interpreter and executing it line by line. And when debugging, I would second Winston Ewert's comment to not discard exceptions.