Logging Guard to limit semi-constant log messages - c++

I'm using boost log in my application for logging.
However, in some sections of my code I have some log statements that could occur very often if something goes wrong. I'd want some kind of guard that can limit log messages when it detects that the same log message appears constantly.
e.g. (This is a simplified example, not actual implementation)
while(!framebuffer.try_pop(frame))
{
BOOST_LOG(trace) << "Buffer underrun.";
}
If for some reason "framebuffer" doesn't receive any frames for a long time the logging will send way to much log messages.
However I'm unsure what strategy to use for limiting log messages, without loosing any important messages, and how to implement it.

How about something simple, you could encapsulate it if you wanted to:
int tooMany = 10;
int count = 0;
while(!framebuffer.try_pop(frame))
{
if(count < tooMany) {
BOOST_LOG(trace) << "Buffer underrun.";
}
count++;
}
if(count >= tooMany) {
BOOST_LOG(trace) << "Message repeated: " << count << " times.";
}
Just be careful of integer overflows on the 'count' variable if you get a absolute bucketload of increments.

Related

C++ SQLite Serialized Mode Questions

I have been through Quite a few pages, and have an ok Idea of whats happening it think, but I have a few Questions just to be sure....
my program uses the -DTHREADSAFE=1 compile options, forks on receiving a database request (Select, Delete, Insert, Update) from a user or my network, then the child process handles the various database tasks, and relaying of messages should that be required and so on,
at the moment my database is not setup for concurrency which I wont lie is a major design flaw, but that's beside the point at the moment, let's say I have a function that prints all the entries in my table LEDGER as follows...
void PersonalDataBase::printAllEntries()
{
//get all entries
const char query [] = "select * from LEDGER";
sqlite3_stmt *stmt;
int error
try
{
if ((error = sqlite3_prepare(publicDB, query, -1, &stmt, 0 )) == SQLITE_OK)
{
int ctotal = sqlite3_column_count(stmt);
int res = 0;
while ( 1 )
{
res = sqlite3_step(stmt);
if ( res == SQLITE_ROW )
{
Entry *temp = loadBlockRow(stmt);
string from, to;
from = getNameForHash(temp -> from);
to = getNameForHash(temp -> to);
temp -> setFromOrTo(from, 0);
temp -> setFromOrTo(to, 1);
temp -> printEntry();
printlnEnd();
delete temp;
}
else if ( res == SQLITE_DONE || res==SQLITE_ERROR)
{
if (res == SQLITE_ERROR) { throw res; }
sqlite3_finalize(stmt);
break;
}
}
}
//problems
else
{
throw error;
}
}
catch (int err)
{
sqlite3_finalize(stmt);
setupOutput();
cout << "Database Error: " << sqlite3_errmsg(publicDB) << ", Error Code: " << (int) error << endl;
cout << "Did Not Find Values Try Again After Fixing Problems Above." << endl;
printlnEnd();
}
println("Done!");
}
my setupOutput(), printlnEnd(), println(), all help with my use of 'non-blocking' keyboard i/o, they work as I want lets not worry about them here, and think of them as just a call to cout
ok so now at this point I figure there are 4 options...
A while around my try/catch, then in catch check if err = 5, if so I need to setup a sqlite3_busy_handler and have it wait for whatever is blocking the current operation (once it returns SQLITE_OK and have cleaned up all my old variables I reiterate through the while/try again), now as only one of these can be setup at a time, let's say for instance Child1 is doing a large write and child2 and child3 are trying to say read and update concurrently on top of the first child's write, so if a SQLITE_BUSY is returned by this function I print out an error, then restart my while loop (restarting the function), of course after I have finalized my old statement, and cleared up any local objects that may have been created, if this a correct line of thinking?
Should I setup a recursive mutex, say screw it to SQLites own locking mechanism, set it up to be shared across processes then only allow one operation on a database at a time? for using my app on a small scale this doesn't seem to bad of an option, however I'm reading a lot of warnings on using a recursive mutex and am wondering if this is is the best option, as many posts say handle mutual exclusion yourself. however then I cannot have concurrent reads, which is a bit of a pain
Use option 1 but instead of using the SQLite busy handler, just call usleep on a random number, clean up data, and restart while?
before/after any function involving my database use sqlite3_exec() with "BEGIN IMMEDIATE"/"COMMIT" respectively, Locking the database for the duration of the code in between those 2 statements. So that nothing enclosed within can(or at least should) return SQLITE_BUSY, then if my "BEGIN IMMEDIATE" returns BUSY (it should be the only one so long as everything is set up correctly), I use the sqlite3_busy_handler which honestly, if only one process can use it at a time seems annoying... or a random number with usleep(), (presumably at this number is rather large 1mil = 1 second the chance of overlap between 1-20 processes is pretty slim) so each process will constantly try to re lock the database at random intervals for their own purposes
Is there a better way? or which one of these is best?
SQLite's internal busy handler (installed with sqlite3_busy_timeout()) already sleeps a more-or-less random number of times; there is no need to write your own handler.
Using your own locking mechanism would be more efficient than random waiting, but only if you have reader/writer locks.
BEGIN or BEGIN IMMEDIATE ensure that no other statement in the same transaction can run into a lock, but only if IMMEDIATE is used for transactions that write.
To allow concurrent readers and writers, consider using WAL mode. (But this does not allow multiple writers either.)

libwebsocket: unable to write frame bigger than 7160 bytes

I'm addressing an issue with WebSocket that I'm not able to understand.
Please, use the code below as reference:
int write_buffer_size = 8000 +
LWS_SEND_BUFFER_PRE_PADDING +
LWS_SEND_BUFFER_POST_PADDING;
char *write_buffer = new unsigned char[write_buffer_size];
/* ... other code
write_buffer is filled in some way that is not important for the question
*/
n = libwebsocket_write(wsi, &write_buffer[LWS_SEND_BUFFER_PRE_PADDING], write_len,
(libwebsocket_write_protocol)write_mode);
if (n < 0) {
cerr << "ERROR " << n << " writing to socket, hanging up" << endl;
if (utils) {
log = "wsmanager::error: hanging up writing to websocket";
utils->writeLog(log);
}
return -1;
}
if (n < write_len) {
cerr << "Partial write: " << n << " < " << write_len << endl;
if (utils) {
log = "wsmanager-error: websocket partial write";
utils->writeLog(log);
}
return -1;
}
When I try to send data bigger than 7160 bytes I receive always the same error, e.g. Partial write: 7160 < 8000.
Do you have any kind of explanation for that behavior?
I have allocated a buffer with 8000 bytes reserved for the payload so I was expecting to be able to send a maximum amount of data of 8K, but 7160 (bytes) seems to be the maximum amount of data I can send.
Any help is appreciated, thanks!
I have encountered similar problem with an older version of libwebsockets. Although I didn't monitor the limit, it was pretty much the same thing: n < write_len. I think my limit was way lower, below 2048B, and I knew that the same code worked fine with newer version of libwebsockets (on different machine).
Since Debian Jessie doesn't have lws v1.6 in repositories, I've built it from github sources. Consider upgrading, it may help solve your problem. Beware, they have changed api. It was mostly renaming of methods' names from libwebsocket_* to lws_*, but also some arguments changed. Check this pull request which migrates boilerplate libwebsockets server to version 1.6. Most of these changes will affect your code.
We solved the issue updating libwebsockets to 1.7.3 version.
We also optimized the code using a custom callback called when the channel is writable
void
WSManager::onWritable() {
int ret, n;
struct fragment *frg;
pthread_mutex_lock(&send_queue_mutex);
if (!send_queue.empty() && !lws_partial_buffered(wsi)) {
frg = send_queue.front();
n = lws_write(wsi, frg->content + LWS_PRE, frg->len, (lws_write_protocol)frg->mode);
ret = checkWsWrite(n, frg->len);
if (ret >= 0 && !lws_partial_buffered(wsi)) {
if (frg->mode == WS_SINGLE_FRAGMENT || frg->mode == WS_LAST_FRAGMENT)
signalResponseSent();
// pop fragment and free memory only if lws_write was successful
send_queue.pop();
delete(frg);
}
}
pthread_mutex_unlock(&send_queue_mutex);
}

Brain Computer Interface P300 Machine Learning

I am currently working a P300 (basically there is detectable increase in a brain wave when a user sees something they are interested) detection system in C++ using the Emotiv EPOC. The system works but to improve accuracy I'm attempting to use Wekinator for machine learning, using an support vector machine (SVM).
So for my P300 system I have three stimuli (left, right and forward arrows). My program keeps track of the stimulus index and performs some filtering on the incoming "brain wave" and then calculates which index has the highest average area under the curve to determine which stimuli the user is looking at.
For my integration with Wekinator: I have setup Wekinator to receive a custom OSC message with 64 features (the length of the brain wave related to the P300) and set up three parameters with discrete values of 1 or 0. For training I have I have been sending the "brain wave" for each stimulus index in a trial and setting the relevant parameters to 0 or 1, then training it and running it. The issue is that when the OSC message is received by the the program from Wekinator it is returning 4 messages, rather than just the one most likely.
Here is the code for the training (and input to Wekinator during run time):
for(int s=0; s < stimCount; s++){
for(int i=0; i < stimIndexes[s].size(); i++) {
int eegIdx = stimIndexes[s][i];
ofxOscMessage wek;
wek.setAddress("/oscCustomFeatures");
if (eegIdx + winStart + winLen < sig.size()) {
int winIdx = 0;
for(int e=eegIdx + winStart; e < eegIdx + winStart + winLen; e++) {
wek.addFloatArg(sig[e]);
//stimAvgWins[s][winIdx++] += sig[e];
}
validWindowCount[s]++;
}
std::cout << "Num args: " << wek.getNumArgs() << std::endl;
wekinator.sendMessage(wek);
}
}
Here is the receipt of messages from Wekinator:
if(receiver.hasWaitingMessages()){
ofxOscMessage msg;
while(receiver.getNextMessage(&msg)) {
std::cout << "Wek Args: " << msg.getNumArgs() << std::endl;
if (msg.getAddress() == "/OSCSynth/params"){
resultReceived = true;
if(msg.getArgAsFloat(0) == 1){
result = 0;
} else if(msg.getArgAsFloat(1) == 1){
result = 1;
} else if(msg.getArgAsFloat(2) == 1){
result = 2;
}
std::cout << "Wek Result: " << result << std::endl;
}
}
}
Full code for both is at the following Gist:
https://gist.github.com/cilliand/f716c92933a28b0bcfa4
Main query is basically whether something is wrong with the code: Should I send the full "brain wave" for a trial to Wekinator? Or should I train Wekinator on different features? Does the code look right or should it be amended? Is there a way to only receive one OSC message back from Wekinator based on smaller feature sizes i.e. 64 rather than 4 x 64 per stimulus or 9 x 64 per stimulus index.

C++: How Can I keep my program (output console) alive

I am writing a simple program (my 1st program) to display the laptop battery, however, I would like to keep it active to monitor the battery %.:
using namespace std;
int main(int argc, char *argv[]) {
id:
SYSTEM_POWER_STATUS spsPwr;
if (GetSystemPowerStatus(&spsPwr)) {
cout << "\nAC Status : " << static_cast<double>(spsPwr.ACLineStatus)
<< "\nBattery Status : " << static_cast<double>(spsPwr.BatteryFlag)
<< "\nBattery Life % : " << static_cast<double>(spsPwr.BatteryLifePercent)
<< endl;
system("CLS");
goto id;
return 0;
}
else return 1;
}
using goto seems to be a bad idea as the CPU utilization jump to 99% ! :(, I am sure this is not the right way to do it.
Any suggestion?
Thanks
while (true) {
// do the stuff
::Sleep(2000); // suspend thread to 2 sec
}
(you are on Windows according to the API function)
see: Sleep
First of all, the issue you are asking about: of course you get 100% CPU usage, since you're asking the computer to try and get and print the power status of the computer as fast it possibly can. And since computers will happily do what you tell them to, well... you know what happens next.
As others have said, the solution is to use an API that will instruct your application to go to sleep. In Windows, which appears to be your platform of choice, that API is Sleep:
// Sleep for around 1000 milliseconds - it may be slightly more since Windows
// is not a hard real-time operating system.
Sleep(1000);
Second, please do not use goto. There are looping constructs in C and you should use them. I'm not fundamentally opposed to goto (in fact, in my kernel-driver programming days I used it quite frequently) but I am opposed to seeing it used when better alternatives are available. In this case the better alternative is a while loop.
Before I show you that let me point out another issue: DO NOT USE THE system function.
Why? The system function executes the command passed to it; on Windows it happens to execute inside the context of the command interpreter (cmd.exe) which supports and internal command called cls which happens to clear the screen. At least on your system. But yours isn't the only system in the world. On some other system, there might be a program called cls.exe which would get executed instead, and who knows what that would do? It could clear the screen, or it could format the hard drive. So please, don't use the system function. It's almost always the wrong thing to do. If you find yourself looking for that command stop and think about what you're doing and whether you need to do it.
So, you may ask, how do I clear the screen if I can't use system("cls")? There's a way to do it which should be portable across various operating systems:
int main(int, char **)
{
SYSTEM_POWER_STATUS spsPwr;
while (GetSystemPowerStatus(&spsPwr))
{
std::string status = "unknown";
if (spsPwr.ACLineStatus == 0)
status = "offline";
else if (spsPwr.ACLineStatus == 1)
status = "online";
// The percent of battery life left is returned as a value
// between 0 and 255 so we normalize it by multiplying it
// by 100.0 and dividing by 255.0 which is ~0.39.
std::cout << "Current Status: " << status << " ("
<< static_cast<int>(spsPwr.BatteryFlag) << "): "
<< 0.39 * static_cast<int>(spsPwr.BatteryLifePercent)
<< "% of battery remaining.\r" << std::flush;
// Sleep for around 1000 milliseconds - it may be slightly more
// since Windows is not a hard real-time operating system.
Sleep(1000);
}
// Print a new line before exiting.
std::cout << std::endl;
return 0;
}
What this does is print the information in a single line, then move back to the beginning of that line, sleep for around one second and then write the next line, overwriting what was previously there.
If the new line you write is shorter than the previous line, you may see some visual artifacts. Removing them should not be difficult but I'll leave it for you as an exercise. Here's a hint: what happens if you output a space where a letter used to be?
In order to do this across lines, you will need to use more advanced techniques to manipulate the console, and this exercise becomes a lot trickier.
You are having 100% CPU usage because your program is always running.
I don't want to get into details, and given that this is your first program, I'll recommend to put a call to usleep before the goto.
And, of course, avoid goto, use a proper loop instead.
int milliseconds2wait = 3000;
while (!flag_exit) {
// code
usleep( 1000 * milliseconds2wait )
}
Update: This is windows, use Sleep instead of usleep:
Sleep( milliseconds2wait );

Looking for a more efficient way to generate/print a "progress bar" in C++

I am working on some grid generation code, during which I really want to see where I am, so I download a piece of progress bar code from internet and then inserted it into my code, something like:
std::string bar;
for(int i = 0; i < 50; i++)
{
if( i < (percent/2))
{
bar.replace(i,1,"=");
}
else if( i == (percent/2))
{
bar.replace(i,1,">");
}
else
{
bar.replace(i,1," ");
}
}
std::cout<< "\r" "[" << bar << "] ";
std::cout.width( 3 );
std::cout<< percent << "% "
<< " ieration: " << iterationCycle << std::flush;
This is very straightforward. However, it GREATLY slows down the whole process, note percent=iterI/nIter.
I am really get annoyed with this, I am wondering if there is any smarter and more efficient way to print a progress bar to the screen.
Thanks a million.
Firstly you could consider only updating it on every 100 or 1000 iterations. Secondly, I don't think the division is the bottleneck, but much rather the string operations and the outputting itself.
I guess the only significant improvement would be to just output less often.
Oh and just for good measure - an efficient way to only execute the code every, say, 1024 iterations, would be not to see if 1024 is a divisor using the modulo operations, but rather using bitwise calls. Something along the lines of
if (iterationCycle & 1024) {
would work. You'd be computing the bitwise AND of iterationCycle and 1024, only returning positive for every time the bit on the 10th position would be a 1. These kind of operations are done extremely fast, as your CPU has specific hardware for them.
You might be overthinking this. I would just output a single character every however-many cycles of your main application code. Run some tests to see how many (hundreds? millions?), but you shouldn't print more than say once a second. Then just do:
std::fputc('*', stdout);
std::fflush(stdout);
You should really check "efficiency", but what would work almost the same ist boost.progress:
#include <boost/progress.hpp>
...
boost::progress_display pd(50);
for (int i=0; i<=60; i++) {
++pd;
}
and as Joost already answered, output less often