I'm debugging a very old application (Visual C++ 6) due to a crash. I have isolated the crash into a while loop. This app reads a bunch of strings from a database and outputs them into a file for processing.
This while loop appears to be looping through all strings retrieved from the DB and wrapping them in a delimiter/end character. However, the while loop will randomly crash.
I have a particular set of data containing around 2000 strings, and at some point, during processing, the loop will crash (or give a debug assertion error in debug mode).
What's strange is that this loop does not stop in the same location each time (bad string was my first thought), nor is this a common occurrence. This app has processed hundreds of thousands of rows over the years, and yet it chokes on this particular data set somewhat randomly.
I added some logging code initially (which I've stripped out, and it appears that the crash is occurring on the actual execution/evaluation of the WHILE line.
Is there anything, in particular, I should look for? I was thinking perhaps this is an index out of bounds error, but I would think it would happen consistently in the same spot with the same data set if that were the case. Does anyone see anything with this code I should double-check, or can offer any advice on how to better troubleshoot this?
//Definitions
CStringList m_stlMessageList;
CStringList m_stlMessageIDList;
//Populating list
rec.Open(CRecordset::forwardOnly, strSQL);
while (!rec.IsEOF())
{
rec.GetFieldValue("string_id", strMessageID);
strMessageID.TrimRight();
rec.GetFieldValue("string_text", strMessage);
strMessage.TrimRight();
rec.GetFieldValue("string_destination", strDestination);
strDestination.TrimRight();
rec.GetFieldValue("string_ip", strIP);
strIP.TrimRight();
//Add to the Lists...
m_stlMessageIDList.AddTail(strMessageID);
m_stlMessageList.AddTail(strMessage);
rec.MoveNext();
}
//Code thats crashing.
CString strSegment, strTemp, strTmp;
BOOL bFirst = TRUE;
POSITION POS = m_stlMessageList.GetHeadPosition();
while (POS)
{
strSegment = m_stlMessageList.GetNext(POS);
strTemp = strSegment.Left(4);
strSegment.Format(strSegment + "%c", 13);
if (POS)
{
strTmp = m_stlMessageList.GetPrev(POS);
m_stlMessageList.SetAt(POS, strSegment);
strTmp = m_stlMessageList.GetNext(POS);
}
else
{
m_stlMessageList.RemoveTail();
m_stlMessageList.AddTail(strSegment);
}
}
Related
Completely new to C++.
I'm comparing various aspects of C++, C# and Ruby to see if there's need for mirroring a library. Currently, simple read of a file (post update).
Compiling C++ and C# in VS 2017. C++ is in release(x64) mode (or at least compile then run)
The libraries more or less read a file and split the lines into three which make up the members of an object which are then stored in an array member.
For stress testing I tried a large file 380MB(7M lines) (after update) now getting similar performance with C++ and Ruby,
Purely reading the file and doing nothing else the performance is as below:
Ruby: 7s
C#: 2.5s
C++: 500+s (stopped running after awhile, something's clearly wrong)
C++(release build x64): 7.5s
The code:
#Ruby
file = File.open "test_file.txt"
while !file.eof
line = file.readline
end
//C#
StreamReader file = new StreamReader("test_file.txt");
file.Open();
while((line = file.ReadLine()) != null){
}
//C++
#include "stdafx.h"
#include "string"
#include "iostream"
#include "ctime"
#include "fstream"
int main()
{
std::ios::sync_with_stdio(false);
std::ifstream file;
file.open("c:/sandboxCPP/test_file.txt");
std::string line;
std::clock_t start;
double duration;
start = std::clock();
while (std::getline(file, line)) {
}
duration = (std::clock() - start) / (double)CLOCKS_PER_SEC;
std::cout << "\nDuration: " << duration;
while (true)
{
}
return 0;
}
Edit: The following performed incredibly well. 0.03s
vector<string> lines;
string tempString = str.str();
boost::split(lines, tempString, boost::is_any_of("\n"));
start = clock();
cout << "\nCount: " << lines.size();
int count = lines.size();
string s;
for (int i = 0; i < count; i++) {
s = lines[i];
}
s = on the likelihood that I don't know what boost is doing. Changed performance.
Tested with a cout of a random record at the end of the loop.
Thanks
Based on the comments and the originally posted code (it has now been fixed [now deleted]) there was previously a coding error (i++ missing) that stopped the C++ program from outputting anything. This plus the while(true) loop in the complete code sample would present symptoms consistent with those stated in the question (i.e. user waits 500s sees no output and force terminates the program). This is because it would complete reading the file without outputting anything and enter into the deliberately added infinite loop.
The revised complete source code correctly completes (according to the comments) in ~1.6s for a 1.2 million file. My advice for improving performance would be as follows:
Make sure you are compiling in release mode (not debug mode). Given the user has specified they are using Visual Studio 2017, I would recommend viewing the official Microsoft documentation (https://msdn.microsoft.com/en-us/library/wx0123s5.aspx) for a thorough explanation.
To make it easier to diagnose problems do not add an infinite loop at the end of your program. Instead run the executable from powershell / (cmd) and confirm that it terminates correctly.
EDIT: I would also add:
For accurate timings you also need to take into account the OS disk cache. Run each benchmark multiple times to 'warm-up' the disk cache.
C++ doesn’t automatically write everything the instant you tell it to. Instead, it buffers the data so it can write it all at once, which is usually faster. To say “I really want to write this now.”, you need to say something like std::cout << std::flush (if you use std::endl to end your lines it does this automatically).
Usually you don’t need to do this; the buffers are flushed when the program exits, or when you ask for input from the user, or things like that. However, your program doesn’t exit, so it never flushes its buffer. You read the input, and then the program is executing while(true) forever, never giving the output.
The solution to this is simple: remove the while loop at the end of the program. You should not have that; people usually assume a console program exits when it’s finished. I would’ve guessed you had that because Visual Studio automatically closed the console window when the program was finished, but apparently it doesn’t do that with Ctrl+F5, which you use, so I’m not sure.
I have a static method that searches (and returns) into String msg the value between a TAG
this is the code function:
static String genericCutterMessage(String TAG, String msg){
Serial.print("a-----");
Serial.println(msg);
Serial.print("b-----");
Serial.println(TAG);
if(msg.indexOf(TAG) >= 0){
Serial.print("msg ");
Serial.println(msg);
int startTx = msg.indexOf(TAG)+3;
int endTx = msg.indexOf(TAG,startTx)-2;
Serial.print("startTx ");
Serial.println(startTx);
Serial.print("endTx ");
Serial.println(endTx);
String newMsg = msg.substring(startTx,endTx);
Serial.print("d-----");
Serial.println(newMsg);
Serial.println("END");
Serial.println(newMsg.length());
return newMsg;
} else {
Serial.println("d-----TAG NOT FOUND");
return "";
}
}
and this is output
a-----[HS][TS]5132[/TS][TO]5000[/TO][/HS]
b-----HS
msg [HS][TS]5132[/TS][TO]5000[/TO][/HS]
startTx 4
endTx 30
d-----
END
0
fake -_-'....go on! <-- print out of genericCutterMessage
in that case I want return the string between HS tag, so my expected output is
[TS]5132[/TS][TO]5000[/TO]
but I don't know why I receive a void string.
to understand how substring works I just followed tutorial on official Arduino site
http://www.arduino.cc/en/Tutorial/StringSubstring
I'm not an expert in C++ and Arduino but this looks like a flushing or buffering problem, isn't it?
Any idea?
Your code is correct, this should not happen. Which forces you to consider the unexpected ways that this could possibly fail. There is really only one candidate mishap I can think of, your Arduino is running out of RAM. It has very little, the Uno only has 2 kilobytes for example. It doesn't take a lot of string munching to fill that up.
This is not reported in a smooth way. All I can do is point you to the relevant company page. Quoting:
If you run out of SRAM, your program may fail in unexpected ways; it will appear to upload successfully, but not run, or run strangely. To check if this is happening, you can try commenting out or shortening the strings or other data structures in your sketch (without changing the code). If it then runs successfully, you're probably running out of SRAM. There are a few things you can do to address this problem:
If your sketch talks to a program running on a (desktop/laptop) computer, you can try shifting data or calculations to the computer, reducing the load on the Arduino.
If you have lookup tables or other large arrays, use the smallest data type necessary to store the values you need; for example, an int takes up two bytes, while a byte uses only one (but can store a smaller range of values).
If you don't need to modify the strings or data while your sketch is running, you can store them in flash (program) memory instead of SRAM; to do this, use the PROGMEM keyword.
That's not very helpful in your specific case, you'll have to look at the rest of the program for candidates. Or upgrade your hardware, StackExchange has a dedicated site for Arduino enthusiasts, surely the best place to get advice.
I am currently stuck in this problem that I do not have any idea to fix. It is regarding a previous question that I have asked here before. But I will reiterate again as I found out the problem but have no idea to fix it.
My program accesses a text file that is updated constantly every millisecond 24/7. It grabs the data line by line and does comparison on each of the line. If any thing is "amiss"(defined by me), then I log that data into a .csv file. This program can be run at timed intervals(user defined).
My problem is that this program works perfectly fine on my computer but yet it doesnt on my clients computer. I have debug the program and these are my findings. Below is my code that I have reduced as much possible to ease the explanation process
int result;
char ReadLogLine[100000] = "";
FILE *readLOG_fp;
CString LogPathName;
LogPathName = Source_Folder + "\\serco.log"; //Source_Folder is found in a .ini file. Value is C:\\phython25\\xyratex\\serco_logs
readLOG_fp = fopen(LogPathName, "r+t");
while ((result = fscanf(readLOG_fp, "%[^\n]\n", ReadLogLine)) != EOF) // Loops through the file till it reaches the end of file
{
Sort_Array(); // Here is a function to sort the different lines that I grabbed from the text file
Comp_State(); // I compare the lines here and store them into an array to be printed out
}
fclose(readLOG_fp);
GenerateCSV(); // This is my function to generate the csv and print everything out
In Sort_Array(), I sort the lines that I grab from the text file as they could be of different nature. For example,
CStringArray LineType_A, LineType_B, LineTypeC, LineTypeD;
if (ReadLogLine == "Example line a")
{
LineType_A.add(ReadLogLine);
}
else if (ReadLogLine == "Example line b")
{
LineType_B.add(ReadLogLine);
}
and so on.
In CompState(), I compare the different values within each LineType array to see if there are any difference. If it is different, then I store them into a seperate array to print. A simple example would be.
CStringArray PrintCSV_Array;
for (int i = 0; i <= LineType_A.GetUpperBound(); i++)
{
if (LineType_A.GetAt(0) == LineType_A.GetAt(1))
{
LineType_A.RemoveAt(0);
}
else
{
LineType_A.RemoveAt(0);
PrintCSV_Array.Add(LineType_A.GetAt(0);
}
}
This way I dont have an infinite amount of data in the array.
Now to the GenerateCSV function, it is just a normal function where I create a .csv file and print whatever I have in the PrintCSV_Array.
Now to the problem. In my client's computer, it seems to not print anything out to the CSV. I debugged the program and found out that it keeps failing here.
while ((result = fscanf(readLOG_fp, "%[^\n]\n", ReadLogLine)) != EOF) // Loops through the file till it reaches the end of file
{
Sort_Array(); // Here is a function to sort the different lines that I grabbed from the text file
Comp_State(); // I compare the lines here and store them into an array to be printed out
}
fclose(readLOG_fp);
It goes into the while loop fine as I did some error checking there in the actual program. The moment it goes into the while loop it breaks out of it suggesting to me it reach EOF for some reason. When that happens, the program has no chance to go into both the Sort_Array and Comp_State functions thus giving me a blank PrintCSV_Array and nothing to print out.
Things that I have checked is that
I definitely have access to the text file.
My thoughts were because the text file is updated every
millisecond, it may have been opened by the other program to write
into it and thus not giving me access OR the text file is always in
an fopen state therefore not saving any data in for me to read. I
tested this out and the program has value added in as I see the KB's
adding up in front of my eyes.
I tried to copy the text file and paste it somewhere else for my
program to read, this way I definitely have full access to it and
once I am done with it, Ill delete it. This gave me nothing to print
aswell.
Am I right to deduce that it is always giving me EOF thus this is
having problems.
while ((result = fscanf(readLOG_fp, "%[^\n]\n", ReadLogLine)) != EOF) // Loops through the file till it reaches the end of file
If yes, How do I fix this?? What other ways can I make it read every line. I have seriously exhausted all my ideas on this problem and need some help in this.
Thanks everyone for your help.
Error is very obvious ... you might have over looked it..
You forgot to open the file.
FILE *readLOG_fp;
CString LogPathName;
LogPathName = Source_Folder + "\\serco.log";
readLOG_fp = fopen(LogPathName.GetBuffer());
if(readLOG_fp==NULL)
cout<<"Error: opening file\n";
My program only sometimes gets a Segmentation fault: 11 and I can't figure it out for the life of me. I don't know a whole lot in the realm of C++ and pointers, so what kinds of things should I be looking for?
I know it might have to do with some function pointers I'm using.
My question is what kinds of things produce Segmentation faults? I'm desperately lost on this and I have looked through all the code I thought could cause this.
The debugger I'm using is lldb and it shows the error being in this code segment:
void Player::update() {
// if there is a smooth animation waiting, do this one
if (queue_animation != NULL) {
// once current animation is done,
// switch it with the queue animation and make the queue NULL again
if (current_animation->Finished()) {
current_animation = queue_animation;
queue_animation = NULL;
}
}
current_animation->update(); // <-- debug says program halts on this line
game_object::update();
}
current_animation and queue_animation are both pointers of class Animation.
Also to note, within Animation::update() is a function pointer that gets passed to Animation in the constructor.
If you need to see all of the code, it's over here.
EDIT:
I changed the code to use a bool:
void Player::update() {
// if there is a smooth animation waiting, do this one
if (is_queue_animation) {
// once current animation is done,
// switch it with the queue animation and make the queue NULL again
if (current_animation->Finished()) {
current_animation = queue_animation;
is_queue_animation = false;
}
}
current_animation->update();
game_object::update();
}
It didn't help anything because I still sometimes get a Segmentation fault.
EDIT 2:
Modified code to this:
void Player::update() {
// if there is a smooth animation waiting, do this one
if (is_queue_animation) {
std::cout << "queue" << std::endl;
// once current animation is done,
// switch it with the queue animation and make the queue NULL again
if (current_animation->Finished()) {
if (queue_animation != NULL) // make sure this is never NULL
current_animation = queue_animation;
is_queue_animation = false;
}
}
current_animation->update();
game_object::update();
}
Just to see when this function would output without any user input. Every time I got a Segmentation fault this would output twice right before the fault. This is my debug output:
* thread #1: tid = 0x1421bd4, 0x0000000000000000, queue = 'com.apple.main-thread, stop reason = EXC_BAD_ACCESS (code=1, address=0x0)
frame #0: 0x0000000000000000
error: memory read failed for 0x0
Some causes of segmentation fault:
You dereference a pointer that is uninitialized or that points to NULL
You dereference a deleted pointer
You write outside the bounds of the scope of allocated memory (e.g. after the last element of an array)
Run valgrind with your software (warning, it really slows things down). Its likely that memory has been overwritten in some way. Valgrind (and other tools) can help track down some of these kinds of issues, but not everything.
If its a large program, this could get very difficult as everything is suspect since anything can corrupt anything in memory. You might try to minimize the code paths run by limiting the program in some way and see if you can make the problem happen. This can help reduce the amount of suspect code.
If you have a previous version of the code that didn't have the problem, see if you can revert back to that and then look to see what changed. If you are using git, it has a way to bisect search into the revision where a failure first occurred.
Warning, this kind of thing is the bane of C/C++ developers, which is one of the reason that languages such as Java are "safer".
You might just start looking through the code and see if you can find things that look suspicious, including possible race conditions. Hopefully this won't take to much time. I don't want to freak you out, but these kinds of bugs can be some of the most difficult to track down.
I have a function that creates and insert some numbers in a vector.
if(Enemy2.dEnemy==true)
{
pt.y=4;
pt.x=90;
pt2.y=4;
pt2.x=125;
for(int i=0; i<6; i++)
{
Enemy2.vS1Enemy.push_back(pt);
Enemy2.vS2Enemy.push_back(pt2);
y-=70;
pt.y=y;
pt2.y=y;
}
Enemy2.dEnemy=false;
Enemy3.cEnemy=0;
}
It should insert 6 numbers in two vectors, the only problem is that it doesn't - it actually inserts more.
I don't think the snippet will run unless Enemy2.dEnemy == true, and it won't stay true for ever.
The first time the snippet runs, then Enemy2.dEnemy is set to false and it shouldn't run again.
I don't set Enemy2.dEnemy to true anywhere except when the window is created.
If I insert a break point any where in the snippet, the program will work fine - it will insert ONLY 6 numbers in the two vectors.
Any ideas what's wrong here?
ok so i did some debugging.
i found that Enemy2.dEnemy=false; is being skipped for some reason.
i tried to do this to see if it was.
if(Enemy2.dEnemy)
{
pt.y=4;
pt.x=90;
pt2.y=4;
pt2.x=125;
for(int i=0; i<6; i++)
{
Enemy2.vS1Enemy.push_back(pt);
Enemy2.vS2Enemy.push_back(pt2);
y-=70;
pt.y=y;
pt2.y=y;
}
TCHAR s[244];
Enemy2.dEnemy=false;
if(Enemy2.dEnemy)
{
MessageBox(hWnd, _T("0"), _T(""), MB_OK);
}
else
{
MessageBox(hWnd, _T("1"), _T(""), MB_OK);
}
Enemy3.cEnemy=0;
}
well the message box popped saying 1 and my code worked fine. it seems that Enemy2.dEnemy=false; doesn't have time to run ;/
blahblahblahblahblahblahblahblahblahblahblahblahblahblahblahblah!
ok i found where is the real problem which was causing to insert more than 6 numbers..
it was where i was asigning Enemy2.dEnemy=true;
if(Enemy2.e1)
{
Enemy2.now=time(NULL);
Enemy2.tEnemy=Enemy2.now+4;
Enemy2.e1=false;
}
if(Enemy2.tEnemy==time(NULL))
{
check=1;
Enemy2.aEnemy=0;
Enemy2.dEnemy=true;
}
the problem seems that the second if runs more than one time, which is weird!
First things first: get rid of that abominable if (Enemy2.dEnemy == true) - it should be:
if (Enemy2.dEnemy)
(I also prefer to name my booleans as a readable sentence segments like Enemy2.isABerserker or Enemy3.hasHadLeftLegCutOffThreeInchesBelowTheKnee but that's just personal preference).
Other than that, the only thing I can suggest is a threading problem. There's nothing wrong with that code per se, but there is a window in which two threads could enter the if statement and both start pushing values into your vector.
In other words, if thread 1 is doing the pushing when thread 2 encounters the if statement, thread 2 will also start pushing values, since thread 1 has yet to set dEnemy to true. And don't think you can just move the assignment to the top of the if block - that will reduce but not remove the window.
My advice is to print out the contents of the vectors in the situation where they have more than six entries and that may give a clue as to what's happened (post the output here if you wish).
Re your update that the second if below is running twice:
if(Enemy2.e1)
{
Enemy2.now=time(NULL);
Enemy2.tEnemy=Enemy2.now+4;
Enemy2.e1=false;
}
if(Enemy2.tEnemy==time(NULL))
{
check=1;
Enemy2.aEnemy=0;
Enemy2.dEnemy=true;
}
If this code is executed twice in the same second (and that's not beyond the bounds of possibility), the second if statement will run twice.
That's because time(NULL) give you the number of seconds since the epoch so, until that second is over, you may well be executing the contents of that if thousands of times (or more).
If this problem disappears when you put in a breakpoint or a diagnostic output message, that's a strong clue that the problem is undefined behavior, which is usually caused by something like dereferencing an uninitialized pointer or careless use of const_cast.
The cause of the problem probably has nothing to do with the code you're looking at. It's caused somewhere else and just happens to show up here. It's like someone being hit by a falling brick: the obvious symptom is a man lying unconscious on the sidewalk, but the real problem has nothing to do with the man or the sidewalk, it's several stories up.
If you want to find the cause of the error, remove your diagnostics until the problem reappears, then start removing everything else. Prune away all of the other code. Whenever the error stops, back up until it starts again; if you don't see the cause of the error, start pruning somewhere else. Eventually the bug will have nowhere to hide.