I have a piece of JNI code with a mem leak:
Detected memory leaks!
Dumping objects ->
{76} normal block at 0x277522F8, 52 bytes long.
Data: < "u' "u' "u' > F8 22 75 27 F8 22 75 27 F8 22 75 27 CD CD CD CD
Object dump complete.
So, I set a breakpoint ont the specified memory allocation number (76 in this case).
_crtBreakAlloc = 76;
But the application never stop execution like if that allocation was never performed.
I also took two memory snapshots at the beginnning and at the end of the program, and I compared them.
(At code beginning):
_CrtMemCheckpoint( &s1 );
(At code end):
_CrtMemCheckpoint( &s2 );
_CrtMemState s3;
_CrtMemDifference( &s3, &s1, &s2);
_CrtMemDumpStatistics( &s3 );
Here the result:
0 bytes in 0 Free Blocks.
0 bytes in 0 Normal Blocks.
0 bytes in 0 CRT Blocks.
0 bytes in 0 Ignore Blocks.
0 bytes in 0 Client Blocks.
Largest number used: 2839 bytes.
Total allocations: 101483 bytes.
It seems that all is OK.
I can't figure out what is happening. Is it a false positive mem leak? Or is a memleak of the JVM? If so, is there a way to detect it?
Added after the solution was found:
I modified the initialization of a static map and the problem has been solved.
In particular, I transformed a private static member from map to map*. The problem is that when you initialize a static, it must be initialized with a constant.
Here is how I changed the declaration of the static member:
static const map<wstring, enumValue>* mapParamNames;
So my initialize() method becomes:
map<wstring, paramNames>* m = new map<wstring, paramNames>();
(*m)[L"detectCaptions"] = detectCaptions;
(*m)[L"insertEmptyParagraphsForBigInterlines"] = insertEmptyParagraphsForBigInterlines;
(*m)[L"fastMode"] = fastMode;
(*m)[L"predefinedTextLanguage"] = predefinedTextLanguage;
(*m)[L"detectFontSize"] = detectFontSize;
(*m)[L"saveCharacterRecognitionVariants"] = saveCharacterRecognitionVariants;
(*m)[L"detectBold"] = detectBold;
(*m)[L"saveWordRecognitionVariants"] = saveWordRecognitionVariants;
KernelParamsSetter::mapParamNames = m;
Finally, I inserted the delete of the map in the class destructor:
delete KernelParamsSetter::mapParamNames;
Hope this can be useful for someone.
One possibility would be that memory allocation 76 occurs during the static initialization of a global variable. In that case, you may be setting _crtBreakAlloc too late to catch the allocation.
Related
A program crashes. I use gdb to check, find that a private member of a instance is changed in a very weird way. This variable refreshRank,is only modified at line 283 and 286. I use gdb to watch refreshRank by watch its address, i.e., watch *0x5555559ec278. I get the address by p &refreshRank when I am in a member function of the class.
However, with watch command, gdb says that where refreshRank is modified is before line 594. But it cannot be modified! refreshRank is even not referenced in those lines, as l command gives.
Hardware watchpoint 1: *0x5555559ec278
Old value = 0
New value = 1
DRAMSim::MemoryController::update (this=0x5555559ec020) at MemoryController.cpp:594
594 newTransactionBank, transaction->data, dramsim_log);
(gdb) l
589 totalReads[transaction->core]++;
590 }
591
592 BusPacket *command = new BusPacket(bpType, transaction->address,
593 newTransactionColumn, newTransactionRow, newTransactionRank,
594 newTransactionBank, transaction->data, dramsim_log);
595
596
597
598 commandQueue.enqueue(ACTcommand);
So I wonder, how can this happen? The variable is modified at a place which it cannot be.
As #j6t points out, the last line it executes, totalReads[transaction->core]++;, transaction->core is out of bound. And in the class definition:
uint64_t totalReads[NUM_CPU];
uint64_t totalPrefReads[NUM_CPU];
uint64_t totalWrites[NUM_CPU];
unsigned channelBitWidth;
unsigned rankBitWidth;
unsigned bankBitWidth;
unsigned rowBitWidth;
unsigned colBitWidth;
unsigned byteOffsetWidth;
int totalRead, totalWrite;
unsigned refreshRank;
Those two variables are close.
I am learning lldb and I am curious how you go about setting watchpoints for larger data structures for example a vector. I know that I can use print and that works but I get a message saying that watch points of size "x" are not supported. Is there a way around this? Thanks for the help!
(lldb) s
Process 36110 stopped
* thread #1, queue = 'com.apple.main-thread', stop reason = step in
frame #0: 0x0000000100001600 a.out`main at test.cpp:10
7 vector<int> arr;
8 arr.push_back(1);
9 arr.push_back(2);
-> 10 arr.push_back(3);
11 arr.push_back(4);
12 arr.push_back(5);
13
Target 0: (a.out) stopped.
(lldb) print arr
(std::__1::vector<int, std::__1::allocator<int> >) $2 = size=2 {
[0] = 1
[1] = 2
}
(lldb) w s v arr
error: Watchpoint creation failed (addr=0x7ffeefbff458, size=24, variable expression='arr').
error: watch size of 24 is not supported
If you are on a Mac, the x86_64 architecture allows 4 separate watched regions of at most 8 bytes each. At present, lldb will only use one region per watch request. It could gang multiple watch regions together to handle larger requests which would work for this structure. Feel free to file an enhancement request for this feature with http://bugs.llvm.org. But watchpoints are really limited resources, so you generally have to be very targeted about what you are trying to watch - which is probably why nobody's gotten around to supporting > 8 bytes.
If you want to stop when elements get added to or removed from the vector, it's good enough to watch the end pointer in the vector (i.e. __end_). You can see the actual guts of the vector with the --raw argument to "frame var":
(lldb) fr v --raw arr
(std::__1::vector<int, std::__1::allocator<int> >) arr = {
std::__1::__vector_base<int, std::__1::allocator<int> > = {
__begin_ = 0x0000000100400000
__end_ = 0x000000010040001c
__end_cap_ = {
std::__1::__compressed_pair_elem<int *, 0, false> = {
__value_ = 0x0000000100400038
}
}
}
}
Whenever the vector grows or shrinks, the end marker will get adjusted, so a watchpoint set with:
(lldb) watch set v arr.__end_
Watchpoint created: Watchpoint 1: addr = 0x7ffeefbff1c8 size = 8 state = enabled type = w
declare # '/tmp/vectors.cpp:6'
watchpoint spec = 'arr.__end_'
new value: 0x000000010030020c
will catch push_back, erase, etc.
If you want to stop when the vector values change, you're going to have to watch individual values; given only 32 bytes to play with you're not going to watch all the data in a vector of meaningful size. And of course when the vector resizes, your watchpoint on the old data will now be pointing to freed memory...
I have hard time finding a way to essentially create a buffer that holds a
"key => array of struct" data.
I have 1000+ rows of data.
I cannot change the structure of source data.
The dataset is called often 10+ times/s.
Looping over the data and filtering every time is extremely inefficient.
I do not know size/values ahead of time t0 initialize arrays.
For example the data could have (each row would be a struct):
00100 64 23
01111 22 1
29999 11 54
00100 24 32
29999 44 50
The first value is a char of size 64.
Expected result (mock example):
00100: [{00100 64 23}, {00100 24 32}]
01111: [{01111 22 1}]
29999: [{29999 11 54}, {29999 44 50}]
So i could quickly access, let's say 29999 related data.
In PHP i would just use $buffer[$key][] = $value in a loop. And access $buffer[$key]
Based of some hints in comments i came up with this solutions:
I define a buffer in my class:
map<string, vector<_Item_fld*>> itemBuffer;
Then in a for loop (when file is first loaded):
char * key;
key = pRec->m_Name;
//if key does not exist, init new vector
if (itemBuffer.find(key) == itemBuffer.end()) {
vector<_Item_fld*> tempVector;
itemBuffer[key] = tempVector;
}
else {
itemBuffer[key].push_back(pRec);
}
pRec->m_Name is a char with length 64 in my case.
pRec is a pointer to a single loaded line of type _Item_fld.
Can access buffer by using:
char * key;
key = data->m_Name;
vector<_Item_fld*> itemRows = g_Main.m_tblItem.itemBuffer[key];
for(unsigned i = 0; i < itemRows.size(); i++)
{
_Item_fld* pRec = itemRows [i];
}
I've got a small program on my Arduino, in which I use the HashMap lib from Wiring framework.
Anyways, the HashMap is of key type char *, and value type char * as well.
I read some keys and values from a file and I try to add them to the map. I read them as Strings, and then I use the .c_str() method to convert them into char *.
When I then print the map, I get very weird values, which I think might be garbage from the memory?
Anyone knows how can I fix that, and why that happens? I was thinking it might have to do something with the temporary values in the memory returned by .c_str(), but I'm not sure. Any help?
Here is the code: (st is just a string and I split it into two, where the : is).
String msgC = st.substring(0, st.indexOf(":") - 1);
const char* messageCode = msgC.c_str();
String msg = st.substring(st.indexOf(":") + 1);
const char* message = msg.c_str();
hashMap[messageCode] = message;
When I print the 'message' and 'messageCode' into the console, they are fine.
Output when I print the hashmap:
Key: rently wiping, hig& Value: B0 D8 0 BE A0 9B 86:Wipers currently wiping, hig&
Key: Value: ⸮pers currently wiping, high speed.
Key: Value: ⸮
EDIT:
When I use String types of the hashMap key and values, I get more strange behavior:
if (dataFile) {
String st;
while (dataFile.available() > 0) {
char s = char(dataFile.read());
if (s!='\n'){
st = st+s;
}
else{
String msgC = st.substring(0, st.indexOf(":") - 1);
String msg = st.substring(st.indexOf(":") + 1);
Serial.print(msgC + " " + msg);
hashMap[msgC] = msg;
st = "";
}
}
Now the first three pairs of msgC and msg are read and stored fine. But the rest get split mid sentence or are not loaded at all. The output from the Serial.print:
1D0 834 B0 D8 0 A0 A0 9B 8 Wipers are off.
1D0 834 B0 D8 0 A8 A0 9B 8 Wipers on, single wipe.
1D0 834 B0 D8 0 A1 A0 9B 8 Wipers are on, not wiping.
(Up until here it's ok)
currently wiping, low speed.
currently wiping, low speed.
tly wiping, medium speed.
tly wiping, medium speed.
urrently wiping, high speed.
urrently wiping, high speed.
Weird stuff. And this did not happen when I used char * in the HashMap, and I did not change anything else in the code. How come that changing the type of HashMap keys and values could affect this reading from a file?
You are adding only pointer to data into hashmap:
String msg = st.substring(st.indexOf(":") + 1);
const char* message = msg.c_str();
hashMap[messageCode] = message;
Once msg is destroyed, the memory pointed by message becomes free for usage, so it is Undefined Behavior.
Your hash map should actually contains String values instead of char*.
In the C++ Standard std:string follows an exponential growth policy, therefore I suppose the capacity() of string during concatenation will always be increased when necessary. However, when I test test.cpp, I found that in the for-loop, only every two times will the capacity() be shrunk back to length() during assignment.
Why isn't this behavior depending on the length of string, but depending on how frequent I change the string? Is it some kind of optimization?
The following codes are tested with g++ -std=c++11.
test.cpp:
#include <iostream>
int main(int argc, char **argv) {
std::string s = "";
for (int i = 1; i <= 1000; i++) {
//s += "*";
s = s + "*";
std::cout << s.length() << " " << s.capacity() << std::endl;
}
return 0;
}
And the output will be like this:
1 1
2 2
3 4
4 4
5 8
6 6 // why is capacity shrunk?
7 12
8 8 // and again?
9 16
10 10 // and again?
11 20
12 12 // and again?
13 24
14 14 // and again?
15 28
16 16 // and again?
17 32
...
996 996
997 1992
998 998 // and again?
999 1996
1000 1000 // and again?
When you do this:
s = s + "*";
You're doing two separate things: making a new temporary string, consisting of "*" concatenated onto the end of the contents s, and then copy-assigning that new string to s.
It's not the + that's shrinking, it's the =. When copy-assigning from one string to another, there's no reason to copy the capacity, just the actual used bytes.
Your commented-out code does this:
s += "*";
… is only doing one thing, appending "*" onto the end of s. So, there's nowhere for the "optimization" to happen (if it happened, it would be a pessimization, defeating the entire purpose of the exponential growth).
It's actually not convered by the C++ standard what happens to capacity() when strings are moved, assigned, etc. This could be a defect. The only constraints are those derivable from the time complexity specified for the operation.
See here for similar discussion about vectors.