I am trying to understand the output of !heap -s command. I understand that each process has a default heap where all allocations are been done. An app can create its own heap. Does each row in this output show a different heap? If so, does it mean that app has created so many heap or can windows also create multiple heaps?
0:000> !heap -s
Heap Flags Reserv Commit Virt Free List UCR Virt Lock Fast
(k) (k) (k) (k) length blocks cont. heap
-------------------------------------------------------------------------------------
Virtual block: 000000000a790000 - 000000000a790000 (size 0000000000000000)
Virtual block: 000000000ac50000 - 000000000ac50000 (size 0000000000000000)
0000000000430000 00000002 48256 31976 48256 558 512 7 2 5 LFH
0000000000010000 00008000 64 4 64 1 1 1 0 0
0000000000680000 00001002 1088 368 1088 9 5 2 0 0 LFH
00000000005e0000 00041002 512 8 512 3 1 1 0 0
0000000000380000 00001002 1088 408 1088 5 5 2 0 0 LFH
0000000000840000 00041002 512 16 512 0 1 1 0 0
0000000000d00000 00001002 512 340 512 3 8 1 0 0 LFH
00000000003a0000 00041002 512 8 512 3 1 1 0 0
000000000c3d0000 00001002 512 344 512 3 22 1 0 0 LFH
000000000c5d0000 00001002 512 336 512 5 10 1 0 0 LFH
0000000000c80000 00001002 64 8 64 3 1 1 0 0
000000000c7d0000 00001002 64 8 64 3 1 1 0 0
000000000b770000 00011002 512 32 512 19 7 1 0 0
0000000000b70000 00001002 1088 368 1088 8 9 2 0 0 LFH
000000000d980000 00001002 512 8 512 3 2 1 0 0
000000000db60000 00001002 64448 37556 64448 34615 196 27 0 0 LFH
External fragmentation 92 % (196 free blocks)
000000000f4b0000 00001002 3136 1928 3136 1198 39 3 0 0 LFH
External fragmentation 62 % (39 free blocks)
0000000015780000 00001002 64064 41784 64064 20339 308 16 0 8 LFH
External fragmentation 48 % (308 free blocks)
000000000e360000 00001002 512 8 512 3 1 1 0 0
-------------------------------------------------------------------------------------
Yes, each row represents a different heap. Any code running in your process can create a heap by calling HeapCreate(). That code includes MS DLLs (such as msvcrt.dll), third party DLLs and your own code.
Related
I'm trying to investigate memory leak in a particular process. For that I'm trying to investigate memory footprints using below command.
adb shell perfdump meminfo .
The output i'm getting is
MEMORY OF /usr/bin/PuffinApp (pid 2376)
TOTAL MEMORY USAGE (kB):
Pss 61937
SwapPss 11576
Graphics 0
------
TOTAL (kB) 73513
OTHER MEMORY STATS (kB):
Vss 2276204
Rss 68440
Uss 60120
CachedPss 8149
NonCachedPss 53788
Swap 11576
SwapUss 11576
PROCESS STATS:
Maj faults 21475
Min faults 2453006
Threads 258
PROCESS MAPS:
PSS SwapPSS TotalPSS Private Private Shared Shared Referenced Name
Clean Dirty Clean Dirty
------ ------ ------ ------ ------ ------ ------ ------ ------
45660 4952 50612 64 45596 0 0 40924 [anon:rw-p]
4324 2076 6400 108 4216 0 0 3984 [heap]
1352 2184 3536 0 1352 0 0 1192 [anon:rwxp]
36 0 36 20 16 0 0 36 [anon:-w-p]
4 28 32 0 4 0 0 4 [stack]
1980 48 2028 1800 180 0 0 1980 /usr/lib/libavcodec.so.58.18.100
1276 108 1384 1128 148 0 0 1276 /usr/lib/libpryon.so
1112 156 1268 1020 92 0 0 1048 /usr/lib/libReggaeWidevine.so
774 4 778 432 88 576 0 1096 /usr/lib/libcrypto.so.1.1
552 0 552 524 20 16 0 552 /usr/lib/libReggaeMediaLib.so
500 0 500 0 468 0 64 532 /dev/shm/puffin-micStream
336 32 368 252 84 0 0 336 /usr/lib/libavformat.so.58.12.100
356 4 360 308 48 0 0 340 /usr/bin/PuffinApp
252 16 268 244 8 0 0 72 /usr/lib/libxml2.so.2.9.13
0 248 248 0 0 0 0 0 /usr/lib/libavfilter.so.7.16.100
219 4 223 172 28 76 0 276 /usr/lib/libssl.so.1.1
202 0 202 0 0 0 404 404 /dev/shm/audio_playback_stream_2
164 0 164 140 12 24 0 176 /usr/lib/libreggae-core.so
106 16 122 4 28 220 0 244 /usr/lib/libPuffinExternalCapabilityAPI.so
92 28 120 84 8 0 0 92 /usr/lib/libavutil.so.56.14.100
114 0 114 0 20 656 0 436 /usr/lib/libsqlite3.so.0.8.6
101 0 101 0 16 520 0 520 /usr/lib/libAVSCommon.so
93 0 93 16 12 228 0 256 /usr/lib/libcurl.so.4.7.0
80 4 84 76 4 0 0 80 /usr/lib/libtvm_modelops_ww_rhea_v3_50target_cnet.so
------ ------ ------ ------ ------ ------ ------ ------
61856 11576 73432 7036 53084 7676 640 62368 TOTAL
I'm not able understand which components should I focus on to find memory leaks.Also the sum of components is not equal to the total PSS mentioned i.e 61856. I think totalPss can help but of which component.Should i have to focus on .so also other than heap Pss ? Also is it possible that cachedPss will contributes to memory leaks.
Can someone help me in understanding the output.
I'm trying to do memory profiling on linux for that I'm using command adb shell perfdump meminfo . I'm not able to understand the output for PSS column as totalPSS is not equal to sum of individual PSS values. Below is the process map I got after running the above command.
MEMORY OF /usr/bin/PuffinApp (pid 2376)
TOTAL MEMORY USAGE (kB):
Pss 66695
SwapPss 11388
Graphics 0
------
TOTAL (kB) 78083
OTHER MEMORY STATS (kB):
Vss 2250440
Rss 76560
Uss 64132
CachedPss 12371
NonCachedPss 54324
Swap 11388
SwapUss 11388
PROCESS STATS:
Maj faults 33592
Min faults 4533273
Threads 253
PROCESS MAPS:
PSS SwapPSS TotalPSS Private Private Shared Shared Referenced Name
Clean Dirty Clean Dirty
------ ------ ------ ------ ------ ------ ------ ------ ------
45752 5208 50960 68 45684 0 0 45524 [anon:rw-p]
4452 1948 6400 224 4228 0 0 4380 [heap]
1612 1988 3600 0 1612 0 0 1568 [anon:rwxp]
36 0 36 20 16 0 0 36 [anon:-w-p]
4 28 32 0 4 0 0 4 [stack]
2116 108 2224 1968 148 0 0 1716 /usr/lib/libpryon.so
1984 4 1988 1936 48 0 0 1104 /usr/bin/PuffinApp
832 0 832 804 20 16 0 516 /usr/lib/libReggaeMediaLib.so
756 32 788 624 84 96 0 384 /usr/lib/libavformat.so.58.12.100
484 156 640 392 92 0 0 400 /usr/lib/libReggaeWidevine.so
522 48 570 280 180 124 0 568 /usr/lib/libavcodec.so.58.18.100
500 0 500 0 468 0 64 532 /dev/shm/puffin-micStream
344 0 344 332 12 0 0 136 /usr/lib/libLocaleWakewordAdapter.so
338 4 342 0 88 1148 0 1236 /usr/lib/libcrypto.so.1.1
280 8 288 276 4 0 0 132 /usr/lib/libSpotifyAdapter.so
264 0 264 184 12 136 0 208 /usr/lib/libreggae-core.so
247 8 255 8 28 976 0 784 /usr/lib/libPuffinExternalCapabilityAPI.so
4 248 252 4 0 0 0 4 /usr/lib/libavfilter.so.7.16.100
222 4 226 200 0 68 0 268 /usr/lib/libopus.so.0.8.0
202 0 202 0 0 0 404 404 /dev/shm/audio_playback_stream_2
176 16 192 168 8 0 0 96 /usr/lib/libxml2.so.2.9.13
183 0 183 120 4 128 0 64 /usr/lib/libDefaultClient.so
142 0 142 88 12 172 0 256 /usr/lib/libacsdkAudioPlayer.so
140 0 140 136 4 0 0 44 /usr/lib/libacsdkVisualCharacteristics.so
------ ------ ------ ------ ------ ------ ------ ------
66613 11388 78001 10728 53404 11912 512 71972 TOTAL
If i calculate the sum of individual PSS (1st column ) I will get 61492 which is around 5MB lesser than the total PSS. Can someone explain me why these two values are different and who is using the remaining memory ?
Simulating user mode native memory leak using c++ program, used gflags to enable page heap. Took process memory dump. Following this article https://www.codeproject.com/Articles/31382/Memory-Leak-Detection-Using-Windbg, however unable to get the output of !heap -stat -h command as
size #blocks total ( %) (percent of total busy bytes)
I get the below output, what am I missing?
0:002> !heap -stat -h 00000224be780000
Walking the heap 00000224be780000 .
0: Heap 00000224be780000
Flags 00000002 - HEAP_GROWABLE
Reserved memory in segments 1020 (k)
Commited memory in segments 160 (k)
Virtual bytes (correction for large UCR) 1020 (k)
Free space 1 (k) (1 blocks)
External fragmentation 0% (1 free blocks)
Virtual address fragmentation 84% (1 uncommited ranges)
Virtual blocks 0 - total 0 KBytes
Lock contention 0
Segments 1
Default heap Front heap Unused bytes
Range (bytes) Busy Free Busy Free Total Average
------------------------------------------------------------------
0 - 1024 145 0 0 0 5126 35
1024 - 2048 11 1 0 0 346 31
2048 - 3072 2 0 0 0 64 32
5120 - 6144 2 0 0 0 78 39
7168 - 8192 1 0 0 0 32 32
19456 - 20480 1 0 0 0 40 40
29696 - 30720 1 0 0 0 32 32
38912 - 39936 1 0 0 0 40 40
------------------------------------------------------------------
Total 164 1 0 0 5758 35
I have a data set where the %age of bads are quite low.Can any one suggest a way to balance such a data set using SAS so that the logistic regression run gives a better result? Below is a sample. Thanks in advance!!
ID X1 X2 X3 X4 X5 Target
1 87 400 2 0 0 0
2 70 620 1 0 0 0
3 66 410 3 0 0 0
4 85 300 1 0 0 0
5 100 200 4 0 0 0
6 201 110 1 0 0 0
7 132 513 3 0 0 0
8 98 417 4 0 0 0
9 397 620 1 0 0 1
10 98 700 5 0 0 1
You can oversample the percentage of bads and then use the priorevent option within the score statement of proc logistic to correct the oversampling. There are plenty of examples online that will help you further with this.
I have following data
......
6 4 4 17 154 93 309 0 11930
7 3 2 233 311 0 11936 11932 111874
8 3 1 15 0 11938 11943 211004 11449
9 3 2 55 102 0 11932 11941 111883
10 3 2 197 231 0 11925 11921 111849
11 3 2 160 777 0 11934 11928 111875
......
I hope to replace any values greater than 5000 to 0, from column 4 to column 9. How can I do this work with awk?
To print with lots of spaces like the input, something like this:
awk '{for(i=4;i<=NF;i++)if($i>5000)$i=0; for(i=1;i<=NF;i++)printf "%7d",$i;printf"\n"}' file
Output
6 4 4 17 154 93 309 0 0
7 3 2 233 311 0 0 0 0
8 3 1 15 0 0 0 0 0
9 3 2 55 102 0 0 0 0
10 3 2 197 231 0 0 0 0
11 3 2 160 777 0 0 0 0
For scrunched up together (TM) output, you can use this:
awk '{for(i=4;i<=NF;i++)if($i>5000)$i=0}1' file
6 4 4 17 154 93 309 0 0
7 3 2 233 311 0 0 0 0
8 3 1 15 0 0 0 0 0
9 3 2 55 102 0 0 0 0
10 3 2 197 231 0 0 0 0
11 3 2 160 777 0 0 0 0
An alternative approach (requires gawk4+):
{
patsplit($0, a, "[0-9]+", s)
printf s[0]
for (i=1; i<=length(a); i++){
if(i>4 && a[i]>5000) {
l=length(a[i])
a[i]=0
}
else l=0
printf "%"l"s%s", a[i], s[i]
}
printf "\n"
}
It is more flexible when the spacing would vary, as opposed to the example data. It might also be faster than the accepted answer, in case the number of fields is way bigger than 9.