I'm trying to investigate memory leak in a particular process. For that I'm trying to investigate memory footprints using below command.
adb shell perfdump meminfo .
The output i'm getting is
MEMORY OF /usr/bin/PuffinApp (pid 2376)
TOTAL MEMORY USAGE (kB):
Pss 61937
SwapPss 11576
Graphics 0
------
TOTAL (kB) 73513
OTHER MEMORY STATS (kB):
Vss 2276204
Rss 68440
Uss 60120
CachedPss 8149
NonCachedPss 53788
Swap 11576
SwapUss 11576
PROCESS STATS:
Maj faults 21475
Min faults 2453006
Threads 258
PROCESS MAPS:
PSS SwapPSS TotalPSS Private Private Shared Shared Referenced Name
Clean Dirty Clean Dirty
------ ------ ------ ------ ------ ------ ------ ------ ------
45660 4952 50612 64 45596 0 0 40924 [anon:rw-p]
4324 2076 6400 108 4216 0 0 3984 [heap]
1352 2184 3536 0 1352 0 0 1192 [anon:rwxp]
36 0 36 20 16 0 0 36 [anon:-w-p]
4 28 32 0 4 0 0 4 [stack]
1980 48 2028 1800 180 0 0 1980 /usr/lib/libavcodec.so.58.18.100
1276 108 1384 1128 148 0 0 1276 /usr/lib/libpryon.so
1112 156 1268 1020 92 0 0 1048 /usr/lib/libReggaeWidevine.so
774 4 778 432 88 576 0 1096 /usr/lib/libcrypto.so.1.1
552 0 552 524 20 16 0 552 /usr/lib/libReggaeMediaLib.so
500 0 500 0 468 0 64 532 /dev/shm/puffin-micStream
336 32 368 252 84 0 0 336 /usr/lib/libavformat.so.58.12.100
356 4 360 308 48 0 0 340 /usr/bin/PuffinApp
252 16 268 244 8 0 0 72 /usr/lib/libxml2.so.2.9.13
0 248 248 0 0 0 0 0 /usr/lib/libavfilter.so.7.16.100
219 4 223 172 28 76 0 276 /usr/lib/libssl.so.1.1
202 0 202 0 0 0 404 404 /dev/shm/audio_playback_stream_2
164 0 164 140 12 24 0 176 /usr/lib/libreggae-core.so
106 16 122 4 28 220 0 244 /usr/lib/libPuffinExternalCapabilityAPI.so
92 28 120 84 8 0 0 92 /usr/lib/libavutil.so.56.14.100
114 0 114 0 20 656 0 436 /usr/lib/libsqlite3.so.0.8.6
101 0 101 0 16 520 0 520 /usr/lib/libAVSCommon.so
93 0 93 16 12 228 0 256 /usr/lib/libcurl.so.4.7.0
80 4 84 76 4 0 0 80 /usr/lib/libtvm_modelops_ww_rhea_v3_50target_cnet.so
------ ------ ------ ------ ------ ------ ------ ------
61856 11576 73432 7036 53084 7676 640 62368 TOTAL
I'm not able understand which components should I focus on to find memory leaks.Also the sum of components is not equal to the total PSS mentioned i.e 61856. I think totalPss can help but of which component.Should i have to focus on .so also other than heap Pss ? Also is it possible that cachedPss will contributes to memory leaks.
Can someone help me in understanding the output.
Related
I'm new to C++ and I wanted to compress a vector via ZSTD compression library. I used ZSTD simple API ZSTD_compress and ZSTD_decompress in the same way as the example. But I found a wired issue that when I compressed and decompressed a vector, the decompressed vector was not the same as the original vector. I'm not sure which part of my operation went wrong.
I looked at ZSTD's GitHub homepage and didn't find an answer. Please help or try to give some ideas how to solve it.
Example C code: https://github.com/facebook/zstd/blob/dev/examples/simple_compression.c
//Initialize a vector
vector<int> NumToCompress ;
NumToCompress.resize(10000);
for(int i = 0; i < 10000; i++)
{
NumToCompress[i] = rand()% 255;
}
//compress
int* com_ptr = NULL;
size_t NumSize = NumToCompress.size();
size_t Boundsize = ZSTD_compressBound(NumSize);
com_ptr =(int*) malloc(Boundsize);
size_t ComSize;
ComSize = ZSTD_compress(com_ptr,Boundsize,NumToCompress.data(),NumToCompress.size(),ZSTD_fast);
//decompress
int* decom_ptr = NULL;
unsigned long long decom_Boundsize;
decom_Boundsize = ZSTD_getFrameContentSize(com_ptr, ComSize);
decom_ptr = (int*)malloc(decom_Boundsize);
size_t DecomSize;
DecomSize = ZSTD_decompress(decom_ptr, decom_Boundsize, com_ptr, ComSize);
vector<int> NumAfterDecompress(decom_ptr,decom_ptr+DecomSize);
//check if two vectors are same
if(NumToCompress == NumAfterDecompress)
{
cout << "Two vectors are same" << endl;
}else
{cout << "Two vectors are insame" << endl;}
free(com_ptr);
free(decom_ptr);
case 1: If zstd can compress std::vector directly?
case 2: How to properly compress vectors with zstd if zstd can compress std::vector directly ?
Two vectors are insame
Original vector:
163 151 162 85 83 190 241 252 249 121 107 82 20 19 233 226 45 81 142 31 86 8 87 39 167 5 212 208 82 130 119 117 27 153 74 237 88 61 106 82 54 213 36 74 104 142 173 149 95 60 53 181 196 140 221 108 17 50 61 226 180 180 89 207 206 35 61 39 223 167 249 150 252 30 224 102 44 14 123 140 202 48 66 143 188 159 123 206 209 184 177 135 236 138 214 187 46 21 99 14
Decompressed vector:
163 151 162 85 83 190 241 252 249 121 107 82 20 19 233 226 45 81 142 31 86 8 87 39 167 0 417 0551929248 21916 551935408 21916 551933352 21916 551939512 21916 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 00 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 00 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
Seems likely that your API expects the size to be in bytes but you give the size as the number of elements. So you need to multiply the number of elements by the size of each element. Like this
ComSize = ZSTD_compress(com_ptr, Boundsize, NumToCompress.data(),
NumToCompress.size()*sizeof(int), ZSTD_fast);
and similarly when you decompress you need to divide by the element size
DecomSize = ZSTD_decompress(decom_ptr, decom_Boundsize, com_ptr, ComSize);
vector<int> NumAfterDecompress(decom_ptr, decom_ptr+DecomSize/sizeof(int));
I'm trying to do memory profiling on linux for that I'm using command adb shell perfdump meminfo . I'm not able to understand the output for PSS column as totalPSS is not equal to sum of individual PSS values. Below is the process map I got after running the above command.
MEMORY OF /usr/bin/PuffinApp (pid 2376)
TOTAL MEMORY USAGE (kB):
Pss 66695
SwapPss 11388
Graphics 0
------
TOTAL (kB) 78083
OTHER MEMORY STATS (kB):
Vss 2250440
Rss 76560
Uss 64132
CachedPss 12371
NonCachedPss 54324
Swap 11388
SwapUss 11388
PROCESS STATS:
Maj faults 33592
Min faults 4533273
Threads 253
PROCESS MAPS:
PSS SwapPSS TotalPSS Private Private Shared Shared Referenced Name
Clean Dirty Clean Dirty
------ ------ ------ ------ ------ ------ ------ ------ ------
45752 5208 50960 68 45684 0 0 45524 [anon:rw-p]
4452 1948 6400 224 4228 0 0 4380 [heap]
1612 1988 3600 0 1612 0 0 1568 [anon:rwxp]
36 0 36 20 16 0 0 36 [anon:-w-p]
4 28 32 0 4 0 0 4 [stack]
2116 108 2224 1968 148 0 0 1716 /usr/lib/libpryon.so
1984 4 1988 1936 48 0 0 1104 /usr/bin/PuffinApp
832 0 832 804 20 16 0 516 /usr/lib/libReggaeMediaLib.so
756 32 788 624 84 96 0 384 /usr/lib/libavformat.so.58.12.100
484 156 640 392 92 0 0 400 /usr/lib/libReggaeWidevine.so
522 48 570 280 180 124 0 568 /usr/lib/libavcodec.so.58.18.100
500 0 500 0 468 0 64 532 /dev/shm/puffin-micStream
344 0 344 332 12 0 0 136 /usr/lib/libLocaleWakewordAdapter.so
338 4 342 0 88 1148 0 1236 /usr/lib/libcrypto.so.1.1
280 8 288 276 4 0 0 132 /usr/lib/libSpotifyAdapter.so
264 0 264 184 12 136 0 208 /usr/lib/libreggae-core.so
247 8 255 8 28 976 0 784 /usr/lib/libPuffinExternalCapabilityAPI.so
4 248 252 4 0 0 0 4 /usr/lib/libavfilter.so.7.16.100
222 4 226 200 0 68 0 268 /usr/lib/libopus.so.0.8.0
202 0 202 0 0 0 404 404 /dev/shm/audio_playback_stream_2
176 16 192 168 8 0 0 96 /usr/lib/libxml2.so.2.9.13
183 0 183 120 4 128 0 64 /usr/lib/libDefaultClient.so
142 0 142 88 12 172 0 256 /usr/lib/libacsdkAudioPlayer.so
140 0 140 136 4 0 0 44 /usr/lib/libacsdkVisualCharacteristics.so
------ ------ ------ ------ ------ ------ ------ ------
66613 11388 78001 10728 53404 11912 512 71972 TOTAL
If i calculate the sum of individual PSS (1st column ) I will get 61492 which is around 5MB lesser than the total PSS. Can someone explain me why these two values are different and who is using the remaining memory ?
I have an ESP8266 connected to PMS5003 particulate matter sensor through (hardware) UART.
I'm getting many checksum errors while reading from PMS5003.
Here's the library that I'm using to communicate with PMS5003:
PMS5003.cpp
#include "PMS5003.h"
void PMS5003::processDataOn(HardwareSerial &serial) {
unsigned long timeout = millis();
int count = 0;
byte incomeByte[NUM_INCOME_BYTE];
boolean startcount = false;
byte data;
int timeoutHops = 0;
while (1){
if (((millis() - timeout) > 1000) && (timeoutHops == 0)) {
timeoutHops = 1;
yield();
ESP.wdtFeed();
}
if (((millis() - timeout) > 2000) && (timeoutHops == 1)) {
timeoutHops = 2;
yield();
ESP.wdtFeed();
}
if ((millis() - timeout) > 3000){
Serial.println("SENSOR-ERROR-TIMEOUT");
break;
}
if (serial.available()){
data = serial.read();
if (data == CHAR_PRELIM && !startcount) {
startcount = true;
count++;
incomeByte[0] = data;
} else if (startcount) {
count++;
incomeByte[count - 1] = data;
if (count >= NUM_INCOME_BYTE){
break;
}
}
}
}
unsigned int calcsum = 0;
unsigned int exptsum;
for (int i = 0; i < NUM_DATA_BYTE; i++) {
calcsum += (unsigned int)incomeByte[i];
}
exptsum = ((unsigned int)incomeByte[CHECK_BYTE] << 8) + (unsigned int)incomeByte[CHECK_BYTE + 1];
if (calcsum == exptsum) {
pm1 = ((unsigned int)incomeByte[PM1_BYTE] << 8) + (unsigned int)incomeByte[PM1_BYTE + 1];
pm25 = ((unsigned int)incomeByte[PM25_BYTE] << 8) + (unsigned int)incomeByte[PM25_BYTE + 1];
pm10 = ((unsigned int)incomeByte[PM10_BYTE] << 8) + (unsigned int)incomeByte[PM10_BYTE + 1];
} else {
Serial.println("#[exception] PM2.5 Sensor CHECKSUM ERROR!");
pm1 = -1;
pm25 = -1;
pm10 = -1;
}
return;
}
int PMS5003::getPM1() {
return pm1;
}
int PMS5003::getPM25() {
return pm25;
}
int PMS5003::getPM10() {
return pm10;
}
PMS5003.h
#ifndef _PMS_5003_H
#define _PMS_5003_H
#include <Wire.h>
#include <Arduino.h>
#define VERSION 0.2
#define Sense_PM 6
#define NUM_INCOME_BYTE 32
#define CHAR_PRELIM 0x42
#define NUM_DATA_BYTE 29
#define CHECK_BYTE 30
#define PM1_BYTE 10
#define PM25_BYTE 12
#define PM10_BYTE 14
class PMS5003 {
public:
//void processData(int *PM1, int *PM25, int *PM10);
void processDataOn(HardwareSerial &serial);
int getPM1();
int getPM25();
int getPM10();
private:
int pm1;
int pm25;
int pm10;
};
#endif
Here's how I'm using it:
struct ParticulateMatterMeasurements {
private:
bool _areValid = false;
int PM01Value = 0;
int PM25Value = 0;
int PM10Value = 0;
public:
void setAreValid(bool _areValid) {
ParticulateMatterMeasurements::_areValid = _areValid;
}
bool getAreValid() const {
return _areValid;
}
int getPM01Value() const {
return PM01Value;
}
void setPM01Value(int PM01Value) {
ParticulateMatterMeasurements::PM01Value = PM01Value;
}
int getPM25Value() const {
return PM25Value;
}
void setPM25Value(int PM25Value) {
ParticulateMatterMeasurements::PM25Value = PM25Value;
}
int getPM10Value() const {
return PM10Value;
}
void setPM10Value(int PM10Value) {
ParticulateMatterMeasurements::PM10Value = PM10Value;
}
};
ParticulateMatterMeasurements getMeasurements() {
ParticulateMatterMeasurements measurements;
measurements.setAreValid(false);
pms5003.processDataOn(Serial);
measurements.setPM01Value(pms5003.getPM1());
measurements.setPM25Value(pms5003.getPM25());
measurements.setPM10Value(pms5003.getPM10());
if (measurements.getPM01Value() != -1 && measurements.getPM25Value() != -1 && measurements.getPM10Value() != -1) {
measurements.setAreValid(true);
}
return measurements;
}
The problem is that I get many checksum errors. During 60 measurements I get about 100 of: #[exception] PM2.5 Sensor CHECKSUM ERROR!.
What could be the problem here?
#edit
I ran a test where I print what PMS5003 sends to my ESP8266. It looks like the checksum which is the last byte is sometimes not sent. Instead, I get 66 usually but I can see sometimes 66 77 instead of the last 2 bytes as well.
66 77 0 28 0 13 0 16 0 20 0 13 0 16 0 20 9 201 2 216 0 86 0 8 0 3 0 1 145 0 3 172
16
66 77 0 28 0 13 0 16 0 20 0 13 0 16 0 20 9 201 2 216 0 86 0 8 0 3 0 1 145 0 3 172
16
66 77 0 28 0 14 0 18 0 22 0 14 0 18 0 22 10 32 2 239 0 93 0 9 0 3 0 1 145 0 3 45
18
66 77 0 28 0 14 0 18 0 22 0 14 0 18 0 22 10 32 2 239 0 93 0 9 0 3 0 1 145 0 3 66
18
#[exception] PM2.5 Sensor CHECKSUM ERROR!
66 77 0 28 0 13 0 17 0 20 0 13 0 17 0 20 10 50 2 231 0 88 0 9 0 3 0 1 145 0 3 42
17
66 77 0 28 0 12 0 16 0 20 0 12 0 16 0 20 9 225 2 208 0 90 0 8 0 3 0 1 145 0 3 190
16
66 77 0 28 0 13 0 17 0 21 0 13 0 17 0 21 9 225 2 208 0 90 0 8 0 3 0 1 145 0 3 196
17
66 77 0 28 0 12 0 15 0 17 0 12 0 15 0 17 9 249 2 211 0 76 0 5 0 0 0 0 145 0 3 188
15
66 77 0 28 0 12 0 15 0 17 0 12 0 15 0 17 9 249 2 211 0 76 0 5 0 0 0 0 145 0 3 188
15
66 77 0 28 0 12 0 15 0 16 0 12 0 15 0 16 9 210 2 188 0 70 0 5 0 0 0 0 145 0 3 118
15
66 77 0 28 0 12 0 15 0 16 0 12 0 15 0 16 9 210 2 188 0 70 0 5 0 0 0 0 145 0 3 118
15
66 77 0 28 0 13 0 16 0 17 0 13 0 16 0 17 9 198 2 183 0 78 0 5 0 0 0 0 145 0 3 115
16
66 77 0 28 0 13 0 16 0 17 0 13 0 16 0 17 9 198 2 183 0 78 0 5 0 0 0 0 145 0 3 66
16
#[exception] PM2.5 Sensor CHECKSUM ERROR!
66 77 0 28 0 12 0 16 0 18 0 12 0 16 0 18 9 234 2 195 0 87 0 8 0 1 0 0 145 0 3 176
16
66 77 0 28 0 12 0 16 0 18 0 12 0 16 0 18 9 234 2 195 0 87 0 8 0 1 0 0 145 0 3 176
16
66 77 0 28 0 12 0 15 0 17 0 12 0 15 0 17 9 186 2 184 0 77 0 6 0 1 0 0 145 0 3 101
15
66 77 0 28 0 13 0 16 0 18 0 13 0 16 0 18 9 186 2 184 0 77 0 6 0 1 0 0 145 0 3 107
16
66 77 0 28 0 13 0 16 0 18 0 13 0 16 0 18 9 186 2 184 0 77 0 6 0 1 0 0 145 0 3 107
16
66 77 0 28 0 13 0 17 0 18 0 13 0 17 0 18 9 165 2 180 0 76 0 6 0 1 0 0 145 0 3 83
17
66 77 0 28 0 13 0 17 0 18 0 13 0 17 0 18 9 165 2 180 0 76 0 6 0 1 0 0 145 0 3 83
17
66 77 0 28 0 13 0 17 0 18 0 13 0 17 0 18 9 156 2 186 0 76 0 6 0 1 0 0 145 0 3 80
17
66 77 0 28 0 12 0 16 0 17 0 12 0 16 0 17 9 156 2 186 0 76 0 6 0 1 0 0 145 0 3 66
16
#[exception] PM2.5 Sensor CHECKSUM ERROR!
66 77 0 28 0 12 0 16 0 17 0 12 0 16 0 17 9 249 2 200 0 67 0 6 0 1 0 0 145 0 3 172
16
66 77 0 28 0 12 0 16 0 17 0 12 0 16 0 17 9 249 2 200 0 67 0 6 0 1 0 0 145 0 3 172
16
66 77 0 28 0 12 0 16 0 17 0 12 0 16 0 17 9 231 2 197 0 73 0 7 0 1 0 0 145 0 3 158
16
66 77 0 28 0 12 0 16 0 17 0 12 0 16 0 17 9 231 2 197 0 73 0 7 0 1 0 0 145 0 3 158
16
66 77 0 28 0 13 0 17 0 18 0 13 0 17 0 18 9 243 2 194 0 73 0 7 0 1 0 0 145 0 3 173
17
66 77 0 28 0 13 0 17 0 18 0 13 0 17 0 18 9 243 2 194 0 73 0 7 0 1 0 0 145 0 3 173
17
66 77 0 28 0 13 0 17 0 18 0 13 0 17 0 18 9 231 2 190 0 70 0 8 0 1 0 0 145 0 3 155
17
66 77 0 28 0 13 0 17 0 18 0 13 0 17 0 18 9 231 2 190 0 70 0 8 0 1 0 0 145 0 3 155
17
66 77 0 28 0 13 0 17 0 19 0 13 0 17 0 19 10 65 2 219 0 68 0 7 0 1 0 0 145 0 3 66
17
#[exception] PM2.5 Sensor CHECKSUM ERROR!
66 77 0 28 0 13 0 17 0 19 0 13 0 17 0 19 10 65 2 219 0 68 0 7 0 1 0 0 145 0 3 18
17
66 77 0 28 0 13 0 18 0 20 0 13 0 18 0 20 10 86 2 218 0 79 0 11 0 2 0 0 145 0 3 58
18
66 77 0 28 0 13 0 18 0 20 0 13 0 18 0 20 10 86 2 218 0 79 0 11 0 2 0 0 145 0 3 58
18
66 77 0 28 0 13 0 18 0 19 0 13 0 18 0 19 10 86 2 216 0 76 0 8 0 1 0 0 145 0 3 47
18
66 77 0 28 0 13 0 18 0 19 0 13 0 18 0 19 10 86 2 216 0 76 0 8 0 1 0 0 145 0 3 47
18
66 77 0 28 0 14 0 18 0 20 0 14 0 18 0 20 10 212 2 250 0 75 0 8 0 1 0 0 145 0 3 210
18
66 77 0 28 0 14 0 18 0 20 0 14 0 18 0 20 10 212 2 250 0 75 0 8 0 1 0 0 145 0 3 210
18
66 77 0 28 0 12 0 17 0 20 0 12 0 17 0 20 10 137 2 234 0 86 0 10 0 1 0 0 145 0 3 126
17
66 77 0 28 0 12 0 17 0 20 0 12 0 17 0 20 10 137 2 234 0 86 0 10 0 1 0 0 145 0 3 126
17
66 77 0 28 0 12 0 17 0 20 0 12 0 17 0 20 10 137 2 234 0 86 0 10 0 1 0 0 145 0 3 66
17
#[exception] PM2.5 Sensor CHECKSUM ERROR!
66 77 0 28 0 14 0 19 0 21 0 14 0 19 0 21 10 164 2 240 0 95 0 10 0 1 0 0 145 0 3 178
19
66 77 0 28 0 14 0 19 0 21 0 14 0 19 0 21 10 164 2 240 0 95 0 10 0 1 0 0 145 0 3 178
19
66 77 0 28 0 13 0 19 0 21 0 13 0 19 0 21 10 110 2 223 0 95 0 10 0 1 0 0 145 0 3 105
19
66 77 0 28 0 13 0 19 0 21 0 13 0 19 0 21 10 110 2 223 0 95 0 10 0 1 0 0 145 0 3 105
19
66 77 0 28 0 13 0 19 0 21 0 13 0 19 0 21 10 128 2 227 0 101 0 10 0 1 0 0 145 0 3 133
19
66 77 0 28 0 13 0 19 0 21 0 13 0 19 0 21 10 128 2 227 0 101 0 10 0 1 0 0 145 0 3 133
19
66 77 0 28 0 14 0 20 0 24 0 14 0 20 0 24 10 158 2 254 0 106 0 13 0 4 0 0 145 0 3 211
20
66 77 0 28 0 13 0 19 0 23 0 13 0 19 0 23 10 158 2 254 0 106 0 13 0 4 0 0 145 0 3 205
19
66 77 0 28 0 13 0 19 0 23 0 13 0 19 0 23 10 212 3 10 0 107 0 12 0 4 0 0 145 0 3 66
19
#[exception] PM2.5 Sensor CHECKSUM ERROR!
66 77 0 28 0 14 0 20 0 24 0 14 0 20 0 24 10 212 3 10 0 107 0 12 0 4 0 0 145 0 3 22
20
66 77 0 28 0 14 0 20 0 24 0 14 0 20 0 24 10 236 3 11 0 109 0 13 0 4 0 0 145 0 3 50
20
66 77 0 28 0 14 0 20 0 24 0 14 0 20 0 24 10 236 3 11 0 109 0 13 0 4 0 0 145 0 3 50
20
66 77 0 28 0 15 0 20 0 23 0 15 0 20 0 23 10 254 3 29 0 105 0 9 0 3 0 0 145 0 3 77
20
66 77 0 28 0 15 0 20 0 23 0 15 0 20 0 23 10 254 3 29 0 105 0 9 0 3 0 0 145 0 3 77
20
66 77 0 28 0 14 0 19 0 22 0 14 0 19 0 22 11 22 3 40 0 99 0 9 0 3 0 0 145 0 2 101
19
66 77 0 28 0 14 0 19 0 22 0 14 0 19 0 22 11 22 3 40 0 99 0 9 0 3 0 0 145 0 2 101
19
66 77 0 28 0 14 0 19 0 21 0 14 0 19 0 21 10 149 3 6 0 93 0 8 0 3 0 0 145 0 2 184
19
66 77 0 28 0 14 0 19 0 21 0 14 0 19 0 21 10 149 3 6 0 93 0 8 0 3 0 0 145 0 2 66
19
#[exception] PM2.5 Sensor CHECKSUM ERROR!
66 77 0 28 0 14 0 18 0 20 0 14 0 18 0 20 10 140 3 2 0 77 0 5 0 3 0 0 145 0 2 148
18
66 77 0 28 0 13 0 17 0 19 0 13 0 17 0 19 10 140 3 2 0 77 0 5 0 3 0 0 145 0 2 142
17
66 77 0 28 0 13 0 17 0 19 0 13 0 17 0 19 10 104 2 251 0 77 0 5 0 3 0 0 145 0 3 98
17
66 77 0 28 0 13 0 17 0 19 0 13 0 17 0 19 10 104 2 251 0 77 0 5 0 3 0 0 145 0 3 98
17
66 77 0 28 0 13 0 17 0 19 0 13 0 17 0 19 10 104 2 251 0 77 0 5 0 3 0 0 145 0 3 98
17
66 77 0 28 0 13 0 18 0 20 0 13 0 18 0 20 10 116 3 11 0 77 0 8 0 3 0 0 145 0 2 134
18
66 77 0 28 0 14 0 19 0 21 0 14 0 19 0 21 10 116 3 11 0 77 0 8 0 3 0 0 145 0 2 140
19
66 77 0 28 0 14 0 19 0 21 0 14 0 19 0 21 10 137 3 18 0 76 0 7 0 3 0 0 145 0 2 166
19
[update] This is the newest version.
66 77 0 28 0 14 0 19 0 21 0 14 0 19 0 21 10 137 3 18 0 76 0 7 0 3 0 0 145 0 2 166
19
66 77 0 28 0 12 0 17 0 18 0 12 0 17 0 18 10 77 2 241 0 77 0 5 0 0 0 0 145 0 3 66
17
#[exception] PM2.5 Sensor CHECKSUM ERROR!
66 77 0 28 0 12 0 17 0 18 0 12 0 17 0 18 10 77 2 241 0 77 0 5 0 0 0 0 145 0 3 54
17
66 77 0 28 0 13 0 18 0 19 0 13 0 18 0 19 10 32 2 224 0 80 0 6 0 0 0 0 145 0 3 2
18
66 77 0 28 0 13 0 18 0 19 0 13 0 18 0 19 10 32 2 224 0 80 0 6 0 0 0 0 145 0 3 2
18
66 77 0 28 0 13 0 19 0 19 0 13 0 19 0 19 10 47 2 240 0 81 0 5 0 0 0 0 145 0 3 35
19
66 77 0 28 0 12 0 18 0 18 0 12 0 18 0 18 10 47 2 240 0 81 0 5 0 0 0 0 145 0 3 29
18
66 77 0 28 0 14 0 20 0 20 0 14 0 20 0 20 10 173 3 6 0 90 0 6 0 1 0 1 145 0 2 202
20
66 77 0 28 0 14 0 20 0 20 0 14 0 20 0 20 10 173 3 6 0 90 0 6 0 1 0 1 145 0 2 202
20
66 77 0 28 0 14 0 21 0 21 0 14 0 21 0 21 10 233 3 24 0 90 0 9 0 1 0 1 145 0 3 31
21
66 77 0 28 0 13 0 19 0 22 0 13 0 19 0 22 10 242 3 25 0 84 0 12 0 4 0 2 145 0 3 38
19
66 77 0 28 0 13 0 19 0 22 0 13 0 19 0 22 10 242 3 25 0 84 0 12 0 4 0 2 145 0 3 38
19
66 77 0 28 0 13 0 19 0 22 0 13 0 19 0 22 10 242 3 25 0 84 0 12 0 4 0 2 145 0 3 66
19
#[exception] PM2.5 Sensor CHECKSUM ERROR!
66 77 0 28 0 15 0 20 0 23 0 15 0 20 0 23 11 79 3 40 0 88 0 9 0 4 0 2 145 0 2 156
20
66 77 0 28 0 15 0 20 0 23 0 15 0 20 0 23 11 79 3 40 0 88 0 9 0 4 0 2 145 0 2 156
20
66 77 0 28 0 14 0 19 0 21 0 14 0 19 0 21 11 67 3 38 0 83 0 9 0 4 0 2 145 0 2 129
19
66 77 0 28 0 14 0 19 0 21 0 14 0 19 0 21 11 67 3 38 0 83 0 9 0 4 0 2 145 0 2 129
19
66 77 0 28 0 14 0 18 0 21 0 14 0 18 0 21 11 58 3 43 0 82 0 8 0 4 0 2 145 0 2 121
18
66 77 0 28 0 15 0 19 0 22 0 15 0 19 0 22 11 58 3 43 0 82 0 8 0 4 0 2 145 0 2 127
19
66 77 0 28 0 15 0 19 0 21 0 15 0 19 0 21 11 31 3 37 0 79 0 7 0 4 0 2 145 0 2 88
19
66 77 0 28 0 15 0 19 0 21 0 15 0 19 0 21 11 31 3 37 0 79 0 7 0 4 0 2 145 0 2 88
19
66 77 0 28 0 14 0 18 0 21 0 14 0 18 0 21 10 203 3 11 0 84 0 8 0 4 0 2 145 0 2 66
18
#[exception] PM2.5 Sensor CHECKSUM ERROR!
66 77 0 28 0 14 0 18 0 21 0 14 0 18 0 21 10 203 3 11 0 84 0 8 0 4 0 2 145 0 2 235
18
66 77 0 28 0 14 0 18 0 21 0 14 0 18 0 21 10 197 3 5 0 84 0 9 0 4 0 2 145 0 2 224
18
66 77 0 28 0 14 0 18 0 21 0 14 0 18 0 21 10 197 3 5 0 84 0 9 0 4 0 2 145 0 2 224
18
66 77 0 28 0 14 0 18 0 21 0 14 0 18 0 21 10 179 3 8 0 81 0 8 0 3 0 1 145 0 2 203
18
66 77 0 28 0 14 0 18 0 21 0 14 0 18 0 21 10 179 3 8 0 81 0 8 0 3 0 1 145 0 2 203
18
66 77 0 28 0 14 0 17 0 20 0 14 0 17 0 20 10 98 2 243 0 75 0 8 0 3 0 1 145 0 3 90
17
66 77 0 28 0 14 0 17 0 20 0 14 0 17 0 20 10 98 2 243 0 75 0 8 0 3 0 1 145 0 3 90
17
66 77 0 28 0 14 0 17 0 19 0 14 0 17 0 19 10 116 2 246 0 77 0 5 0 3 0 1 145 0 3 108
17
66 77 0 28 0 14 0 17 0 19 0 14 0 17 0 19 10 116 2 246 0 77 0 5 0 3 0 1 145 0 3 66
17
#[exception] PM2.5 Sensor CHECKSUM ERROR!
66 77 0 28 0 14 0 17 0 19 0 14 0 17 0 19 10 74 2 232 0 80 0 3 0 1 0 1 145 0 3 51
17
66 77 0 28 0 14 0 17 0 19 0 14 0 17 0 19 10 74 2 232 0 80 0 3 0 1 0 1 145 0 3 51
17
66 77 0 28 0 14 0 17 0 19 0 14 0 17 0 19 10 74 2 232 0 80 0 3 0 1 0 1 145 0 3 51
17
66 77 0 28 0 13 0 17 0 18 0 13 0 17 0 18 9 246 2 220 0 79 0 3 0 1 0 1 145 0 3 205
17
66 77 0 28 0 13 0 17 0 18 0 13 0 17 0 18 9 246 2 220 0 79 0 3 0 1 0 1 145 0 3 205
17
66 77 0 28 0 13 0 17 0 19 0 13 0 17 0 19 9 255 2 222 0 82 0 6 0 1 0 1 145 0 3 224
17
66 77 0 28 0 13 0 17 0 19 0 13 0 17 0 19 9 255 2 222 0 82 0 6 0 1 0 1 145 0 3 224
17
66 77 0 28 0 14 0 18 0 20 0 14 0 18 0 20 10 89 2 242 0 85 0 6 0 1 0 1 145 0 3 88
18
66 77 0 28 0 14 0 18 0 20 0 14 0 18 0 20 10 89 2 242 0 85 0 6 0 1 0 1 145 0 3 66
18
#[exception] PM2.5 Sensor CHECKSUM ERROR!
After some time I get more errors:
66 77 0 28 0 13 0 17 0 66 77 0 28 0 13 0 17 0 17 0 13 0 17 0 17 9 129 2 177 0 65 0
7168
#[exception] PM2.5 Sensor CHECKSUM ERROR!
66 77 0 28 0 13 0 17 0 17 0 66 77 0 28 0 13 0 17 0 17 0 13 0 17 0 17 9 129 2 177 0
19712
#[exception] PM2.5 Sensor CHECKSUM ERROR!
66 77 0 28 0 13 0 17 0 17 0 66 77 0 28 0 12 0 16 0 16 0 12 0 16 0 16 9 186 2 186 0
19712
#[exception] PM2.5 Sensor CHECKSUM ERROR!
66 77 0 28 0 12 0 16 0 16 0 12 0 16 0 16 9 186 2 186 0 67 0 6 0 0 0 0 145 0 3 92
16
66 77 0 28 0 12 0 16 0 16 0 12 0 16 0 16 9 174 2 193 0 62 0 5 0 0 0 0 145 0 3 81
16
66 77 0 28 0 13 0 17 0 17 0 13 0 17 0 17 9 174 2 193 0 62 0 5 0 0 0 0 145 0 3 87
17
66 77 0 28 0 13 0 17 0 18 0 13 0 17 0 18 9 180 2 197 0 62 0 6 0 1 0 1 145 0 3 102
17
66 77 0 28 0 13 0 17 0 18 0 13 0 17 0 18 9 180 2 197 0 62 0 6 0 1 0 1 145 0 3 102
17
66 77 0 28 0 13 0 17 0 18 0 13 0 17 0 18 9 165 2 190 0 62 0 6 0 1 0 1 145 0 3 80
17
66 77 0 28 0 13 0 17 0 66 77 0 28 0 13 0 16 0 18 0 13 0 16 0 18 9 135 2 182 0 65 0
7168
#[exception] PM2.5 Sensor CHECKSUM ERROR!
66 77 0 28 0 13 0 16 0 18 0 66 77 0 28 0 13 0 16 0 18 0 13 0 16 0 18 9 135 2 182 0
19712
#[exception] PM2.5 Sensor CHECKSUM ERROR!
66 77 0 28 0 13 0 16 0 18 0 13 0 16 0 18 9 117 2 180 0 64 0 3 0 2 0 1 145 0 3 20
16
66 77 0 28 0 13 0 16 0 18 0 66 77 0 28 0 13 0 16 0 18 0 13 0 16 0 18 9 117 2 180 0
19712
#[exception] PM2.5 Sensor CHECKSUM ERROR!
66 77 0 28 0 12 0 15 0 17 0 12 0 15 0 17 9 153 2 196 0 64 0 3 0 2 0 1 145 0 3 66
15
66 77 0 28 0 12 0 15 0 17 0 12 0 15 0 17 9 153 2 196 0 64 0 3 0 2 0 1 145 0 3 66
15
[update] This is the newest version.
66 77 0 28 0 12 0 16 0 19 0 12 0 16 0 19 9 162 2 206 0 71 0 6 0 5 0 2 145 0 3 105
16
66 77 0 28 0 13 0 17 0 20 0 13 0 17 0 20 9 162 2 206 0 71 0 6 0 5 0 2 145 0 3 111
17
66 77 0 28 0 13 0 16 0 19 0 13 0 16 0 19 9 123 2 189 0 74 0 6 0 5 0 2 145 0 3 54
16
66 77 0 28 0 13 0 16 0 66 77 0 28 0 12 0 16 0 20 0 12 0 16 0 20 9 54 2 166 0 76 0
7168
#[exception] PM2.5 Sensor CHECKSUM ERROR!
66 77 0 28 0 12 0 16 0 20 0 12 0 16 0 20 9 54 2 166 0 76 0 7 0 6 0 2 145 0 2 222
16
66 77 0 28 0 11 0 15 0 19 0 66 77 0 28 0 11 0 15 0 19 0 11 0 15 0 19 9 21 2 166 0
19712
#[exception] PM2.5 Sensor CHECKSUM ERROR!
66 77 0 28 0 11 0 15 0 19 0 66 77 0 28 0 12 0 16 0 20 0 12 0 16 0 20 9 21 2 166 0
19712
#[exception] PM2.5 Sensor CHECKSUM ERROR!
66 77 0 28 0 13 0 16 0 20 0 13 0 16 0 20 9 120 2 189 0 83 0 9 0 6 0 1 145 0 3 65
16
66 77 0 28 0 13 0 16 0 20 0 13 0 16 0 20 9 120 2 189 0 83 0 9 0 6 0 1 145 0 3 65
16
66 77 0 28 0 13 0 16 0 20 0 13 0 16 0 20 9 186 2 198 0 74 0 9 0 5 0 1 145 0 3 130
16
66 77 0 28 0 13 0 16 0 20 0 13 0 16 0 20 9 186 2 198 0 74 0 9 0 5 0 1 145 0 3 130
16
66 77 0 28 0 13 0 15 0 19 0 13 0 15 0 19 9 171 2 184 0 74 0 9 0 5 0 1 145 0 3 97
15
66 77 0 28 0 13 0 15 0 66 77 0 28 0 13 0 16 0 20 0 13 0 16 0 20 9 198 2 190 0 77 0
7168
#[exception] PM2.5 Sensor CHECKSUM ERROR!
66 77 0 28 0 13 0 16 0 20 0 13 0 16 0 20 9 198 2 190 0 77 0 8 0 5 0 1 145 0 3 136
16
66 77 0 28 0 13 0 16 0 19 0 66 77 0 28 0 13 0 16 0 19 0 13 0 16 0 19 9 225 2 190 0
19712
#[exception] PM2.5 Sensor CHECKSUM ERROR!
66 77 0 28 0 13 0 17 0 20 0 13 0 17 0 20 10 23 2 214 0 81 0 9 0 3 0 0 145 0 2 246
17
66 77 0 28 0 13 0 17 0 20 0 13 0 17 0 20 10 23 2 214 0 81 0 9 0 3 0 0 145 0 2 246
17
66 77 0 28 0 13 0 17 0 20 0 13 0 17 0 20 10 23 2 214 0 81 0 9 0 3 0 0 145 0 2 246
17
66 77 0 28 0 13 0 17 0 20 0 66 77 0 28 0 13 0 17 0 20 0 13 0 17 0 20 10 98 2 234 0
19712
#[exception] PM2.5 Sensor CHECKSUM ERROR!
The lonely number in the new line after 32 bytes is the value of PM2.5 which shouldn't get high. However, it gets sometimes high and sometimes not when the checksum is incorrect.
I wonder why the situation changes over time... Maybe I could keep resetting the UART somehow?
For each of the packets that fail the checksum test, you find a CHAR_PRELIM (66) either in the middle or at the end. This means the sensor is occasionally dropping packets and causing misalignment.
One solution is to restart packet reading each time a 66 is read. This code should do it:
UPDATE: as per #sawdust's comment, the presence of both 66 and 77 should be used as a start condition because it may be possible for 66 to appear by itself in the data. The other consideration is to use the packet length provided by the 3rd and 4th bytes instead of assuming the length to be 32. Hopefully these improvements make the code more durable.
size_t length;
incomingByte[0] = 66; // the first two bytes are always known
incomingByte[1] = 77;
...
if (serial.available()) {
if (serial.read() == 66 && serial.read() == 77)
incomingByte[2] = serial.read(); // length high byte
incomingByte[3] = serial.read(); // length low byte
int length = (incomingByte[2] << 8) + incomingByte[3];
// starting at index 4, read `length` bytes
serial.readBytes(incomingByte + 4, length);
break;
}
}
// when the code breaks out of the while(1) loop, you still need to evaluate the checksum.
According to the protocol defined by this source, the packet length is fixed at 32 bytes, so the encoded frame length (bytes 3 and 4) should always equal 0 28 (32 bytes - 2 start bytes - 2 frame length bytes = 28).
However, this code should work even for variable length packets (thanks #sawdust).
Fair warning: I do not have one of these sensors, so obviously I didn't test this, but the concept remains.
I recognize that this code won't solve the issue of characters being dropped, since it just ignores incomplete packets and you still rely on the validity of the checksum.
Finally, I find it interesting that the reason that the checksum is failing is because the checksum bytes are not even being received in those cases!
Hope this helps!
UPDATE #2: This is more or less a revised answer in it of itself.
Using this code to read packets, the following criteria (which are defined by the protocol) are guaranteed:
The packets begins with [66 77]
The packet contains 32 bytes
The start condition [66 77] will never occur in the body of the packet.
Here's the code. I manage to reduce it down to a few if statements
void PMS5003::processDataOn(HardwareSerial &serial) {
bool possibleStart = false;
incomeByte[0] = 66;
incomeByte[1] = 77;
uint8_t count = 0;
...
while (1) {
...
if (serial.available()) {
uint8_t c = serial.read();
if (possibleStart) {
possibleStart = false;
if (c == 77) count = 2;
}
if (c == 66) possibleStart = true;
if (count >= 2) incomeByte[count++] = c;
if (count == NUM_DATA_BYTE) break;
}
}
// at this point, incomeByte must:\
// > begin with [66 77]
// > contain 32 bytes
// > not contain [66 77] anywhere after the first two bytes
// > therefore, it is guaranteed to contain a checksum
// now is the right time to evaluate the checksum.
// I expect all of the checksums to match, but you might as well check
}
At the time of posting, the OP has already coded a solution which fulfills the requirements. I am posting this because I believe this code improves upon the OP's by being more concise, more readable/declarative, and hopefully more easily manageable.
This code can also serve as a general solution for any case in which two characters define a start condition, provided the packet length is known or can be determined.
While the question why the data comes corrupted still remains, here is a workaround I managed to achieve:
I'm checking the checksum, if it's incorrect, then:
I'm looking for 66 77 in the whole data. When I find it:
I'm checking if in the next 16 bytes there's another 66 77. If it's not found:
I'm presuming the values that are distanced by 10-15 bytes from 66 77 are the ones I'm looking for (PM1, PM2.5, PM10).
Here's the code:
void PMS5003::processDataOn(HardwareSerial &serial) {
unsigned long timeout = millis();
int count = 0;
byte incomeByte[NUM_INCOME_BYTE];
boolean startcount = false;
byte data;
int timeoutHops = 0;
while (1){
if (((millis() - timeout) > 1000) && (timeoutHops == 0)) {
timeoutHops = 1;
yield();
ESP.wdtFeed();
}
if (((millis() - timeout) > 2000) && (timeoutHops == 1)) {
timeoutHops = 2;
yield();
ESP.wdtFeed();
}
if ((millis() - timeout) > 3000) {
yield();
ESP.wdtFeed();
Serial.println("SENSOR-ERROR-TIMEOUT");
break;
}
if (serial.available()) {
data = serial.read();
if (data == CHAR_PRELIM && !startcount) {
startcount = true;
count++;
incomeByte[0] = data;
} else if (startcount) {
count++;
incomeByte[count - 1] = data;
if (count >= NUM_INCOME_BYTE){
break;
}
}
}
}
unsigned int calcsum = 0;
unsigned int exptsum;
for (int a = 0; a < NUM_INCOME_BYTE; a++) {
Serial.print((unsigned int)incomeByte[a]);
Serial.print(" ");
}
Serial.println();
Serial.println(((unsigned int)incomeByte[PM25_BYTE] << 8) + (unsigned int)incomeByte[PM25_BYTE + 1]);
for (int i = 0; i < NUM_DATA_BYTE; i++) {
calcsum += (unsigned int)incomeByte[i];
}
exptsum = ((unsigned int)incomeByte[CHECK_BYTE] << 8) + (unsigned int)incomeByte[CHECK_BYTE + 1];
if (calcsum == exptsum) {
pm1 = ((unsigned int)incomeByte[PM1_BYTE] << 8) + (unsigned int)incomeByte[PM1_BYTE + 1];
pm25 = ((unsigned int)incomeByte[PM25_BYTE] << 8) + (unsigned int)incomeByte[PM25_BYTE + 1];
pm10 = ((unsigned int)incomeByte[PM10_BYTE] << 8) + (unsigned int)incomeByte[PM10_BYTE + 1];
} else {
Serial.println("#[exception] PM2.5 Sensor CHECKSUM ERROR!");
pm1 = -1;
pm25 = -1;
pm10 = -1;
for (int a = 0; a < NUM_INCOME_BYTE; a++) {
bool valid = true;
if (((unsigned int)incomeByte[a] == 66) && ((unsigned int)incomeByte[a+1] == 77)) {
if (a+16 < NUM_INCOME_BYTE) {
for (int b = a+1; b < a+15; b++) {
if (((unsigned int)incomeByte[b] == 66) && ((unsigned int)incomeByte[b+1] == 77)) {
valid = false;
break;
}
}
if (valid) {
pm1 = ((unsigned int)incomeByte[a+10] << 8) + (unsigned int)incomeByte[a+11];
pm25 = ((unsigned int)incomeByte[a+12] << 8) + (unsigned int)incomeByte[a+13];
pm10 = ((unsigned int)incomeByte[a+14] << 8) + (unsigned int)incomeByte[a+15];
Serial.println("valid: ");
Serial.print(pm1);
Serial.print(" ");
Serial.print(pm25);
Serial.print(" ");
Serial.print(pm10);
Serial.println();
break;
}
}
}
}
}
return;
}
Theoretically, it may produce false positives or negatives but in practice, it just works.
66 77 0 28 0 12 0 15 0 17 0 12 0 15 0 17 9 102 2 176 66 77 0 28 0 12 0 15 0 16 0 12
15
#[exception] PM2.5 Sensor CHECKSUM ERROR!
valid:
12 15 17
66 77 0 28 0 12 0 15 0 16 0 12 0 15 0 16 9 114 2 175 0 73 0 4 0 1 0 0 145 0 3 12
15
66 77 0 28 0 12 0 15 0 16 0 12 0 15 0 16 9 114 2 175 0 73 0 4 0 1 0 0 145 0 3 12
15
66 77 0 28 0 12 0 15 0 16 0 12 0 15 0 16 9 141 2 190 0 72 0 3 0 1 0 0 145 0 3 52
15
66 77 0 28 0 12 0 15 0 16 0 12 0 15 0 16 9 141 2 190 0 72 0 3 0 1 0 0 145 0 3 52
15
66 77 0 28 0 12 0 16 0 16 0 12 0 16 0 16 9 198 2 202 0 75 0 3 0 0 0 0 145 0 3 125
16
66 77 0 28 0 12 0 16 0 16 0 66 77 0 28 0 12 0 16 0 16 0 12 0 16 0 16 9 198 2 202 0
19712
#[exception] PM2.5 Sensor CHECKSUM ERROR!
valid:
12 16 16
66 77 0 28 0 12 0 15 0 16 0 12 0 15 0 16 9 174 2 199 0 71 0 3 0 0 0 0 145 0 3 92
15
66 77 0 28 0 12 0 15 0 16 0 12 0 15 0 16 9 174 2 199 0 71 0 3 0 0 0 0 145 0 3 92
15
66 77 0 28 0 12 0 15 0 16 0 12 0 15 0 16 9 174 2 199 66 77 0 28 0 13 0 16 0 16 0 13
15
#[exception] PM2.5 Sensor CHECKSUM ERROR!
valid:
12 15 16
66 77 0 28 0 13 0 16 0 16 0 13 0 16 0 16 9 213 2 205 0 72 0 3 0 0 0 0 145 0 3 142
16
66 77 0 28 0 13 0 16 0 16 0 13 0 16 0 16 9 213 2 205 0 72 0 3 0 0 0 0 145 0 3 142
16
66 77 0 28 0 13 0 16 0 17 0 13 0 16 0 17 9 207 2 208 0 83 0 6 0 1 0 0 145 0 3 156
16
66 77 0 28 0 13 0 16 0 17 0 13 0 16 0 17 9 207 2 208 0 83 0 6 0 1 0 0 145 0 3 156
16
66 77 0 28 0 13 0 17 0 17 0 13 0 17 0 17 9 159 2 202 0 87 0 5 0 1 0 0 145 0 3 107
17
66 77 0 28 0 13 0 17 0 17 0 66 77 0 28 0 13 0 17 0 17 0 13 0 17 0 17 9 159 2 202 0
19712
#[exception] PM2.5 Sensor CHECKSUM ERROR!
valid:
13 17 17
I want to give a big input in command prompt i.e.
100 100
437 1
68 0
319 0
565 0
307 1
512 0
493 0
30 0
557 0
367 0
547 1
263 0
481 0
78 0
492 1
56 1
81 0
154 0
503 1
375 0
152 0
401 0
226 0
482 0
264 0
52 0
9 0
145 0
72 0
293 0
15 0
42 1
305 0
34 0
509 0
156 0
321 0
437 0
510 1
449 0
79 0
492 0
191 0
354 1
529 0
315 0
384 0
371 0
100 0
480 0
408 0
221 0
286 0
551 0
106 0
123 0
549 0
183 1
428 0
435 0
370 0
46 0
289 0
246 0
414 1
159 0
442 0
286 0
21 0
119 0
263 0
572 0
334 0
96 0
307 0
323 0
554 1
487 0
455 1
399 0
559 0
276 0
357 1
586 0
346 0
240 1
492 0
63 0
262 0
489 0
124 1
526 0
350 1
243 0
35 0
292 0
418 0
364 1
41 1
519 0
This is the part of my code that takes the input :
for (int i = 0; i < n; i++)
{
cin >> a >> b;
A.insert(std::pair<unsigned int, bool>(a, b));
total += a;
if (b == 1)
totalI++;
}
I can obviously just type it all but there must be a shorter way and I want to know what it is. Or is there some other tool / IDE I can use. I am using Visual Studio 2015 (Community)
Just redirect input to the program.
C:\> my_litte_program <mydata.txt
To run the program that way from within Visual Studio 2015, just configure it in the project's debug options:
This works also for running without debugging, e.g. via Ctrl+F5.
Stop using cin; instead, use a stream declared by you. Say, myin.
Put all that "big inputs" of yours in a text file, say, myBigInputs.txt
Make the main() function of your program accept an argument, which is going to be a filename. Use that filename to open myin with. Edit your launch configuration and make it so that when you launch your program from within your IDE, the argument myBigInputs.txt is passed to it.
You can also make the argument to your program optional. If you detect that no argument was passed, then just do myin = cin; in which case your program will work exactly as previously. Then, you will also be able to run your program as follows: myprogram < myBigInput.txt
Open Visual Studio. Load the project. Go to Debug ... Properties ... Configuration Properties ... Debugging
That gives you all the options for setting up the command line. Add "< data.txt" as "command argument"
I need to perform a matching between an image and an histogram I receive as a text.
I do the cdf for both of them:
//Calculating cumulative histogram of src
double total = src.rows*src.cols;
double probSrc[255];
int newValuesSrc[255];
double cuml = 0;
for(int j = 0; j < 256; j++)
{
probSrc[j] = imageHistogram[j]/total; // Probability of each value in image
cuml = cuml + probSrc[j]; // Cumulative probability of current and all previous values
double cdfmax = cuml * 255; // Cumulative probability * max value
newValuesSrc[j] = (int) round(cdfmax);
cout << imageHistogram[j] << " "<< probSrc[j] << " " << newValuesSrc[j] << endl;
}
readHistogramFromFile();
//Calculating cumulative histogram from file
double probDst[255];
int newValuesDst[255];
cuml = 0;
for(int j = 0; j < 256; j++)
{
probDst[j] = receivedHistogram[j]/total; // Probability of each value in image
cuml = cuml + probDst[j]; // Cumulative probability of current and all previous values
double cdfmax = cuml * 255; // Cumulative probability * max value
newValuesDst[j] = (int) round(cdfmax);
cout << receivedHistogram[j] << " "<< probDst[j] << " " << newValuesDst[j] << endl;
}
and I get this values:
For the src image:
207677 0.0901376 23
37615 0.016326 27
19098 0.00828906 29
11955 0.0051888 31
8744 0.00379514 32
7386 0.00320573 32
6546 0.00284115 33
6178 0.00268142 34
5967 0.00258984 34
5437 0.00235981 35
5280 0.00229167 36
5127 0.00222526 36
5002 0.00217101 37
4839 0.00210026 37
4754 0.00206337 38
4676 0.00202951 38
4547 0.00197352 39
4517 0.0019605 39
4484 0.00194618 40
4290 0.00186198 40
4197 0.00182161 41
4188 0.00181771 41
4265 0.00185113 42
4229 0.0018355 42
4233 0.00183724 43
4245 0.00184245 43
4358 0.00189149 44
4330 0.00187934 44
4400 0.00190972 45
4474 0.00194184 45
4519 0.00196137 46
4415 0.00191623 46
4477 0.00194314 47
4468 0.00193924 47
4580 0.00198785 48
4416 0.00191667 48
4558 0.0019783 49
4674 0.00202865 49
4705 0.0020421 50
4998 0.00216927 50
4848 0.00210417 51
4782 0.00207552 51
4883 0.00211936 52
4989 0.00216536 52
4957 0.00215148 53
4987 0.0021645 53
5133 0.00222786 54
4967 0.00215582 54
5217 0.00226432 55
5185 0.00225043 56
5140 0.0022309 56
5236 0.00227257 57
5291 0.00229644 57
5458 0.00236892 58
5473 0.00237543 59
5464 0.00237153 59
5495 0.00238498 60
5439 0.00236068 60
5458 0.00236892 61
5557 0.00241189 62
5881 0.00255252 62
5900 0.00256076 63
5935 0.00257595 64
5902 0.00256163 64
6040 0.00262153 65
6203 0.00269227 66
6146 0.00266753 66
6140 0.00266493 67
6075 0.00263672 68
6054 0.0026276 68
6238 0.00270747 69
6060 0.00263021 70
6153 0.00267057 70
6303 0.00273568 71
6231 0.00270443 72
6278 0.00272483 72
6360 0.00276042 73
6359 0.00275998 74
6368 0.00276389 75
6438 0.00279427 75
6329 0.00274696 76
6408 0.00278125 77
6360 0.00276042 77
6378 0.00276823 78
6329 0.00274696 79
6394 0.00277517 79
6517 0.00282856 80
6521 0.0028303 81
6707 0.00291102 82
6788 0.00294618 82
6761 0.00293446 83
6878 0.00298524 84
7004 0.00303993 85
6963 0.00302214 85
7050 0.0030599 86
6940 0.00301215 87
6875 0.00298394 88
7073 0.00306988 89
7035 0.00305339 89
7146 0.00310156 90
7007 0.00304123 91
7159 0.0031072 92
7089 0.00307682 92
7185 0.00311849 93
7410 0.00321615 94
7237 0.00314106 95
7334 0.00318316 96
7364 0.00319618 97
7452 0.00323437 97
7760 0.00336806 98
7839 0.00340234 99
7882 0.00342101 100
7885 0.00342231 101
8055 0.00349609 102
7923 0.0034388 103
8165 0.00354384 103
8306 0.00360503 104
8271 0.00358984 105
8275 0.00359158 106
8634 0.0037474 107
8684 0.0037691 108
8752 0.00379861 109
9080 0.00394097 110
8958 0.00388802 111
9094 0.00394705 112
9279 0.00402734 113
9234 0.00400781 114
9348 0.00405729 115
9440 0.00409722 116
9431 0.00409332 117
9662 0.00419358 118
9842 0.0042717 119
9816 0.00426042 121
9957 0.00432161 122
10353 0.00449349 123
10626 0.00461198 124
10764 0.00467187 125
10832 0.00470139 126
10767 0.00467318 128
11222 0.00487066 129
11469 0.00497786 130
11661 0.0050612 131
11731 0.00509158 133
12023 0.00521832 134
12086 0.00524566 135
12094 0.00524913 137
12362 0.00536545 138
12364 0.00536632 139
12659 0.00549436 141
12587 0.00546311 142
12776 0.00554514 144
13037 0.00565842 145
13252 0.00575174 147
13425 0.00582682 148
13595 0.00590061 150
13795 0.00598741 151
14308 0.00621007 153
14232 0.00617708 154
14657 0.00636155 156
14966 0.00649566 157
14867 0.00645269 159
15051 0.00653255 161
15510 0.00673177 162
15357 0.00666536 164
15326 0.00665191 166
15308 0.0066441 168
15316 0.00664757 169
15321 0.00664974 171
15298 0.00663976 173
15435 0.00669922 174
15496 0.00672569 176
15307 0.00664366 178
15343 0.00665929 179
15356 0.00666493 181
15315 0.00664714 183
15444 0.00670312 185
15346 0.00666059 186
15583 0.00676345 188
15429 0.00669661 190
15641 0.00678863 191
15661 0.00679731 193
15638 0.00678733 195
15689 0.00680946 197
15866 0.00688628 198
15552 0.00675 200
15150 0.00657552 202
15185 0.00659071 203
14941 0.00648481 205
14989 0.00650564 207
14585 0.0063303 208
14718 0.00638802 210
14553 0.00631641 212
14612 0.00634201 213
14520 0.00630208 215
14358 0.00623177 216
13931 0.00604644 218
13580 0.0058941 220
13370 0.00580295 221
13281 0.00576432 222
13053 0.00566536 224
12711 0.00551693 225
12556 0.00544965 227
12556 0.00544965 228
12125 0.00526259 229
12184 0.00528819 231
11975 0.00519748 232
12198 0.00529427 233
11919 0.00517318 235
11898 0.00516406 236
11589 0.00502995 237
11348 0.00492535 239
11011 0.00477908 240
10523 0.00456727 241
10388 0.00450868 242
9795 0.0042513 243
9251 0.00401519 244
9014 0.00391233 245
8436 0.00366146 246
8266 0.00358767 247
7851 0.00340755 248
7299 0.00316797 249
6996 0.00303646 250
6303 0.00273568 250
5625 0.00244141 251
5375 0.0023329 251
5102 0.00221441 252
4747 0.00206033 253
4313 0.00187196 253
3809 0.00165321 253
3307 0.00143533 254
2756 0.00119618 254
2276 0.000987847 254
1935 0.000839844 255
1617 0.000701823 255
1087 0.000471788 255
547 0.000237413 255
217 9.4184e-05 255
31 1.34549e-05 255
4 1.73611e-06 255
0 0 255
0 0 255
0 0 255
0 0 255
0 0 255
0 0 255
0 0 255
0 0 255
0 0 255
0 0 255
0 0 255
0 0 255
0 0 255
0 0 255
0 0 255
0 0 255
0 0 255
And for the histogram I receive:
10 4.34028e-06 0
11 4.77431e-06 0
12 5.20833e-06 0
13 5.64236e-06 0
14 6.07639e-06 0
15 6.51042e-06 0
16 6.94444e-06 0
17 7.37847e-06 0
18 7.8125e-06 0
19 8.24653e-06 0
20 8.68056e-06 0
22 9.54861e-06 0
24 1.04167e-05 0
26 1.12847e-05 0
28 1.21528e-05 0
30 1.30208e-05 0
34 1.47569e-05 0
38 1.64931e-05 0
42 1.82292e-05 0
50 2.17014e-05 0
60 2.60417e-05 0
70 3.03819e-05 0
80 3.47222e-05 0
90 3.90625e-05 0
100 4.34028e-05 0
120 5.20833e-05 0
140 6.07639e-05 0
160 6.94444e-05 0
160 6.94444e-05 0
150 6.51042e-05 0
140 6.07639e-05 0
130 5.64236e-05 0
120 5.20833e-05 0
110 4.77431e-05 0
100 4.34028e-05 0
90 3.90625e-05 0
80 3.47222e-05 0
70 3.03819e-05 0
60 2.60417e-05 0
50 2.17014e-05 0
40 1.73611e-05 0
30 1.30208e-05 0
20 8.68056e-06 0
10 4.34028e-06 0
10 4.34028e-06 0
10 4.34028e-06 0
10 4.34028e-06 0
11 4.77431e-06 0
12 5.20833e-06 0
13 5.64236e-06 0
14 6.07639e-06 0
15 6.51042e-06 0
16 6.94444e-06 0
17 7.37847e-06 0
18 7.8125e-06 0
19 8.24653e-06 0
20 8.68056e-06 0
22 9.54861e-06 0
24 1.04167e-05 0
26 1.12847e-05 0
28 1.21528e-05 0
30 1.30208e-05 0
34 1.47569e-05 0
38 1.64931e-05 0
42 1.82292e-05 0
50 2.17014e-05 0
60 2.60417e-05 0
70 3.03819e-05 0
80 3.47222e-05 0
90 3.90625e-05 0
100 4.34028e-05 0
120 5.20833e-05 0
140 6.07639e-05 0
160 6.94444e-05 0
160 6.94444e-05 0
150 6.51042e-05 0
140 6.07639e-05 0
130 5.64236e-05 1
120 5.20833e-05 1
110 4.77431e-05 1
100 4.34028e-05 1
90 3.90625e-05 1
80 3.47222e-05 1
70 3.03819e-05 1
60 2.60417e-05 1
50 2.17014e-05 1
40 1.73611e-05 1
30 1.30208e-05 1
20 8.68056e-06 1
10 4.34028e-06 1
10 4.34028e-06 1
11 4.77431e-06 1
12 5.20833e-06 1
13 5.64236e-06 1
14 6.07639e-06 1
15 6.51042e-06 1
16 6.94444e-06 1
17 7.37847e-06 1
18 7.8125e-06 1
19 8.24653e-06 1
20 8.68056e-06 1
22 9.54861e-06 1
24 1.04167e-05 1
26 1.12847e-05 1
28 1.21528e-05 1
30 1.30208e-05 1
34 1.47569e-05 1
38 1.64931e-05 1
42 1.82292e-05 1
50 2.17014e-05 1
60 2.60417e-05 1
70 3.03819e-05 1
80 3.47222e-05 1
90 3.90625e-05 1
100 4.34028e-05 1
120 5.20833e-05 1
140 6.07639e-05 1
160 6.94444e-05 1
160 6.94444e-05 1
150 6.51042e-05 1
140 6.07639e-05 1
130 5.64236e-05 1
120 5.20833e-05 1
110 4.77431e-05 1
100 4.34028e-05 1
90 3.90625e-05 1
80 3.47222e-05 1
70 3.03819e-05 1
60 2.60417e-05 1
50 2.17014e-05 1
40 1.73611e-05 1
30 1.30208e-05 1
20 8.68056e-06 1
10 4.34028e-06 1
10 4.34028e-06 1
10 4.34028e-06 1
10 4.34028e-06 1
11 4.77431e-06 1
12 5.20833e-06 1
13 5.64236e-06 1
14 6.07639e-06 1
15 6.51042e-06 1
16 6.94444e-06 1
17 7.37847e-06 1
18 7.8125e-06 1
19 8.24653e-06 1
20 8.68056e-06 1
22 9.54861e-06 1
24 1.04167e-05 1
26 1.12847e-05 1
28 1.21528e-05 1
30 1.30208e-05 1
34 1.47569e-05 1
38 1.64931e-05 1
42 1.82292e-05 1
50 2.17014e-05 1
60 2.60417e-05 1
70 3.03819e-05 1
80 3.47222e-05 1
90 3.90625e-05 1
100 4.34028e-05 1
120 5.20833e-05 1
140 6.07639e-05 1
160 6.94444e-05 1
160 6.94444e-05 1
150 6.51042e-05 1
140 6.07639e-05 1
130 5.64236e-05 1
120 5.20833e-05 1
110 4.77431e-05 1
100 4.34028e-05 1
90 3.90625e-05 1
80 3.47222e-05 1
70 3.03819e-05 1
60 2.60417e-05 1
50 2.17014e-05 1
40 1.73611e-05 1
30 1.30208e-05 1
20 8.68056e-06 1
10 4.34028e-06 1
10 4.34028e-06 1
10 4.34028e-06 1
20 8.68056e-06 1
30 1.30208e-05 1
40 1.73611e-05 1
50 2.17014e-05 1
60 2.60417e-05 1
70 3.03819e-05 1
80 3.47222e-05 1
90 3.90625e-05 1
100 4.34028e-05 1
120 5.20833e-05 1
140 6.07639e-05 1
160 6.94444e-05 1
160 6.94444e-05 1
150 6.51042e-05 1
140 6.07639e-05 1
130 5.64236e-05 1
120 5.20833e-05 1
110 4.77431e-05 1
100 4.34028e-05 1
90 3.90625e-05 1
80 3.47222e-05 1
70 3.03819e-05 1
60 2.60417e-05 1
50 2.17014e-05 1
40 1.73611e-05 1
30 1.30208e-05 1
20 8.68056e-06 1
10 4.34028e-06 1
10 4.34028e-06 1
10 4.34028e-06 1
20 8.68056e-06 1
30 1.30208e-05 1
40 1.73611e-05 1
40 1.73611e-05 1
50 2.17014e-05 1
55 2.38715e-05 1
60 2.60417e-05 1
65 2.82118e-05 1
70 3.03819e-05 1
75 3.25521e-05 1
80 3.47222e-05 1
85 3.68924e-05 2
90 3.90625e-05 2
95 4.12326e-05 2
90 3.90625e-05 2
80 3.47222e-05 2
70 3.03819e-05 2
60 2.60417e-05 2
50 2.17014e-05 2
40 1.73611e-05 2
30 1.30208e-05 2
20 8.68056e-06 2
10 4.34028e-06 2
10 4.34028e-06 2
10 4.34028e-06 2
20 8.68056e-06 2
30 1.30208e-05 2
40 1.73611e-05 2
40 1.73611e-05 2
50 2.17014e-05 2
55 2.38715e-05 2
60 2.60417e-05 2
65 2.82118e-05 2
70 3.03819e-05 2
75 3.25521e-05 2
80 3.47222e-05 2
85 3.68924e-05 2
90 3.90625e-05 2
95 4.12326e-05 2
100 4.34028e-05 2
105 4.55729e-05 2
110 4.77431e-05 2
115 4.99132e-05 2
120 5.20833e-05 2
As you can see, the src histogram are a lot more distributed than the histrogram I receive ([0-255] against [0-2]).
My question is, what do I do now? How do I match them?
Why don't you scale [0-2] histogram to [0-255]? oldValue * 255 / 2.