I need to create DXF files containing curves and spline.
I am since long able to create simple DXF R12 File like this:
0
SECTION
2
ENTITIES
0
POLYLINE
8
0
66
1
10
0.0
20
0.0
30
0.0
70
0
75
6
62
1
0
VERTEX
8
0
10
1044.52
20
825.596
30
0.0
0
VERTEX
8
0
10
1044.52
20
700.099
30
0.0
0
SEQEND
0
ENDSEC
0
EOF
According to the DXF documentation (https://www.autodesk.com/techpubs/autocad/acad2000/dxf/polyline_dxf_06.htm and also http://paulbourke.net/dataformats/dxf/dxf10.html) even R12 POLYLINES can be given a "SPLINE" attribute, so i tried this:
0
SECTION
2
ENTITIES
0
POLYLINE
8
0
66
1
10
0.0
20
0.0
30
0.0
70
4
62
1
0
VERTEX
8
0
10
1044.52
20
1825.596
30
0.0
70
8
0
VERTEX
8
0
10
644.52
20
1025.596
30
0.0
70
8
0
VERTEX
8
0
10
544.52
20
325.596
30
0.0
70
8
0
VERTEX
8
0
10
44.52
20
25.596
30
0.0
70
8
0
VERTEX
8
0
10
1044.52
20
700.099
30
0.0
0
SEQEND
0
ENDSEC
0
EOF
But this still shows only straight lines.
Is it possible to create DXF Files with SPLINE content in DXF 12
Can anybody point to SEIMPLE sample of how to create a spline in
a DXF file
Started with the information in
https://forums.autodesk.com/t5/visual-lisp-autolisp-and-general/manually-writing-a-minimal-dxf/td-p/2081140
and was able to make a headerless, tableless file.dxf
which succesfully draws two lines
MinTwoLines.dxf
0
SECTION
2
ENTITIES
0
LINE
8
DXF_Lines_LayerName
10
0.0
20
0.0
30
0.0
11
1.0
21
0.0
31
0.0
0
LINE
8
DXF_Lines_LayerName
10
1.0
20
0.0
30
0.0
11
1.0
21
0.0
31
2.0
0
ENDSEC
0
EOF
Plopping a SPLINE ENTITY into the same location as the LINE failed.
so nabbed the splineA.dxf file from
https://github.com/jscad/sample-files/blob/master/dxf/dxf-parser/splines.dxf
and incrementally hacked off bits until no more could be successfully removed.
The files resulting from laminating the below files can be successfully imported into Rhino 7.
The file lamination commands in Windows are:\
to make a single file with only the FourPoint spline
copy MinSplineHead.txt+FourPointSplineBody.txt+MinSplineTail.txt MinSpline.dxf
to make a single file with only the TwentyOnePoint spline
copy MinSplineHead.txt+TwentyOnePointSplineBody.txt+MinSplineTail.txt MinSpline.dxf
to make a single file with both splines
copy MinSplineHead.txt+FourPointSplineBody.txt+TwentyOnePointSplineBody.txt+MinSplineTail.txt MinSpline.dxf
MinSplineHead.txt
0
SECTION
2
HEADER
9
$ACADVER
1
AC1021
0
ENDSEC
0
SECTION
2
TABLES
0
TABLE
2
LAYER
5
2
330
0
100
AcDbSymbolTable
70
1
0
LAYER
5
10
330
2
100
AcDbSymbolTableRecord
100
AcDbLayerTableRecord
2
0
70
0
62
7
6
CONTINUOUS
370
0
390
F
0
ENDTAB
0
TABLE
2
BLOCK_RECORD
5
1
330
0
100
AcDbSymbolTable
70
2
0
BLOCK_RECORD
5
1F
330
1
100
AcDbSymbolTableRecord
100
AcDbBlockTableRecord
2
*Model_Space
70
0
280
1
281
0
0
BLOCK_RECORD
5
1E
330
1
100
AcDbSymbolTableRecord
100
AcDbBlockTableRecord
2
*Paper_Space
70
0
280
1
281
0
0
ENDTAB
0
ENDSEC
0
SECTION
2
ENTITIES
FourPointSplineBody.txt
0
SPLINE
8
DXF_Spline_LayerName
100
AcDbSpline
70
8
71
2
72
7
73
4
74
0
42
1e-07
43
1e-07
40
0
40
0
40
0
40
0.5
40
1
40
1
40
1
10
0
20
0
30
0
10
122.4
20
-38.5
30
0
10
77.5
20
149.5
30
0
10
200
20
100
30
0
TwentyOnePointSplineBody.txt
0
SPLINE
8
DXF_Spline_LayerName
100
AcDbSpline
70
0
71
3
72
25
73
21
74
0
42
0.00000
43
0.00000
40
0.0
40
0.0
40
0.0
40
0.0
40
0.02777
40
0.05555
40
0.08333
40
0.11111
40
0.13888
40
0.16666
40
0.19444
40
0.22222
40
0.25
40
0.27777
40
0.30555
40
0.33333
40
0.36111
40
0.38888
40
0.41666
40
0.44444
40
0.47222
40
0.5
40
0.5
40
0.5
40
0.5
10
7.000
20
-4.000
30
-5.500
10
7.288
20
-4.008
30
-5.492
10
7.955
20
-4.029
30
-5.416
10
8.881
20
-4.062
30
-5.168
10
9.750
20
-4.097
30
-4.763
10
10.535
20
-4.136
30
-4.213
10
11.213
20
-4.177
30
-3.535
10
11.763
20
-4.222
30
-2.750
10
12.168
20
-4.270
30
-1.881
10
12.416
20
-4.321
30
-0.955
10
12.500
20
-4.375
30
-0.000
10
12.416
20
-4.432
30
0.955
10
12.168
20
-4.492
30
1.881
10
11.763
20
-4.556
30
2.750
10
11.213
20
-4.622
30
3.535
10
10.535
20
-4.691
30
4.213
10
9.750
20
-4.764
30
4.763
10
8.881
20
-4.840
30
5.168
10
7.955
20
-4.918
30
5.416
10
7.288
20
-4.975
30
5.492
10
7.000
20
-5.000
30
5.500
MinSplineTail.txt
0
ENDSEC
0
EOF
Related
I have an ESP8266 connected to PMS5003 particulate matter sensor through (hardware) UART.
I'm getting many checksum errors while reading from PMS5003.
Here's the library that I'm using to communicate with PMS5003:
PMS5003.cpp
#include "PMS5003.h"
void PMS5003::processDataOn(HardwareSerial &serial) {
unsigned long timeout = millis();
int count = 0;
byte incomeByte[NUM_INCOME_BYTE];
boolean startcount = false;
byte data;
int timeoutHops = 0;
while (1){
if (((millis() - timeout) > 1000) && (timeoutHops == 0)) {
timeoutHops = 1;
yield();
ESP.wdtFeed();
}
if (((millis() - timeout) > 2000) && (timeoutHops == 1)) {
timeoutHops = 2;
yield();
ESP.wdtFeed();
}
if ((millis() - timeout) > 3000){
Serial.println("SENSOR-ERROR-TIMEOUT");
break;
}
if (serial.available()){
data = serial.read();
if (data == CHAR_PRELIM && !startcount) {
startcount = true;
count++;
incomeByte[0] = data;
} else if (startcount) {
count++;
incomeByte[count - 1] = data;
if (count >= NUM_INCOME_BYTE){
break;
}
}
}
}
unsigned int calcsum = 0;
unsigned int exptsum;
for (int i = 0; i < NUM_DATA_BYTE; i++) {
calcsum += (unsigned int)incomeByte[i];
}
exptsum = ((unsigned int)incomeByte[CHECK_BYTE] << 8) + (unsigned int)incomeByte[CHECK_BYTE + 1];
if (calcsum == exptsum) {
pm1 = ((unsigned int)incomeByte[PM1_BYTE] << 8) + (unsigned int)incomeByte[PM1_BYTE + 1];
pm25 = ((unsigned int)incomeByte[PM25_BYTE] << 8) + (unsigned int)incomeByte[PM25_BYTE + 1];
pm10 = ((unsigned int)incomeByte[PM10_BYTE] << 8) + (unsigned int)incomeByte[PM10_BYTE + 1];
} else {
Serial.println("#[exception] PM2.5 Sensor CHECKSUM ERROR!");
pm1 = -1;
pm25 = -1;
pm10 = -1;
}
return;
}
int PMS5003::getPM1() {
return pm1;
}
int PMS5003::getPM25() {
return pm25;
}
int PMS5003::getPM10() {
return pm10;
}
PMS5003.h
#ifndef _PMS_5003_H
#define _PMS_5003_H
#include <Wire.h>
#include <Arduino.h>
#define VERSION 0.2
#define Sense_PM 6
#define NUM_INCOME_BYTE 32
#define CHAR_PRELIM 0x42
#define NUM_DATA_BYTE 29
#define CHECK_BYTE 30
#define PM1_BYTE 10
#define PM25_BYTE 12
#define PM10_BYTE 14
class PMS5003 {
public:
//void processData(int *PM1, int *PM25, int *PM10);
void processDataOn(HardwareSerial &serial);
int getPM1();
int getPM25();
int getPM10();
private:
int pm1;
int pm25;
int pm10;
};
#endif
Here's how I'm using it:
struct ParticulateMatterMeasurements {
private:
bool _areValid = false;
int PM01Value = 0;
int PM25Value = 0;
int PM10Value = 0;
public:
void setAreValid(bool _areValid) {
ParticulateMatterMeasurements::_areValid = _areValid;
}
bool getAreValid() const {
return _areValid;
}
int getPM01Value() const {
return PM01Value;
}
void setPM01Value(int PM01Value) {
ParticulateMatterMeasurements::PM01Value = PM01Value;
}
int getPM25Value() const {
return PM25Value;
}
void setPM25Value(int PM25Value) {
ParticulateMatterMeasurements::PM25Value = PM25Value;
}
int getPM10Value() const {
return PM10Value;
}
void setPM10Value(int PM10Value) {
ParticulateMatterMeasurements::PM10Value = PM10Value;
}
};
ParticulateMatterMeasurements getMeasurements() {
ParticulateMatterMeasurements measurements;
measurements.setAreValid(false);
pms5003.processDataOn(Serial);
measurements.setPM01Value(pms5003.getPM1());
measurements.setPM25Value(pms5003.getPM25());
measurements.setPM10Value(pms5003.getPM10());
if (measurements.getPM01Value() != -1 && measurements.getPM25Value() != -1 && measurements.getPM10Value() != -1) {
measurements.setAreValid(true);
}
return measurements;
}
The problem is that I get many checksum errors. During 60 measurements I get about 100 of: #[exception] PM2.5 Sensor CHECKSUM ERROR!.
What could be the problem here?
#edit
I ran a test where I print what PMS5003 sends to my ESP8266. It looks like the checksum which is the last byte is sometimes not sent. Instead, I get 66 usually but I can see sometimes 66 77 instead of the last 2 bytes as well.
66 77 0 28 0 13 0 16 0 20 0 13 0 16 0 20 9 201 2 216 0 86 0 8 0 3 0 1 145 0 3 172
16
66 77 0 28 0 13 0 16 0 20 0 13 0 16 0 20 9 201 2 216 0 86 0 8 0 3 0 1 145 0 3 172
16
66 77 0 28 0 14 0 18 0 22 0 14 0 18 0 22 10 32 2 239 0 93 0 9 0 3 0 1 145 0 3 45
18
66 77 0 28 0 14 0 18 0 22 0 14 0 18 0 22 10 32 2 239 0 93 0 9 0 3 0 1 145 0 3 66
18
#[exception] PM2.5 Sensor CHECKSUM ERROR!
66 77 0 28 0 13 0 17 0 20 0 13 0 17 0 20 10 50 2 231 0 88 0 9 0 3 0 1 145 0 3 42
17
66 77 0 28 0 12 0 16 0 20 0 12 0 16 0 20 9 225 2 208 0 90 0 8 0 3 0 1 145 0 3 190
16
66 77 0 28 0 13 0 17 0 21 0 13 0 17 0 21 9 225 2 208 0 90 0 8 0 3 0 1 145 0 3 196
17
66 77 0 28 0 12 0 15 0 17 0 12 0 15 0 17 9 249 2 211 0 76 0 5 0 0 0 0 145 0 3 188
15
66 77 0 28 0 12 0 15 0 17 0 12 0 15 0 17 9 249 2 211 0 76 0 5 0 0 0 0 145 0 3 188
15
66 77 0 28 0 12 0 15 0 16 0 12 0 15 0 16 9 210 2 188 0 70 0 5 0 0 0 0 145 0 3 118
15
66 77 0 28 0 12 0 15 0 16 0 12 0 15 0 16 9 210 2 188 0 70 0 5 0 0 0 0 145 0 3 118
15
66 77 0 28 0 13 0 16 0 17 0 13 0 16 0 17 9 198 2 183 0 78 0 5 0 0 0 0 145 0 3 115
16
66 77 0 28 0 13 0 16 0 17 0 13 0 16 0 17 9 198 2 183 0 78 0 5 0 0 0 0 145 0 3 66
16
#[exception] PM2.5 Sensor CHECKSUM ERROR!
66 77 0 28 0 12 0 16 0 18 0 12 0 16 0 18 9 234 2 195 0 87 0 8 0 1 0 0 145 0 3 176
16
66 77 0 28 0 12 0 16 0 18 0 12 0 16 0 18 9 234 2 195 0 87 0 8 0 1 0 0 145 0 3 176
16
66 77 0 28 0 12 0 15 0 17 0 12 0 15 0 17 9 186 2 184 0 77 0 6 0 1 0 0 145 0 3 101
15
66 77 0 28 0 13 0 16 0 18 0 13 0 16 0 18 9 186 2 184 0 77 0 6 0 1 0 0 145 0 3 107
16
66 77 0 28 0 13 0 16 0 18 0 13 0 16 0 18 9 186 2 184 0 77 0 6 0 1 0 0 145 0 3 107
16
66 77 0 28 0 13 0 17 0 18 0 13 0 17 0 18 9 165 2 180 0 76 0 6 0 1 0 0 145 0 3 83
17
66 77 0 28 0 13 0 17 0 18 0 13 0 17 0 18 9 165 2 180 0 76 0 6 0 1 0 0 145 0 3 83
17
66 77 0 28 0 13 0 17 0 18 0 13 0 17 0 18 9 156 2 186 0 76 0 6 0 1 0 0 145 0 3 80
17
66 77 0 28 0 12 0 16 0 17 0 12 0 16 0 17 9 156 2 186 0 76 0 6 0 1 0 0 145 0 3 66
16
#[exception] PM2.5 Sensor CHECKSUM ERROR!
66 77 0 28 0 12 0 16 0 17 0 12 0 16 0 17 9 249 2 200 0 67 0 6 0 1 0 0 145 0 3 172
16
66 77 0 28 0 12 0 16 0 17 0 12 0 16 0 17 9 249 2 200 0 67 0 6 0 1 0 0 145 0 3 172
16
66 77 0 28 0 12 0 16 0 17 0 12 0 16 0 17 9 231 2 197 0 73 0 7 0 1 0 0 145 0 3 158
16
66 77 0 28 0 12 0 16 0 17 0 12 0 16 0 17 9 231 2 197 0 73 0 7 0 1 0 0 145 0 3 158
16
66 77 0 28 0 13 0 17 0 18 0 13 0 17 0 18 9 243 2 194 0 73 0 7 0 1 0 0 145 0 3 173
17
66 77 0 28 0 13 0 17 0 18 0 13 0 17 0 18 9 243 2 194 0 73 0 7 0 1 0 0 145 0 3 173
17
66 77 0 28 0 13 0 17 0 18 0 13 0 17 0 18 9 231 2 190 0 70 0 8 0 1 0 0 145 0 3 155
17
66 77 0 28 0 13 0 17 0 18 0 13 0 17 0 18 9 231 2 190 0 70 0 8 0 1 0 0 145 0 3 155
17
66 77 0 28 0 13 0 17 0 19 0 13 0 17 0 19 10 65 2 219 0 68 0 7 0 1 0 0 145 0 3 66
17
#[exception] PM2.5 Sensor CHECKSUM ERROR!
66 77 0 28 0 13 0 17 0 19 0 13 0 17 0 19 10 65 2 219 0 68 0 7 0 1 0 0 145 0 3 18
17
66 77 0 28 0 13 0 18 0 20 0 13 0 18 0 20 10 86 2 218 0 79 0 11 0 2 0 0 145 0 3 58
18
66 77 0 28 0 13 0 18 0 20 0 13 0 18 0 20 10 86 2 218 0 79 0 11 0 2 0 0 145 0 3 58
18
66 77 0 28 0 13 0 18 0 19 0 13 0 18 0 19 10 86 2 216 0 76 0 8 0 1 0 0 145 0 3 47
18
66 77 0 28 0 13 0 18 0 19 0 13 0 18 0 19 10 86 2 216 0 76 0 8 0 1 0 0 145 0 3 47
18
66 77 0 28 0 14 0 18 0 20 0 14 0 18 0 20 10 212 2 250 0 75 0 8 0 1 0 0 145 0 3 210
18
66 77 0 28 0 14 0 18 0 20 0 14 0 18 0 20 10 212 2 250 0 75 0 8 0 1 0 0 145 0 3 210
18
66 77 0 28 0 12 0 17 0 20 0 12 0 17 0 20 10 137 2 234 0 86 0 10 0 1 0 0 145 0 3 126
17
66 77 0 28 0 12 0 17 0 20 0 12 0 17 0 20 10 137 2 234 0 86 0 10 0 1 0 0 145 0 3 126
17
66 77 0 28 0 12 0 17 0 20 0 12 0 17 0 20 10 137 2 234 0 86 0 10 0 1 0 0 145 0 3 66
17
#[exception] PM2.5 Sensor CHECKSUM ERROR!
66 77 0 28 0 14 0 19 0 21 0 14 0 19 0 21 10 164 2 240 0 95 0 10 0 1 0 0 145 0 3 178
19
66 77 0 28 0 14 0 19 0 21 0 14 0 19 0 21 10 164 2 240 0 95 0 10 0 1 0 0 145 0 3 178
19
66 77 0 28 0 13 0 19 0 21 0 13 0 19 0 21 10 110 2 223 0 95 0 10 0 1 0 0 145 0 3 105
19
66 77 0 28 0 13 0 19 0 21 0 13 0 19 0 21 10 110 2 223 0 95 0 10 0 1 0 0 145 0 3 105
19
66 77 0 28 0 13 0 19 0 21 0 13 0 19 0 21 10 128 2 227 0 101 0 10 0 1 0 0 145 0 3 133
19
66 77 0 28 0 13 0 19 0 21 0 13 0 19 0 21 10 128 2 227 0 101 0 10 0 1 0 0 145 0 3 133
19
66 77 0 28 0 14 0 20 0 24 0 14 0 20 0 24 10 158 2 254 0 106 0 13 0 4 0 0 145 0 3 211
20
66 77 0 28 0 13 0 19 0 23 0 13 0 19 0 23 10 158 2 254 0 106 0 13 0 4 0 0 145 0 3 205
19
66 77 0 28 0 13 0 19 0 23 0 13 0 19 0 23 10 212 3 10 0 107 0 12 0 4 0 0 145 0 3 66
19
#[exception] PM2.5 Sensor CHECKSUM ERROR!
66 77 0 28 0 14 0 20 0 24 0 14 0 20 0 24 10 212 3 10 0 107 0 12 0 4 0 0 145 0 3 22
20
66 77 0 28 0 14 0 20 0 24 0 14 0 20 0 24 10 236 3 11 0 109 0 13 0 4 0 0 145 0 3 50
20
66 77 0 28 0 14 0 20 0 24 0 14 0 20 0 24 10 236 3 11 0 109 0 13 0 4 0 0 145 0 3 50
20
66 77 0 28 0 15 0 20 0 23 0 15 0 20 0 23 10 254 3 29 0 105 0 9 0 3 0 0 145 0 3 77
20
66 77 0 28 0 15 0 20 0 23 0 15 0 20 0 23 10 254 3 29 0 105 0 9 0 3 0 0 145 0 3 77
20
66 77 0 28 0 14 0 19 0 22 0 14 0 19 0 22 11 22 3 40 0 99 0 9 0 3 0 0 145 0 2 101
19
66 77 0 28 0 14 0 19 0 22 0 14 0 19 0 22 11 22 3 40 0 99 0 9 0 3 0 0 145 0 2 101
19
66 77 0 28 0 14 0 19 0 21 0 14 0 19 0 21 10 149 3 6 0 93 0 8 0 3 0 0 145 0 2 184
19
66 77 0 28 0 14 0 19 0 21 0 14 0 19 0 21 10 149 3 6 0 93 0 8 0 3 0 0 145 0 2 66
19
#[exception] PM2.5 Sensor CHECKSUM ERROR!
66 77 0 28 0 14 0 18 0 20 0 14 0 18 0 20 10 140 3 2 0 77 0 5 0 3 0 0 145 0 2 148
18
66 77 0 28 0 13 0 17 0 19 0 13 0 17 0 19 10 140 3 2 0 77 0 5 0 3 0 0 145 0 2 142
17
66 77 0 28 0 13 0 17 0 19 0 13 0 17 0 19 10 104 2 251 0 77 0 5 0 3 0 0 145 0 3 98
17
66 77 0 28 0 13 0 17 0 19 0 13 0 17 0 19 10 104 2 251 0 77 0 5 0 3 0 0 145 0 3 98
17
66 77 0 28 0 13 0 17 0 19 0 13 0 17 0 19 10 104 2 251 0 77 0 5 0 3 0 0 145 0 3 98
17
66 77 0 28 0 13 0 18 0 20 0 13 0 18 0 20 10 116 3 11 0 77 0 8 0 3 0 0 145 0 2 134
18
66 77 0 28 0 14 0 19 0 21 0 14 0 19 0 21 10 116 3 11 0 77 0 8 0 3 0 0 145 0 2 140
19
66 77 0 28 0 14 0 19 0 21 0 14 0 19 0 21 10 137 3 18 0 76 0 7 0 3 0 0 145 0 2 166
19
[update] This is the newest version.
66 77 0 28 0 14 0 19 0 21 0 14 0 19 0 21 10 137 3 18 0 76 0 7 0 3 0 0 145 0 2 166
19
66 77 0 28 0 12 0 17 0 18 0 12 0 17 0 18 10 77 2 241 0 77 0 5 0 0 0 0 145 0 3 66
17
#[exception] PM2.5 Sensor CHECKSUM ERROR!
66 77 0 28 0 12 0 17 0 18 0 12 0 17 0 18 10 77 2 241 0 77 0 5 0 0 0 0 145 0 3 54
17
66 77 0 28 0 13 0 18 0 19 0 13 0 18 0 19 10 32 2 224 0 80 0 6 0 0 0 0 145 0 3 2
18
66 77 0 28 0 13 0 18 0 19 0 13 0 18 0 19 10 32 2 224 0 80 0 6 0 0 0 0 145 0 3 2
18
66 77 0 28 0 13 0 19 0 19 0 13 0 19 0 19 10 47 2 240 0 81 0 5 0 0 0 0 145 0 3 35
19
66 77 0 28 0 12 0 18 0 18 0 12 0 18 0 18 10 47 2 240 0 81 0 5 0 0 0 0 145 0 3 29
18
66 77 0 28 0 14 0 20 0 20 0 14 0 20 0 20 10 173 3 6 0 90 0 6 0 1 0 1 145 0 2 202
20
66 77 0 28 0 14 0 20 0 20 0 14 0 20 0 20 10 173 3 6 0 90 0 6 0 1 0 1 145 0 2 202
20
66 77 0 28 0 14 0 21 0 21 0 14 0 21 0 21 10 233 3 24 0 90 0 9 0 1 0 1 145 0 3 31
21
66 77 0 28 0 13 0 19 0 22 0 13 0 19 0 22 10 242 3 25 0 84 0 12 0 4 0 2 145 0 3 38
19
66 77 0 28 0 13 0 19 0 22 0 13 0 19 0 22 10 242 3 25 0 84 0 12 0 4 0 2 145 0 3 38
19
66 77 0 28 0 13 0 19 0 22 0 13 0 19 0 22 10 242 3 25 0 84 0 12 0 4 0 2 145 0 3 66
19
#[exception] PM2.5 Sensor CHECKSUM ERROR!
66 77 0 28 0 15 0 20 0 23 0 15 0 20 0 23 11 79 3 40 0 88 0 9 0 4 0 2 145 0 2 156
20
66 77 0 28 0 15 0 20 0 23 0 15 0 20 0 23 11 79 3 40 0 88 0 9 0 4 0 2 145 0 2 156
20
66 77 0 28 0 14 0 19 0 21 0 14 0 19 0 21 11 67 3 38 0 83 0 9 0 4 0 2 145 0 2 129
19
66 77 0 28 0 14 0 19 0 21 0 14 0 19 0 21 11 67 3 38 0 83 0 9 0 4 0 2 145 0 2 129
19
66 77 0 28 0 14 0 18 0 21 0 14 0 18 0 21 11 58 3 43 0 82 0 8 0 4 0 2 145 0 2 121
18
66 77 0 28 0 15 0 19 0 22 0 15 0 19 0 22 11 58 3 43 0 82 0 8 0 4 0 2 145 0 2 127
19
66 77 0 28 0 15 0 19 0 21 0 15 0 19 0 21 11 31 3 37 0 79 0 7 0 4 0 2 145 0 2 88
19
66 77 0 28 0 15 0 19 0 21 0 15 0 19 0 21 11 31 3 37 0 79 0 7 0 4 0 2 145 0 2 88
19
66 77 0 28 0 14 0 18 0 21 0 14 0 18 0 21 10 203 3 11 0 84 0 8 0 4 0 2 145 0 2 66
18
#[exception] PM2.5 Sensor CHECKSUM ERROR!
66 77 0 28 0 14 0 18 0 21 0 14 0 18 0 21 10 203 3 11 0 84 0 8 0 4 0 2 145 0 2 235
18
66 77 0 28 0 14 0 18 0 21 0 14 0 18 0 21 10 197 3 5 0 84 0 9 0 4 0 2 145 0 2 224
18
66 77 0 28 0 14 0 18 0 21 0 14 0 18 0 21 10 197 3 5 0 84 0 9 0 4 0 2 145 0 2 224
18
66 77 0 28 0 14 0 18 0 21 0 14 0 18 0 21 10 179 3 8 0 81 0 8 0 3 0 1 145 0 2 203
18
66 77 0 28 0 14 0 18 0 21 0 14 0 18 0 21 10 179 3 8 0 81 0 8 0 3 0 1 145 0 2 203
18
66 77 0 28 0 14 0 17 0 20 0 14 0 17 0 20 10 98 2 243 0 75 0 8 0 3 0 1 145 0 3 90
17
66 77 0 28 0 14 0 17 0 20 0 14 0 17 0 20 10 98 2 243 0 75 0 8 0 3 0 1 145 0 3 90
17
66 77 0 28 0 14 0 17 0 19 0 14 0 17 0 19 10 116 2 246 0 77 0 5 0 3 0 1 145 0 3 108
17
66 77 0 28 0 14 0 17 0 19 0 14 0 17 0 19 10 116 2 246 0 77 0 5 0 3 0 1 145 0 3 66
17
#[exception] PM2.5 Sensor CHECKSUM ERROR!
66 77 0 28 0 14 0 17 0 19 0 14 0 17 0 19 10 74 2 232 0 80 0 3 0 1 0 1 145 0 3 51
17
66 77 0 28 0 14 0 17 0 19 0 14 0 17 0 19 10 74 2 232 0 80 0 3 0 1 0 1 145 0 3 51
17
66 77 0 28 0 14 0 17 0 19 0 14 0 17 0 19 10 74 2 232 0 80 0 3 0 1 0 1 145 0 3 51
17
66 77 0 28 0 13 0 17 0 18 0 13 0 17 0 18 9 246 2 220 0 79 0 3 0 1 0 1 145 0 3 205
17
66 77 0 28 0 13 0 17 0 18 0 13 0 17 0 18 9 246 2 220 0 79 0 3 0 1 0 1 145 0 3 205
17
66 77 0 28 0 13 0 17 0 19 0 13 0 17 0 19 9 255 2 222 0 82 0 6 0 1 0 1 145 0 3 224
17
66 77 0 28 0 13 0 17 0 19 0 13 0 17 0 19 9 255 2 222 0 82 0 6 0 1 0 1 145 0 3 224
17
66 77 0 28 0 14 0 18 0 20 0 14 0 18 0 20 10 89 2 242 0 85 0 6 0 1 0 1 145 0 3 88
18
66 77 0 28 0 14 0 18 0 20 0 14 0 18 0 20 10 89 2 242 0 85 0 6 0 1 0 1 145 0 3 66
18
#[exception] PM2.5 Sensor CHECKSUM ERROR!
After some time I get more errors:
66 77 0 28 0 13 0 17 0 66 77 0 28 0 13 0 17 0 17 0 13 0 17 0 17 9 129 2 177 0 65 0
7168
#[exception] PM2.5 Sensor CHECKSUM ERROR!
66 77 0 28 0 13 0 17 0 17 0 66 77 0 28 0 13 0 17 0 17 0 13 0 17 0 17 9 129 2 177 0
19712
#[exception] PM2.5 Sensor CHECKSUM ERROR!
66 77 0 28 0 13 0 17 0 17 0 66 77 0 28 0 12 0 16 0 16 0 12 0 16 0 16 9 186 2 186 0
19712
#[exception] PM2.5 Sensor CHECKSUM ERROR!
66 77 0 28 0 12 0 16 0 16 0 12 0 16 0 16 9 186 2 186 0 67 0 6 0 0 0 0 145 0 3 92
16
66 77 0 28 0 12 0 16 0 16 0 12 0 16 0 16 9 174 2 193 0 62 0 5 0 0 0 0 145 0 3 81
16
66 77 0 28 0 13 0 17 0 17 0 13 0 17 0 17 9 174 2 193 0 62 0 5 0 0 0 0 145 0 3 87
17
66 77 0 28 0 13 0 17 0 18 0 13 0 17 0 18 9 180 2 197 0 62 0 6 0 1 0 1 145 0 3 102
17
66 77 0 28 0 13 0 17 0 18 0 13 0 17 0 18 9 180 2 197 0 62 0 6 0 1 0 1 145 0 3 102
17
66 77 0 28 0 13 0 17 0 18 0 13 0 17 0 18 9 165 2 190 0 62 0 6 0 1 0 1 145 0 3 80
17
66 77 0 28 0 13 0 17 0 66 77 0 28 0 13 0 16 0 18 0 13 0 16 0 18 9 135 2 182 0 65 0
7168
#[exception] PM2.5 Sensor CHECKSUM ERROR!
66 77 0 28 0 13 0 16 0 18 0 66 77 0 28 0 13 0 16 0 18 0 13 0 16 0 18 9 135 2 182 0
19712
#[exception] PM2.5 Sensor CHECKSUM ERROR!
66 77 0 28 0 13 0 16 0 18 0 13 0 16 0 18 9 117 2 180 0 64 0 3 0 2 0 1 145 0 3 20
16
66 77 0 28 0 13 0 16 0 18 0 66 77 0 28 0 13 0 16 0 18 0 13 0 16 0 18 9 117 2 180 0
19712
#[exception] PM2.5 Sensor CHECKSUM ERROR!
66 77 0 28 0 12 0 15 0 17 0 12 0 15 0 17 9 153 2 196 0 64 0 3 0 2 0 1 145 0 3 66
15
66 77 0 28 0 12 0 15 0 17 0 12 0 15 0 17 9 153 2 196 0 64 0 3 0 2 0 1 145 0 3 66
15
[update] This is the newest version.
66 77 0 28 0 12 0 16 0 19 0 12 0 16 0 19 9 162 2 206 0 71 0 6 0 5 0 2 145 0 3 105
16
66 77 0 28 0 13 0 17 0 20 0 13 0 17 0 20 9 162 2 206 0 71 0 6 0 5 0 2 145 0 3 111
17
66 77 0 28 0 13 0 16 0 19 0 13 0 16 0 19 9 123 2 189 0 74 0 6 0 5 0 2 145 0 3 54
16
66 77 0 28 0 13 0 16 0 66 77 0 28 0 12 0 16 0 20 0 12 0 16 0 20 9 54 2 166 0 76 0
7168
#[exception] PM2.5 Sensor CHECKSUM ERROR!
66 77 0 28 0 12 0 16 0 20 0 12 0 16 0 20 9 54 2 166 0 76 0 7 0 6 0 2 145 0 2 222
16
66 77 0 28 0 11 0 15 0 19 0 66 77 0 28 0 11 0 15 0 19 0 11 0 15 0 19 9 21 2 166 0
19712
#[exception] PM2.5 Sensor CHECKSUM ERROR!
66 77 0 28 0 11 0 15 0 19 0 66 77 0 28 0 12 0 16 0 20 0 12 0 16 0 20 9 21 2 166 0
19712
#[exception] PM2.5 Sensor CHECKSUM ERROR!
66 77 0 28 0 13 0 16 0 20 0 13 0 16 0 20 9 120 2 189 0 83 0 9 0 6 0 1 145 0 3 65
16
66 77 0 28 0 13 0 16 0 20 0 13 0 16 0 20 9 120 2 189 0 83 0 9 0 6 0 1 145 0 3 65
16
66 77 0 28 0 13 0 16 0 20 0 13 0 16 0 20 9 186 2 198 0 74 0 9 0 5 0 1 145 0 3 130
16
66 77 0 28 0 13 0 16 0 20 0 13 0 16 0 20 9 186 2 198 0 74 0 9 0 5 0 1 145 0 3 130
16
66 77 0 28 0 13 0 15 0 19 0 13 0 15 0 19 9 171 2 184 0 74 0 9 0 5 0 1 145 0 3 97
15
66 77 0 28 0 13 0 15 0 66 77 0 28 0 13 0 16 0 20 0 13 0 16 0 20 9 198 2 190 0 77 0
7168
#[exception] PM2.5 Sensor CHECKSUM ERROR!
66 77 0 28 0 13 0 16 0 20 0 13 0 16 0 20 9 198 2 190 0 77 0 8 0 5 0 1 145 0 3 136
16
66 77 0 28 0 13 0 16 0 19 0 66 77 0 28 0 13 0 16 0 19 0 13 0 16 0 19 9 225 2 190 0
19712
#[exception] PM2.5 Sensor CHECKSUM ERROR!
66 77 0 28 0 13 0 17 0 20 0 13 0 17 0 20 10 23 2 214 0 81 0 9 0 3 0 0 145 0 2 246
17
66 77 0 28 0 13 0 17 0 20 0 13 0 17 0 20 10 23 2 214 0 81 0 9 0 3 0 0 145 0 2 246
17
66 77 0 28 0 13 0 17 0 20 0 13 0 17 0 20 10 23 2 214 0 81 0 9 0 3 0 0 145 0 2 246
17
66 77 0 28 0 13 0 17 0 20 0 66 77 0 28 0 13 0 17 0 20 0 13 0 17 0 20 10 98 2 234 0
19712
#[exception] PM2.5 Sensor CHECKSUM ERROR!
The lonely number in the new line after 32 bytes is the value of PM2.5 which shouldn't get high. However, it gets sometimes high and sometimes not when the checksum is incorrect.
I wonder why the situation changes over time... Maybe I could keep resetting the UART somehow?
For each of the packets that fail the checksum test, you find a CHAR_PRELIM (66) either in the middle or at the end. This means the sensor is occasionally dropping packets and causing misalignment.
One solution is to restart packet reading each time a 66 is read. This code should do it:
UPDATE: as per #sawdust's comment, the presence of both 66 and 77 should be used as a start condition because it may be possible for 66 to appear by itself in the data. The other consideration is to use the packet length provided by the 3rd and 4th bytes instead of assuming the length to be 32. Hopefully these improvements make the code more durable.
size_t length;
incomingByte[0] = 66; // the first two bytes are always known
incomingByte[1] = 77;
...
if (serial.available()) {
if (serial.read() == 66 && serial.read() == 77)
incomingByte[2] = serial.read(); // length high byte
incomingByte[3] = serial.read(); // length low byte
int length = (incomingByte[2] << 8) + incomingByte[3];
// starting at index 4, read `length` bytes
serial.readBytes(incomingByte + 4, length);
break;
}
}
// when the code breaks out of the while(1) loop, you still need to evaluate the checksum.
According to the protocol defined by this source, the packet length is fixed at 32 bytes, so the encoded frame length (bytes 3 and 4) should always equal 0 28 (32 bytes - 2 start bytes - 2 frame length bytes = 28).
However, this code should work even for variable length packets (thanks #sawdust).
Fair warning: I do not have one of these sensors, so obviously I didn't test this, but the concept remains.
I recognize that this code won't solve the issue of characters being dropped, since it just ignores incomplete packets and you still rely on the validity of the checksum.
Finally, I find it interesting that the reason that the checksum is failing is because the checksum bytes are not even being received in those cases!
Hope this helps!
UPDATE #2: This is more or less a revised answer in it of itself.
Using this code to read packets, the following criteria (which are defined by the protocol) are guaranteed:
The packets begins with [66 77]
The packet contains 32 bytes
The start condition [66 77] will never occur in the body of the packet.
Here's the code. I manage to reduce it down to a few if statements
void PMS5003::processDataOn(HardwareSerial &serial) {
bool possibleStart = false;
incomeByte[0] = 66;
incomeByte[1] = 77;
uint8_t count = 0;
...
while (1) {
...
if (serial.available()) {
uint8_t c = serial.read();
if (possibleStart) {
possibleStart = false;
if (c == 77) count = 2;
}
if (c == 66) possibleStart = true;
if (count >= 2) incomeByte[count++] = c;
if (count == NUM_DATA_BYTE) break;
}
}
// at this point, incomeByte must:\
// > begin with [66 77]
// > contain 32 bytes
// > not contain [66 77] anywhere after the first two bytes
// > therefore, it is guaranteed to contain a checksum
// now is the right time to evaluate the checksum.
// I expect all of the checksums to match, but you might as well check
}
At the time of posting, the OP has already coded a solution which fulfills the requirements. I am posting this because I believe this code improves upon the OP's by being more concise, more readable/declarative, and hopefully more easily manageable.
This code can also serve as a general solution for any case in which two characters define a start condition, provided the packet length is known or can be determined.
While the question why the data comes corrupted still remains, here is a workaround I managed to achieve:
I'm checking the checksum, if it's incorrect, then:
I'm looking for 66 77 in the whole data. When I find it:
I'm checking if in the next 16 bytes there's another 66 77. If it's not found:
I'm presuming the values that are distanced by 10-15 bytes from 66 77 are the ones I'm looking for (PM1, PM2.5, PM10).
Here's the code:
void PMS5003::processDataOn(HardwareSerial &serial) {
unsigned long timeout = millis();
int count = 0;
byte incomeByte[NUM_INCOME_BYTE];
boolean startcount = false;
byte data;
int timeoutHops = 0;
while (1){
if (((millis() - timeout) > 1000) && (timeoutHops == 0)) {
timeoutHops = 1;
yield();
ESP.wdtFeed();
}
if (((millis() - timeout) > 2000) && (timeoutHops == 1)) {
timeoutHops = 2;
yield();
ESP.wdtFeed();
}
if ((millis() - timeout) > 3000) {
yield();
ESP.wdtFeed();
Serial.println("SENSOR-ERROR-TIMEOUT");
break;
}
if (serial.available()) {
data = serial.read();
if (data == CHAR_PRELIM && !startcount) {
startcount = true;
count++;
incomeByte[0] = data;
} else if (startcount) {
count++;
incomeByte[count - 1] = data;
if (count >= NUM_INCOME_BYTE){
break;
}
}
}
}
unsigned int calcsum = 0;
unsigned int exptsum;
for (int a = 0; a < NUM_INCOME_BYTE; a++) {
Serial.print((unsigned int)incomeByte[a]);
Serial.print(" ");
}
Serial.println();
Serial.println(((unsigned int)incomeByte[PM25_BYTE] << 8) + (unsigned int)incomeByte[PM25_BYTE + 1]);
for (int i = 0; i < NUM_DATA_BYTE; i++) {
calcsum += (unsigned int)incomeByte[i];
}
exptsum = ((unsigned int)incomeByte[CHECK_BYTE] << 8) + (unsigned int)incomeByte[CHECK_BYTE + 1];
if (calcsum == exptsum) {
pm1 = ((unsigned int)incomeByte[PM1_BYTE] << 8) + (unsigned int)incomeByte[PM1_BYTE + 1];
pm25 = ((unsigned int)incomeByte[PM25_BYTE] << 8) + (unsigned int)incomeByte[PM25_BYTE + 1];
pm10 = ((unsigned int)incomeByte[PM10_BYTE] << 8) + (unsigned int)incomeByte[PM10_BYTE + 1];
} else {
Serial.println("#[exception] PM2.5 Sensor CHECKSUM ERROR!");
pm1 = -1;
pm25 = -1;
pm10 = -1;
for (int a = 0; a < NUM_INCOME_BYTE; a++) {
bool valid = true;
if (((unsigned int)incomeByte[a] == 66) && ((unsigned int)incomeByte[a+1] == 77)) {
if (a+16 < NUM_INCOME_BYTE) {
for (int b = a+1; b < a+15; b++) {
if (((unsigned int)incomeByte[b] == 66) && ((unsigned int)incomeByte[b+1] == 77)) {
valid = false;
break;
}
}
if (valid) {
pm1 = ((unsigned int)incomeByte[a+10] << 8) + (unsigned int)incomeByte[a+11];
pm25 = ((unsigned int)incomeByte[a+12] << 8) + (unsigned int)incomeByte[a+13];
pm10 = ((unsigned int)incomeByte[a+14] << 8) + (unsigned int)incomeByte[a+15];
Serial.println("valid: ");
Serial.print(pm1);
Serial.print(" ");
Serial.print(pm25);
Serial.print(" ");
Serial.print(pm10);
Serial.println();
break;
}
}
}
}
}
return;
}
Theoretically, it may produce false positives or negatives but in practice, it just works.
66 77 0 28 0 12 0 15 0 17 0 12 0 15 0 17 9 102 2 176 66 77 0 28 0 12 0 15 0 16 0 12
15
#[exception] PM2.5 Sensor CHECKSUM ERROR!
valid:
12 15 17
66 77 0 28 0 12 0 15 0 16 0 12 0 15 0 16 9 114 2 175 0 73 0 4 0 1 0 0 145 0 3 12
15
66 77 0 28 0 12 0 15 0 16 0 12 0 15 0 16 9 114 2 175 0 73 0 4 0 1 0 0 145 0 3 12
15
66 77 0 28 0 12 0 15 0 16 0 12 0 15 0 16 9 141 2 190 0 72 0 3 0 1 0 0 145 0 3 52
15
66 77 0 28 0 12 0 15 0 16 0 12 0 15 0 16 9 141 2 190 0 72 0 3 0 1 0 0 145 0 3 52
15
66 77 0 28 0 12 0 16 0 16 0 12 0 16 0 16 9 198 2 202 0 75 0 3 0 0 0 0 145 0 3 125
16
66 77 0 28 0 12 0 16 0 16 0 66 77 0 28 0 12 0 16 0 16 0 12 0 16 0 16 9 198 2 202 0
19712
#[exception] PM2.5 Sensor CHECKSUM ERROR!
valid:
12 16 16
66 77 0 28 0 12 0 15 0 16 0 12 0 15 0 16 9 174 2 199 0 71 0 3 0 0 0 0 145 0 3 92
15
66 77 0 28 0 12 0 15 0 16 0 12 0 15 0 16 9 174 2 199 0 71 0 3 0 0 0 0 145 0 3 92
15
66 77 0 28 0 12 0 15 0 16 0 12 0 15 0 16 9 174 2 199 66 77 0 28 0 13 0 16 0 16 0 13
15
#[exception] PM2.5 Sensor CHECKSUM ERROR!
valid:
12 15 16
66 77 0 28 0 13 0 16 0 16 0 13 0 16 0 16 9 213 2 205 0 72 0 3 0 0 0 0 145 0 3 142
16
66 77 0 28 0 13 0 16 0 16 0 13 0 16 0 16 9 213 2 205 0 72 0 3 0 0 0 0 145 0 3 142
16
66 77 0 28 0 13 0 16 0 17 0 13 0 16 0 17 9 207 2 208 0 83 0 6 0 1 0 0 145 0 3 156
16
66 77 0 28 0 13 0 16 0 17 0 13 0 16 0 17 9 207 2 208 0 83 0 6 0 1 0 0 145 0 3 156
16
66 77 0 28 0 13 0 17 0 17 0 13 0 17 0 17 9 159 2 202 0 87 0 5 0 1 0 0 145 0 3 107
17
66 77 0 28 0 13 0 17 0 17 0 66 77 0 28 0 13 0 17 0 17 0 13 0 17 0 17 9 159 2 202 0
19712
#[exception] PM2.5 Sensor CHECKSUM ERROR!
valid:
13 17 17
My data looks like this:
2 PresentationID 12954
5 Attendees 65
6 Downloads 0
7 Questions 0
8 Likes 11
9 Tweets 0
10 Polls 0
73 PresentationID 12953
76 Attendees 64
77 Downloads 31
78 Questions 0
79 Likes 11
80 Tweets 0
81 Polls 0
143 PresentationID 12951
146 Attendees 64
147 Downloads 28
148 Questions 2
149 Likes 2
150 Tweets 0
151 Polls 0
And i need to get it to this format:
PresentationID Attendees Downloads Questions Likes Tweets Polls
0 12954 65 0 0 11 0 0
1 12953 64 31 6 0 4
2 12892 204 0 0 14 0 0
I have tried several combinations of groupby, pivot, and stack with no avail. Any advice greatly appreciated. Thanks.
You can use cumcount with pivot:
print (df)
A B C
0 2 PresentationID 12954
1 5 Attendees 65
2 6 Downloads 0
3 7 Questions 0
4 8 Likes 11
5 9 Tweets 0
6 10 Polls 0
7 73 PresentationID 12953
8 76 Attendees 64
9 77 Downloads 31
10 78 Questions 0
11 79 Likes 11
12 80 Tweets 0
13 81 Polls 0
14 143 PresentationID 12951
15 146 Attendees 64
16 147 Downloads 28
17 148 Questions 2
18 149 Likes 2
19 150 Tweets 0
20 151 Polls 0
df['G'] = df.groupby('B').cumcount()
df = df.pivot(index='G', columns='B', values='C')
print (df)
B Attendees Downloads Likes Polls PresentationID Questions Tweets
G
0 65 0 11 0 12954 0 0
1 64 31 11 0 12953 0 0
2 64 28 2 0 12951 2 0
df = pd.pivot(index=df.groupby('B').cumcount(), columns=df.B, values=df.C)
print (df)
B Attendees Downloads Likes Polls PresentationID Questions Tweets
0 65 0 11 0 12954 0 0
1 64 31 11 0 12953 0 0
2 64 28 2 0 12951 2 0
I am trying to replace values in a dataframe by 0. the first column I need to replace the 1st 3 values, the next column the 1st 6 values so on so forth increasing by 3 every time
a=np.array([133,124,156,189,132,176,189,192,100,120,130,140,150,50,70,133,124,156,189,132])
b = pd.DataFrame(a.reshape(10,2), columns= ['s','t'])
for columns in b:
yy = 3
for i in xrange(yy):
b[columns][i] = 0
yy += 3
print b
the outcome is the following
s t
0 0 0
1 0 0
2 0 0
3 189 189
4 132 132
5 176 176
6 189 189
7 192 192
8 100 100
9 120 120
I am clearly missing something really simple, to make the loop replace 6 values instead of only 3 in column t, any ideas?
i would do it this way:
i = 1
for c in b.columns:
b.ix[0 : 3*i-1, c] = 0
i += 1
Demo:
In [86]: b = pd.DataFrame(np.random.randint(0, 100, size=(20, 4)), columns=list('abcd'))
In [87]: %paste
i = 1
for c in b.columns:
b.ix[0 : 3*i-1, c] = 0
i += 1
## -- End pasted text --
In [88]: b
Out[88]:
a b c d
0 0 0 0 0
1 0 0 0 0
2 0 0 0 0
3 10 0 0 0
4 8 0 0 0
5 49 0 0 0
6 55 48 0 0
7 99 43 0 0
8 63 29 0 0
9 61 65 74 0
10 15 29 41 0
11 79 88 3 0
12 91 74 11 4
13 56 71 6 79
14 15 65 46 81
15 81 42 60 24
16 71 57 95 18
17 53 4 80 15
18 42 55 84 11
19 26 80 67 59
You need inicialize yy=3 before loop:
yy = 3
for columns in b:
for i in xrange(yy):
b[columns][i] = 0
yy += 3
print b
Python 3 solution:
yy = 3
for columns in b:
for i in range(yy):
b[columns][i] = 0
yy += 3
print (b)
s t
0 0 0
1 0 0
2 0 0
3 189 0
4 100 0
5 130 0
6 150 50
7 70 133
8 124 156
9 189 132
Another solution:
yy= 3
for i, col in enumerate(b.columns):
b.ix[:i*yy+yy-1, col] = 0
print (b)
s t
0 0 0
1 0 0
2 0 0
3 189 0
4 100 0
5 130 0
6 150 50
7 70 133
8 124 156
9 189 132
I need to perform a matching between an image and an histogram I receive as a text.
I do the cdf for both of them:
//Calculating cumulative histogram of src
double total = src.rows*src.cols;
double probSrc[255];
int newValuesSrc[255];
double cuml = 0;
for(int j = 0; j < 256; j++)
{
probSrc[j] = imageHistogram[j]/total; // Probability of each value in image
cuml = cuml + probSrc[j]; // Cumulative probability of current and all previous values
double cdfmax = cuml * 255; // Cumulative probability * max value
newValuesSrc[j] = (int) round(cdfmax);
cout << imageHistogram[j] << " "<< probSrc[j] << " " << newValuesSrc[j] << endl;
}
readHistogramFromFile();
//Calculating cumulative histogram from file
double probDst[255];
int newValuesDst[255];
cuml = 0;
for(int j = 0; j < 256; j++)
{
probDst[j] = receivedHistogram[j]/total; // Probability of each value in image
cuml = cuml + probDst[j]; // Cumulative probability of current and all previous values
double cdfmax = cuml * 255; // Cumulative probability * max value
newValuesDst[j] = (int) round(cdfmax);
cout << receivedHistogram[j] << " "<< probDst[j] << " " << newValuesDst[j] << endl;
}
and I get this values:
For the src image:
207677 0.0901376 23
37615 0.016326 27
19098 0.00828906 29
11955 0.0051888 31
8744 0.00379514 32
7386 0.00320573 32
6546 0.00284115 33
6178 0.00268142 34
5967 0.00258984 34
5437 0.00235981 35
5280 0.00229167 36
5127 0.00222526 36
5002 0.00217101 37
4839 0.00210026 37
4754 0.00206337 38
4676 0.00202951 38
4547 0.00197352 39
4517 0.0019605 39
4484 0.00194618 40
4290 0.00186198 40
4197 0.00182161 41
4188 0.00181771 41
4265 0.00185113 42
4229 0.0018355 42
4233 0.00183724 43
4245 0.00184245 43
4358 0.00189149 44
4330 0.00187934 44
4400 0.00190972 45
4474 0.00194184 45
4519 0.00196137 46
4415 0.00191623 46
4477 0.00194314 47
4468 0.00193924 47
4580 0.00198785 48
4416 0.00191667 48
4558 0.0019783 49
4674 0.00202865 49
4705 0.0020421 50
4998 0.00216927 50
4848 0.00210417 51
4782 0.00207552 51
4883 0.00211936 52
4989 0.00216536 52
4957 0.00215148 53
4987 0.0021645 53
5133 0.00222786 54
4967 0.00215582 54
5217 0.00226432 55
5185 0.00225043 56
5140 0.0022309 56
5236 0.00227257 57
5291 0.00229644 57
5458 0.00236892 58
5473 0.00237543 59
5464 0.00237153 59
5495 0.00238498 60
5439 0.00236068 60
5458 0.00236892 61
5557 0.00241189 62
5881 0.00255252 62
5900 0.00256076 63
5935 0.00257595 64
5902 0.00256163 64
6040 0.00262153 65
6203 0.00269227 66
6146 0.00266753 66
6140 0.00266493 67
6075 0.00263672 68
6054 0.0026276 68
6238 0.00270747 69
6060 0.00263021 70
6153 0.00267057 70
6303 0.00273568 71
6231 0.00270443 72
6278 0.00272483 72
6360 0.00276042 73
6359 0.00275998 74
6368 0.00276389 75
6438 0.00279427 75
6329 0.00274696 76
6408 0.00278125 77
6360 0.00276042 77
6378 0.00276823 78
6329 0.00274696 79
6394 0.00277517 79
6517 0.00282856 80
6521 0.0028303 81
6707 0.00291102 82
6788 0.00294618 82
6761 0.00293446 83
6878 0.00298524 84
7004 0.00303993 85
6963 0.00302214 85
7050 0.0030599 86
6940 0.00301215 87
6875 0.00298394 88
7073 0.00306988 89
7035 0.00305339 89
7146 0.00310156 90
7007 0.00304123 91
7159 0.0031072 92
7089 0.00307682 92
7185 0.00311849 93
7410 0.00321615 94
7237 0.00314106 95
7334 0.00318316 96
7364 0.00319618 97
7452 0.00323437 97
7760 0.00336806 98
7839 0.00340234 99
7882 0.00342101 100
7885 0.00342231 101
8055 0.00349609 102
7923 0.0034388 103
8165 0.00354384 103
8306 0.00360503 104
8271 0.00358984 105
8275 0.00359158 106
8634 0.0037474 107
8684 0.0037691 108
8752 0.00379861 109
9080 0.00394097 110
8958 0.00388802 111
9094 0.00394705 112
9279 0.00402734 113
9234 0.00400781 114
9348 0.00405729 115
9440 0.00409722 116
9431 0.00409332 117
9662 0.00419358 118
9842 0.0042717 119
9816 0.00426042 121
9957 0.00432161 122
10353 0.00449349 123
10626 0.00461198 124
10764 0.00467187 125
10832 0.00470139 126
10767 0.00467318 128
11222 0.00487066 129
11469 0.00497786 130
11661 0.0050612 131
11731 0.00509158 133
12023 0.00521832 134
12086 0.00524566 135
12094 0.00524913 137
12362 0.00536545 138
12364 0.00536632 139
12659 0.00549436 141
12587 0.00546311 142
12776 0.00554514 144
13037 0.00565842 145
13252 0.00575174 147
13425 0.00582682 148
13595 0.00590061 150
13795 0.00598741 151
14308 0.00621007 153
14232 0.00617708 154
14657 0.00636155 156
14966 0.00649566 157
14867 0.00645269 159
15051 0.00653255 161
15510 0.00673177 162
15357 0.00666536 164
15326 0.00665191 166
15308 0.0066441 168
15316 0.00664757 169
15321 0.00664974 171
15298 0.00663976 173
15435 0.00669922 174
15496 0.00672569 176
15307 0.00664366 178
15343 0.00665929 179
15356 0.00666493 181
15315 0.00664714 183
15444 0.00670312 185
15346 0.00666059 186
15583 0.00676345 188
15429 0.00669661 190
15641 0.00678863 191
15661 0.00679731 193
15638 0.00678733 195
15689 0.00680946 197
15866 0.00688628 198
15552 0.00675 200
15150 0.00657552 202
15185 0.00659071 203
14941 0.00648481 205
14989 0.00650564 207
14585 0.0063303 208
14718 0.00638802 210
14553 0.00631641 212
14612 0.00634201 213
14520 0.00630208 215
14358 0.00623177 216
13931 0.00604644 218
13580 0.0058941 220
13370 0.00580295 221
13281 0.00576432 222
13053 0.00566536 224
12711 0.00551693 225
12556 0.00544965 227
12556 0.00544965 228
12125 0.00526259 229
12184 0.00528819 231
11975 0.00519748 232
12198 0.00529427 233
11919 0.00517318 235
11898 0.00516406 236
11589 0.00502995 237
11348 0.00492535 239
11011 0.00477908 240
10523 0.00456727 241
10388 0.00450868 242
9795 0.0042513 243
9251 0.00401519 244
9014 0.00391233 245
8436 0.00366146 246
8266 0.00358767 247
7851 0.00340755 248
7299 0.00316797 249
6996 0.00303646 250
6303 0.00273568 250
5625 0.00244141 251
5375 0.0023329 251
5102 0.00221441 252
4747 0.00206033 253
4313 0.00187196 253
3809 0.00165321 253
3307 0.00143533 254
2756 0.00119618 254
2276 0.000987847 254
1935 0.000839844 255
1617 0.000701823 255
1087 0.000471788 255
547 0.000237413 255
217 9.4184e-05 255
31 1.34549e-05 255
4 1.73611e-06 255
0 0 255
0 0 255
0 0 255
0 0 255
0 0 255
0 0 255
0 0 255
0 0 255
0 0 255
0 0 255
0 0 255
0 0 255
0 0 255
0 0 255
0 0 255
0 0 255
0 0 255
And for the histogram I receive:
10 4.34028e-06 0
11 4.77431e-06 0
12 5.20833e-06 0
13 5.64236e-06 0
14 6.07639e-06 0
15 6.51042e-06 0
16 6.94444e-06 0
17 7.37847e-06 0
18 7.8125e-06 0
19 8.24653e-06 0
20 8.68056e-06 0
22 9.54861e-06 0
24 1.04167e-05 0
26 1.12847e-05 0
28 1.21528e-05 0
30 1.30208e-05 0
34 1.47569e-05 0
38 1.64931e-05 0
42 1.82292e-05 0
50 2.17014e-05 0
60 2.60417e-05 0
70 3.03819e-05 0
80 3.47222e-05 0
90 3.90625e-05 0
100 4.34028e-05 0
120 5.20833e-05 0
140 6.07639e-05 0
160 6.94444e-05 0
160 6.94444e-05 0
150 6.51042e-05 0
140 6.07639e-05 0
130 5.64236e-05 0
120 5.20833e-05 0
110 4.77431e-05 0
100 4.34028e-05 0
90 3.90625e-05 0
80 3.47222e-05 0
70 3.03819e-05 0
60 2.60417e-05 0
50 2.17014e-05 0
40 1.73611e-05 0
30 1.30208e-05 0
20 8.68056e-06 0
10 4.34028e-06 0
10 4.34028e-06 0
10 4.34028e-06 0
10 4.34028e-06 0
11 4.77431e-06 0
12 5.20833e-06 0
13 5.64236e-06 0
14 6.07639e-06 0
15 6.51042e-06 0
16 6.94444e-06 0
17 7.37847e-06 0
18 7.8125e-06 0
19 8.24653e-06 0
20 8.68056e-06 0
22 9.54861e-06 0
24 1.04167e-05 0
26 1.12847e-05 0
28 1.21528e-05 0
30 1.30208e-05 0
34 1.47569e-05 0
38 1.64931e-05 0
42 1.82292e-05 0
50 2.17014e-05 0
60 2.60417e-05 0
70 3.03819e-05 0
80 3.47222e-05 0
90 3.90625e-05 0
100 4.34028e-05 0
120 5.20833e-05 0
140 6.07639e-05 0
160 6.94444e-05 0
160 6.94444e-05 0
150 6.51042e-05 0
140 6.07639e-05 0
130 5.64236e-05 1
120 5.20833e-05 1
110 4.77431e-05 1
100 4.34028e-05 1
90 3.90625e-05 1
80 3.47222e-05 1
70 3.03819e-05 1
60 2.60417e-05 1
50 2.17014e-05 1
40 1.73611e-05 1
30 1.30208e-05 1
20 8.68056e-06 1
10 4.34028e-06 1
10 4.34028e-06 1
11 4.77431e-06 1
12 5.20833e-06 1
13 5.64236e-06 1
14 6.07639e-06 1
15 6.51042e-06 1
16 6.94444e-06 1
17 7.37847e-06 1
18 7.8125e-06 1
19 8.24653e-06 1
20 8.68056e-06 1
22 9.54861e-06 1
24 1.04167e-05 1
26 1.12847e-05 1
28 1.21528e-05 1
30 1.30208e-05 1
34 1.47569e-05 1
38 1.64931e-05 1
42 1.82292e-05 1
50 2.17014e-05 1
60 2.60417e-05 1
70 3.03819e-05 1
80 3.47222e-05 1
90 3.90625e-05 1
100 4.34028e-05 1
120 5.20833e-05 1
140 6.07639e-05 1
160 6.94444e-05 1
160 6.94444e-05 1
150 6.51042e-05 1
140 6.07639e-05 1
130 5.64236e-05 1
120 5.20833e-05 1
110 4.77431e-05 1
100 4.34028e-05 1
90 3.90625e-05 1
80 3.47222e-05 1
70 3.03819e-05 1
60 2.60417e-05 1
50 2.17014e-05 1
40 1.73611e-05 1
30 1.30208e-05 1
20 8.68056e-06 1
10 4.34028e-06 1
10 4.34028e-06 1
10 4.34028e-06 1
10 4.34028e-06 1
11 4.77431e-06 1
12 5.20833e-06 1
13 5.64236e-06 1
14 6.07639e-06 1
15 6.51042e-06 1
16 6.94444e-06 1
17 7.37847e-06 1
18 7.8125e-06 1
19 8.24653e-06 1
20 8.68056e-06 1
22 9.54861e-06 1
24 1.04167e-05 1
26 1.12847e-05 1
28 1.21528e-05 1
30 1.30208e-05 1
34 1.47569e-05 1
38 1.64931e-05 1
42 1.82292e-05 1
50 2.17014e-05 1
60 2.60417e-05 1
70 3.03819e-05 1
80 3.47222e-05 1
90 3.90625e-05 1
100 4.34028e-05 1
120 5.20833e-05 1
140 6.07639e-05 1
160 6.94444e-05 1
160 6.94444e-05 1
150 6.51042e-05 1
140 6.07639e-05 1
130 5.64236e-05 1
120 5.20833e-05 1
110 4.77431e-05 1
100 4.34028e-05 1
90 3.90625e-05 1
80 3.47222e-05 1
70 3.03819e-05 1
60 2.60417e-05 1
50 2.17014e-05 1
40 1.73611e-05 1
30 1.30208e-05 1
20 8.68056e-06 1
10 4.34028e-06 1
10 4.34028e-06 1
10 4.34028e-06 1
20 8.68056e-06 1
30 1.30208e-05 1
40 1.73611e-05 1
50 2.17014e-05 1
60 2.60417e-05 1
70 3.03819e-05 1
80 3.47222e-05 1
90 3.90625e-05 1
100 4.34028e-05 1
120 5.20833e-05 1
140 6.07639e-05 1
160 6.94444e-05 1
160 6.94444e-05 1
150 6.51042e-05 1
140 6.07639e-05 1
130 5.64236e-05 1
120 5.20833e-05 1
110 4.77431e-05 1
100 4.34028e-05 1
90 3.90625e-05 1
80 3.47222e-05 1
70 3.03819e-05 1
60 2.60417e-05 1
50 2.17014e-05 1
40 1.73611e-05 1
30 1.30208e-05 1
20 8.68056e-06 1
10 4.34028e-06 1
10 4.34028e-06 1
10 4.34028e-06 1
20 8.68056e-06 1
30 1.30208e-05 1
40 1.73611e-05 1
40 1.73611e-05 1
50 2.17014e-05 1
55 2.38715e-05 1
60 2.60417e-05 1
65 2.82118e-05 1
70 3.03819e-05 1
75 3.25521e-05 1
80 3.47222e-05 1
85 3.68924e-05 2
90 3.90625e-05 2
95 4.12326e-05 2
90 3.90625e-05 2
80 3.47222e-05 2
70 3.03819e-05 2
60 2.60417e-05 2
50 2.17014e-05 2
40 1.73611e-05 2
30 1.30208e-05 2
20 8.68056e-06 2
10 4.34028e-06 2
10 4.34028e-06 2
10 4.34028e-06 2
20 8.68056e-06 2
30 1.30208e-05 2
40 1.73611e-05 2
40 1.73611e-05 2
50 2.17014e-05 2
55 2.38715e-05 2
60 2.60417e-05 2
65 2.82118e-05 2
70 3.03819e-05 2
75 3.25521e-05 2
80 3.47222e-05 2
85 3.68924e-05 2
90 3.90625e-05 2
95 4.12326e-05 2
100 4.34028e-05 2
105 4.55729e-05 2
110 4.77431e-05 2
115 4.99132e-05 2
120 5.20833e-05 2
As you can see, the src histogram are a lot more distributed than the histrogram I receive ([0-255] against [0-2]).
My question is, what do I do now? How do I match them?
Why don't you scale [0-2] histogram to [0-255]? oldValue * 255 / 2.
I have two images, A and B, and I need to estimate B's quantization table and compress A using this table. What is the best way to do this, using libjpeg or, even better, opencv?
I've used libjpeg's utility 'djpeg' to find the quantization table of a image, but I'm not sure how to interpret its output and use it with libjpeg. Besides, I need to find this matrix and do the compression from inside my program, which renders (I think) 'djpeg' unusable in this case.
Following, is the output of 'djpeg' for a test image, running it with:
djpeg -v -v cat1.jpg > /dev/null
Start of Image
JFIF APP0 marker: version 1.01, density 96x96 1
Define Quantization Table 0 precision 0
5 3 3 5 7 12 15 18
4 4 4 6 8 17 18 17
4 4 5 7 12 17 21 17
4 5 7 9 15 26 24 19
5 7 11 17 20 33 31 23
7 11 17 19 24 31 34 28
15 19 23 26 31 36 36 30
22 28 29 29 34 30 31 30
Define Quantization Table 1 precision 0
5 5 7 14 30 30 30 30
5 6 8 20 30 30 30 30
7 8 17 30 30 30 30 30
14 20 30 30 30 30 30 30
30 30 30 30 30 30 30 30
30 30 30 30 30 30 30 30
30 30 30 30 30 30 30 30
30 30 30 30 30 30 30 30
Start Of Frame 0xc0: width=450, height=320, components=3
Component 1: 2hx2v q=0
Component 2: 1hx1v q=1
Component 3: 1hx1v q=1
Define Huffman Table 0x00
0 1 5 1 1 1 1 1
1 0 0 0 0 0 0 0
Define Huffman Table 0x10
0 2 1 3 3 2 4 3
5 5 4 4 0 0 1 125
Define Huffman Table 0x01
0 3 1 1 1 1 1 1
1 1 1 0 0 0 0 0
Define Huffman Table 0x11
0 2 1 2 4 4 3 4
7 5 4 4 0 1 2 119
Start Of Scan: 3 components
Component 1: dc=0 ac=0
Component 2: dc=1 ac=1
Component 3: dc=1 ac=1
Ss=0, Se=63, Ah=0, Al=0
End Of Image
Thanks in advance!