Related
I'm using Eigen3 version 3.3.1 and g++ version (Ubuntu 7.3.0-27ubuntu1~18.04) 7.3.0. I'm finding that I get different results from JacobiSVD::singularValues(), depending on whether -march=native is part of the compile command. It seems as though the actual significant flag within the "-march=native" umbrella is -mavx. Here is a test case:
using dictionary_t = Eigen::Matrix<float, Eigen::Dynamic, Eigen::Dynamic, Eigen::ColMajor>;
const float halfroot = std::sqrt(2.0f)/2.0f;
Eigen::Matrix<float, 37, 38, Eigen::ColMajor> m;
m << 0.5, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0.5, 0, 0, 0, 0, 0, 0, 0, 0, 0, halfroot, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0.5, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0.5, 0, 0, 0, 0, 0, 0, 0, 0, 0, -halfroot, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0.5, 0, 0, 0, 0, 0, 0, 0, 0, 0, -0.5, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, halfroot, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0.5, 0, 0, 0, 0, 0, 0, 0, 0, 0, -0.5, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, -halfroot, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0.5, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0.5, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, halfroot, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0.5, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0.5, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, -halfroot, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0.5, 0, 0, 0, 0, 0, 0, 0, 0, 0, -0.5, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, halfroot, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0.5, 0, 0, 0, 0, 0, 0, 0, 0, 0, -0.5, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, -halfroot, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0.5, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0.5, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, halfroot, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0.5, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0.5, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, -halfroot, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0.5, 0, 0, 0, 0, 0, 0, 0, 0, 0, -0.5, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, halfroot, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0.5, 0, 0, 0, 0, 0, 0, 0, 0, 0, -0.5, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, -halfroot, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0.5, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0.5, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, halfroot, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0.5, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0.5, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, -halfroot, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0.5, 0, 0, 0, 0, 0, 0, 0, 0, 0, -0.5, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, halfroot, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0.5, 0, 0, 0, 0, 0, 0, 0, 0, 0, -0.5, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, -halfroot, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0.5, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0.5, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, halfroot, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0.5, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0.5, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, -halfroot, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0.5, 0, 0, 0, 0, 0, 0, 0, 0, 0, -0.5, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, halfroot, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0.5, 0, 0, 0, 0, 0, 0, 0, 0, 0, -0.5, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, -halfroot, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0.5, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0.5, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, halfroot, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0.5, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0.5, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, -halfroot, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0.5, 0, 0, 0, 0, 0, 0, 0, 0, 0, -0.5, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, halfroot, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0.5, 0, 0, 0, 0, 0, 0, 0, 0, 0, -0.5, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, -halfroot, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0.5, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0.5, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, halfroot, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0.5, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0.5, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, -halfroot, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0.5, 0, 0, 0, 0, 0, 0, 0, 0, 0, -0.5, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, halfroot, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0.5, 0, 0, 0, 0, 0, 0, 0, 0, 0, -0.5, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, -halfroot, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0.5, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0.5, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, halfroot, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0.5, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0.5, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, -halfroot, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0.5, 0, 0, 0, 0, 0, 0, 0, 0, 0, -0.5, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, halfroot, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0.5, 0, 0, 0, 0, 0, 0, 0, 0, 0, -0.5, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, -halfroot, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0.5, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0.5, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, halfroot, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0.5, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0.5, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, -halfroot, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0.5, 0, 0, 0, 0, 0, 0, 0, 0, 0, -0.5, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, halfroot,
0, 0, 0, 0, 0, 0, 0, 0, 0.5, 0, 0, 0, 0, 0, 0, 0, 0, 0, -0.5, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, -halfroot,
0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1;
Eigen::JacobiSVD<dictionary_t> svdDi{m, Eigen::ComputeFullU|Eigen::ComputeFullV};
Eigen::VectorXf singVals = svdDi.singularValues();
Eigen::IOFormat fmt{Eigen::StreamPrecision, Eigen::DontAlignCols, ", "};
std::cout << "singular values of m: \n" << std::setprecision(10)
<< singVals.format(fmt) << std::endl;
And here is its output without -march=native set:
singular values of m:
1.84775877
1.000000238
1.000000119
1.000000119
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
0.9999999404
0.9999999404
0.9999999404
0.9999999404
0.9999999404
0.9999999404
0.9999999404
0.9999999404
0.9999999404
0.9999999404
0.9999999404
0.9999999404
0.9999998808
0.9999998212
0.7653669715
If I compile with -march=native, the first couple of singular values are different:
singular values of m:
1.847759128
1.000000119
1.000000119
1.000000119
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
0.9999999404
0.9999999404
0.9999999404
0.9999999404
0.9999999404
0.9999999404
0.9999999404
0.9999999404
0.9999999404
0.7653669715
Sorry about the bulkiness of my example. So, is this expected behavior? If so, is there a reason to prefer one result over the other?
These Eigen values are close enough that they can be considered as identical (especially for float).
Eigen can use a different set of intrinsics depending on the flags, so the computation order can be different, and of course floating point math is broken.
All these numbers are close enough compared to machine precision and the size and type of your matrix.
In the following code, I have two histograms, that for this simple example, I hard-coded in the source. Each histogram have 128 bins, where the 64 firsts bins correspond to one histogram, and the 64 other correspond to another histogram. However the resultant distance is 0, even though there are clear differences in the latter 64 bins of the 128 bins of each vector. I don't understand how it's possible why two different vectors have a null distance.
#include <iostream>
#include <opencv2/core.hpp>
#include <opencv2/shape/emdL1.hpp>
using Vec128f = cv::Vec<float, 128>;
float sum_of_emd_dists(const Vec128f& a, const Vec128f& b)
{
const cv::Mat a_color(cv::Size{64, 1}, CV_32FC1, (void*)(&a.val[0]));
const cv::Mat a_label(cv::Size{64, 1}, CV_32FC1, (void*)(&a.val[64]));
const cv::Mat b_color(cv::Size{64, 1}, CV_32FC1, (void*)(&b.val[0]));
const cv::Mat b_label(cv::Size{64, 1}, CV_32FC1, (void*)(&b.val[64]));
float dist = cv::EMDL1(a_color, b_color) + cv::EMDL1(a_label, b_label);
return dist;
}
int main()
{
Vec128f a = {64, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0.265625, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0.734375, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0};
Vec128f b = {64, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0.109375, 0, 0, 0, 0.109375, 0, 0, 0.09375, 0, 0, 0, 0, 0, 0, 0, 0, 0.0625, 0, 0, 0.09375, 0, 0, 0, 0.046875, 0.046875, 0, 0, 0, 0, 0, 0, 0, 0.078125, 0.140625, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0.09375, 0, 0, 0.0625, 0, 0, 0, 0.0625, 0, 0, 0, 0, 0, 0};
std::cerr << "dist = " << sum_of_emd_dists(a, b) << std::endl;
return 0;
}
Result:
dist = 0
Thank you for any help explaining why theEMD-L1 distance is 0.
Thats because your matrix size is 1 row and 64 cols and you need single col martix.
I'm trying to optimize this code, in particular:
bool interpolate(const Mat &im, float ofsx, float ofsy, float a11, float a12, float a21, float a22, Mat &res)
{
bool ret = false;
// input size (-1 for the safe bilinear interpolation)
const int width = im.cols-1;
const int height = im.rows-1;
// output size
const int halfWidth = res.cols >> 1;
const int halfHeight = res.rows >> 1;
float *out = res.ptr<float>(0);
for (int j=-halfHeight; j<=halfHeight; ++j)
{
const float rx = ofsx + j * a12;
const float ry = ofsy + j * a22;
for(int i=-halfWidth; i<=halfWidth; ++i)
{
float wx = rx + i * a11;
float wy = ry + i * a21;
const int x = (int) floor(wx);
const int y = (int) floor(wy);
if (x >= 0 && y >= 0 && x < width && y < height)
{
// compute weights
wx -= x; wy -= y;
// bilinear interpolation
*out++ =
(1.0f - wy) * ((1.0f - wx) * im.at<float>(y,x) + wx * im.at<float>(y,x+1)) +
( wy) * ((1.0f - wx) * im.at<float>(y+1,x) + wx * im.at<float>(y+1,x+1));
} else {
*out++ = 0;
ret = true; // touching boundary of the input
}
}
}
return ret;
}
According to Intel Advisor, this is a very time consuming function. In this question I asked how I could optimize this, and someone made me notice that this is warp-affine transformation.
Now, since I'm not the image processing guy, I had to read this article to understand what a warp-affine transformation is.
To my understanding, given a point p=(x,y), you apply a transformation A (a 2x2 matrix) and then translate it by a vector b. So the obtained point after the transformation p' can be expressed as p' = A*p+b. So far so good.
However, I'm a little bit confused on how to apply cv::warpAffine() to this case. First of all, from the function above interpolate() I can see only the 4 A components (a11, a12, a21, a22), while I can't see the 2 b components...Are they ofsx and ofy?
In addition notice that this function returns a bool value, which is not returned by warpAffine (this boolean value is used here at line 126), so I don't know I could this with the OpenCV function.
But most of all I'm so confused by for (int j=-halfHeight; j<=halfHeight; ++j) and for(int i=-halfWidth; i<=halfWidth; ++i) and all the crap that happens inside.
I understand that:
// bilinear interpolation
*out++ =
(1.0f - wy) * ((1.0f - wx) * im.at<float>(y,x) + wx * im.at<float>(y,x+1)) +
( wy) * ((1.0f - wx) * im.at<float>(y+1,x) + wx * im.at<float>(y+1,x+1));
Is what INTER_LINEAR does, but apart from that I'm totally lost.
So, to test my approach, I tried to do the equivalent of line 131 of this as:
bool touchesBoundary = interpolate(smoothed, (float)(patchImageSize>>1), (float)(patchImageSize>>1), imageToPatchScale, 0, 0, imageToPatchScale, patch);
Mat warp_mat( 2, 3, CV_32FC1 );
float a_11 = imageToPatchScale;
float a_12 = 0;
float a_21 = 0;
float a_22 = imageToPatchScale;
float ofx = (float)(patchImageSize>>1);
float ofy = (float)(patchImageSize>>1);
float ofx_new = ofx - a12*halfHeight - a11*halfWidth;
float ofy_new = ofy - a22*halfHeight - a21*halfWidth;
warp_mat.at<float>(0,0) = imageToPatchScale;
warp_mat.at<float>(0,1) = 0;
warp_mat.at<float>(0,2) = ofx_new;
warp_mat.at<float>(1,0) = 0;
warp_mat.at<float>(1,1) = imageToPatchScale;
warp_mat.at<float>(1,2) = ofy_new;
cv::Mat myPatch;
std::cout<<"Applying warpAffine"<<std::endl;
warpAffine(smoothed, myPatch, warp_mat, patch.size());
std::cout<<"WarpAffineApplied patch size="<<patch.size()<<" myPatch size="<<myPatch.size()<<std::endl;
cv::Mat diff = patch!=myPatch;
if(cv::countNonZero(diff) != 0){
throw std::runtime_error("Warp affine doesn't work!");
}
else{
std::cout<<"It's working!"<<std::endl;
}
And of course at the first time the this is executed, the exception is thrown (so the two methods are not equivalent)...How can I solve this?
Can someone help me please?
As I already written in the comments, the resulting matrix by using the code above is a zero matrix. While this is maPatch obtained by using ofx and ofy instead of ofx_new and ofy_new, while patch has all the values different from zero:
[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0;
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0;
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0;
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0;
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0;
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0;
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0;
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0;
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0;
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0;
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0;
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 229.78679, 229.5752, 229.11732, 229.09612, 229.84615, 230.28633, 230.35257, 230.70955, 230.99368, 231.00777, 231.20511, 231.63196, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0;
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 230.60367, 230.16417, 230.07034, 230.06793, 230.02016, 230.14925, 230.60413, 230.84822, 230.92368, 231.02249, 230.99162, 230.9149, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0;
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 232.76547, 231.39716, 231.26674, 231.34512, 230.746, 230.25253, 229.65276, 227.83998, 225.43642, 229.57695, 230.31363, 230.16011, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0;
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 234.01663, 232.88118, 232.15475, 231.40129, 223.21553, 208.22626, 205.58975, 214.53882, 220.32681, 228.11552, 229.31509, 228.86545, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0;
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 234.04565, 233.00443, 231.9902, 230.14912, 198.0849, 114.86175, 97.901344, 160.0218, 217.38528, 231.07045, 231.13109, 231.10185, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0;
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 233.293, 232.69095, 217.03873, 190.56714, 167.61592, 94.968391, 81.302032, 150.72263, 194.79535, 215.15564, 230.01717, 232.37894, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0;
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 231.70988, 227.81319, 207.59377, 173.35149, 113.88276, 73.171112, 71.523285, 103.05875, 160.05588, 194.65132, 226.4287, 231.45871, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0;
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 231.93924, 224.24269, 199.1693, 150.65695, 103.33984, 79.489555, 77.509094, 87.893059, 122.01918, 168.37506, 219.22086, 231.05161, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0;
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 232.2706, 232.12926, 206.97635, 127.69308, 92.714355, 81.512207, 74.89402, 75.968353, 84.518105, 157.07962, 223.18773, 229.92766, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0;
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 232.64882, 222.16704, 161.95021, 92.577881, 83.757164, 76.764214, 67.041054, 66.195595, 71.112335, 131.66878, 188.27278, 217.6635, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0;
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 234.77202, 231.75511, 178.64326, 104.27015, 95.664223, 82.791382, 67.68969, 72.78054, 72.355469, 104.77696, 172.32361, 204.92691, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0;
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 236.49684, 235.5802, 185.34337, 115.96995, 106.85963, 82.980408, 61.703068, 69.540627, 76.200562, 82.429321, 101.46993, 119.75877, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0;
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0;
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0;
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0;
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0;
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0;
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0;
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0;
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0;
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0;
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0;
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0;
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0;
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0;
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0;
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0;
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0;
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0;
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]
Example of smoothed:
[229.78679, 229.67955, 229.56825, 229.40576, 229.08748, 228.90848, 229.13086, 229.53154, 229.91875, 230.1864, 230.31964, 230.34709, 230.35471, 230.51445, 230.81174, 230.97459, 231.00513, 231.00487, 231.01001, 231.08649, 231.30977, 231.55736, 231.71651;
229.71237, 229.63612, 229.65092, 229.72298, 229.65163, 229.58559, 229.68594, 229.8093, 229.91052, 230.0466, 230.22325, 230.43683, 230.67668, 230.87794, 230.98672, 231.02505, 231.03383, 231.03091, 231.02097, 231.03201, 231.09761, 231.17659, 231.23175;
230.66309, 230.37627, 230.1866, 230.1675, 230.09061, 230.03766, 230.10495, 230.09256, 230.01401, 230.03775, 230.18376, 230.42041, 230.67554, 230.82742, 230.84885, 230.87372, 230.94225, 231.01442, 231.02843, 231.00027, 230.97455, 230.9254, 230.86211;
232.00514, 231.33768, 230.82791, 230.77686, 230.84599, 230.88741, 230.84238, 230.58279, 230.27737, 230.22282, 230.2531, 230.28053, 230.33743, 230.24406, 229.8969, 229.53674, 229.66661, 230.42201, 230.86761, 230.84827, 230.7677, 230.72296, 230.69333;
232.84413, 232.07454, 231.4113, 231.24339, 231.31792, 231.42, 231.39203, 231.09439, 230.71797, 230.52229, 230.16359, 229.71872, 229.5307, 228.81219, 226.98767, 224.92525, 225.05101, 228.29745, 230.37059, 230.39821, 230.14323, 230.08817, 230.12051;
233.69714, 233.27977, 232.63216, 231.97507, 231.61856, 231.50835, 231.37958, 230.94897, 230.22003, 229.17024, 227.78331, 226.92528, 227.3483, 226.49516, 223.07671, 219.54231, 220.02966, 225.84485, 229.56601, 229.69946, 229.2941, 228.91028, 228.47911;
234.07579, 233.56334, 232.87689, 232.33269, 232.23909, 232.26355, 231.24196, 227.51971, 220.59465, 210.97746, 202.39467, 198.75334, 202.68945, 209.23911, 214.57399, 218.0966, 221.80714, 226.69366, 229.27985, 229.35699, 229.21922, 229.04704, 228.72176;
234.02943, 233.1526, 232.62421, 232.68416, 232.63794, 232.74126, 230.84375, 220.47586, 197.81956, 164.03839, 136.08931, 125.05849, 134.9079, 158.19888, 186.67014, 209.67909, 223.89606, 229.51706, 230.72685, 230.50046, 230.31461, 230.29973, 230.30855;
234.04939, 233.55843, 233.05295, 232.52957, 231.76837, 231.33992, 229.65753, 220.00912, 191.89427, 140.79909, 97.534477, 80.921623, 93.553299, 127.26912, 171.24872, 205.13603, 224.29935, 230.74513, 231.68158, 231.38503, 231.22385, 231.26157, 231.31372;
233.67462, 233.69278, 233.09642, 230.73448, 225.79077, 220.33292, 216.52835, 212.6403, 192.7964, 142.2917, 93.74559, 73.776016, 92.972778, 136.18417, 183.40891, 209.98003, 220.25392, 225.67984, 229.14565, 230.97379, 231.68997, 231.87923, 231.80464;
233.16579, 232.95818, 232.5157, 227.84683, 212.53104, 193.47, 179.53844, 171.00941, 154.97589, 118.29485, 82.342369, 67.311531, 83.867973, 119.85723, 158.53325, 180.67912, 191.74194, 203.44008, 216.87592, 227.59789, 231.31285, 232.24002, 232.91658;
232.21611, 231.93192, 231.80423, 227.06053, 208.82571, 183.86725, 160.27481, 136.63663, 112.56454, 89.978371, 73.328209, 66.652176, 73.406273, 90.259987, 113.70027, 138.08961, 159.2791, 178.08627, 201.78604, 223.79007, 230.86775, 231.59146, 232.17819;
231.5118, 230.38042, 225.97289, 217.07312, 205.34308, 192.29631, 174.19812, 142.59843, 105.71719, 80.45845, 68.488274, 67.021088, 73.29406, 86.493896, 110.19484, 145.04185, 174.52554, 187.26851, 202.64322, 221.51042, 229.94238, 231.48595, 231.08746;
231.67564, 229.07423, 217.57478, 197.87076, 181.8385, 167.48799, 148.19232, 124.3977, 100.57513, 83.081154, 73.410683, 71.723045, 77.010704, 85.107651, 98.029099, 121.88382, 145.77963, 161.43314, 184.43152, 212.01347, 227.27411, 231.84755, 231.33319;
232.0773, 231.27109, 227.09813, 218.50165, 206.31781, 182.26494, 144.46196, 115.64604, 99.402679, 87.584351, 79.348366, 76.547188, 79.332504, 82.244148, 86.3069, 100.71764, 122.39668, 147.5081, 179.02258, 210.10269, 226.37909, 231.12947, 230.34335;
232.11732, 231.67418, 231.89207, 229.20001, 213.83904, 180.2238, 134.82561, 107.20949, 97.260231, 88.765694, 80.533333, 75.941055, 76.372505, 77.851997, 78.464508, 81.875244, 96.896721, 131.28108, 175.47084, 213.05406, 227.81297, 230.31032, 229.60373;
232.36255, 232.00981, 232.29773, 226.30051, 199.48029, 156.13557, 112.30969, 91.346344, 88.295509, 85.21006, 79.416222, 74.552238, 73.894844, 75.069275, 74.349594, 72.166176, 85.453522, 128.47208, 180.33452, 218.87312, 229.58446, 229.77406, 230.03587;
232.52425, 231.2455, 226.65468, 210.90804, 174.35748, 128.79022, 92.861343, 79.050415, 78.796555, 76.526512, 71.317635, 67.324234, 67.506172, 69.193619, 68.941025, 67.913399, 82.488945, 124.88449, 171.48178, 203.84958, 215.13747, 221.22523, 228.15715;
232.74571, 229.80283, 217.69687, 189.34862, 145.52664, 104.71513, 84.893997, 83.699814, 88.473457, 86.446617, 77.834595, 68.74688, 65.925613, 65.426163, 63.241882, 61.236107, 69.682426, 97.213646, 131.60564, 160.99944, 180.75278, 202.22523, 223.85883;
233.80923, 232.82767, 227.83594, 209.05493, 166.58002, 120.64989, 94.880188, 89.971268, 93.209671, 90.605591, 80.354561, 69.243584, 67.490875, 70.700516, 72.353569, 70.053764, 70.773293, 86.577957, 121.76624, 160.51776, 182.91074, 203.17424, 224.06786;
235.62155, 235.22169, 234.91901, 223.3783, 181.88362, 132.80327, 104.59508, 97.904762, 98.472153, 91.749123, 79.65731, 69.025223, 66.806007, 70.64135, 75.239159, 74.961838, 73.406227, 83.469612, 118.84832, 161.62743, 181.61127, 192.7933, 203.54196;
236.851, 236.1096, 235.65253, 224.02559, 182.0352, 134.56085, 111.10134, 106.82736, 105.87054, 95.272148, 80.614365, 68.017456, 61.20583, 62.735069, 69.976379, 72.687195, 71.943336, 75.369637, 89.042145, 106.32064, 116.6455, 127.58019, 139.77493;
236.09546, 235.84727, 235.44041, 223.06668, 180.65508, 134.57915, 114.13975, 110.49339, 107.15049, 93.355858, 77.559898, 65.277794, 58.067509, 62.642029, 76.700447, 81.800919, 80.054298, 80.085251, 82.980927, 87.177017, 92.031647, 100.26192, 109.12404]
I'm more familiar with this warpAffine, whose basic statement is
cv::warpAffine (InputArray src, // input mat
OutputArray dst, // output mat
InputArray M, // affine transformation mat
Size dsize) // size of the output mat
where M is the matrix
a11 a12 ofx
a21 a22 ofy
In your term, the first two columns is the linear transformation matrix A, the last is the translation vector b.
The cv::hal::warpAffine() is just the same, where double M[6] corresponds to the above affine transformation matrix, but I'm not sure in which order it is flatten (most likely, [a11,a12,ofx,a21,a22,ofy]).
In OpenCV, the origin (0,0) is the top-left conner as usual, while in Intel's code, the origin (0,0) is in the middle of the image. That's what the part
for (int j=-halfHeight; j<=halfHeight; ++j)
{
for(int i=-halfWidth; i<=halfWidth; ++i)
{
const int y = (int) floor(wy);
//...
}
}
does: (i,j) is the coordinate in res, j from -halfHeight to halfHeight and i from -halfHeight to halfHeight. So in this case (0,0) is in the center of the res image.
In the provided code, if you want to map src onto res (i guess), you would need to do:
bool touchesBoundary = interpolate(smoothed, (float)(imageSize>>1), (float)(imageSize>>1), imageToPatchScale, 0, 0, imageToPatchScale, patch);
Notice here imageSize>>1 instead of patchImageSize>>1. Why? You want the center of the res (i=0,j=0) maps to the center of src, i.e. the value src.at<float>(src.cols/2, src.rows/2) (why?)
Now to make that work in your example, the equivalent of cv::warpedAffine() would be
warpAffine(smoothed, myPatch, warp_mat, patch.size(),WARP_INVERSE_MAP);
where the warp_mat has ofsx=0,ofsy=0.
Finally, here's an illustration of what I tried:
where diff = mypatch - patch >5 and smoothed is scaled up by OS. Notice the black border in patch, it is because the restrictions x < width and y<height in the code.
I am working with pressure-sensing of human feet, and I need to transmit frames in realtime over serial.
The typical frame is like below, consisting of a flat background and blobs of non-flat data:
The speed of transmission is currently a bottleneck due to micro-controller overhead caused by Serial.send commands, so the engineer is using Run Length Encoding to compress the image, which seems good due to the flat, continuous background, but we would like to compress it even further.
I tried the "Coordinate List" encoding format (List<i, j, val> where val > 0), but the size is similar enough to RLE to not make a significant difference.
While researching a bit on SO, people say "don't reinvent the wheel, there are a lot of tried-and-tested compression algorithms for any kind of image", so I wonder what would be the best for the type of image displayed below, considering:
Compression performance (since it is to be performed by a micro-controller);
Size - since it is to be sent by serial, which is currently a bottleneck (sic).
Other approach would be to use "sparse-matrix" concepts (instead of "image-compression" concepts), and it looks like there is something like CRS, or CSR, which I couldn't quite understand how to implement and how to serialize properly, and even less how it would compare with image-compression techniques.
UPDATE:
I created a Gist with the data I used to create the image. These were the results of compression methods (one byte per entry):
plain: ([n_rows, n_columns, *data]): 2290 bytes;
coordinate list: ([*(i, j, val)]): 936 bytes;
run length encoding: ([*(rowlength, rle-pairs)]): 846 bytes;
list of lists: 690 bytes;
compact list of lists: (see Gist) 498 bytes;
Proposed algorithm
Below is a possible algorithm that is using only simple operations [1] with a low memory footprint (no pun intended).
It seems to work reasonably well but, of course, it should be tested on several different data sets in order to have a more precise idea of its efficiency.
Divide the matrix into 13x11 blocks of 4x4 pixels
For each block:
If the block is empty, emit bit '0'
If the block is not empty:
emit bit '1'
emit 16-bit bitmask of non-zero pixels in this block
emit 8-bit value representing the minimum value (other than 0) found in this block
if there's only one non-zero pixel, stop here [2]
emit 3-bit value representing the number of bits required to encode each non-zero pixel in this block: b = ceil(log2(max + 1 - min))
emit non-zero pixel data as N x b bits
It is based on the following observations:
Many blocks in the matrix are empty
Non-empty blocks at the frontier of the footprint usually have many empty cells (the 'pressure' / 'no pressure' transition on the sensors is abrupt)
[1] There's notably no floating point operation. The log2() operation that is used in the description of the algorithm can easily be replaced by simple comparisons against 1, 2, 4, 8, 16, ... up to 256.
[2] This is a minor optimization that will not trigger very often. The decoder will have to detect that there's only one bit set in the bitmask by computing for instance: (msk & -msk) == msk.
Block encoding example
Let's consider the following block:
0, 0, 0, 0
12, 0, 0, 0
21, 20, 0, 0
28, 23, 0, 0
The bitmask of non-zero pixels is:
0, 0, 0, 0
1, 0, 0, 0 = 0000100011001100
1, 1, 0, 0
1, 1, 0, 0
The minimum value is 12 (00001100) and the number of bits required to encode each non-zero pixel is 5 (101), as log2(28 + 1 - 12) ~= 4.09.
Finally, let's encode non-zero pixels:
[ 12, 21, 20, 28, 23 ]
- [ 12, 12, 12, 12, 12 ]
------------------------
= [ 0, 9, 8, 16, 11 ] = [ 00000, 01001, 01000, 10000, 01011 ]
So, the final encoding for this block would be:
1 0000100011001100 00001100 101 00000 01001 01000 10000 01011
which is 53 bits long (as opposed to 16 * 8 = 128 bits in uncompressed format).
However, the biggest gain comes from empty blocks which are encoded as one single bit. The fact that there are many empty blocks in the matrix is an important assumption in this algorithm.
Demo
Here is some JS demonstration code working on your original data set:
var nEmpty, nFilled;
function compress(matrix) {
var x, y, data = '';
nEmpty = nFilled = 0;
for(y = 0; y < 44; y += 4) {
for(x = 0; x < 52; x += 4) {
data += compressBlock(matrix, x, y);
}
}
console.log("Empty blocks: " + nEmpty);
console.log("Filled blocks: " + nFilled);
console.log("Average bits per block: " + (data.length / (nEmpty + nFilled)).toFixed(2));
console.log("Average bits per filled block: " + ((data.length - nEmpty) / nFilled).toFixed(2));
console.log("Final packed size: " + data.length + " bits --> " + ((data.length + 7) >> 3) + " bytes");
}
function compressBlock(matrix, x, y) {
var min = 0x100, max = 0, msk = 0, data = [],
width, v, x0, y0;
for(y0 = 0; y0 < 4; y0++) {
for(x0 = 0; x0 < 4; x0++) {
if(v = matrix[y + y0][x + x0]) {
msk |= 1 << (15 - y0 * 4 - x0);
data.push(v);
min = Math.min(min, v);
max = Math.max(max, v);
}
}
}
if(msk) {
nFilled++;
width = Math.ceil(Math.log(max + 1 - min) / Math.log(2));
data = data.map(function(v) { return bin(v - min, width); }).join('');
return '1' + bin(msk, 16) + bin(min, 8) + ((msk & -msk) == msk ? '' : bin(width, 3) + data);
}
nEmpty++;
return '0';
}
function bin(n, sz) {
var b = n.toString(2);
return Array(sz + 1 - b.length).join('0') + b;
}
compress([
[ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 ],
[ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 ],
[ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 ],
[ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 ],
[ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 10, 12, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 ],
[ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 7, 0, 0, 10, 12, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 15, 15, 9, 0, 0, 7, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 ],
[ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 7, 10, 9, 11, 7, 12, 21, 20, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 ],
[ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 13, 15, 13, 0, 0, 15, 28, 23, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 10, 0, 0, 0, 0, 0, 0, 0, 0, 0 ],
[ 0, 0, 0, 0, 0, 0, 0, 0, 0, 12, 7, 8, 0, 0, 0, 0, 14, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 14, 12, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 ],
[ 0, 0, 0, 0, 0, 0, 0, 0, 11, 10, 0, 0, 11, 19, 12, 9, 8, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 10, 12, 12, 14, 24, 26, 21, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 ],
[ 0, 0, 0, 0, 0, 0, 0, 0, 7, 0, 0, 21, 33, 38, 30, 23, 26, 15, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 7, 15, 16, 17, 22, 29, 32, 26, 18, 0, 0, 0, 0, 0, 0, 0, 0, 0 ],
[ 0, 0, 0, 0, 0, 0, 0, 0, 7, 0, 22, 38, 46, 47, 42, 33, 27, 28, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 7, 14, 18, 18, 23, 28, 32, 31, 23, 12, 0, 0, 0, 0, 0, 0, 0, 0 ],
[ 0, 0, 0, 0, 0, 0, 0, 7, 7, 17, 31, 52, 54, 55, 48, 36, 34, 32, 9, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 9, 12, 12, 17, 22, 29, 28, 26, 17, 7, 0, 0, 0, 0, 0, 0, 0 ],
[ 0, 0, 0, 0, 0, 0, 0, 0, 10, 26, 40, 50, 51, 48, 38, 28, 30, 25, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 14, 23, 22, 20, 16, 10, 0, 0, 0, 0, 0, 0, 0 ],
[ 0, 0, 0, 0, 0, 0, 0, 0, 20, 30, 38, 40, 42, 37, 27, 19, 18, 10, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 11, 15, 13, 12, 10, 0, 0, 0, 0, 0, 0, 0 ],
[ 0, 0, 0, 0, 0, 0, 0, 13, 24, 27, 28, 30, 32, 26, 13, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 9, 12, 9, 11, 0, 0, 0, 0, 0, 0, 0 ],
[ 0, 0, 0, 0, 0, 0, 0, 14, 26, 27, 24, 24, 19, 10, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 7, 0, 0, 0, 0, 0, 0, 0, 0 ],
[ 0, 0, 0, 0, 0, 0, 0, 7, 20, 22, 19, 17, 12, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 ],
[ 0, 0, 0, 0, 0, 0, 0, 0, 15, 16, 17, 14, 9, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 ],
[ 0, 0, 0, 0, 0, 0, 0, 0, 0, 15, 14, 15, 11, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 ],
[ 0, 0, 0, 0, 0, 0, 0, 0, 0, 10, 16, 18, 15, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 ],
[ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 17, 19, 17, 9, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 ],
[ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 19, 20, 20, 9, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 ],
[ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 18, 20, 21, 12, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 ],
[ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 12, 19, 16, 10, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 ],
[ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 12, 11, 8, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 ],
[ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 9, 8, 8, 7, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 10, 12, 12, 10, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 ],
[ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 8, 10, 10, 13, 13, 8, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 9, 20, 25, 24, 17, 9, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 ],
[ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 13, 20, 26, 25, 24, 11, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 20, 28, 32, 31, 24, 13, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 ],
[ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 20, 28, 36, 39, 34, 26, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 11, 29, 36, 39, 37, 30, 18, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 ],
[ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 22, 31, 43, 50, 58, 39, 15, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 19, 39, 46, 46, 40, 32, 20, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 ],
[ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 24, 38, 51, 60, 64, 54, 26, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 25, 40, 49, 49, 44, 33, 20, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 ],
[ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 25, 45, 59, 65, 68, 66, 32, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 21, 40, 46, 46, 42, 31, 11, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 ],
[ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 22, 44, 56, 66, 70, 61, 32, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 11, 31, 35, 38, 31, 18, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 ],
[ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 11, 31, 55, 66, 64, 52, 25, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 9, 17, 18, 11, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 ],
[ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 17, 36, 50, 50, 32, 12, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 ],
[ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 13, 22, 21, 12, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 ],
[ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 ],
[ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 ],
[ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 ],
[ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 ],
[ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 ],
[ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 ],
[ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 ]
]);
The final output is 349 bytes long.
Empty blocks: 102
Filled blocks: 41
Average bits per block: 19.50
Average bits per filled block: 65.51
Final packed size: 2788 bits --> 349 bytes
I would test JPEG-LS. It is a very fast algorithm and provides state-of-the art lossless compression results for many types of images. In particular, its prediction algorithm will provide results comparable to RLE for the flat regions, and much better results for the foot areas.
Since you are transmitting several frames, and these frames are likely to be very similar, you may want to try to subtract one frame from the next before applying JPEG-LS (you will probably need to remap the pixels to positive integers before using JPEG-LS, though).
If you don't need strictly lossless compression (i.e., if you can tolerate some distortion in the reconstructed images), you can test the near-lossless mode, which bounds the maximum absolute error introduced in any given pixel.
You can find a very good and complete implementation here https://jpeg.org/jpegls/software.html.
I'm making kind of an histogram stored in a Matrix on OpenCV.
So, if I match one result, I will at +1 on some index.
My mat is:
Mat countFramesMatrix = Mat::zeros(9,9,CV_8U);
when I try to access to sum +1 to the already set index (from 0), I do:
int valueMatrixFrames = countFramesMatrix.at<int>(sliceMatch.j, sliceMatch.i);
valueMatrixFrames++;
countFramesMatrix.at<unsigned char>(sliceMatch.j, sliceMatch.i) = (unsigned char)valueMatrixFrames;
I tried in other ways, as changing unsigned char for int an other problems I had before, but nothing happens.
My results are:
Or all the matrix is zeros.
Or I get something like:
[2.3693558e-38, 0, 0, 0, 0, 0, 0, 0, 0;
2.3693558e-38, 0, 0, 0, 0, 0, 0, 0, 0;
0, 0, 0, 0, 0, 0, 0, 0, 0;
0, 0, 0, 0, 0, 0, 0, 0, 0;
0, 0, 0, 0, 0, 0, 0, 0, 0;
0, 0, 0, 0, 0, 0, 0, 0, 0;
0, 0, 0, 0, 0, 0, 0, 0, 0;
0, 0, 0, 0, 0, 0, 0, 0, 0;
0, 0, 0, 0, 0, 0, 0, 0, 0]
And I'm never storing data at (0,0) or (0,1) or (1,0) or (1,1), :(
What would you suggest? thank you.
You are misting a very simple mistake,
valueMatrixFrames++;
will increment value of valueMatrixFrames, not of the matrix location.
RIGHT WAY
Let's say, if you want to increment at (1,1) you should use,
countFramesMatrix.at<uchar>(1,1)++;
OUTPUT
[0, 0, 0, 0, 0, 0, 0, 0, 0;
0, 1, 0, 0, 0, 0, 0, 0, 0;
0, 0, 0, 0, 0, 0, 0, 0, 0;
0, 0, 0, 0, 0, 0, 0, 0, 0;
0, 0, 0, 0, 0, 0, 0, 0, 0;
0, 0, 0, 0, 0, 0, 0, 0, 0;
0, 0, 0, 0, 0, 0, 0, 0, 0;
0, 0, 0, 0, 0, 0, 0, 0, 0;
0, 0, 0, 0, 0, 0, 0, 0, 0]
Running the above command again will increment the value at (1, 1) to 2.
[0, 0, 0, 0, 0, 0, 0, 0, 0;
0, 2, 0, 0, 0, 0, 0, 0, 0;
0, 0, 0, 0, 0, 0, 0, 0, 0;
0, 0, 0, 0, 0, 0, 0, 0, 0;
0, 0, 0, 0, 0, 0, 0, 0, 0;
0, 0, 0, 0, 0, 0, 0, 0, 0;
0, 0, 0, 0, 0, 0, 0, 0, 0;
0, 0, 0, 0, 0, 0, 0, 0, 0;
0, 0, 0, 0, 0, 0, 0, 0, 0]
So, your histogram is ready!!