Comparison of floating point arrays using google test and google mock - c++

I am new to Google's test products and trying them out with some signal processing code. I am trying to assert that to floating point arrays are equal to within some bounds, using google mock as suggested by the answer to this question. I would like to know the recommended method for adding some error tolerance to an expression like the following . . .
EXPECT_THAT( impulse, testing::ElementsAreArray( std::vector<float>({
0, 0, 0, 1, 1, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0
}) ) );
I want the test to pass if the element-wise values in the arrays are within 10-8 of one another.

The following works for me:
using ::testing::Pointwise;
using ::testing::FloatNear;
auto const max_abs_error = 1 / 1024.f;
ASSERT_THAT(
test,
Pointwise(FloatNear(max_abs_error), ref));
Where test and ref are of type std::vector<float>.

One approach is to use the googletest rather than googlemock macros, which results in a more compact assert:
#define EXPECT_FLOATS_NEARLY_EQ(expected, actual, thresh) \
EXPECT_EQ(expected.size(), actual.size()) << "Array sizes differ.";\
for (size_t idx = 0; idx < std::min(expected.size(), actual.size()); ++idx) \
{ \
EXPECT_NEAR(expected[idx], actual[idx], thresh) << "at index: " << idx;\
}
// define expected_array as in the other answer
EXPECT_FLOATS_NEARLY_EQ(impulse, expected_array, 0.001);

Here is one method. First define a matcher outside of the test scope. According to the documentation, the matcher cannot be defined in a class or function . .
MATCHER_P(FloatNearPointwise, tol, "Out of range") {
return (std::get<0>(arg)>std::get<1>(arg)-tol && std::get<0>(arg)<std::get<1>(arg)+tol) ;
}
Then is can be used with Pointwise int the test . . .
std::vector<float> expected_array({
0, 0, 0, 1, 1, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0
});
EXPECT_THAT( impulse, Pointwise( FloatNearPointwise(1e-8), expected_array ) );
But it would be neater if there was a solution that used the builtin FloatNear directly.

Related

Is there a way to overload a constructor with a 2D array (int)?

When I utilize the default constructor, I expect it to call the constructor that accepts an argument; however, this does not occur correctly. When debugging, as far as I can tell it is assigning the values and then the instance simply isn't maintained. I'm not sure if I need to create a helper method instead to pass the object, array, assign out values, and then pass back the object?
My goal is to have a default constructor that passes a hard-coded set of values and then a constructor that accepts the same type of array passed as values.
I've tried passing the array as an argument for the constructors, and while it seems to work for the derived class, it does not work for the base class. I ended up moving the functionality of the overloaded constructor to the default constructor and that works correctly.
This is the base class:
// Puzzle.h
class Puzzle
{
public:
Puzzle();
Puzzle(int grid[gridLength][gridLength]);
~Puzzle();
void Print_Puzzle(); // Displays puzzle in console
protected:
int grid[gridLength][gridLength]; // Our board
private:
};
This is the definition:
Puzzle::Puzzle()
{
int grid[gridLength][gridLength] = // Taken from https://www.puzzles.ca/sudoku_puzzles/sudoku_easy_505.html
{
{ 0, 7, 0, 0, 0, 2, 0, 0, 0 },
{ 0, 9, 0, 3, 7, 0, 0, 0, 0 },
{ 0, 0, 5, 0, 8, 0, 0, 0, 1 },
{ 0, 0, 4, 7, 0, 0, 0, 0, 9 },
{ 0, 0, 0, 0, 9, 6, 0, 0, 0 },
{ 0, 0, 0, 0, 0, 8, 6, 5, 4 },
{ 0, 2, 0, 0, 0, 0, 0, 0, 0 },
{ 0, 0, 0, 0, 0, 1, 0, 4, 3 },
{ 4, 0, 7, 9, 5, 0, 2, 6, 0 }
};
for (int x = 0; x < gridLength; x++)
{
for (int y = 0; y < gridLength; y++)
Puzzle::grid[x][y] = grid[x][y];
}
// Puzzle::Puzzle(grid) // Doesn't work. Not sure how to properly pass the array values.
}
Puzzle::Puzzle(int grid[gridLength][gridLength])
{
for (int row = 0; row < gridLength; row++)
{
for (int col = 0; col < gridLength; col++)
this->grid[row][col] = grid[row][col];
}
}
I expect the default constructor to pass the grid variable and the receiving constructor to assign those values to the instance's member property.
A more convenient way and no less efficient is to use std::array which can be copied, so that you do not have to copy it element-wise:
#include <array>
constexpr int gridLength = 9;
using Grid = std::array<std::array<int, gridLength>, gridLength>;
Grid const grid = {{ // Taken from https://www.puzzles.ca/sudoku_puzzles/sudoku_easy_505.html
{ 0, 7, 0, 0, 0, 2, 0, 0, 0 },
{ 0, 9, 0, 3, 7, 0, 0, 0, 0 },
{ 0, 0, 5, 0, 8, 0, 0, 0, 1 },
{ 0, 0, 4, 7, 0, 0, 0, 0, 9 },
{ 0, 0, 0, 0, 9, 6, 0, 0, 0 },
{ 0, 0, 0, 0, 0, 8, 6, 5, 4 },
{ 0, 2, 0, 0, 0, 0, 0, 0, 0 },
{ 0, 0, 0, 0, 0, 1, 0, 4, 3 },
{ 4, 0, 7, 9, 5, 0, 2, 6, 0 }
}};
class Puzzle {
Grid grid_;
public:
Puzzle(Grid const& grid)
: grid_(grid) // Copy grid into grid_.
{}
};
int main() {
Puzzle p(grid);
}
Alternatively:
class Puzzle {
Grid grid_;
static Grid make_grid() {
Grid grid;
// Your code to fill the grid.
return grid;
}
public:
Puzzle()
: Puzzle(make_grid())
{}
Puzzle(Grid const& grid)
: grid_(grid) // Copy the grid.
{}
};
In my opinion, you are committing a design error.
Never use C arrays in C++, use std::vector or std::array instead.
Try this.
class Sudoku
{
public:
Sudoku(const std::vector<std::vector<int>> initData = { {/* Write your default values here */} })
{
data = initData;
}
private:
std::vector<std::vector<int>> data;
};
if you want to use C-like arrays you will need to understand the only way to pass arrays is via pointers, and is a messy solution.
template <uint32_t width, uint32_t height>
class Sudoku
{
public:
Sudoku(int** initData, int maxX, int maxY)
{
for (int i = 0; i < maxX; i++) {
for (int j = 0; j < maxY; j++) {
data[i][j] = initData[i][j];
}
}
}
private:
std::array<width, std::array<height, int>> data;
};

How to use an existing DL4J trained model to classify new input

I have a DL4J LSTM model that generates a binary classification of sequential input. i have trained and tested the model and am happy with the precision/recall. Now I want to use this model to predict the binary classification of new inputs. How do I do this? i.e. how do I give the trained neural network a single Input (file containing the sequence of feature rows) and get the binary classification of this input file.
Here is my original training data set iterator:
SequenceRecordReader trainFeatures = new CSVSequenceRecordReader(0, ","); //skip no header lines
try {
trainFeatures.initialize( new NumberedFileInputSplit(featureBaseDir + "/s_%d.csv", 0,this._modelDefinition.getNB_TRAIN_EXAMPLES()-1));
} catch (IOException e) {
trainFeatures.close();
throw new IOException(String.format("IO error %s. during trainFeatures", e.getMessage()));
} catch (InterruptedException e) {
trainFeatures.close();
throw new IOException(String.format("Interrupted exception error %s. during trainFeatures", e.getMessage()));
}
SequenceRecordReader trainLabels = new CSVSequenceRecordReader();
try {
trainLabels.initialize(new NumberedFileInputSplit(labelBaseDir + "/s_%d.csv", 0,this._modelDefinition.getNB_TRAIN_EXAMPLES()-1));
} catch (InterruptedException e) {
trainLabels.close();
trainFeatures.close();
throw new IOException(String.format("Interrupted exception error %s. during trainLabels initialise", e.getMessage()));
}
DataSetIterator trainData = new SequenceRecordReaderDataSetIterator(trainFeatures, trainLabels,
this._modelDefinition.getBATCH_SIZE(),this._modelDefinition.getNUM_LABEL_CLASSES(), false, SequenceRecordReaderDataSetIterator.AlignmentMode.ALIGN_END);
Here is my model:
MultiLayerConfiguration conf = new NeuralNetConfiguration.Builder()
.seed(this._modelDefinition.getRANDOM_SEED()) //Random number generator seed for improved repeatability. Optional.
.weightInit(WeightInit.XAVIER)
.updater(new Nesterovs(this._modelDefinition.getLEARNING_RATE()))
.gradientNormalization(GradientNormalization.ClipElementWiseAbsoluteValue) //Not always required, but helps with this data set
.gradientNormalizationThreshold(0.5)
.list()
.layer(0, new LSTM.Builder().activation(Activation.TANH).nIn(this._modelDefinition.getNB_INPUTS()).nOut(this._modelDefinition.getLSTM_LAYER_SIZE()).build())
.layer(1, new LSTM.Builder().activation(Activation.TANH).nIn(this._modelDefinition.getLSTM_LAYER_SIZE()).nOut(this._modelDefinition.getLSTM_LAYER_SIZE()).build())
.layer(2,new DenseLayer.Builder().nIn(this._modelDefinition.getLSTM_LAYER_SIZE()).nOut(this._modelDefinition.getLSTM_LAYER_SIZE())
.weightInit(WeightInit.XAVIER)
.build())
.layer(3, new RnnOutputLayer.Builder(LossFunctions.LossFunction.MCXENT)
.activation(Activation.SOFTMAX).nIn(this._modelDefinition.getLSTM_LAYER_SIZE()).nOut(this._modelDefinition.getNUM_LABEL_CLASSES()).build())
.pretrain(false).backprop(true).build();
I train the model over N epochs to get my optimal scores. I save the model, now I want to open the model and get classifications for new sequential feature files.
If there is an example of this - please let me know where.
thanks
anton
The answer is to feed the model the exact same input as we trained with, except set the labels to -1. The output will be an INDarray containing the probability of 0 in one array and the probability of 1 in the other array, showing up in the last sequence line.
Here is the code:
public void getOutputsForTheseInputsUsingThisNet(String netFilePath,String inputFileDir) throws Exception {
//open the network file
File locationToSave = new File(netFilePath);
MultiLayerNetwork nNet = null;
logger.info("Trying to open the model");
try {
nNet = ModelSerializer.restoreMultiLayerNetwork(locationToSave);
logger.info("Success: Model opened");
} catch (IOException e) {
throw new Exception(String.format("Unable to open model from %s because of error %s", locationToSave.getAbsolutePath(),e.getMessage()));
}
logger.info("Loading test data");
SequenceRecordReader testFeatures = new CSVSequenceRecordReader(0, ","); //skip no lines at the top - i.e. no header
try {
testFeatures.initialize(new NumberedFileInputSplit(inputFileDir + "/features/s_4180%d.csv", 0, 4));
} catch (InterruptedException e) {
testFeatures.close();
throw new Exception(String.format("IO error %s. during testFeatures", e.getMessage()));
}
logger.info("Loading label data");
SequenceRecordReader testLabels = new CSVSequenceRecordReader();
try {
testLabels.initialize(new NumberedFileInputSplit(inputFileDir + "/labels/s_4180%d.csv", 0,4));
} catch (InterruptedException e) {
testLabels.close();
testFeatures.close();
throw new IOException(String.format("Interrupted exception error %s. during testLabels initialise", e.getMessage()));
}
//DataSetIterator inputData = new Seque
logger.info("creating iterator");
DataSetIterator testData = new SequenceRecordReaderDataSetIterator(testFeatures, testLabels,
this._modelDefinition.getBATCH_SIZE(),this._modelDefinition.getNUM_LABEL_CLASSES(), false, SequenceRecordReaderDataSetIterator.AlignmentMode.ALIGN_END);
//now use it to classify some data
logger.info("classifying examples");
INDArray output = nNet.output(testData);
logger.info("outputing the classifications");
if(output==null||output.isEmpty())
throw new Exception("There is no output");
System.out.println(output);
//sample output
// [[[ 0, 0, 0, 0, 0.9882, 0, 0, 0, 0],
// [ 0, 0, 0, 0, 0.0118, 0, 0, 0, 0]],
//
// [[ 0, 0.1443, 0, 0, 0, 0, 0, 0, 0],
// [ 0, 0.8557, 0, 0, 0, 0, 0, 0, 0]],
//
// [[ 0, 0, 0, 0, 0, 0, 0, 0, 0.9975],
// [ 0, 0, 0, 0, 0, 0, 0, 0, 0.0025]],
//
// [[ 0, 0, 0, 0, 0, 0, 0.8482, 0, 0],
// [ 0, 0, 0, 0, 0, 0, 0.1518, 0, 0]],
//
// [[ 0, 0, 0, 0.8760, 0, 0, 0, 0, 0],
// [ 0, 0, 0, 0.1240, 0, 0, 0, 0, 0]]]
}

D3D11_INPUT_ELEMENT_DESC: Element types / ordering / packing

Are there concerns (performance or other) related to the types / ordering of elements in a D3D11_INPUT_ELEMENT_DESC structure? For example, I now have an input layout defined like this:
D3D11_INPUT_ELEMENT_DESC QuadInputLayoutDescription[] = {
{ "PRECT", 0, DXGI_FORMAT_R32G32B32A32_FLOAT, 0, 0, D3D11_INPUT_PER_INSTANCE_DATA, 1 },
{ "DEPTH", 0, DXGI_FORMAT_R32_FLOAT, 0, D3D11_APPEND_ALIGNED_ELEMENT, D3D11_INPUT_PER_INSTANCE_DATA, 1 },
{ "TEXID", 0, DXGI_FORMAT_R32_UINT, 0, D3D11_APPEND_ALIGNED_ELEMENT, D3D11_INPUT_PER_INSTANCE_DATA, 1 },
{ "SLICE", 0, DXGI_FORMAT_R32_FLOAT, 0, D3D11_APPEND_ALIGNED_ELEMENT, D3D11_INPUT_PER_INSTANCE_DATA, 1 },
{ "UVRCT", 0, DXGI_FORMAT_R32G32B32A32_FLOAT, 0, D3D11_APPEND_ALIGNED_ELEMENT, D3D11_INPUT_PER_INSTANCE_DATA, 1 },
{ "COLOR", 0, DXGI_FORMAT_R32G32B32A32_FLOAT, 0, D3D11_APPEND_ALIGNED_ELEMENT, D3D11_INPUT_PER_INSTANCE_DATA, 1 } };
Does it matter that the TEXID element is of type UINT rather than FLOAT here? Is "alignment" against a four component vector struct (float4) preferred; Should I "pack" all the R32G32B32A32_FLOATs together at the beginning of the layout? Is anything padded?
The primary performance issue for input layouts is the total size of them as it directly impacts the amount of data that has to pass through the pre- and post-transform vertex cache. Historically, the idea total size is 32 bytes or 64 bytes.
Otherwise, the types you use don't usually make a lot of difference in terms of performance. The limit on types used is your target hardware Feature Level.

Creating an empty .zip file in C++

I am using a code snippet from this page on how to create a zip file and add and a compress a directory to that zip file. I am running the following on Windows 7 but it does not seem to create the zip file at all.
BSTR bstrFolderOutName = L"C:\\Test\\Archive.zip";
BYTE startBuffer[] = {80, 75, 5, 6, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0};
FILE *f = _wfopen(bstrFolderOutName, L"wb");
fwrite(startBuffer,sizeof(startBuffer),1,f);
fclose(f);
The stated problem, that no file is created, is impossible to answer with the information given. It is most likely due to an invalid file path. However, the OP states in a comment that the path in his example is not the real code.
EDIT: the hex string example that I cited originally was wrong, I just tested.
This code works:
#include <stdio.h>
auto main() -> int
{
FILE* f = fopen("foo.zip", "wb");
//fwrite( "\x80\x75\x05\x06\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0", 22, 1, f );
fwrite( "\x50\x4B\x05\x06\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0", 22, 1, f );
fclose(f);
}
Harumph, one cannot even trust Stack Overflow comments. Not to mention accepted answers.
Original text:
Assuming that the OP now has edited the code so that the part below is the real code, then this constant
{80, 75, 5, 6, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}
is not identical to
"\x80\x75\x05\x06\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"
Can the OP spot the relevant difference?
Further, given that, can the OP infer anything about his source of information?
My example from a comment elsewhere.

Initialize large two dimensional array in C++

I want to have static and constant two dimensional array inside a class. The array is relatively large, but I only want to initialize a few elements and others may be whatever compiler initializes them to.
For example, if a class is defined like:
class A {
public:
static int const test[10][10];
};
int const A::test[10][10] = {
{0, 0, 0, 0, 0, 0, 0, 0, 0, 0},
{0, 0, 0, 0, 0, 0, 0, 0, 0, 0},
{0, 0, 0, 0, 0, 0, 0, 0, 0, 0},
{0, 0, 0, 7, 7, 7, 7, 0, 0, 0},
{0, 0, 0, 7, 7, 7, 7, 0, 0, 0},
{0, 0, 0, 7, 7, 7, 7, 0, 0, 0},
{0, 0, 0, 7, 7, 7, 7, 0, 0, 0},
{0, 0, 0, 0, 0, 0, 0, 0, 0, 0},
{0, 0, 0, 0, 0, 0, 0, 0, 0, 0},
{0, 0, 0, 0, 0, 0, 0, 0, 0, 0}
};
and I am interested only to initialize the elements marked with '7', how do I do this on the same elements, but with array of larger size, like array[1024][1024]?
Any part of an array which is initialized, that is beyond the initialization, is initialized to 0. Hence:
int const A::test[10][10]; // uninitialized
int const A::test[10][10] = { {0} }; // all elements initialized to 0.
int const A::test[10][10] = {1,2}; // test[0][0] ==1, test[0][1]==2, rest==0
That means all you have to initialize is up to the last non-zero:
int const A::test[10][10] = {
{0, 0, 0, 0, 0, 0, 0, 0, 0, 0},
{0, 0, 0, 0, 0, 0, 0, 0, 0, 0},
{0, 0, 0, 0, 0, 0, 0, 0, 0, 0},
{0, 0, 0, 7, 7, 7, 7, 0, 0, 0},
{0, 0, 0, 7, 7, 7, 7, 0, 0, 0},
{0, 0, 0, 7, 7, 7, 7, 0, 0, 0},
{0, 0, 0, 7, 7, 7, 7, 0, 0, 0}
};
It is not the best solution, but will save some work.
There is no way to assign an int to const array after it's been initialized. So you will have to script it up:
And include your file this way:
class A {
public:
static const int test[10][10];
};
const int A::test[10][10] = {
#include "data.inc" // points to the file generated by script.
};
Coincidentally, a couple of hours after reading your question, I bumped into a possible solution while looking for something else in the book "C - A Reference Manual" 5th ed., Harbison/Steele (this is a fantastic C reference, by the way).
According to the book,
C99 allows you to name the components of an aggregate (structs, union or array) to be initialized within an initializer list.
... and it gives an example:
int a1[5] = { [2]=100, [1]=3 }; /* eqv. to {0, 3, 100, 0, 0} */
So, depending on the compliance of your compiler and on the size of the non-zero elements in your array, you may be able to use this syntax to init your matrix efficiently. That said, the book doesn't give an example for 2D arrays. Unfortunately, I couldn't test this idea since MSVC++ 2005 doesn't seem to support C99.
When I do this, I use a method to read in data. Generically, it looks like:
extern void ReadElements(string sFile, Matrix a)
{
int x;
int y;
double value;
ifstream myInFile;
myInFile.open(sFile, ifstream::in);
while(!myInFile.eof())
{
myInFile >> x >> y >> value;
a[x][y] = value;
}
myInFile.close();
return;
}
You could access the array only through accessor functions/macros and arrange the internal storage so, that the initialzed part goes first.
A solution would be to hide non-const array somewhere, load it from file or resource, and then use const reference to access it. I.e.
#include <stdlib.h>
#include <string.h>
#include <stdio.h>
typedef int Array[1024][1024];
namespace DontTouch{
Array arr;
void initArray(){
for (int i = 0; i < 1024; i++)
for (int j = 0; j < 1024; j++)
arr[i][j] = rand() & 0xff;
}
}
const Array &arr = DontTouch::arr;
int main(int argc, char** argv){
DontTouch::initArray();
//arr[4][4] = 0;//compiler error
for (int i = 0; i < 1024; i++){
for (int j = 0; j < 1024; j++)
printf(" 0x%02x", arr[i][j]);
printf("\n");
}
return 0;
}
It will be (IMO) more readable than script-generated huge array.
You can do same thing with class that acts like 2D array (easy enough to write). Again - have non-const object somewhere, and use const reference to access the data. It should be easy to make non-const array completely invisibile outside of just one cpp.
Another way to do that is to generate array using script. If you think that big arrays are ugly, put entire thing into *.h file (make sure it is included in only one *.cpp file), so it won't scare people away. Compiler doesn't care what you write in your code as long as it is syntactically correct.
I don't think there are any other options.
It only takes four lines by using std::fill_n
using std::fill_n;
using std::begin;
fill_n(begin(test[3])+3, 4, 7);
fill_n(begin(test[4])+3, 4, 7);
fill_n(begin(test[5])+3, 4, 7);
fill_n(begin(test[6])+3, 4, 7);
install R software, it's free !
then call function defined below with
writetable(data,"test","myfile.h")
if data is your matrix then you're done
writetable<-function(data,varname="test",file="myFile.hpp"){
cat('const static double CONST_array_',varname," [][] = { \n \t\t\t\t {",file=file,append=TRUE,sep='')
for (j in 1:(dim(data)[2]-1)){
for (i in 1:(dim(data)[1]-1) ){
cat(data[i,j],',',file=file,append=TRUE)
}
cat(data[dim(data)[1],j],'},\n \t\t\t\t\t{',file=file,append=TRUE)
}
for (i in 1:(dim(data)[1]-1) ){
cat(data[i,dim(data)[2]],',',file=file,append=TRUE)
}
cat(data[dim(data)[1],dim(data)[2]],'}\n }; \n',file=file,append=TRUE)
}