I have an acoustic model that successfully converted from ONNX to OpenVino. However, in OpenVino this model outputs tensor that consists of zeroes from some position.
#include <iostream>
#include <fstream>
#include <iterator>
#include <inference_engine.hpp>
typedef struct {
float* data;
size_t size;
size_t timeLen;
} Fbank;
using namespace InferenceEngine;
using std::cout;
using std::endl;
void print_arr(std::string text, const float* arr, int l, int r) {
cout << text << endl;
for (int i = l; i < r; i++) {
cout << arr[i] << " ";
}
cout << endl;
}
void doInference(ExecutableNetwork& executable_network, const std::string& input_name, const std::string& output_name, Fbank* fbank) {
InferRequest infer_request = executable_network.CreateInferRequest();
InferenceEngine::TensorDesc tDesc(InferenceEngine::Precision::FP32,
{fbank->size, fbank->timeLen}, InferenceEngine::Layout::HW);
Blob::Ptr blob = InferenceEngine::make_shared_blob<float>(tDesc, fbank->data);
infer_request.SetBlob(input_name, blob);
infer_request.Infer();
Blob::Ptr output_blob = infer_request.GetBlob(output_name);
auto dims = output_blob->getTensorDesc().getDims();
size_t batchSize = dims[0];
size_t T = dims[1];
size_t D = dims[2];
MemoryBlob::CPtr moutput = as<MemoryBlob>(output_blob);
if (!moutput) {
return;
}
auto moutputHolder = moutput->rmap();
const float *pred = moutputHolder.as<const float*>();
print_arr("AM output:", pred, D*29, D*31);
}
int main() {
Fbank* fbank = new Fbank;
fbank->size = 64;
fbank->timeLen = 2000;
fbank->data = new float[64*2000];
Core ie;
CNNNetwork network = ie.ReadNetwork("quartznet_random.xml", "quartznet_random.bin");
std::string input_name = network.getInputsInfo().begin()->first;
std::string output_name = network.getOutputsInfo().begin()->first;
network.getOutputsInfo().begin()->second->setPrecision(Precision::FP32);
ExecutableNetwork executable_network = ie.LoadNetwork(network, "cpu");
doInference(executable_network, input_name, output_name, fbank);
return 0;
}
Outputs:
AM output:
0.138650 -5.833140 -8.023724 -7.637482 -8.001101 -9.033963 -8.029905 -8.132050 -9.186495 -8.537528 -8.788505 -9.240234 -8.547676 -8.673388 0.000000 0.000000 -0.000000 0.000000 -0.000000 0.000000 0.000000 -0.000000 -0.000000 0.000000 -0.000000 0.000000 0.000000 -0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 -0.000000 0.000000 0.000000 -0.000000 0.000000 0.000000 -0.000000 -0.000000 -0.000000 -0.000000 -0.000000 0.000000 -0.000000 -0.000000 -0.000000 0.000000 -0.000000 -0.000000 -0.000000 0.000000 0.000000 -0.000000 -0.000000 0.000000 -0.000000 0.000000 -0.000000 -0.000000 -0.000000 0.000000 -0.000000 -0.000000 0.000000 -0.000000 0.000000
If I run ONNX model in Python using onnxruntime, the output will be correct. (Example).
Is it possible to fix it?
P.S. Command to convert the model from ONNX: python3 mo_onnx.py —input_model model.onnx —output="output" —input="fbanks[64 2000]"
Tested provided ONNX model in OpenVINO for Linux, couple of findings while testing OpenVINO 2020.1 and new 2020.2 version (released today 4/14/2020, release notes).
Using same command to convert from ONNX. Although its unclear what would be the expected output (probability between 0.0 and 1.0?), OpenVINO 2020.2 seems to affect the output results.
On 2020.1, observed similar results to yours (one can assume this is the OpenVINO version you used).
AM output: -3.55062 -3.5114 -3.50925 -3.52013 -3.51791 -3.54656 -3.53908 -3.54239 -3.53626 -3.50982 -3.54193 -3.55593 -3.52877 -3.53786 -1.546e-07 -6.14673e-08 -8.56817e-08 -1.41561e-07 -6.14673e-08 -1.16415e-07 -9.30158e-08 -9.12696e-08 -1.29454e-07 -1.04774e-07 -6.14673e-08 -5.58794e-08 -1.71363e-07 -1.02445e-07 -5.7742e-08 -1.35042e-07 -9.26666e-08 -1.00583e-07 -1.04308e-07 -1.2666e-07 -1.39698e-07 -7.26432e-08 -9.68575e-08 -1.47149e-07 -9.40636e-08 -9.77889e-08 -9.49949e-08 -1.16415e-07 -9.54606e-08 -8.3819e-08 -1.28523e-07 -1.35973e-07 -7.66013e-08 -1.12224e-07 -1.546e-07 -6.14673e-08 -8.56817e-08 -1.41561e-07 -6.14673e-08 -1.16415e-07 -9.30158e-08 -9.12696e-08 -1.29454e-07 -1.04774e-07 -6.14673e-08 -5.58794e-08 -1.71363e-07 -1.02445e-07 -5.7742e-08 -1.35042e-07 -9.26666e-08 -1.00583e-07 -1.04308e-07 -1.2666e-07
On OpenVINO 2020.2 had to change ExecutableNetwork executable_network = ie.LoadNetwork(network, "cpu"); to ExecutableNetwork executable_network = ie.LoadNetwork(network, "CPU"); as Inference Engine didnt't recognize lowercase CPU device, error was "terminate called after throwing an instance of 'InferenceEngine::details::InferenceEngineException'
what(): Device with "cpu" name is not registered in the InferenceEngine
Aborted (core dumped)"
On OpenVINO 2020.2, the results differ and are not close to zero (although all seem negative).
AM output: -3.55062 -3.5114 -3.50925 -3.52013 -3.51791 -3.54656 -3.53908 -3.54239 -3.53626 -3.50982 -3.54193 -3.55593 -3.52877 -3.53786 -3.52153 -3.52563 -3.51142 -3.54885 -3.52137 -3.54384 -3.53411 -3.55188 -3.5477 -3.52514 -3.51171 -3.5022 -3.5138 -3.50823 -3.50125 -3.51817 -3.53914 -3.50173 -3.50603 -3.51917 -3.55062 -3.5114 -3.50925 -3.52013 -3.51791 -3.54656 -3.53908 -3.54239 -3.53626 -3.50982 -3.54193 -3.55593 -3.52877 -3.53786 -3.52153 -3.52563 -3.51142 -3.54885 -3.52137 -3.54384 -3.53411 -3.55188 -3.5477 -3.52514 -3.51171 -3.5022 -3.5138 -3.50823 -3.50125 -3.51817 -3.53914 -3.50173 -3.50603 -3.51917
It's uncertain if the output results of OpenVINO 2020.2 are expected/correct. I am unable to test Python example with the ONNX model using onnxruntime, script expects /kek/fbank.out file. Clarify/share what output is expected, i.e. correct AM output.
The problem was due to numerical instability in our implementation of LogSoftmax. Without log of softmax all works fine.
Related
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 2 years ago.
Improve this question
I want to replace a line of a file, but only if the 3rd (last) value is above 0.000000.
Specifically I have an Ascii .stl file, like below, where I want to be able to check the last value of each vertex line, and replace the entire line with a different string if it is above 0.000000.
I am really struggling with understanding how to create a parser using regex, to look for this. I also don't know if that's the best way.
Any help with the parsing would be very much appreciated.
facet normal -1.000000 0.000000 0.000000
outer loop
vertex 26.000000 6.000000 0.000000
vertex 26.000000 6.000000 1.000000
vertex 26.000000 7.000000 1.000000
endloop
endfacet
facet normal -1.000000 0.000000 0.000000
outer loop
vertex 26.000000 6.000000 0.000000
vertex 26.000000 7.000000 1.000000
vertex 26.000000 7.000000 0.000000
endloop
endfacet
facet normal 0.000000 1.000000 0.000000
outer loop
vertex 26.000000 7.000000 0.000000
vertex 26.000000 7.000000 1.000000
vertex 27.000000 7.000000 1.000000
endloop
endfacet
endsolid
If it matters, I'm working in Microsoft Visual Studio 2019
For something as simple as detecting which line starts with "vertex", I don't think you need something as heavy as a regular expression.
Here is a very simple example of detecting the line that starts with "vertex":
#include <sstream>
#include <string>
#include <iostream>
std::string test = "facet normal -1.000000 0.000000 0.000000\n"
"outer loop\n"
"vertex 26.000000 6.000000 0.000000\n"
"vertex 26.000000 6.000000 1.000000\n"
"vertex 26.000000 7.000000 1.000000\n"
"endloop\n"
"endfacet\n"
"facet normal -1.000000 0.000000 0.000000\n"
"outer loop\n"
"vertex 26.000000 6.000000 0.000000\n"
"vertex 26.000000 7.000000 1.000000\n"
"vertex 26.000000 7.000000 0.000000\n"
"endloop\n"
"endfacet\n"
"facet normal 0.000000 1.000000 0.000000\n"
"outer loop\n"
"vertex 26.000000 7.000000 0.000000\n"
"vertex 26.000000 7.000000 1.000000\n"
"vertex 27.000000 7.000000 1.000000\n"
"endloop\n"
"endfacet\n"
"endsolid";
int main()
{
std::istringstream strm(test);
std::string line;
int lineNumber = 0;
while (std::getline(strm, line))
{
++lineNumber;
if (line.compare( 0, 6, "vertex", 6) == 0) // check if line starts with "vertex"
{
std::istringstream strm2(line); // read the line
std::string type;
double num1, num2, num3;
strm2 >> type >> num1 >> num2 >> num3;
if ( num3 > 0 ) // checks the last number
std::cout << "Line " << lineNumber << ": " << line << "\n"; // just outputs that line
}
}
}
Output:
Line 4: vertex 26.000000 6.000000 1.000000
Line 5: vertex 26.000000 7.000000 1.000000
Line 11: vertex 26.000000 7.000000 1.000000
Line 18: vertex 26.000000 7.000000 1.000000
Line 19: vertex 27.000000 7.000000 1.000000
In lieu of a file, the code reads the data from a stringstream.
It's just a matter of detecting the line that starts with "vertex", and then using a std::istringstream to read the data.
I would agree with the comments of others that Regex may not be the best approach depending on the nature of your solution. But for a Regex to find what you're after:
vertex.+\s[0-9\.]*[1-9][0-9\.]*$
Should do the trick.
(regex101)
I want to understand how OBJ format dealing with texture coordinates.
example:
vt 1.000000 1.005200
vt 0.467300 1.709900
vt 0.923800 1.994400
vt 0.500000 1.002600
vt 0.371400 1.000000
vt 0.438100 2.000000
vt 0.000000 1.000000
vt 0.467300 1.709900
vt 0.105000 1.159500
vt 0.434600 1.002300
i understand values should range from (0) to (1.000000) to cover Texture image file from 0% to 100% for each X and Y. (tu tv)
But i find some values in array are above 1.000000, and sometime below 0.000000
How should i deal with these values to stay between 0 and 1 ?
This kind of texture coodinate value indicates that the texture shall be repeated. In the cases of the obj-file
vt 0.438100 2.000000
the v-part shall be repeated twice.
I have written a piece of code to plot a graph using GNUplot from a data file.
It's giving a warning:
warning: Skipping data file with no valid points
The code is:
{
FILE *gnuplotPipe, *tempDataFile;
FILE * pFile;
char *tempDataFileName;
char *Datafile;
double x, y;
int i;
tempDataFileName = "Pulse.txt";
Datafile = "PulseFinal.dat";
gnuplotPipe = _popen("gnuplot", "w");
if (gnuplotPipe)
{
fprintf(gnuplotPipe, "plot \"%s\" '-' using 1:2 with lines", Datafile);
fflush(gnuplotPipe);
printf("press enter to continue...");
getchar();
fprintf(gnuplotPipe, "exit \n");
}
else
{
printf("gnuplot not found...");
}
}
The data file is:
0.000000 0.018519
1.000000 0.000000
2.000000 0.000000
3.000000 0.000000
4.000000 0.000000
5.000000 0.000000
6.000000 0.000000
7.000000 0.000000
8.000000 0.000000
9.000000 0.000000
10.000000 -0.006173
Can someone please help me with this?
You have tried
plot \"%s\" '-' using 1:2 with lines
which means for Gnuplot
plot "YPulseFinal.dat" '-' using 1:2 with lines
You can't plot a file and the stream at the same time. You can
plot "YPulseFinal.dat" using 1:2 with lines
or
plot '-' using 1:2 with lines
I recommend you
fprintf(gnuplotPipe, "plot \"%s\" using 1:2 with lines", Datafile);
Side note: hey everyone, if you found my question/answer helpful, please don't forget to up vote. I kind of need it...
So there seems to be something different with my implementation of both matrix [projection and model] (other than the stuff I've commented out for debugging purposes). Below is a screenshot of the bug I see when drawing a cube. Keep in mind I do keep the viewport and matrix up to date with the window size and calculate screen ratio with float and not int, so don't bother asking, I've checked the usual suspects.....
Screen Shot
Files (linux build, see readme in ./build)
side note: while debugging, I've changed the cube's distance. To reproduce the screen shot, on line 76 of workspace.cpp set mDistance to about 90 and stretch the window frame to dimensions noted at lower right corner of the window.
Please keep in mind the screen shot and the debug text output are seperate events as I'm constantly debugging this problem and getting new numbers.
The code:
#define _AP_MAA 0
#define _AP_MAB 1
#define _AP_MAC 2
#define _AP_MAD 3
#define _AP_MBA 4
#define _AP_MBB 5
#define _AP_MBC 6
#define _AP_MBD 7
#define _AP_MCA 8
#define _AP_MCB 9
#define _AP_MCC 10
#define _AP_MCD 11
#define _AP_MDA 12
#define _AP_MDB 13
#define _AP_MDC 14
#define _AP_MDD 15
Setting up the camera perspective:
void APCamera::setPerspective(GMFloat_t fov, GMFloat_t aspect, GMFloat_t near, GMFloat_t far)
{
GMFloat_t difZ = near - far;
GMFloat_t *data;
mProjection->clear(); //set to identity matrix
data = mProjection->getData();
GMFloat_t v = 1.0f / tan(fov / 2.0f);
data[_AP_MAA] = v / aspect;
data[_AP_MBB] = v;
data[_AP_MCC] = (far + near) / (difZ);
data[_AP_MCD] = -1.0f;
data[_AP_MDD] = 0.0f;
data[_AP_MDC] = (2.0f * far * near)/ (difZ);
mRatio = aspect;
mInvProjOutdated = true;
mIsPerspective = true;
}
Setting up the camera direction:
bool APCamera::lookTo(Coordinate &to, Coordinate &from, Coordinate &up)
{
Coordinate f, unitUp, right;
GMFloat_t *data;
CoordinateOp::diff(&to, &from, &f);
VectorOp::toUnit(&f, &f);
VectorOp::toUnit(&up, &unitUp);
VectorOp::cross(&f, &unitUp, &right);
if((fabs(right.x) < FLOAT_THRESHOLD) && (fabs(right.y) < FLOAT_THRESHOLD) && (fabs(right.z) < FLOAT_THRESHOLD))
{
return false;
}
mCamPt = from;
VectorOp::toUnit(&right, &mRight);
mForward = f;
VectorOp::cross(&mRight, &mForward, &mUp);
mModelView->clear();
data = mModelView->getData();
data[_AP_MAA] = mRight.x;
data[_AP_MBA] = mRight.y;
data[_AP_MCA] = mRight.z;
data[_AP_MAB] = mUp.x;
data[_AP_MBB] = mUp.y;
data[_AP_MCB] = mUp.z;
data[_AP_MAC] = -mForward.x;
data[_AP_MBC] = -mForward.y;
data[_AP_MCC] = -mForward.z;
//translation part is commented out to narrow bugs down, "camera" is kept at the center (0,0,0)
//data[_AP_MDA] = (data[_AP_MAA] * -mCamPt.x) + (data[_AP_MBA] * -mCamPt.y) + (data[_AP_MCA] * -mCamPt.z);
//data[_AP_MDB] = (data[_AP_MAB] * -mCamPt.x) + (data[_AP_MBB] * -mCamPt.y) + (data[_AP_MCB] * -mCamPt.z);
//data[_AP_MDC] = (data[_AP_MAC] * -mCamPt.x) + (data[_AP_MBC] * -mCamPt.y) + (data[_AP_MCC] * -mCamPt.z);
mInvViewOutdated = true;
return true;
}
The debug output:
LookTo() From:<0,0,0> To:<-1,0,0>:
0.000000 0.000000 -1.000000 0.000000
0.000000 1.000000 0.000000 0.000000
1.000000 -0.000000 -0.000000 0.000000
0.000000 0.000000 0.000000 1.000000
setPerspective() fov:0.785398 ratio:1.185185 near:0.500000 far:100.000000:
2.036993 0.000000 0.000000 0.000000
0.000000 2.414213 0.000000 0.000000
0.000000 0.000000 -1.010050 -1.005025
0.000000 0.000000 -1.000000 0.000000
In the end, it looks like the trouble maker was just the FOV. So the quick answer is NO I didn't do anything different from the documented perspective and look at function. For anyone having a similar problem 2.0f * atan(tan(DEFAULT_FOV_RAD/mRatio) * mRatio) did the job for me.
This has to be some stupid error, but I have been working at this bug for days and have finally narrowed it down to a bizarre issue. Maybe I'm just not seeing it because most of my experience is programming in java, not C++. I am animating a snowman to move on some terrain in openGL, and whenever I set the velocity of a snowman to be on a random walk, it is not preserved for the next time that the update function is called. I do this by calculating a probability, and if that probability check succeeds, the snowman is given a random velocity and that is added to the position every update. Somehow, the velocities keep getting reset to 0 every time the update function exits, causing them to jump a little bit for one frame, and then teleport back to their previous position for future frames, ultimately not making them move. The x and y position are also restored to their previous value across calls. A sample of the command prompt output is here:
DEBUG: new frame
2) dx dy 0.000000 0.000000
256.000000 15.000000 0.000000 0.000000
2) dx dy 0.000000 0.000000
104.000000 17.000000 0.000000 0.000000
2) dx dy 0.000000 0.000000
256.000000 256.000000 0.000000 0.000000
1) dx dy -0.100000 -0.400000
2) dx dy -0.100000 -0.400000
306.000000 498.000000 -0.100000 -0.400000
2)dx dy 0.000000 0.000000
6.000000 256.000000 0.000000 0.000000
DEBUG: new frame
2) dx dy 0.000000 0.000000
256.000000 15.000000 0.000000 0.000000
2) dx dy 0.000000 0.000000
104.000000 17.000000 0.000000 0.000000
2) dx dy 0.000000 0.000000
256.000000 256.000000 0.000000 0.000000
2) dx dy 0.000000 0.000000
256.000000 256.000000 0.000000 0.000000
2)dx dy 0.000000 0.000000
6.000000 256.000000 0.000000 0.000000
These print out a bunch of values, but the thing to note is that the 4th element should have the same values between calls, and they don't. They get reset.
Snowman.h
#pragma once
#include "ppm_canvas.h"
class Snowman
{
public:
Snowman(canvas_t);
~Snowman(void);
void setWireframe(bool);
void toggleWireframe(void);
void setAnimate(bool);
void toggleAnimate(void);
void setArmSegments(int);
void addArmSegment(void);
void subtractArmSegment(void);
void update(canvas_t);
void draw(void);
private:
bool wireFrame;
bool animate;
bool walking;
int armSegments;
int animationFrameNumber;
float manualUserOffset;
float x, y, z;
float dx, dy;
inline float f(void);
inline void drawMouth(int headRadius);
inline void drawFace(int headRadius);
void drawArm(int remainingSegments);
inline void drawBody();
inline float getHeight(canvas_t);
};
From Snowman.cpp
void Snowman::update(canvas_t texture){
//randomly toggle the walking variable
int probability = rand() % 100;
if(probability <= 2){
walking = !walking;
this->dx = static_cast<float>(( (rand() % 100) - 50))/10.0;
this->dy = static_cast<float>(( (rand() % 100) - 50))/10.0;
printf("1) dx dy %f %f\n", dx, dy);
}
printf("2) dx dy %f %f\n", dx, dy);
//code to control movement
if(walking){
this->animate = true;
this->x += this->dx;
this->y += this->dy;
constrain(this->x, 0, texture.width * 2);
constrain(this->y, 0, texture.height * 2);
}else{
animate = false;
}
//set the height after x and y are resolved
this->z = getHeight(texture);
}
void Snowman::draw(void){
glPushMatrix();{
//turns out x is correct, y is vertical, and z is the other horozintal. oops...
float alittlebit = 10.0;//offset because the center of the bottom sphere is below 0
glRotatef(atan2(dy, dx), 0, 1, 0);
glTranslatef(x, z + alittlebit, y);
drawBody();
}glPopMatrix();
printf(" %f %f %f %f\n", this->x, this->y, this->dx, this->dy );
}
From Main.cpp
inline void drawSnowmen(){
printf("DEBUG: new frame\n");
glPushMatrix();{
glScalef(0.4, 0.4, 0.4);
glBindTexture(GL_TEXTURE_2D, 0);
for(Snowman i:snowmen){
i.update(terrain);
i.draw();
}
glBindTexture(GL_TEXTURE_2D, 1);
}glPopMatrix();
}
I've also included a video of the bug in action below. I'm so desperate to fix this having already spent over 8 hours on this alone, any help would be greatly appreciated.
http://www.youtube.com/watch?v=76uuu1sage0&feature=youtu.be
Also, there is no way to condense the problem to only this, but here is the entire source code:
http://people.ucsc.edu/~cchilder/openGLBug.zip
Try using a ref to the vector element in drawSnowmen(). See my comments above regarding the copy constructor.
for(Snowman &i:snowmen){
i.update(terrain);
i.draw();
}
Without the ref, the update() is modifying a temporary instance of Snowman, not the instance stored in the vector:
for(Snowman i:snowmen){
i.update(terrain);
i.draw();
}