I'm new to libtorch and I need to load a LSTM network in C++. Before that, I have already tested with the following Python script and it is working well:
actuator_net_file = "resources/actuator_nets/anydrive_v3_lstm.pt"
actuator_network = torch.jit.load(actuator_net_file)
actuator_network.eval()
num_envs = 1
num_actions = 1
sea_input = torch.zeros(num_envs*num_actions, 1, 2, requires_grad=False)
sea_hidden_state = torch.zeros(2, num_envs*num_actions, 8, requires_grad=False)
sea_cell_state = torch.zeros(2, num_envs*num_actions, 8, requires_grad=False)
torques, (sea_hidden_state[:], sea_cell_state[:]) = actuator_network(sea_input, (sea_hidden_state, sea_cell_state))
And the next step is to write a simple C++ program to test the forward evaluation of the network. But I don't know how to give arguments to the forward function. Here is what I got:
#include <torch/script.h> // One-stop header.
#include <torch/torch.h>
#include <iostream>
#include <memory>
#include <vector>
int main(int argc, const char* argv[]) {
if (argc != 2) {
std::cerr << "usage: example-app <path-to-exported-script-module>\n";
return -1;
}
std::string actuator_net_file = "/home/fenglongsong/Desktop/example-app/anydrive_v3_lstm.pt";
torch::jit::script::Module actuator_network;
try {
actuator_network = torch::jit::load(actuator_net_file);
actuator_network.eval();
}
catch (const c10::Error& e) {
std::cerr << "error loading the model\n";
return -1;
}
std::cout << "load model ok\n";
const int num_envs = 1;
const int num_actions = 1;
auto u0 = torch::zeros({num_envs*num_actions, 1, 2});
auto h0 = torch::zeros({2, num_envs*num_actions, 8});
auto c0 = torch::zeros({2, num_envs*num_actions, 8});
std::vector<torch::jit::IValue> inputs;
inputs.push_back(u0);
std::vector<torch::jit::IValue> tuple;
tuple.push_back(h0);
tuple.push_back(c0);
inputs.push_back(c10::ivalue::Tuple::create(tuple));
std::cout << "before forward" << std::endl;
actuator_network.forward(inputs).toTensor();
}
The compile passes successfully but when running the executable, the following error occurs:
fenglongsong#alvaro-rsl ~/Desktop/example-app/build $ ./example-app .
load model ok
before forward
terminate called after throwing an instance of 'c10::Error'
what(): Expected Tensor but got Tuple
Exception raised from reportToTensorTypeError at ../aten/src/ATen/core/ivalue.cpp:908 (most recent call first):
frame #0: c10::Error::Error(c10::SourceLocation, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >) + 0x6b (0x7f9153dc07ab in /home/fenglongsong/Documents/ocs2_ws/src/libtorch/lib/libc10.so)
frame #1: c10::detail::torchCheckFail(char const*, char const*, unsigned int, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) + 0xce (0x7f9153dbc15e in /home/fenglongsong/Documents/ocs2_ws/src/libtorch/lib/libc10.so)
frame #2: c10::IValue::reportToTensorTypeError() const + 0x64 (0x7f913dd6d304 in /home/fenglongsong/Documents/ocs2_ws/src/libtorch/lib/libtorch_cpu.so)
frame #3: c10::IValue::toTensor() && + 0x4b (0x55d1ce3cd311 in ./example-app)
frame #4: main + 0x54e (0x55d1ce3ca0ec in ./example-app)
frame #5: __libc_start_main + 0xf3 (0x7f913c7c6083 in /lib/x86_64-linux-gnu/libc.so.6)
frame #6: _start + 0x2e (0x55d1ce3c986e in ./example-app)
Aborted (core dumped)
My question is, what should be the equivalence in C++ of
torques, (sea_hidden_state[:], sea_cell_state[:]) = actuator_network(sea_input, (sea_hidden_state, sea_cell_state)) ? Any suggessions will be much appreciated!
I have experience only directly with torch::Tensor, not with torch::jit::IValue, but generally, the signature is:
network.forward(const Tensor & input, torch::optional<std::tuple<Tensor,Tensor>>)
So in your example, you would call it like this:
actuator_network.forward(u0, std::make_tuple(h0, c0)).toTensor();
But it depends on the actual architecture, next time, include the actuator network architecture to your question to make it more clear.
I am trying to run a math intensive C++ program on Ubuntu server and surprisingly the Ubuntu Server 16 running on a baremetal Core i7 6700 is taking more time than a dual core Ubuntu server 12.04.5 running on a VM over windows 10 on the same machine. Its totally surprising to see this result. Am using the GCC version 5.4.1 on both. Also tried compiling using the -Ofast and -ffast-math but didn't make any difference. Also tried fetching the latest gcc 7.2 on the bare metal but again it didn't make any difference whatsoever. Also tried fetching the latest libm (glibc) and tried with no difference in the numbers at all. Can someone please help in letting me know where things are going wrong?
Also running callgrind over the program (am using a third party so library and so have no control over it), I see most of the time being spent in libm. The only difference between the two environments other than the server version is the libm version. On VM which performed well it was 2.15 and on the bare metal which takes more time it is 2.23. Any suggestions will be greatly appreciated. Thanks.
The build command is :
g++ -std=c++14 -O3 -o scicomplintest EuroFutureOption_test.cpp -L. -lFEOption
The program is to calculate option greeks for a set of 22 strike prices using a library whose source code isn't available. However would be able to answer any questions w.r.t the test code.
Have simplified the latency calculation using the class below:
typedef std::chrono::high_resolution_clock::time_point TimePoint;
typedef std::chrono::high_resolution_clock SteadyClock;
template <typename precision = std::chrono::microseconds>
class EventTimerWithPrecision
{
public:
EventTimerWithPrecision() { _beg = SteadyClock::now(); }
long long elapsed() {
return std::chrono::duration_cast<precision>(SteadyClock::now()
- _beg).count();
}
void reset() { _beg = SteadyClock::now(); }
private:
TimePoint _beg;
};
typedef EventTimerWithPrecision<> EventTimer;
Now am getting the times as below:
Ubuntu server 12.04.5 on VM with dual core (over windows 10):
siril#ubuntu:/media/sf_workshare/scicompeurofuturestest$ ./scicomplintest
Mean time: 61418 us
Min time: 44990 us
Max time: 79033 us
Ubuntu server 16 on Core i7 6700 bare metal:
Mean time: 104888 us
Min time: 71015 us
Max time: 125928 us
on Windows 10 (MSVC 14) on Core i7 6700 bare metal:
D:\workshare\scicompeurofuturestest\x64\Release>scicompwintest.exe
Mean time: 53322 us
Min time: 39655 us
Max time: 64506 us
I can understand windows 10 performing faster than linux on VM but why is the baremetal ubuntu so slow?
Unable to get to any conclusion am pasting the whole test code below. Please help (really curious to know why its behaving so).
#include <iostream>
#include <vector>
#include <numeric>
#include <algorithm>
#include "FEOption.h"
#include <chrono>
#define PRINT_VAL(x) std::cout << #x << " = " << (x) << std::endl
typedef std::chrono::high_resolution_clock::time_point TimePoint;
typedef std::chrono::high_resolution_clock SteadyClock;
template <typename precision = std::chrono::microseconds>
class EventTimerWithPrecision
{
public:
EventTimerWithPrecision() { _beg = SteadyClock::now(); }
long long elapsed() {
return std::chrono::duration_cast<precision>(SteadyClock::now() - _beg).count();
}
void reset() { _beg = SteadyClock::now(); }
private:
TimePoint _beg;
};
typedef EventTimerWithPrecision<> EventTimer;
int main(){
int cnt, nWarmup = 10, nTimer = 100000;
double CompuTime;
// Option Parameters
double Omega[] = {
-1,
-1,
-1,
1,
1,
1,
1,
-1,
-1,
-1,
1,
1,
1,
1,
-1,
-1,
-1,
1,
1,
1,
1,
-1,
-1,
-1,
1,
1,
1,
1,
-1,
-1,
-1,
1,
1,
1,
1,
-1,
-1,
-1,
1,
1,
1,
1
};
double Strike[] = {
92.77434863,
95.12294245,
97.5309912,
100,
102.5315121,
105.1271096,
107.7884151,
89.93652726,
93.17314234,
96.52623599,
100,
103.598777,
107.327066,
111.1895278,
85.61884708,
90.16671558,
94.95615598,
100,
105.311761,
110.90567,
116.796714,
80.28579206,
86.38250571,
92.9421894,
100,
107.5937641,
115.7641807,
124.5550395,
76.41994703,
83.58682355,
91.4258298,
100,
109.3782799,
119.6360811,
130.8558876,
73.30586976,
81.30036598,
90.16671558,
100,
110.90567,
123.0006763,
136.4147241
};
double Expiration[] = {
7,
7,
7,
7,
7,
7,
7,
14,
14,
14,
14,
14,
14,
14,
30,
30,
30,
30,
30,
30,
30,
60,
60,
60,
60,
60,
60,
60,
90,
90,
90,
90,
90,
90,
90,
120,
120,
120,
120,
120,
120,
120
};
int TradeDaysPerYr = 252;
// Market Parameters
double ValueDate = 0;
double Future = 100;
double annualSigma = 0.3;
double annualIR = 0.05;
// Numerical Parameters
int GreekSwitch = 2;
double annualSigmaBump = 0.01;
double annualIRBump = 0.0001;
double ValueDateBump = 1;
double PV;
double Delta;
double Gamma;
double Theta;
double Vega;
double Rho;
sciStatus_t res;
int nData = sizeof(Strike) / sizeof(double);
std::vector<long long> v(nData);
for (int i = 0; i < nData; i++)
{
for (cnt = 0; cnt < nWarmup; ++cnt){
res = EuroFutureOptionFuncC(annualIR, annualSigma, Omega[i], ValueDate, Expiration[i], Future, Strike[i], TradeDaysPerYr, annualIRBump + cnt*1.0e-16,
annualSigmaBump, ValueDateBump, GreekSwitch,
&PV,
&Delta,
&Gamma,
&Theta,
&Vega,
&Rho
);
if (res != SCI_STATUS_SUCCESS) {
std::cout << "Failure with error code " << res << std::endl;
return -1;
}
}
EventTimer sci;
for (cnt = 0; cnt < nTimer; ++cnt){
res = EuroFutureOptionFuncC(annualIR, annualSigma, Omega[i], ValueDate, Expiration[i], Future, Strike[i], TradeDaysPerYr, annualIRBump + cnt*1.0e-16,
annualSigmaBump, ValueDateBump, GreekSwitch,
&PV,
&Delta,
&Gamma,
&Theta,
&Vega,
&Rho
);
if (res != SCI_STATUS_SUCCESS) {
std::cout << "Failure with error code " << res << std::endl;
return -1;
}
}
v[i] = sci.elapsed();
}
long long sum = std::accumulate(v.begin(), v.end(), 0);
long long mean_t = (double)sum / v.size();
long long max_t = *std::max_element(v.begin(), v.end());
long long min_t = *std::min_element(v.begin(), v.end());
std::cout << "Mean time: " << mean_t << " us" << std::endl;
std::cout << "Min time: " << min_t << " us" << std::endl;
std::cout << "Max time: " << max_t << " us" << std::endl;
std::cout << std::endl;
PRINT_VAL(PV);
PRINT_VAL(Delta);
PRINT_VAL(Gamma);
PRINT_VAL(Theta);
PRINT_VAL(Vega);
PRINT_VAL(Rho);
return 0;
}
The callgrind graph is as follow:
callgrind graph
More updates:
Tried -fopenacc and -fopenmp on both baremetal and vm ubuntu on the same g++ 7.2. The vm showed a little improvement but the baremetal ubuntu is showing the same number again and again. Also since the majority of the time spent is in libm, is there any way to upgrade that library ?(glibc) ? Don't see any new version of it in apt-cache though
Used callgrind and plotted a graph using dot. According to that it takes 42.27% time in libm exp (version 2.23) and 15.18% time in libm log.
Finally found a similar post (so pasting it here for others): The program runs 3 times slower when compiled with g++ 5.3.1 than the same program compiled with g++ 4.8.4, the same command
The problem as suspected was from the libs (according to the post). And by setting the LD_BIND_NOW the execution times came down drastically (and now less than VM). Also that post has couple of links to bugs that were filed for that version of glibc. Will go through and will give more details here. However thanks for all the valuable inputs.
This question is similar to cuModuleLoadDataEx options but I would like to bring the topic up again and in addition provide more information.
When loading a PTX string with the NV driver via cuModuleLoadDataEx it seems to ignore all options all together. I provide full working examples so that anyone interested can directly and with no effort reproduce this. First a small PTX kernel (save this as small.ptx) then the C++ program that loads the PTX kernel.
.version 3.1
.target sm_20, texmode_independent
.address_size 64
.entry main()
{
ret;
}
main.cc
#include<cstdlib>
#include<iostream>
#include<fstream>
#include<sstream>
#include<string>
#include<map>
#include "cuda.h"
int main(int argc,char *argv[])
{
CUdevice cuDevice;
CUcontext cuContext;
CUfunction func;
CUresult ret;
CUmodule cuModule;
cuInit(0);
std::cout << "trying to get device 0\n";
ret = cuDeviceGet(&cuDevice, 0);
if (ret != CUDA_SUCCESS) { exit(1);}
std::cout << "trying to create a context\n";
ret = cuCtxCreate(&cuContext, 0, cuDevice);
if (ret != CUDA_SUCCESS) { exit(1);}
std::cout << "loading PTX string from file " << argv[1] << "\n";
std::ifstream ptxfile( argv[1] );
std::stringstream buffer;
buffer << ptxfile.rdbuf();
ptxfile.close();
std::string ptx_kernel = buffer.str();
std::cout << "Loading PTX kernel with driver\n" << ptx_kernel;
const unsigned int jitNumOptions = 3;
CUjit_option *jitOptions = new CUjit_option[jitNumOptions];
void **jitOptVals = new void*[jitNumOptions];
// set up size of compilation log buffer
jitOptions[0] = CU_JIT_INFO_LOG_BUFFER_SIZE_BYTES;
int jitLogBufferSize = 1024*1024;
jitOptVals[0] = (void *)&jitLogBufferSize;
// set up pointer to the compilation log buffer
jitOptions[1] = CU_JIT_INFO_LOG_BUFFER;
char *jitLogBuffer = new char[jitLogBufferSize];
jitOptVals[1] = jitLogBuffer;
// set up wall clock time
jitOptions[2] = CU_JIT_WALL_TIME;
float jitTime = -2.0;
jitOptVals[2] = &jitTime;
ret = cuModuleLoadDataEx( &cuModule , ptx_kernel.c_str() , jitNumOptions, jitOptions, (void **)jitOptVals );
if (ret != CUDA_SUCCESS) { exit(1);}
std::cout << "walltime: " << jitTime << "\n";
std::cout << std::string(jitLogBuffer) << "\n";
}
Build (assuming CUDA is installed under /usr/local/cuda, I use CUDA 5.0):
g++ -I/usr/local/cuda/include -L/usr/local/cuda/lib64/ main.cc -o main -lcuda
If someone is able to extract any sensible information from the compilation process that would be great! The documentation of CUDA driver API where cuModuleLoadDataEx is explained (and which options it is supposed to accept) http://docs.nvidia.com/cuda/cuda-driver-api/index.html
If I run this, the log is empty and jitTime wasn't even touched by the NV driver:
./main small.ptx
trying to get device 0
trying to create a context
loading PTX string from file empty.ptx
Loading PTX kernel with driver
.version 3.1
.target sm_20, texmode_independent
.address_size 64
.entry main()
{
ret;
}
walltime: -2
EDIT:
I managed to get the JIT compile time. However it seems that the driver expects an array of 32bit values as OptVals. Not as stated in the manual as an array of pointers (void *) which are on my system 64 bits. So, this works:
const unsigned int jitNumOptions = 1;
CUjit_option *jitOptions = new CUjit_option[jitNumOptions];
int *jitOptVals = new int[jitNumOptions];
jitOptions[0] = CU_JIT_WALL_TIME;
// here the call to cuModuleLoadDataEx
std::cout << "walltime: " << (float)jitOptions[0] << "\n";
I believe that it is not possible to do the same with an array of void *. The following code does not work:
const unsigned int jitNumOptions = 1;
CUjit_option *jitOptions = new CUjit_option[jitNumOptions];
void **jitOptVals = new void*[jitNumOptions];
jitOptions[0] = CU_JIT_WALL_TIME;
// here the call to cuModuleLoadDataEx
// here I also would have a problem casting a 64 bit void * to a float (32 bit)
EDIT
Looking at the JIT compilation time jitOptVals[0] was misleading. As mentioned in the comments, the JIT compiler caches previous translations and won't update the JIT compile time if it finds a cached compilation. Since I was looking whether this value has changed or not I assumed that the call ignores the options all together. Which it doesn't. It's works fine.
Your jitOptVals should not contain pointers to your values, instead cast the values to void*:
// set up size of compilation log buffer
jitOptions[0] = CU_JIT_INFO_LOG_BUFFER_SIZE_BYTES;
int jitLogBufferSize = 1024*1024;
jitOptVals[0] = (void *)jitLogBufferSize;
// set up pointer to the compilation log buffer
jitOptions[1] = CU_JIT_INFO_LOG_BUFFER;
char *jitLogBuffer = new char[jitLogBufferSize];
jitOptVals[1] = jitLogBuffer;
// set up wall clock time
jitOptions[2] = CU_JIT_WALL_TIME;
float jitTime = -2.0;
//Keep jitOptVals[2] empty as it only an Output value:
//jitOptVals[2] = (void*)jitTime;
and after cuModuleLoadDataEx, you get your jitTime like jitTime = (float)jitOptions[2];
I'am actually trying to extract images from a gifFile using the giflib with the following code.
t_gif initGif(const char *filename){
t_gif gif;
int *error;
GifFileType *GifFile = DGifOpenFileName(filename, error);
assert(error != NULL);
int ret = DGifSlurp(GifFile);
assert(ret == GIF_OK);
gif.h = (int)GifFile->SHeight;
gif.w = GifFile->SWidth;
gif.nbFrames = GifFile->ImageCount;
gif.colorSize = GifFile->SColorResolution;
GifImageDesc Image = GifFile->Image;
SavedImage *img = &GifFile->SavedImages[0];
cout << "width: " << gif.w << endl;
cout << "height: " << gif.h << endl;
cout << "Image Count: " << gif.nbFrames << endl;
cout << "SColor Resolution: " << gif.colorSize << endl;
Mat color = Mat(Size(gif.w, gif.h), CV_8UC1, img->RasterBits);
imwrite("./test.png", color);
return gif;
}
But this causes a segfault. I am using opencv in v2.4.5 and giflib in v5.0.4.
I think this is not caused by Opencv because with giflib in v4 I had no problem here.
MoreOver the following test also causes segfault.
printf("%u\n", (unsigned int)img->RasterBits[0]);
Gdb output:
(gdb) run bsd.gif
Starting program: /home/matt/Code/perso/utils/gif/a.out bsd.gif
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
Gif Analyser tool
started by MG in may 2013
compiled with giflib: v5.0.4
width: 1000
height: 907
Image Count: 0
SColor Resolution: 8
Program received signal SIGSEGV, Segmentation fault.
0x0000000000401af3 in initGif (filename=0x7fffffffe2be "bsd.gif") at main.cpp:40
40 Mat color = Mat(Size(gif.w, gif.h), CV_8UC1, img->RasterBits);
(gdb)
I also don't find how to get color back using the colorGlobal table.
Can someone Help me ?
Thanks
I just get back to version 4.1.6 and everything is good. The v5 is probably bugged.
To get the color information , you can use the rasterbits of the image as index to the colormap hence getting the RGB information for the corresponding pixel.Then you can simply use CV_8UC3 data type for storing the gif frame as a colored png .
I'm following these tutorials on modern OpenGL. I've done them up to number 15 "Camera Control - Part 2". The tutorial suggests using glutWarpPointer(). The problem is, my program crashes at that call. This is my code:
c_camera::c_camera(int width, int height, const c_vector3f& Pos, const c_vector3f& Target, const c_vector3f& Up){
m_windowWidth = width;
m_windowHeight = height;
m_pos = Pos;
m_target = Target;
m_target.Normalize();
m_up = Up;
m_up.Normalize();
Init();
}
void c_camera::Init(){
c_vector3f HTarget(m_target.x, 0.0, m_target.z);
HTarget.Normalize();
if (HTarget.z >= 0.0f){
if (HTarget.x >= 0.0f){
m_AngleH = 360.0f - (asin(HTarget.z) TO_DEG);
} else {
m_AngleH = 180.0f + (asin(HTarget.z) TO_DEG);
}
} else {
if (HTarget.x >= 0.0f){
m_AngleH = (asin(-HTarget.z) TO_DEG);
} else {
m_AngleH = 90.0f + (asin(-HTarget.z) TO_DEG);
}
}
m_AngleV = -(asin(m_target.y) TO_DEG);
m_OnUpperEdge = false;
m_OnLowerEdge = false;
m_OnLeftEdge = false;
m_OnRightEdge = false;
m_mousePos.x = m_windowWidth / 2;
m_mousePos.y = m_windowHeight / 2;
cout << "this gets printed just fine" << endl;
glutWarpPointer(500,400); //program crashes
cout << "this doesn't get printed" << endl;
}
I'm not sure if I'm doing something weird here, or if I just have a bad glut version (seems unlikely to me) or if the tutorial is just wrong... Do I need to set up something glut specific before I can call glutWarpPointer()? I am new to glut, and new to modern OpenGL (I learned immediate mode first).
A quick google search didn't help me much. Any help would be appreciated.
Edit: I am on windows, and I'm using mingw 4.5
Edit2: These are the details windows gives me about the crash:
Problem Event Name: APPCRASH
Application Name: modern_opengl.exe
Application Version: 0.0.0.0
Application Timestamp: 51044575
Fault Module Name: glut32.dll
Fault Module Version: 0.0.0.0
Fault Module Timestamp: 3bea4ff3
Exception Code: c0000005
Exception Offset: 0000a879
OS Version: 6.2.9200.2.0.0.256.48
Locale ID: 1043
Additional Information 1: 5861
Additional Information 2: 5861822e1919d7c014bbb064c64908b2
Additional Information 3: f3d5
Additional Information 4: f3d5be0cad2787556264647dc02181c3
Edit3: This is my call stack:
0 1000A879 glutWarpPointer() (C:\Windows\system\glut32.dll:??)
1 004033FB c_camera::Init(this=0x4aa0e0) (C:\CodeBlocks\projects\modern_opengl\c_camera.cpp:50)
2 00403164 c_camera::c_camera(this=0x4aa0e0, width=800, height=600, Pos=..., Target=..., Up=...) (C:\CodeBlocks\projects\modern_opengl\c_camera.cpp:18)
3 00402F4B __static_initialization_and_destruction_0(__initialize_p=1, __priority=65535) (C:\CodeBlocks\projects\modern_opengl\main.cpp:55)
4 00403004 GLOBAL_sub_I_vertices() (C:\CodeBlocks\projects\modern_opengl\main.cpp:177)
5 0043595B __do_global_ctors() (../mingw/gccmain.c:59)
6 00401098 __mingw_CRTStartup() (../mingw/crt1.c:236)
7 00401284 mainCRTStartup() (../mingw/crt1.c:264)
Your function seems to be in c_camera::Init, which seems to be called before main probably due to it being instantiated as a global object (globals are constructed before main is entered). You should delay glut calls till after you enter main and called glutInit is called.