How to set the input to a LSTM network in C++ - c++

Question

I'm new to libtorch and I need to load a LSTM network in C++. Before that, I have already tested with the following Python script and it is working well:
actuator_net_file = "resources/actuator_nets/anydrive_v3_lstm.pt"
actuator_network = torch.jit.load(actuator_net_file)
actuator_network.eval()
num_envs = 1
num_actions = 1
sea_input = torch.zeros(num_envs*num_actions, 1, 2, requires_grad=False)
sea_hidden_state = torch.zeros(2, num_envs*num_actions, 8, requires_grad=False)
sea_cell_state = torch.zeros(2, num_envs*num_actions, 8, requires_grad=False)
torques, (sea_hidden_state[:], sea_cell_state[:]) = actuator_network(sea_input, (sea_hidden_state, sea_cell_state))
And the next step is to write a simple C++ program to test the forward evaluation of the network. But I don't know how to give arguments to the forward function. Here is what I got:
#include <torch/script.h> // One-stop header.
#include <torch/torch.h>
#include <iostream>
#include <memory>
#include <vector>
int main(int argc, const char* argv[]) {
if (argc != 2) {
std::cerr << "usage: example-app <path-to-exported-script-module>\n";
return -1;
}
std::string actuator_net_file = "/home/fenglongsong/Desktop/example-app/anydrive_v3_lstm.pt";
torch::jit::script::Module actuator_network;
try {
actuator_network = torch::jit::load(actuator_net_file);
actuator_network.eval();
}
catch (const c10::Error& e) {
std::cerr << "error loading the model\n";
return -1;
}
std::cout << "load model ok\n";
const int num_envs = 1;
const int num_actions = 1;
auto u0 = torch::zeros({num_envs*num_actions, 1, 2});
auto h0 = torch::zeros({2, num_envs*num_actions, 8});
auto c0 = torch::zeros({2, num_envs*num_actions, 8});
std::vector<torch::jit::IValue> inputs;
inputs.push_back(u0);
std::vector<torch::jit::IValue> tuple;
tuple.push_back(h0);
tuple.push_back(c0);
inputs.push_back(c10::ivalue::Tuple::create(tuple));
std::cout << "before forward" << std::endl;
actuator_network.forward(inputs).toTensor();
}
The compile passes successfully but when running the executable, the following error occurs:
fenglongsong#alvaro-rsl ~/Desktop/example-app/build $ ./example-app .
load model ok
before forward
terminate called after throwing an instance of 'c10::Error'
what(): Expected Tensor but got Tuple
Exception raised from reportToTensorTypeError at ../aten/src/ATen/core/ivalue.cpp:908 (most recent call first):
frame #0: c10::Error::Error(c10::SourceLocation, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >) + 0x6b (0x7f9153dc07ab in /home/fenglongsong/Documents/ocs2_ws/src/libtorch/lib/libc10.so)
frame #1: c10::detail::torchCheckFail(char const*, char const*, unsigned int, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) + 0xce (0x7f9153dbc15e in /home/fenglongsong/Documents/ocs2_ws/src/libtorch/lib/libc10.so)
frame #2: c10::IValue::reportToTensorTypeError() const + 0x64 (0x7f913dd6d304 in /home/fenglongsong/Documents/ocs2_ws/src/libtorch/lib/libtorch_cpu.so)
frame #3: c10::IValue::toTensor() && + 0x4b (0x55d1ce3cd311 in ./example-app)
frame #4: main + 0x54e (0x55d1ce3ca0ec in ./example-app)
frame #5: __libc_start_main + 0xf3 (0x7f913c7c6083 in /lib/x86_64-linux-gnu/libc.so.6)
frame #6: _start + 0x2e (0x55d1ce3c986e in ./example-app)
Aborted (core dumped)
My question is, what should be the equivalence in C++ of
torques, (sea_hidden_state[:], sea_cell_state[:]) = actuator_network(sea_input, (sea_hidden_state, sea_cell_state)) ? Any suggessions will be much appreciated!

Answer 1

I have experience only directly with torch::Tensor, not with torch::jit::IValue, but generally, the signature is:
network.forward(const Tensor & input, torch::optional<std::tuple<Tensor,Tensor>>)
So in your example, you would call it like this:
actuator_network.forward(u0, std::make_tuple(h0, c0)).toTensor();
But it depends on the actual architecture, next time, include the actuator network architecture to your question to make it more clear.