I have a nested table in my lua code that I want to pass to C++ so the native code can manipulate it:
-- Some persistent data in my game
local data = {
{ 44, 34, 0, 7, },
{ 4, 4, 1, 3, },
}
-- Pass it into a C++ function that can modify the input data.
TimelineEditor(data)
How do I write my C++ code to read the nested table and modify its values?
Reading Lua nested tables in C++ and lua c read nested tables both describe how I can read from nested tables, but not how to write to them.
Short answer
Lua uses a stack to get values in and out of tables. To modify table values you'll need to push the table you want to modify with lua_rawgeti, push a value you want to insert with lua_pushinteger, and then set the value in the table with lua_rawseti.
When writing this, it's important to visualize the stack to ensure you use the right indexes:
lua_rawgeti()
stack:
table
lua_rawgeti()
stack:
number <-- top of the stack
table
lua_tonumber()
stack:
number
table
lua_pop()
stack:
table
lua_pushinteger()
stack:
number
table
lua_rawseti()
stack:
table
Negative indexes are stack positions and positive indexes are argument positions. So we'll often pass -1 to access the table at the stack. When calling lua_rawseti to write to the table, we'll pass -2 since the table is under the value we're writing.
Example
I'll add inspect.lua to the lua code to print out the table values so we can see that the values are modified.
local inspect = require "inspect"
local data = {
{ 44, 34, 0, 7, },
{ 4, 4, 1, 3, },
}
print("BEFORE =", inspect(data, { depth = 5, }))
TimelineEditor(data)
print("AFTER =", inspect(data, { depth = 5, }))
Assuming you've figured out BindingCodeToLua, you can implement the function like so:
// Replace LOG with whatever you use for logging or use this:
#define LOG(...) printf(__VA_ARGS__); printf("\n")
// I bound with Lunar. I don't think it makes a difference for this example.
int TimelineEditor(lua_State* L)
{
LOG("Read the values and print them out to show that it's working.");
{
int entries_table_idx = 1;
luaL_checktype(L, entries_table_idx, LUA_TTABLE);
int n_entries = static_cast<int>(lua_rawlen(L, entries_table_idx));
LOG("%d entries", n_entries);
for (int i = 1; i <= n_entries; ++i)
{
// Push inner table onto stack.
lua_rawgeti(L, entries_table_idx, i);
int item_table_idx = 1;
luaL_checktype(L, -1, LUA_TTABLE);
int n_items = static_cast<int>(lua_rawlen(L, -1));
LOG("%d items", n_items);
for (int i = 1; i <= n_items; ++i)
{
// Push value from table onto stack.
lua_rawgeti(L, -1, i);
int is_number = 0;
// Read value
int x = static_cast<int>(lua_tonumberx(L, -1, &is_number));
if (!is_number)
{
// fire an error
luaL_checktype(L, -1, LUA_TNUMBER);
}
LOG("Got: %d", x);
// pop value off stack
lua_pop(L, 1);
}
// pop table off stack
lua_pop(L, 1);
}
}
LOG("Overwrite the values");
{
int entries_table_idx = 1;
luaL_checktype(L, entries_table_idx, LUA_TTABLE);
int n_entries = static_cast<int>(lua_rawlen(L, entries_table_idx));
LOG("%d entries", n_entries);
for (int i = 1; i <= n_entries; ++i)
{
// Push inner table onto stack.
lua_rawgeti(L, entries_table_idx, i);
int item_table_idx = 1;
luaL_checktype(L, -1, LUA_TTABLE);
int n_items = static_cast<int>(lua_rawlen(L, -1));
LOG("%d items", n_items);
for (int j = 1; j <= n_items; ++j)
{
int x = j + 10;
// Push new value onto stack.
lua_pushinteger(L, x);
// rawseti pops the value off. Need to go -2 to get to the
// table because the value is on top.
lua_rawseti(L, -2, j);
LOG("Wrote: %d", x);
}
// pop table off stack
lua_pop(L, 1);
}
}
// No return values
return 0;
}
Output:
BEFORE = { { 44, 34, 0, 7 }, { 4, 4, 1, 3 } }
Read the values and print them out to show that it's working.
2 entries
4 items
Got: 44
Got: 34
Got: 0
Got: 7
4 items
Got: 4
Got: 4
Got: 1
Got: 3
Overwrite the values
2 entries
4 items
Wrote: 11
Wrote: 12
Wrote: 13
Wrote: 14
4 items
Wrote: 11
Wrote: 12
Wrote: 13
Wrote: 14
AFTER = { { 11, 12, 13, 14 }, { 11, 12, 13, 14 } }
Related
so i was trying to push a table on a function inside arguments
*lua
function test1(varlist)
print(varlist[1])
print(varlist[2])
print(varlist[3])
end
addHook("string", "string2", test1)
*cpp
static int lua_addHook(lua_State* L) {
if (lua_isstring(L, 1) && lua_isstring(L, 2) && lua_isfunction(L, 3)) {
lua_newtable(L);
lua_newtable(L);
for (size_t i = 0; i < 3; ++i) {
lua_pushinteger(L, i + 1);
lua_pushstring(L, string("string varlist: " + to_string(i)).c_str());
lua_settable(L, -3);
}
if (lua_pcall(L, 1, 0, 0) != 0) {
printf("error: %s\n", lua_tostring(L, -1));
lua_pop(L, 1);
}
}
return 1;
}
so it should printing
string varlist: 0
string varlist: 1
string varlist: 2
but i keep getting error "attempt to call a table value"
u know what the problem is?
The stack looks like this at the point of lua_pcall:
table (constructed by the loop above) # STACK TOP and arg1 to function call
table (empty) # interpreted as the function to call
function test1
string "string1"
string "string"
Getting rid of one of the lua_newtable calls should fix it.
My problem is : i gave a series of queries, and a series of references, and i want to count the number of occurance of said keys between them, but only if they have matching keys. I choose to have an LUT because i think it will help me efficiently, but im not sure if there are better ways, or the way im using LUT is not efficient enough.
i have the following data structures.
unordered_map <int, set<int>> Reference_map1, Reference_map2, ... , Reference_mapM;
// to story all the reference maps by their neightas
unordered_map <string, unordered_map <int, set<int>>> ReferenceMaps;
unordered_map <int, set<string>> LUT // look up table
// N is significantly greater than M
unordered_map <int, set<int>> query_map1, query_map2, ... , query_mapN;
Example of Reference_mapi and query_mapj
Reference_map1[111] = {0, 1, 2};
Reference_map1[333] = {1, 2, 3};
Reference_map1[888] = {2, 8, 0};
Reference_map2[111] = {1, 5, 9};
Reference_map2[999] = {0, 7, 4};
ReferenceMaps['Reference_map1']=Reference_map1;
ReferenceMaps['Reference_map2']=Reference_map2;
query_map1[111] = {8, 2, 6};
query_map1[333] = {4, 7, 3};
query_map2[222] = {3, 6, 8};
query_map2[999] = {2, 3, 5};
How i store my look up table LUT
This is so that for whatever keys i get from 'query_mapj', i only get the necessary Reference_mapis
LUT[111] = {'Reference_map1', 'Reference_map2'}
LUT[333] = {'Reference_map1'}
LUT[888] = {'Reference_map1'}
LUT[999] = {'Reference_map2'}
For example 111 from query_map1 gives both 'Reference_map1', 'Reference_map2' as they have the key 111.
On the other hand, 999 from query_map2 only gives 'Reference_map2' as only it have the key 999.
So it will go like this:
unordered_map<string, int> MakeCounter(
unordered_map <int, set<int>> &query_map,
unordered_map <string, unordered_map <int, set<int>>> &ReferenceMaps
){
unordered_map<string, int> RefName_Counter;
set<string> ReferenceNameSet;
// Update the RefName_Counter
for (const auto &key2nameset:query_map) {
// Check if this hash is in the LUT
if (LUT.count(key2nameset.first) <= 0){ continue; }
// Update Counter
ReferenceNameSet = LUT[key2nameset.first];
for (const auto &it : ReferenceNameSet){
if (RefName_Counter.count(it) > 0)
RefName_Counter[it]++;
else
RefName_Counter[it] = 1;
}
}
return RefName_Counter;
}
// The results should be like this
Counter1 = MakeCounter(query_map1, ReferenceMaps);
/*
Counter1['Reference_map1'] = 2; // because they share keys : 111 and 333
Counter1['Reference_map2'] = 1; // because they share keys : 111
*/
Counter2 = MakeCounter(query_map2, ReferenceMaps);
/*
Counter1['Reference_map2'] = 1; // because they share keys : 999
*/
Is there a better way to get Counteri for each respective query_mapi ?
Considering my comment above, here is what I got.
Adjusted to keep original data, but this feels like an extra credit :)
unordered_map<int,int> MakeCounter(const unordered_map <int, set<int>>& qs,
const vector<unordered_map <int, set<int>>>& refs)
{
unordered_map<int, int> counters;
for (size_t i = 0; i < refs.size(); ++i)
{
for (auto q : qs)
{
if (refs[i].find(q.first) != refs[i].end())
counters[i]++;
}
}
return counters;
}
int main()
{
vector<unordered_map <int, set<int>>> References = {
{ {111, { 0, 1, 2 } },
{333, { 1, 2, 3 } },
{888, { 2, 8, 0 } },
},
{ {111, { 1, 5, 9 } },
{999, { 0, 7, 4 } },
}
};
vector<unordered_map <int, set<int>>> Queries = {
{ {111, { 8, 2, 6 } },
{333, { 4, 7, 3 } },
},
{ {222, { 3, 6, 8 } },
{999, { 2, 3, 5 } },
}
};
unordered_map<int, int> m0 = MakeCounter(Queries[0], References);
unordered_map<int, int> m1 = MakeCounter(Queries[1], References);
}
This is rather a comment; I am using "answer" to properly format code fragments.
This searches your LUT twice:
// Check if this hash is in the LUT
if (LUT.count(key2nameset.first) <= 0){ continue; }
// Update Counter
ReferenceNameSet = LUT[key2nameset.first];
It also makes a copy of the set for no reason.
I believe that indexing a non-existing element in the map inserts that element, so
if (RefName_Counter.count(it) > 0)
RefName_Counter[it]++;
else
RefName_Counter[it] = 1;
is effectively:
RefName_Counter[it]++;
So your outer for loop becomes:
for (const auto &key2nameset:query_map) {
// Check if this hash is in the LUT
auto iter = LUT.find(key2nameset.first);
if(iter == LUT.end()) { continue; }
// Update Counter
for (const auto &it : *iter){
RefName_Counter[it]++;
}
I'm trying to merge a collection of dictionaries into the root process. Here's a short example:
#define MAX_CF_LENGTH 55
map<string, int> dict;
if (rank == 0)
{
dict = {
{"Accelerator Defective", 33},
{"Aggressive Driving/Road Rage", 27},
{"Alcohol Involvement", 19},
{"Animals Action", 30}};
}
if (rank == 1)
{
dict = {
{"Driver Inexperience", 6},
{"Driverless/Runaway Vehicle", 46},
{"Drugs (Illegal)", 38},
{"Failure to Keep Right", 24}};
}
if (rank == 2)
{
dict = {
{"Lost Consciousness", 1},
{"Obstruction/Debris", 8},
{"Other Electronic Device", 25},
{"Other Lighting Defects", 43},
{"Other Vehicular", 7}};
}
Scatterer scatterer(rank, MPI_COMM_WORLD, num_workers);
scatterer.gatherDictionary(dict, MAX_CF_LENGTH);
The idea inside gatherDictionary() is to put every key in a char array at each process (duplicates are allowed). After that, gathering all keys into the root and creating the final (merged) dictionary before broadcasting it. Here's the code:
void Scatterer::gatherDictionary(map<string,int> &dict, int maxKeyLength)
{
// Calculate destination dictionary size
int numKeys = dict.size();
int totalLength = numKeys * maxKeyLength;
int finalNumKeys = 0;
MPI_Reduce(&numKeys, &finalNumKeys, 1, MPI_INT, MPI_SUM, 0, comm);
// Computing number of elements that are received from each process
int *recvcounts = NULL;
if (rank == 0)
recvcounts = new int[num_workers];
MPI_Gather(&totalLength, 1, MPI_INT, recvcounts, 1, MPI_INT, 0, comm);
// Computing displacement relative to recvbuf at which to place the incoming data from each process
int *displs = NULL;
if (rank == 0)
{
displs = new int[num_workers];
displs[0] = 0;
for (int i = 1; i < num_workers; i++)
displs[i] = displs[i - 1] + recvcounts[i - 1] + 1;
}
char(*dictKeys)[maxKeyLength];
char(*finalDictKeys)[maxKeyLength];
dictKeys = (char(*)[maxKeyLength])malloc(numKeys * sizeof(*dictKeys));
if (rank == 0)
finalDictKeys = (char(*)[maxKeyLength])malloc(finalNumKeys * sizeof(*finalDictKeys));
// Collect keys for each process
int i = 0;
for (auto pair : dict)
{
strncpy(dictKeys[i], pair.first.c_str(), maxKeyLength);
i++;
}
MPI_Gatherv(dictKeys, totalLength, MPI_CHAR, finalDictKeys, recvcounts, displs, MPI_CHAR, 0, comm);
// Create new dictionary and distribute it to all processes
dict.clear();
if (rank == 0)
{
for (int i = 0; i < finalNumKeys; i++)
dict[finalDictKeys[i]] = dict.size();
}
delete[] dictKeys;
if (rank == 0)
{
delete[] finalDictKeys;
delete[] recvcounts;
delete[] displs;
}
broadcastDictionary(dict, maxKeyLength);
}
I'm sure of broadcastDicitonary() correctness as I've already tested it. Debugging into the gathering function I'm getting the following partial results:
Recvcounts:
220
220
275
Displacements:
0
221
442
FinalDictKeys:
Rank:0 Accelerator Defective
Rank:0 Aggressive Driving/Road Rage
Rank:0 Alcohol Involvement
Rank:0 Animals Action
Rank:0
Rank:0
Rank:0
Rank:0
Rank:0
Rank:0
Rank:0
Rank:0
Rank:0
Since only root data is being collected I'm wondering if this has something to do with the characters allocation even if it should be contiguous. I don't think this is related to a missing null character at the end since there's already a lot of padding for each string/key.
Thanks in advance for pointing out any missings or improvements and please comment if you need any extra infos.
If you wish to test it yourself I've put in a one-file only the code all together, it is compile&run ready (of course this works with 3 mpi processes). Code Here
displs[i] = displs[i - 1] + recvcounts[i - 1] + 1;
That + 1 at the end is superfluous. Change it to:
displs[i] = displs[i - 1] + recvcounts[i - 1];
I have successfully created a scalar valued attribute whose value is a variable length array of const char*. I do not understand how to read this attribute however!
This is code I used to create the attribute:
void create_attribute_with_vector_of_strings_as_value()
{
using namespace H5;
// Create some test strings.
std::vector<std::string> strings;
for (int iii = 0; iii < 10; iii++)
{
strings.push_back("this is " + boost::lexical_cast<std::string>(iii));
}
// Part 1: grab pointers to the chars
std::vector<const char*> chars;
for (auto si = strings.begin(); si != strings.end(); ++si)
{
std::string &s = (*si);
chars.push_back(s.c_str());
}
BOOST_TEST_MESSAGE("Size of char* array is: " << chars.size());
// Part 2: create the variable length type
hvl_t hdf_buffer;
hdf_buffer.p = chars.data();
hdf_buffer.len = chars.size();
// Part 3: create the type
auto s_type = H5::StrType(H5::PredType::C_S1, H5T_VARIABLE);
auto svec_type = H5::VarLenType(&s_type);
try
{
// Open an existing file and dataset.
H5File file(m_file_name.c_str(), H5F_ACC_RDWR);
// Part 4: write the output to a scalar attribute
DataSet dataset = file.openDataSet(m_dataset_name.c_str());
std::string filter_names = "multi_filters";
Attribute attribute = dataset.createAttribute( filter_names.c_str(), svec_type, H5S_SCALAR);
attribute.write(svec_type, &hdf_buffer);
file.close();
}
Here is the dataset with attribute as seen from h5dump:
HDF5 "d:\tmp\hdf5_tutorial\h5tutr_dset.h5" {
GROUP "/" {
DATASET "dset" {
DATATYPE H5T_STD_I32BE
DATASPACE SIMPLE { ( 4, 6 ) / ( 4, 6 ) }
DATA {
(0,0): 1, 7, 13, 19, 25, 31,
(1,0): 2, 8, 14, 20, 26, 32,
(2,0): 3, 9, 15, 21, 27, 33,
(3,0): 4, 10, 16, 22, 28, 34
}
ATTRIBUTE "multi_filters" {
DATATYPE H5T_VLEN { H5T_STRING {
STRSIZE H5T_VARIABLE;
STRPAD H5T_STR_NULLTERM;
CSET H5T_CSET_ASCII;
CTYPE H5T_C_S1;
}}
DATASPACE SCALAR
DATA {
(0): ("this is 0", "this is 1", "this is 2", "this is 3", "this is 4", "this is 5", "this is 6", "this is 7", "this is 8", "this is 9")
}
}
}
}
}
I do not understand how to read this data. The code I've experimented with so far is below. It compiles, but I've hardwired the array-size to the known length and the variable-length cstrings are empty? Does anyone have any suggestions as to where I'm going wrong? In particular, how do I query for the length of the array of const char* and how do I read the actual const char* cstrings contained in the array?
void read_attribute_with_vector_of_strings_as_value()
{
using namespace H5;
std::vector<std::string> strings;
try
{
// Open an existing file and dataset readonly
H5File file(m_file_name.c_str(), H5F_ACC_RDONLY);
// Part 4: Open the dataset
DataSet dataset = file.openDataSet(m_dataset_name.c_str());
// Atribute_name
std::string filter_names = "multi_filters";
Attribute attribute = dataset.openAttribute(filter_names.c_str());
size_t sz = attribute.getInMemDataSize();
size_t sz_1 = attribute.getStorageSize();
auto t1 = attribute.getDataType();
VarLenType t2 = attribute.getVarLenType();
H5T_class_t type_class = attribute.getTypeClass();
if (type_class == H5T_STRING)
BOOST_TEST_MESSAGE("H5T_STRING");
int length = 10;
std::vector<char*> tmp_vec(length);
auto s_type = H5::StrType(H5::PredType::C_S1, H5T_VARIABLE);
auto svec_type = H5::VarLenType(&s_type);
hvl_t hdf_buffer;
hdf_buffer.p = tmp_vec.data();
hdf_buffer.len = length;
attribute.read(svec_type, &hdf_buffer);
//attribute.read(s_type, &hdf_buffer);
//attribute.read(tmp_vec.data(), s_type);
for(size_t x = 0; x < tmp_vec.size(); ++x)
{
fprintf(stdout, "GOT STRING [%s]\n", tmp_vec[x] );
strings[x] = tmp_vec[x];
}
file.close();
}
If you are not required to use specific technologies to implement what you have in mind, you may consider HDFql (http://www.hdfql.com) which is a high-level language to manage HDF files easily (think SQL). That way you can be alleviated from all the low-level details of manipulating HDF files that you describe. Using HDFql in C++, reading an array of variable-length char is done like this:
// include HDFql C++ header file (make sure it can be found by the C++ compiler)
#include <iostream>
#include "HDFql.hpp"
int main(int argc, char *argv[])
{
// create an HDF file named "example.h5" and use (i.e. open) it
HDFql::execute("CREATE AND USE FILE example.h5");
// create an attribute named "multi_filters" of type varchar of one dimension (size 5)
HDFql::execute("CREATE ATTRIBUTE multi_filters AS VARCHAR(5)");
// insert (i.e. write) values "Red", "Green", "Blue", "Orange" and "Yellow" into attribute "multi_filters"
HDFql::execute("INSERT INTO multi_filters VALUES(Red, Green, Blue, Orange, Yellow)");
// select (i.e. read) attribute "multi_filters" into HDFql default cursor
HDFql::execute("SELECT FROM multi_filters");
// display content of HDFql default cursor
while(HDFql::cursorNext() == HDFql::Success)
{
std::cout << "Color " << HDFql::cursorGetChar() << " has a size of " << HDFql::cursorGetSize() << std::endl;
}
return 0;
}
I'm creating a C/C++ function which will be called from Lua. My function must call a library function who's signature is like this:
void libFunction( int val1, int val2, tSETTINGS * pSettings );
I'm given these C/C++ structs:
typedef struct
{
int cmd;
int arg;
} tCOMMAND;
typedef struct
{
int numberCommands;
int id;
tCOMMAND commands[1];
} tSETTINGS;
Maybe my thinking is all wrong on this, but from Lua I'm calling like this:
id = 42
val1 = 1
val2 = 2
cmd1 = { 3, 4 }
cmd2 = { 5, 6 }
commands = { cmd1, cmd2 }
settings = { #commands, id, commands }
mycfunction( val1, val2, settings )
I'm sure that I'm still not understanding the Lua stack as referenced from C++, since what I'm trying just doesn't work. My solution is able to retrieve val1, val2, #commands and id, but when I try to retrieve commands[0] and commands[1] I get {1, 2} and {2, 42} respectively.
My C++ is essentially like this (for this sample I'm discarding the values). I've already retrieved val1 and val2:
int stkNdx = 1;
lua_rawgeti(L, 3, stkNdx++ );
int numcmds = lua_tointeger(L, -1); // this successfully retrieves numberCommands 2
lua_pop(L, 1);
lua_rawgeti(L, 3, stkNdx++ );
int id = lua_tointeger(L, -1); // this successfully retrieves id 42
lua_pop(L, 1);
lua_pushvalue(L, -1 );
lua_pushnil(L);
int cmdNbr = 0;
for( lua_next(L, -2); cmdNbr < numcmds; cmdNbr++ )
{
lua_pushvalue(L, -2);
int cmd = lua_tointeger(L, -1);
int arg = lua_tointeger(L, -1);
lua_pop(L, 2);
lua_next(L, -2);
}
lua_pop(L, 1);
I've tried various permutations of lua_rawgeti() followed by lua_tonumber() and lua_pop(), with basically the same result.
This seems similar to this question, and my solution is modeled after that with no success.
Experimenting more I inserted this:
lua_pushnil(L);
while( lua_next(L, -2) )
{
if( ! lua_istable(L, -1) )
{
int v = lua_tointeger(L, -1);
}
lua_pop(L, 1);
}
This loop executes 4 times. The first 2 times the values 2 and 42 are assigned to v. The next 2 iterations skip the assignment (lua_istable returned true). So it seems that although I've already retrieved numcmds and id, they're still there on the stack. I also clearly don't understand how to iterate over the subtables when they're encountered.
Lua table indices range from [1 .. N] instead of [0 .. N-1].
Your loop should be:
int cmdNbr = 1;
for( lua_next(L, -2); cmdNbr <= numcmds; cmdNbr++ )
{
...
}
or as I prefer it:
lua_rawgeti(L, 3, 2 );
int id = lua_tointeger(L, -1); // this successfully retrieves id 42
lua_pop(L, 1);
lua_rawgeti(L, 3, 3);
{
// commands table at stack top
size_t N = lua_objlen(L,-1); // size of the table
for (int i = 1; i <= N; ++i)
{
lua_rawgeti(L,-1, i); // cmd# at stack top
{
lua_rawgeti(L,-1,1); // first entry
int cmd = lua_tointeger(L,-1);
lua_pop(L,1);
lua_rawgeti(L,-1,2); // second entry
int arg = lua_tointeger(L,-1);
lua_pop(L,1);
}
lua_pop(L, 1); // pop cmd#
}
}
lua_pop(L, 1); // pop commands table
Note that, with the function lua_objlen(L,idx), it's not necessary to pass numcmds.