In a C++ program I am trying to set a custom comparison function for a Berkeley DB, using the Db::set_bt_function member function (the DB is opened as a BTREE type). My code works fine when I'm not changing the comparison function; I can put and get keys/values using Db::put and Db::get.
To try the set_bt_function method, I defined my own "lexicographic comparison" as follows:
int compkeys(Db *db, const Dbt *dbt1, const Dbt *dbt2, size_t *locp) {
size_t s = dbt1->get_size() > dbt2->get_size() ? dbt2->get_size() : dbt1->get_size();
int c = std::memcmp(dbt1->get_data(), dbt2->get_data(), s);
if(c != 0) return c;
if(dbt1->get_size() < dbt2->get_size()) return -1;
if(dbt1->get_size() > dbt2->get_size()) return 1;
return 0;
}
So this should lead to exactly the same behavior as my reference code, when the comparison function isn't changed, since by default Berkeley DB uses lexicographical order.
Yet, when using this comparison function, Db::get doesn't work anymore. It returns -30999 (DB_BUFFER_SMALL).
Here is what I am doing to get the value associated with a given key:
Db* _dbm = ... /* DB is open */
std::vector<char> mykey;
... /* mykey is set to some content */
Dbt db_key((void*)(mykey.data()), uint32_t(mykey.size()));
Dbt db_data;
db_key.set_flags(DB_DBT_USERMEM);
db_data.set_flags(DB_DBT_MALLOC);
int status = _dbm->get(NULL, &db_key, &db_data, 0);
... /* check status, do something with db_data */
free(db_data.get_data());
Any idea why this code works when I'm not setting the comparison function, and doesn't when I am?
Note: if I access key/values using a cursor (Dbc::get) I don't have this issue.
The DB_BUFFER_SMALL error in this case is complaining about your db_key Dbt. You need to call db_key.set_ulen(uint32_t(mykey.size())) to tell BDB how much space you've allocated to hold the keys that come out of the database.
Things get a little weirder when you're using a custom comparison function. You can have data in the key that's not part of the compare - and not in the key that you passed in to get(). For this reason, BDB returns the key it found in the database in your db_key.
When setting the ulen, make it large enough to hold any key that can come back from the database. You may find that it's saner to just keep a char array on the stack to deal with this key in/out behavior.
Related
I am trying to use tbb’s concurrent_hash_map to increase my application’s concurrent performance. Read up about it and implemented it according to my application but I am seeing crashes..
So, my application is a multi-threadedd application where I am storing pairs,the key is char* and the value is an integer. Pseudocode looks like this:
In .h file,
typedef tbb::concurrent_hash_map<const void*, unsigned, Hasher> tbb_concurrent_hash;
tbb_concurrent_hash concurrent_hash_table;
tbb_concurrent_hash::accessor write_lock;
tbb_concurrent_hash::const_accessor read_lock;
In .c file,
void storeName(const char * name) {
int id=0;
// This creates a pair object of name and index
std::pair object(name, 0);
// read_lock is a const accessor for reading. This find function searches for char* in the table and if not found, create a write_lock.
bool found = concurrent_hash_table.find(read_lock, name);
if (found == FALSE) {
concurrent_hash_table.insert(write_lock, name);
// Get integer id somehow.
id = somefunction();
write_lock->second = id;
write_lock.release();
} else {
// if the name is found in the table then get the value and release it later
id = read_lock->second;
read_lock.release();
}
}
As per my understanding, I am good with the implementation but as I said, there are multiple crashes coming when find returns me FALSE. Crash have traces of mutexs as well.
Your 'locks', i.e. accessors are declared global in .h file. so, basically you write to a shared scoped_lock instance... which logically leads to a data race. Accessors are like fused std::shared_ptr and std::scoped_lock classes in one, or simpler - a pointer for your result and a lock guard for the data it points. You don't want to use one global pointer from multiple threads. Declare them locally in a scope you want to have that access (and you'd not need to call .release() as well)
Another problem is the data race between find() and insert (). Two or more threads can decide that they have to insert since they found nothing. In this case, the first thread will insert the new element while other threads will return existing element because insert() acts as find() if there's existing element. The problem is that your code doesn't account for that.
I can see why you might want to double check using const_accessor as the read lock is more scalable. But instead, you might want to use bool insert( const_accessor& result, const value_type& value ); with read lock (const_accessor) and value_type instead of a key only, which will initialize the whole pair in the case when a new element is added.
Good morning, I'm stuck using a map in the correct way.
Situation
A database table with unique ID and two other codes
ID (long) | Type (long) | Name (string)
to fill the map correctly I've defined it in this way:
map<long, MyObject>
where key is my ID and the object holds all the stuff. The map works correctly, I load all rows and I navigate easily inside of it.
Troubles
Troubles come when I need to sort the rows using a criteria which is not the key but:
Type
Name
Looking around the Internet I found that I should:
Define the operator< for MyObject or...
Define another type of comparator for my map.
I did the step 1., but with no success (it is never called). I'm trying to do the point 2. but with even less success.
I'll paste some code to help:
class CSitoWebRigaVariante
{
public:
bool m_bSriDelete;
bool m_bSriVisibile;
long m_lSriId;
long m_lSriIdTipol;
long m_lSriCodGes;
CString m_strSriCodMat;
public:
CSitoWebRigaVariante(void);
CSitoWebRigaVariante(const CSitoWebRigaVariante& cRiga);
~CSitoWebRigaVariante(void);
bool operator<(const CSitoWebRigaVariante& cRiga);
void operator=(const CSitoWebRigaVariante& cRiga);
void Azzera(void);
static void CaricaDaMDB(CDB* pDB, long lIdVM, map<long, CSitoWebRigaVariante>& cRighe);
};
typedef map<long, CSitoWebRigaVariante> CSWRighe;
///> Static method to fill a map.
void CSitoWebRigaVariante::CaricaDaMDB(CADODatabase* pDB, long lIdVM, map<long, CSitoWebRigaVariante>& cRighe)
{
BOOL bValRit;
CRecordset* pRS;
CSitoWebRigaVariante riga;
CString strInt;
pRS = new CADORecordset(pDB);
strInt.Format(_T("SELECT * FROM SITOWEB_RIVARMAT WHERE sri_idvarmat = %ld;"), lIdVM);
cRighe.clear();
if (pRS->Open(strInt, CADORecordset::openQuery) == TRUE && pRS->GetRecordCount() > 0)
{
while (pRS->IsEOF() == FALSE)
{
bValRit = pRS->GetFieldValue(_T("sri_id"), riga.m_lSriId);
bValRit &= pRS->GetFieldValue(_T("sri_idtipol"), riga.m_lSriIdTipol);
bValRit &= pRS->GetFieldValue(_T("sri_codges"), riga.m_lSriCodGes);
bValRit &= pRS->GetFieldValue(_T("sri_codmat"), riga.m_strSriCodMat);
bValRit &= pRS->GetFieldValue(_T("sri_delete"), riga.m_bSriDelete);
bValRit &= pRS->GetFieldValue(_T("sri_visibile"), riga.m_bSriVisibile);
cRighe.insert(pair<long, CSitoWebRigaVariante>(riga.m_lSriCodGes, riga));
pRS->MoveNext();
}
}
pRS->Close();
delete pRS;
}
I'm using Visual Studio 2010, MFC.
Any help is appreciated.
std::map is not a multi-index associative container. Its find method (and other things) uses the key as a search criteria. There's no possibility to specify another search criteria. It's why it's a "single-index lookup table".
You can use Boost.MultiIndex. It was designed for your case and supports multiple indexes (as the name suggests), both unique and not-unique.
Or you can use multiple map instances with different keys. If keys are not unique you need std::multimap.
I would recommend you to use map for model (for storing data). When you need to display information, you can just output it in the order you need it to be shown. Sorting must be done not at level of storing items, but at level of displaying them.
Although, in each situation you will need to do reordering only once.
Also, if any help is appreciated, I would strongly recommend you to do a
typedef long MyId;
and to use VS 2015.
The map class provides a Compare parameter of the constructor. You could not complete your target by setting Compare as map has only support key compare function.
The first idea of mine is construct a class support your tables schema.
class Example
{
public:
<your code>
void sortByID();
void sortByType();
void sortByName();
private:
long ID_;
long Type_;
string Name_;
};
But it sounds terrible. As once your table change, you should hard copy.So why you just get result using database order by?
I'm currently using luarpc in my program to make interprocess communication. The problem now is that due to my tolua++ binding which stores class instances as userdata im unable to use any of those functions cause luarpc cant handle userdata. My question now is if would be possible (and how) to transmit userdata if you know that its always only a pointer (4 Bytes) and has a metatable attached for call and indexing operations.
You can't.
It doesn't matter if the userdata is a pointer or an object. The reason you can't arbitrarily RPC through them is because the data is not stored in Lua. And therefore LuaRPC cannot transmit it properly.
A pointer into your address space is absolutely worthless for some other process; even moreso if it's running on another machine. You have to actually transmit the data itself to make the RPC work. LuaRPC can do this transmission, but only for data that it can understand. And the only data it understands is data stored in Lua.
Ok i got it working now. What i did is for userdata args/returns i send the actual ptr + metatable name(typename) to the client. the client then attaches a metatable with an __index method that creates a new helper with the typename and appends a helper with the field you want to access. when you then call or read a field from that userdata the client sends the data for calling a field of the typetable and the userdata to the server.
ReadVariable:
lua_pushlightuserdata(L,msg.read<void*>());
#ifndef RPC_SERVER
luaL_getmetatable(L,"rpc.userdata");
int len = msg.read<int>();
char* s = new char[len];
msg.read((uint8*)s,len);
s[len] = '\0';
lua_pushlstring(L,s,len);
lua_setfield(L,-2,"__name");
lua_pushlightuserdata(L,TlsGetValue(transporttls));
lua_setfield(L,-2,"__transport");
lua_setmetatable(L,-2);
#endif
Write Variable:
else
{
msg.append<RPCType>(RPC_USERDATA);
msg.append<void*>(lua_touserdata(L,idx));
#ifdef RPC_SERVER
lua_getmetatable(L,idx);
lua_rawget(L,LUA_REGISTRYINDEX);
const char* s = lua_tostring(L,-1);
int len = lua_strlen(L,-1);
msg.append<int>(len);
msg.append(s,len);
#endif
lua_settop(L,stack_at_start);
}
userdata indexing:
checkNumArgs(L,2);
ASSERT(lua_isuserdata(L,1) && isMetatableType(L,1,"rpc.userdata"));
if(lua_type(L,2) != LUA_TSTRING)
return luaL_error( L, "can't index a handle with a non-string" );
const char* s = lua_tostring(L,2);
if(strlen(s) > MAX_PATH - 1)
return luaL_error(L,"string to long");
int stack = lua_gettop(L);
lua_getmetatable(L,1);
lua_getfield(L,-1,"__name");
const char* name = lua_tostring(L,-1);
if(strlen(name) > MAX_PATH - 1)
return luaL_error(L,"string to long");
lua_pop(L,1); // remove name
lua_getfield(L,-1,"__transport");
Transport* t = reinterpret_cast<Transport*>(lua_touserdata(L,-1));
lua_pop(L,1);
Helper* h = Helper::create(L,t,name);
Helper::append(L,h,s);
return 1;
well i more or less rewrote the complete rpc library to work with named pipes and windows but i think the code should give anyone enough information to implement it.
this allows code like:
local remote = rpc.remoteobj:getinstance()
remote:dosmthn()
on the clientside. it currently doesnt allow to add new fields but well this is all i need for now :D
I'm using luabind 0.9.1 from Ryan Pavlik's master distribution with Lua 5.1, cygwin on Win XP SP3 + latest patches x86, boost 1.48, gcc 4.3.4. Lua and boost are cygwin pre-compiled versions.
I've successfully built luabind in both static and shared versions.
Both versions pass all the tests EXCEPT for the test_object_identity.cpp test which fails in both versions.
I've tracked down the problem to the following issue:
If an entry in a table is created for NON built-in class (i.e., not int, string, etc), the value CANNOT be retrieved.
Here's a code piece that demonstrates this:
#include "test.hpp"
#include <luabind/luabind.hpp>
#include <luabind/detail/debug.hpp>
using namespace luabind;
struct test_param
{
int obj;
};
void test_main(lua_State* L)
{
using namespace luabind;
module(L)
[
class_<test_param>("test_param")
.def_readwrite("obj", &test_param::obj)
];
test_param temp_object;
object tabc = newtable(L);
tabc[1] = 10;
tabc[temp_object] = 30;
TEST_CHECK( tabc[1] == 10 ); // passes
TEST_CHECK( tabc[temp_object] == 30 ); // FAILS!!!
}
tabc[1] is indeed 10 while tabc[temp_object] is NOT 30! (actually, it seems to be nil)
However, if I use iterate to go over tabc entries, there're the two entries with the CORRECT key/value pairs.
Any ideas?
BTW, overloading the == operator like this:
#include <luabind/operator.hpp>
struct test_param
{
int obj;
bool operator==(test_param const& rhs) const
{
return obj == rhs.obj;
}
};
and
module(L)
[
class_<test_param>("test_param")
.def_readwrite("obj", &test_param::obj)
.def(const_self == const_self)
];
Doesn't change the result.
I also tried switching to settable() and gettable() from the [] operator. The result is the same. I can see with the debugger that default conversion of the key is invoked, so I guess the error arises from somewhere therein, but it's beyond me to figure out what exactly the problem is.
As the following simple test case show, there're definitely a bug in Luabind's conversion for complex types:
struct test_param : wrap_base
{
int obj;
bool operator==(test_param const& rhs) const
{ return obj == rhs.obj ; }
};
void test_main(lua_State* L)
{
using namespace luabind;
module(L)
[
class_<test_param>("test_param")
.def(constructor<>())
.def_readwrite("obj", &test_param::obj)
.def(const_self == const_self)
];
object tabc, zzk, zzv;
test_param tp, tp1;
tp.obj = 123456;
// create new table
tabc = newtable(L);
// set tabc[tp] = 5;
// o k v
settable( tabc, tp, 5);
// get access to entry through iterator() API
iterator zzi(tabc);
// get the key object
zzk = zzi.key();
// read back the value through gettable() API
// o k
zzv = gettable(tabc, zzk);
// check the entry has the same value
// irrespective of access method
TEST_CHECK ( *zzi == 5 &&
object_cast<int>(zzv) == 5 );
// convert key to its REAL type (test_param)
tp1 = object_cast<test_param>(zzk);
// check two keys are the same
TEST_CHECK( tp == tp1 );
// read the value back from table using REAL key type
zzv = gettable(tabc, tp1);
// check the value
TEST_CHECK( object_cast<int>(zzv) == 5 );
// the previous call FAILS with
// Terminated with exception: "unable to make cast"
// this is because gettable() doesn't return
// a TRUE value, but nil instead
}
Hopefully, someone smarter than me can figure this out,
Thx
I've traced the problem to the fact that Luabind creates a NEW DISTINCT object EVERY time you use a complex value as key (but it does NOT if you use a primitive one or an object).
Here's a small test case that demonstrates this:
struct test_param : wrap_base
{
int obj;
bool operator==(test_param const& rhs) const
{ return obj == rhs.obj ; }
};
void test_main(lua_State* L)
{
using namespace luabind;
module(L)
[
class_<test_param>("test_param")
.def(constructor<>())
.def_readwrite("obj", &test_param::obj)
.def(const_self == const_self)
];
object tabc, zzk, zzv;
test_param tp;
tp.obj = 123456;
tabc = newtable(L);
// o k v
settable( tabc, tp, 5);
iterator zzi(tabc), end;
std::cerr << "value = " << *zzi << "\n";
zzk = zzi.key();
// o k v
settable( tabc, tp, 6);
settable( tabc, zzk, 7);
for (zzi = iterator(tabc); zzi != end; ++zzi)
{
std::cerr << "value = " << *zzi << "\n";
}
}
Notice how tabc[tp] first has the value 5 and then is overwritten with 7 when accessed through the key object. However, when accessed AGAIN through tp, a new entry gets created. This is why gettable() fails subsequently.
Thx,
David
Disclaimer: I'm not an expert on luabind. It's entirely possible I've missed something about luabind's capabilities.
First of all, what is luabind doing when converting test_param to a Lua key? The default policy is copy. To quote the luabind documentation:
This will make a copy of the parameter. This is the default behavior when passing parameters by-value. Note that this can only be used when passing from C++ to Lua. This policy requires that the parameter type has an accessible copy constructor.
In pratice, what this means is that luabind will create a new object (called "full userdata") which is owned by the Lua garbage collector and will copy your struct into it. This is a very safe thing to do because it no longer matters what you do with the c++ object; the Lua object will stick around without really any overhead. This is a good way to do bindings for by-value sorts of objects.
Why does luabind create a new object each time you pass it to Lua? Well, what else could it do? It doesn't matter if the address of the passed object is the same, because the original c++ object could have changed or been destroyed since it was first passed to Lua. (Remember, it was copied to Lua by value, not by reference.) So, with only ==, luabind would have to maintain a list of every object of that type which had ever been passed to Lua (possibly weakly) and compare your object against each one to see if it matches. luabind doesn't do this (nor do I think should it).
Now, let's look at the Lua side. Even though luabind creates two different objects, they're still equal, right? Well, the first problem is that, besides certain built-in types, Lua can only hold objects by reference. Each of those "full userdata" that I mentioned before is actually a pointer. That means that they are not identical.
But they are equal, if we define an __eq meta operation. Unfortunately, Lua itself simply does not support this case. Userdata when used as table keys are always compared by identity, no matter what. This actually isn't special for userdata; it is also true for tables. (Note that to properly support this case, Lua would need to override the hashcode operation on the object in addition to __eq. Lua also does not support overriding the hashcode operation.) I can't speak for the authors of Lua why they did not allow this (and it has been suggested before), but there it is.
So, what are the options?
The simplest thing would be to convert test_param to an object once (explicitly), and then use that object to index the table both times. However, I suspect that while this fixes your toy example, it isn't very helpful in practice.
Another option is simply not to use such types as keys. Actually, I think this is a very good suggestion, since this kind of light-weight binding is quite useful, and the only other option is to discard it.
It looks like you can define a custom conversion on your type. In your example, it might be reasonable to convert your type to a Lua number which will behave well as a table index.
Use a different kind of binding. There will be some overhead, but if you want identity, you'll have to live with it. It sounds like luabind has some support for wrappers, which you may need to use to preserve identity:
When a pointer or reference to a registered class with a wrapper is passed to Lua, luabind will query for it's dynamic type. If the dynamic type inherits from wrap_base, object identity is preserved.
I am working with the OpenSSL library's X509 certificate class, and I need to query the "key usage" extension.
After abandoning OpenSSL's vapourware "documentation", some shot-in-the-dark web searching eventually revealed that I needed to call
X509_get_ext_d2i(X509 *x, int nid, int *crit, int *idx)
and searching through the objects.h header revealed the proper NID.
Problem is, this call returns a pointer to void, which apparently can point to a variety of structs depending upon what extension one has asked for.
Since none of these appear to be documented, one is left without a way to figure out how to parse what the function returns.
Can anyone point me to a document that actually talks about this, instead of just listing things I can find out for myself (the function profile, which file it comes from, etc)?
The easiest solutions to read the key usage seems to be
X509* x509_cert = ...
// without X509_check_ca x509_cert->ex_kusage always returns 0 (no idea why)
int ca = X509_check_ca(x509_cert);
unsigned long usage = x509_cert->ex_kusage;
The resulting values are defined in opensc/pkcs15-init.h
SC_PKCS15INIT_X509_DIGITAL_SIGNATURE 0x0080UL
SC_PKCS15INIT_X509_NON_REPUDIATION 0x0040UL
SC_PKCS15INIT_X509_KEY_ENCIPHERMENT 0x0020UL
SC_PKCS15INIT_X509_DATA_ENCIPHERMENT 0x0010UL
SC_PKCS15INIT_X509_KEY_AGREEMENT 0x0008UL
SC_PKCS15INIT_X509_KEY_CERT_SIGN 0x0004UL
SC_PKCS15INIT_X509_CRL_SIGN 0x0002UL
I came to that soultion by finding the following code of an openssl source file
/* Handle key usage */
if((usage=X509_get_ext_d2i(x, NID_key_usage, NULL, NULL))) {
if(usage->length > 0) {
x->ex_kusage = usage->data[0];
if(usage->length > 1)
x->ex_kusage |= usage->data[1] << 8;
} else x->ex_kusage = 0;
x->ex_flags |= EXFLAG_KUSAGE;
ASN1_BIT_STRING_free(usage);
}
The second parameter (nid) determines the returned type.
Looking at this code:
http://opengrok.creo.hu/dragonfly/xref/src/crypto/openssl-0.9/crypto/x509v3/v3_purp.c#X509_check_purpose
It appears that for the key usage NID, it returns a ASN1_BIT_STRING. (line 361).