My application sends varints many times. Every time I have to allocate memory for 2 objetcs: CodedOutputStream and FileOutputStream and later relese it. IMO it is unnecessary time loss. How can I send the varint without all this process? (I don't want to do it manually but with protobuf)
I have found it:
delete coded_output;
/* delete raw_output;*/
((FileOutputStream*)raw_output)->Flush();
but still there is one object to allocate every time
void Connection::send(const Message& msg) throw(EmptyMessage) {
//CodedOutputStream* coded_output = new CodedOutputStream(raw_output);
CodedOutputStream coded_output(raw_output);
int n = msg.ByteSize();
if(n<=0) throw EmptyMessage();
//coded_output->WriteVarint32(n);
coded_output.WriteVarint32(n);
//delete coded_output;
coded_output.~CodedOutputStream();
raw_output->Flush();
msg.SerializeToArray(buffer, n);
SocketMaintenance::write(buffer, n);
}
Annoucement Connection::receive() throw(EmptySocket) {
//CodedInputStream* coded_input = new CodedInputStream(raw_input);
CodedInputStream coded_input(raw_input);
google::protobuf::uint32 n;
//coded_input->ReadVarint32(&n);
coded_input.ReadVarint32(&n);
char *b;
int m;
//coded_input->GetDirectBufferPointer((const void**)&b, &m);
coded_input.GetDirectBufferPointer((const void**)&b, &m);
Annoucement ann;
ann.ParseFromArray(b, n);
return ann;
}
When I use the code above, I get this error (runtime error) from my client application (this apps uses only the send function):
libprotobuf FATAL
google/protobuf/io/zero_copy_stream_impl_lite.cc:346]
CHECK failed: (buffer_used_) ==
(buffer_size_): BackUp() can only be
called after Next(). terminate called
after throwing an instance of
'google::protobuf::FatalException'
what(): CHECK failed: (buffer_used_)
== (buffer_size_): BackUp() can only be called after Next(). Stopped
When I use the commented out part of code instead the corresponding one all works fine.
You don't have to allocate the CodedOutputStream on the heap, you can just declare it as a local variable (or class member) where you need it. It doesn't look like the constructor is particularly expensive.
Are you always writing to the same file? If so you can just use a single CodedOuputStream and FileOutputStream for all the writes.
Related
I am trying to create a python object in c++ and put it in a file to be used by another program. the problem I am facing that PyObject is a pointer and I don't know how to save a pointer to a file and reload it later. I have tried multiple solutions but nothing works for me, boost for example.
func = PyDict_GetItemString(dict, "create_and_train");
pValue = PyObject_CallFunctionObjArgs(func, PyUnicode_FromString(file_path.c_str()), PyLong_FromLong(size), NULL);
//return feature_values, training_set
if (pValue != NULL)
{
PyObject *object = PyTuple_GetItem(pValue, 0);
}
I want to save object pointer to a file and load it later.
Could any one help me? thanks in advance.
Pointers are basically addresses of memory. Since they are dynamically allocated (meaning where ever there is space in the memory when you plan on executing it). This means that unless you are guaranteed that the location will not change whatsoever, you cant use a saved pointer.
One option is to copy the object's memory footprint to the file in binary. This means that you can load the whole object from the file at once at runtime. But be careful because if the object has pointers, it might not work properly because of the reason I stated before.
Saving the object's memory footprint is pretty simple,
std::ofstream file("sim.bin", std::ios::binary);
if (!file.is_open()) return; // Bail if failed.
// Dummy object as an example.
struct DummyObject { int x = 100; };
DummyObject obj;
file.write(reinterpret_cast<char*>(&obj), sizeof(DummyObject));
file.close();
And when loading the content,
std::ifstream iFile("sim.bin", std::ios::binary);
if (!iFile.is_open()) return; // Bail if failed.
DummyObject iobj;
iFile.read(reinterpret_cast<char*>(&iobj), sizeof(DummyObject));
iFile.close();
Now if that DummyObject had a pointer to some other thing, then we need to copy the content of that data to the file too. It'll slightly complicate things.
I have build a set C++ containing classes on top of the BluetoothAPIs apis.
I can enumerate open handles to services, characteristics and descriptors. I can read characteristic values. The issue that I have is that I cannot write to a characteristic value.
Below is the code use to write the characteristic value
void BleGattCharacteristic::setValue(UCHAR * data, ULONG size){
if (pGattCharacteristic->IsSignedWritable || pGattCharacteristic->IsWritable || pGattCharacteristic->IsWritableWithoutResponse)
{
size_t required_size = sizeof(BTH_LE_GATT_CHARACTERISTIC_VALUE) + size;
PBTH_LE_GATT_CHARACTERISTIC_VALUE gatt_value = (PBTH_LE_GATT_CHARACTERISTIC_VALUE)malloc(required_size);
ZeroMemory(gatt_value, required_size);
gatt_value->DataSize = (ULONG)size;
memcpy(gatt_value->Data, data, size);
HRESULT hr = BluetoothGATTSetCharacteristicValue(bleDeviceContext.getBleServiceHandle(), pGattCharacteristic, gatt_value, NULL, BLUETOOTH_GATT_FLAG_NONE);
free(gatt_value);
if (HRESULT_FROM_WIN32(S_OK) != hr)
{
stringstream msg;
msg << "Unable to write the characeristic value. Reason: ["
<< Util.getLastError(hr) << "]";
throw BleException(msg.str());
}
}
else
{
throw BleException("characteristic is not writable");
}}
The call to bleDeviceContext.getBleServiceHandle() returns the open handle to the device info service.
pGattCharacteristics is the pointer to the characteristic to write too. It was opened with a call to BluetoothGATTGetCharacteristics.
I have tried different combinations of the flags with no difference in the return code.
I have also tried using the handle to the device not to the service. In that case I get an ERROR_INVALID_FUNCTION return error code.
I would appreciate any pointers as to what I am doing wrong or what other possible options I could try.
1- You have to use the Service Handle, right.
2- I don't know how you designed your class, and then how you allocate some memory for the Characteristic's Value itself.
What I do (to be sure to have enough and proper memory for Value's data):
a) at init of the Value object, call ::BluetoothGATTGetCharacteristicValue twice, to get the needed size and then actually allocate some internal memory for it.
b) when using it, set the inner memory to what it may , then call ::BluetoothGATTSetCharacteristicValue
hr=::BluetoothGATTSetCharacteristicValue(
handle,
(PBTH_LE_GATT_CHARACTERISTIC)Characteristic,
value,//actually a (PBTH_LE_GATT_CHARACTERISTIC_VALUE) to allocated memory
0,//BTH_LE_GATT_RELIABLE_WRITE_CONTEXT ReliableWriteContext,
BLUETOOTH_GATT_FLAG_NONE)
So a few things:
typedef struct _BTH_LE_GATT_CHARACTERISTIC_VALUE {
ULONG DataSize;
UCHAR Data[];
} BTH_LE_GATT_CHARACTERISTIC_VALUE, *PBTH_LE_GATT_CHARACTERISTIC_VALUE;
is how the data structure used in the parameter CharacteristicValue is defined. Please note that Data is NOT an allocated array, but rather a pointer. So accessing Data[0] is undefined behavior and could be accessing anywhere in memory. Rather you need to do gatt_value.Data = &data; setting the pointer to the address of the input parameter.
Secondly the documentation is quite clear as to why you might get ERROR_INVALID_FUNCTION; if another reliable write is already pending then this write will fail. You should consider retry logic in that case.
As for E_INVALIDARG I'd assume it's related to the undefined behavior but I'd check after fixing the other issues previously mentioned.
I'm having problems creating a tmx map from string input.
bool LevelManager::initLevel(int currentLevel)
{
const char* map;
try {
map = LevelManager::getLevel(currentLevel);
} catch (int) {
throw 1;
}
if(map != NULL){
CCLog("%s", map);
tileMap = CCTMXTiledMap::create(map);
tileMap->setAnchorPoint(ccp(0,0));
tileMap->setPosition(ccp(15,20));
this->addChild(tileMap, 5);
backgoundLayer = tileMap->layerNamed("Background");
} else {
throw 1;
}
return true;
}
Thats my code.
It is very unstable. Most of the times it crashes and sometimes it doesn't.
I'm loading my map from the string map. Wich is a const *char.
My map is named Level1.tmx and when i load the map like this: tileMap = CCTMXTiledMap::create("Level1.tmx"); it always works and never crashes.
And i know for a fact that the value of map is Level1.tmx because i log it in the line before the load.
When it crashes the log outputs this: (lldb)
and on the line tileMap->setAnchorPoint(ccp(0,0)); it says "Thread 1: EXC_BAD_ACCESS (code=2, adress=0x0)
Does anyone know why this happens and how to fix it?
Many thanks.
Ps: i'm using xcode, the latest cocos2d-x release and the iPhone simulator
Edit:
Using breakpoints i checked where things go bad while loading the tilemap.
on the line tileMap = CCTMXTiledMap::create(map);
my variable map is still fine
but on line tileMap->setAnchorPoint(ccp(0,0));
it is suddenly corrupted (most of the time)
It sounds like you're returning a char* string created on the stack, which means the memory may or may not be corrupted, depending on circumstances, moon phases, and what not.
So the question is: How is getLevel defined and what does it do (post the code)?
If you do something like this:
const char* LevelManager::getLevel(int level)
{
char* levelName = "default.tmx";
return levelName;
}
…then that's going to be the culprit. The levelName variable is created on the stack, no memory (on the heap) is allocated for it. The levelName variable and the memory it points to become invalid as soon as the method returns.
Hence when the method leaves this area of memory where levelName points to can be allocated by other parts of the program or other method's stack memory. Whatever is in that area of memory may still be the string, or it may be (partially) overridden by other bits and bytes.
PS: Your exception handling code is …. well it shows a lack of understanding what exception handling does, how to use it and especially when. I hope these are just remnants of trying to get to the bottom of the issue, otherwise get rid of it. I recommend reading a tutorial and introductions on C++ exception handling if you want to continue to use exceptions. Especially something like (map != NULL) should be an assertion, not an exception.
I fixed it.
const char* was to blame.
When returning my map as a char * it worked flawless.
const char *levelFileName = level.attribute("file").value();
char *levelChar = new char[strlen(levelFileName) + 1];
std:: strcpy (levelChar, levelFileName);
return levelChar;
Thats how i now return the map.
I'm relatively new to the MySQL++ connector in C++, and have an really annoying issue with it already!
I've managed to get stored procedures working, however i'm having issues with the delete statements. I've looked high and low and have found no documentation with examples.
First I thought maybe the code needs to free the query/connection results after calling the stored procedure, but of course MySQL++ doesn't have a free_result method... or does it?
Anyways, here's what I've got:
#include <iostream>
#include <stdio.h>
#include <queue>
#include <deque>
#include <sys/stat.h>
#include <mysql++/mysql++.h>
#include <boost/thread/thread.hpp>
#include "RepositoryQueue.h"
using namespace boost;
using namespace mysqlpp;
class RepositoryChecker
{
private:
bool _isRunning;
Connection _con;
public:
RepositoryChecker()
{
try
{
this->_con = Connection(false);
this->_con.set_option(new MultiStatementsOption(true));
this->_con.set_option(new ReconnectOption(true));
this->_con.connect("**", "***", "***", "***");
this->ChangeRunningState(true);
}
catch(const Exception& e)
{
this->ChangeRunningState(false);
}
}
/**
* Thread method which runs and creates the repositories
*/
void CheckRepositoryQueues()
{
//while(this->IsRunning())
//{
std::queue<RepositoryQueue> queues = this->GetQueue();
if(queues.size() > 0)
{
while(!queues.empty())
{
RepositoryQueue &q = queues.front();
char cmd[256];
sprintf(cmd, "svnadmin create /home/svn/%s/%s/%s", q.GetPublicStatus().c_str(),
q.GetUsername().c_str(), q.GetRepositoryName().c_str());
if(this->DeleteQueuedRepository(q.GetQueueId()))
{
printf("query deleted?\n");
}
printf("Repository created!\n");
queues.pop();
}
}
boost::this_thread::sleep(boost::posix_time::milliseconds(500));
//}
}
protected:
/**
* Gets the latest queue of repositories from the database
* and returns them inside a cool queue defined with the
* RepositoryQueue class.
*/
std::queue<RepositoryQueue> GetQueue()
{
std::queue<RepositoryQueue> queues;
Query query = this->_con.query("CALL sp_GetRepositoryQueue();");
StoreQueryResult result = query.store();
RepositoryQueue rQ;
if(result.num_rows() > 0)
{
for(unsigned int i = 0;i < result.num_rows(); ++i)
{
rQ = RepositoryQueue((unsigned int)result[i][0],
(unsigned int)result[i][1],
(String)result[i][2],
(String)result[i][3],
(String)result[i][4],
(bool)result[i][5]);
queues.push(rQ);
}
}
return queues;
}
/**
* Allows the thread to be shut off.
*/
void ChangeRunningState(bool isRunning)
{
this->_isRunning = isRunning;
}
/**
* Returns the running value of the active thread.
*/
bool IsRunning()
{
return this->_isRunning;
}
/**
* Deletes the repository from the mysql queue table. This is
* only called once it has been created.
*/
bool DeleteQueuedRepository(unsigned int id)
{
char cmd[256];
sprintf(cmd, "DELETE FROM RepositoryQueue WHERE Id = %d LIMIT 1;", id);
Query query = this->_con.query(cmd);
return (query.exec());
}
};
I've removed all the other methods as they're not needed...
Basically it's the DeleteQueuedRepository method which isn't working, the GetQueue works fine.
PS: This is on a Linux OS (Ubuntu server)
Many thanks,
Shaun
MySQL++ doesn't have a free_result method... or does it?
It doesn't need one. When the result object goes out of scope at the end of GetQueue(), all memory associated with it is automatically freed.
this->_con = Connection(false);
Three problems here:
When you create the RepositoryChecker object, you already have created a Connection object. If you need to pass different parameters to its constructor, you'd do that in the initialization list of the RepositoryChecker constructor, not in its body. Read your C++ book.
What you've done here instead is a) create a default Connection object, then b) create a different Connection object with exceptions turned off, then c) overwrite the first with the second. If that works, it's highly inefficient. MySQL++ Connection objects have had problems with their copy ctors in the past, so if you're using an old version of the library, it could explain your problems.
You're telling the Connection object (and every object it creates, even indirectly, which means pretty much everything in MySQL++) you don't want it to throw exceptions, but then you wrap it in a big try block. Pick one.
I'd suggest using exceptions — the default in MySQL++ — given the way your code is currently structured. If there is a query error way down in DeleteQueuedRepository(), there's no way to see what happened because you'd just pass false up to the caller, which is ignored because there is no else clause on the call. If you do this, log the e.what() message in your catch block. You're just throwing that information away right now.
There are several places where you're using constructs that look more like Python (or perhaps JavaScript) than C++. This makes me wonder if your problem isn't damage caused by some other misuse of C++.
On this line in particular, you're using the this pointer explicitly, for which there is no need in C++. This code does exactly the same thing:
_con = Connection(false);
Though again, the line should be replaced entirely, using the RepositoryChecker ctor initializer list instead.
Moving on...
sprintf(cmd, "DELETE FROM RepositoryQueue WHERE Id = %d LIMIT 1;", id);
As others have commented, you'd be better off using the Query stream interface:
Query q = _con.query();
q << "DELETE FROM RepositoryQueue WHERE Id = " << id << " LIMIT 1";
This has several advantages:
Fixes the type safety problem implied by the one who suggested changing your %d to %u. C++ IOStreams take care of that for you.
Automatic quoting of data inserted into the stream, if needed. (Which it isn't, in this case.)
Prevents any possibility of running off the end of the buffer. You could use the nonportable snprintf() instead here, but why?
If you're really happy with printf(), there's the template query interface instead.
boost::this_thread::sleep(boost::posix_time::milliseconds(500));
Have you read the threads chapter in the user manual? You don't get thread safety for free in MySQL++. Your problem could be due to memory corruption.
Warren Young, MySQL++ Maintainer
Try changing "%d" to "%u" in sprintf.
I'm reading from a binary data file which is written by invoking the following lines in Matlab m-file:
disp(sprintf('template = %d', fwrite(fid, template_1d, 'uint8')));
AFAIK, uint8 is the same size as the types BYTE, unsigned char, and unsigned short. Hence I have written the following code in a file-reading method in a C++ class instantiated in the mexfunction called by Matlab:
template1D = (unsigned short*) malloc(Nimgs*sizeof(unsigned short));
printf("template1D = %d\n", fread(template1D, sizeof(unsigned short), Nimgs, dfile));
and the following is how I deallocated this member variable in the class destructor's helper function:
free((void*) template1D);
In the main mexfunction, when I did not instantiate the class object to persist in memory after mex-function completes by calling mexMakeMemoryPersistent() function, template1D gets cleared properly without segmentation error messages from Matlab. However, if I did instantiate the class to persist in memory as follows:
if (!dasani)
{
dasani = new NeedleUSsim;
mexMakeMemoryPersistent((void*) dasani);
mexAtExit(ExitFcn);
}
with ExitFcn being:
void ExitFcn()
{
delete dasani;
}
then when I'm at the line of free((void*) template1D);, Matlab gives me an error message about the segmentation fault. I have checked the memory sizes and they seem to be consistent. For the malloc/calloc/free functions, I'm using Matlab's mxMalloc/mxCalloc/mxFree functions when I'm executing the C++ project as a Matlab mex function.
Based on this description, what further suggestions would you have for me to solve this problem and ensure this doesn't happen in the future (or at least know how to deal with similar problems like this in the future)?
Thanks in advance.
----------------------------Additions------------------------------------------------------
The following block of code basically shows the jists of my mex file. A mex file is basically an executable that is run in Matlab and compiled from C/C++ code with some Matlab headers.
void ExitFcn()
{
delete dasani;
}
void mexFunction(int nlhs, mxArray *plhs[], int nrhs, const mxArray *prhs[])
{
needle_info pin;
// check number of i/o if they are correct
if (nrhs != NUMIN)
{
mexErrMsgTxt("Invalid number of input arguments");
}
else if (nlhs != NUMOUT)
{
mexErrMsgTxt("Invalid number of output arguments");
}
// check if the input is noncomplex
if (mxIsComplex(NEEDLE))
{
mexErrMsgTxt("Input must be a noncomplex scalar integer.");
}
// check if the dimensions of the needle information is valid
int needlerows, needlecols;
needlerows = mxGetM(NEEDLE);
needlecols = mxGetN(NEEDLE);
if (needlerows < 1 || needlecols < 6)
{
mexErrMsgTxt("Needle information's dimensions are invalid");
}
float *needlePoint, *yPoint ;
// retrieving current needle information
// order of the variables are always as follows:
// r, theta, l, rho, alpha, beta
needlePoint = (float*) mxGetData(NEEDLE) ;
pin.r = needlePoint[0];
pin.theta = needlePoint[1];
pin.l = needlePoint[2];
pin.rho = needlePoint[3];
pin.alpha = needlePoint[4];
pin.beta = needlePoint[5];
//// read the file inputs
**//if (!dasani)
//{
// dasani = new NeedleUSsim;
// mexMakeMemoryPersistent((void*) dasani);
// mexAtExit(ExitFcn);
//}
dasani = new NeedleUSsim;
delete dasani;**
// sending an useless output for now (get rid of this if not conceptually needed
plhs[0] = mxCreateNumericMatrix(1,1,mxSINGLE_CLASS,mxREAL) ;
yPoint = (float*) mxGetData(plhs[0]) ;
*yPoint = 1;
}
This code would run after build/compilation if the user invokes "mexfunction" anywhere from the command line or m-file script. The snippet enclosed by "**" (when I was trying to bold the snippet) is the problem that I'm looking at. From a second look at the snippet, I may be allocating the memory for dasani pointer in a different memory from the Matlab memory (as there is the memory with scope limited to the C++ mex function only, and another memory space with scope visible to the Matlab program). Otherwise, I'm not sure why Matlab is complaining about this problem.
The MEX API supports C as well as C++. Because C has no try/catch or destructors, there is no way for a C MEX-function to clean-up memory directly in the event of an error. Therefore, MATLAB tracks the results of the memory allocation routines (mxMalloc, mxCalloc, mxRealloc, mxFree, and all the mxCreate* functions that return mxArrays) on an internal list. If an error occurs during the MEX-function execution (either by calling mexErrMsgIdAndTxt directly, or using something like mexEvalString to call MATLAB code that errors), then MATLAB will automatically free any mx-based allocated memory. But, also, when a MEX-function terminates normally, MATLAB will
also free any mx-based memory allocated by the MEX-function. Before the days of destructors, this was a convenience for MEX authors, though in the modern C++ world it can get really annoying.
Sometimes, as in the case of this question, you don't want MATLAB to auto-free memory. In that case, you have to use mexMakeMemoryPersistent, or mexMakeArrayPersistent for mxArrays.
You should only ever pass a pointer to mexMakeMemoryPersistent if it was originally allocated with mxMalloc, mxCalloc, or mxRealloc. So this code
dasani = new NeedleUSsim;
mexMakeMemoryPersistent((void*) dasani);
is bad with a capital 'B', unless you have overloaded NeedleUSsim::operator new() to use mxMalloc, which I wouldn't recommend. But if the fields of dasani are allocated with mxMalloc et al., then you would want to pass those to mexMakeMemoryPersistent. I'd recommend doing something like that in the NeedleUSsim constructor if at all possible, to keep it near the mxMalloc call.
It feels like it's mexMakeMemoryPersistent() that is causing all this trouble. I guess you have to use it to instruct matlab do not delete the memory once it's done. But why should matlab delete the dasani pointer? How is that pointer provided to matlab and what does matlab need it for?
On top of making dasani to be a persistent pointer, I also need to make its member variables with memory allocated by mxMalloc/mxCalloc to be persistent too, for example:
if (!dasani)
{
dasani = new NeedleUSsim;
mexMakeMemoryPersistent((void*) dasani->tplL);
mexMakeMemoryPersistent((void*) dasani->tplR);
mexMakeMemoryPersistent((void*) dasani->tplRho_deg);
mexMakeMemoryPersistent((void*) dasani->tplAlpha_deg);
mexMakeMemoryPersistent((void*) dasani->tplBeta_deg);
mexMakeMemoryPersistent((void*) dasani->hashTb);
mexMakeMemoryPersistent((void*) dasani->template1D);
mexAtExit(ExitFcn);
}
With the destructor as shown:
void NeedleUSsim::Deallocate()
{
free((void*) tplR); free((void*) tplL);
free((void*) tplRho_deg); free((void*) tplAlpha_deg);
free((void*) tplBeta_deg);
free((void*) hashTb);
free((void*) template1D);
}