I'm new in Linux C and C++ programming. I'm trying to create a C++ class for system v ipc message queue and I have 1 problem.
I wrote class for message like this:
class message
{
friend class queue;
private:
typedef struct
{
long mtype;
char mdata[maxmsg_buf];
}msgbuf_t, *msgbuf_ptr_t;
msgbuf_ptr_t msgbuf_ptr;
public:
message(void):msgbuf_ptr(NULL)
{
msgbuf_ptr = new msgbuf_t;
}
ipc::message::message(const long& type, const std::string& data):msgbuf_ptr(NULL)
{
msgbuf_ptr = new msgbuf_t;
msgbuf_ptr->mtype = type;
if(data.length() <= maxmsg_buf)
{
strncpy(msgbuf_ptr->mdata, data.c_str(), data.length());
}
}
};
class queue
{
private:
mutable qid_t qid;
public:
queue(const key_t& qkey = unique, const perm_t& qperm = usr_rw, const flag_t& qflag = none)
{
qid = msgget(qkey, qperm | qflag);
if(qid == -1)
{
throw errno; //specify exception for queue
}
}
void send(const ipc::message& msg, const int& sflag) const
{
if((msgsnd(qid, &msg.msgbuf_ptr, sizeof(msg.msgbuf_ptr->mdata), sflag)) == -1)
{
throw errno; //specify exception for queue
}
}
};
//Usage:
ipc::queue q(0x0000FFFF, ipc::usr_rw, ipc::create);
ipc::message msg(10L, "First test message for sysVipc queue");
q.send(msg); //throws EFAULT from here
When I send msgbuf_ptr to msgsnd syscall, it returns EFAULT(Bad address) error. So my question is: may I allocate msgbuf using operator new?
P.S. Sorry, if my english is not well.
The problem is in your queue::send method.
&msg.msgbuf_ptr is a pointer to a pointer. Omit the address operator & and you should be fine.
EDIT: no wait
msg.msgbuf_ptr->mdata is your message.
So, you should call it this way:
msgsnd(qid, &msg.msgbuf_ptr + sizeof(msg.msgbuf.mtype), sizeof(msg.msgbuf_ptr->mdata), sflag)
Related
I'm trying to make multithread c++ program.
I've been testing my program for long time, and sometimes got SIGSEGV (segmentation fault).
And I found a pointer variable copied partial memory only, not all.
Here is parts of my code.
Class definition
class Message
{
public:
Message() {}
virtual ~Message() {}
virtual void execute() = 0; //do something...
}
class Job
{
public:
Job();
~Job();
void post(Message *message);
void execute(Message *message);
private:
Message *_queue[1024] = {nullptr,};
volatile int _head = 0;
int _tail = 0;
};
And 4 of threads were running and call method of same Job class instance.
Thread 1 call this method.
void Job::execute(Message *message)
{
if (message != nullptr)
{
int index = __sync_fetch_and_add(&_head, 1) % 1024;
_queue[index] = message;
}
while (_tail != _head)
{
int index = _tail % 1024;
Message *workingMessage = _queue[index];
while (workingMessage == nullptr)
{
sched_yield();
workingMessage = _queue[index];
}
workingMessage->execute();
_queue[index] = nullptr;
++_tail; //this is changed on Thread 1 only!
}
}
Thread 2~4 call this method.
void Job::post(Message *message)
{
int index = __sync_fetch_and_add(&_head, 1) % 1024;
_queue[index] = message;
}
After several minutes, I've got SIGSEGV crash on this part of Thread 1.
workingMessage->execute(); // Crash!
On debugger, workingMessage had only some part of _queue[Index] value.
For example,
_queue[index] : 0x7fffdc09abdb
workingMessage : 0xffdc09abdb
or
_queue[index] : 0x7fffe8048e35
workingMessage : 0x7fffe8040000
like these cases.
I've tried to use std::atomic< Message *> for _queue and workingMessage, same crashes were happened.
What's wrong? I've compiled and tested this on CentOS 7, gcc 4.8.5.
I'm trying to write some server which authenticates clients using linux pam. I wrote the following class:
class Pam
{
public:
Pam(const char *module, const char *username)
{
mConv.appdata_ptr = nullptr;
mConv.conv = &convCallback;
const int res = pam_start("system-auth", username, &mConv, &mPamHandle);
if (res != PAM_SUCCESS)
throw std::runtime_error("Failed to initialize PAM");
}
bool authenticate(char *passwd)
{
pam_response *resp = static_cast<pam_response*>(malloc(sizeof(pam_response)));
resp->resp = passwd;
resp->resp_retcode = 0;
mConv.appdata_ptr = resp;
const int res = pam_authenticate(mPamHandle, 0);
log(res);
return res == PAM_SUCCESS;
}
~Pam()
{
if (mPamHandle)
pam_end(mPamHandle, PAM_SUCCESS);
mPamHandle = nullptr;
}
private:
static int convCallback (int msgId, const pam_message **msg, pam_response **resp, void *appData)
{
*resp = static_cast<pam_response*>(appData);
return PAM_SUCCESS;
}
private:
pam_handle_t *mPamHandle = nullptr;
pam_conv mConv;
};
Which then is used like:
Pam pam("system-auth", username);
if (pam.authenticate(passwd))
return true;
// error handling code here
I discovered that pam_authenticate returns PAM_AUTHTOK_RECOVERY_ERR for valid user/password. Possible return values documented in the man page and on the linux-pam.org http://www.linux-pam.org/Linux-PAM-html/adg-interface-by-app-expected.html#adg-pam_authenticate do not contain this value at all. Documentation says that it can be returned by pam_chauthtok and this means:
PAM_AUTHTOK_RECOVERY_ERR
A module was unable to obtain the old authentication token.
And it's still unclear what does it means in case of authentication. I've tried to run code both as normal user and as root the result was the same.
What's happening is that you're seeing 0 as the value of appData in convCallback, which is where the error is coming from - the reply data is empty, which means bad conversation, which causes the PAM_AUTHTOK_RECOVERY_ERR return value. This is based on reading the support.c file in the current code for the PAM-Linux source code.
Ok, couple of issues.
You can't reassign the conversation appdata_ptr value after initialization - the value of the pointer should be considered a constant after the invocation of pam_start. You should pass in a value there that will never change. If you checked the conversation function you would have noticed that the value of appData is 0.
You must assume that the value being put into the reply is owned by the calling routine - i.e. you'll have to strdup the password string (with all the evil that is connected to that).
With both of these in mind, I slightly altered your code to the following, which should address your problems (again, this is simplified code):
class Pam
{
public:
Pam(const char *module, const char *username)
{
mConv.appdata_ptr = (void *)(this);
mConv.conv = &convCallback;
const int res = pam_start(module, username, &mConv, &mPamHandle);
if (res != PAM_SUCCESS)
throw std::runtime_error("Failed to initialize PAM");
}
bool authenticate(char *passwd)
{
mPassword = passwd;
const int res = pam_authenticate(mPamHandle, 0);
log(res);
return res == PAM_SUCCESS;
}
~Pam()
{
if (mPamHandle)
pam_end(mPamHandle, PAM_SUCCESS);
mPamHandle = 0;
}
private:
static int convCallback (int msgId, const pam_message **msg, pam_response **resp, void *appData)
{
Pam *me = static_cast<Pam *>(appData);
pam_response *reply = static_cast<pam_response *>(calloc(1, sizeof(pam_response)));
reply->resp = strdup(me->mPassword);
reply->resp_retcode = 0;
*resp = reply;
return PAM_SUCCESS;
}
private:
pam_handle_t *mPamHandle = 0;
pam_conv mConv;
const char *mPassword = 0;
};
I have got a really bad memory leak I am trying to fix, but somehow i am not able to delete Objects without triggering this assertation.
I have searched for a solution via Google and have read the Questions on stackoverflow about this Error but I was still not able to find the answer!
Possible reasons to get this Error according to my research:
1. deleting objects more then one
2. shadow copying
3. creating and deleting Objects that are loaded from an external dll
4. creating objects without storing the pointer
BUT:
1. I checked the code and was not able to find double deletion
2. I use a copy constructor to copy Objects
3. The Error relatet classes are build (with MS Visual Studio) to a seperate lib but not to a dll. AND all the classes that are related to this error are located in the same lib.
4. I checked the code and it seems like that's not the problem
It would be great if anybody is able to spot the mistake in the code below, and I appreciate every hint that points me to the solution of the problem.
EDIT:
I forgot to mention the same deleting problem in sendThreadMain of MessageSystem (see code below). If i delete the Message there it causes unexpected errors somewhere else in the code. Might just be wrong data transmission... but i do not really know.
This code is run on Windows and Linux!
Here are the error related parts of the code:
Message
class Message
{
public:
Message (char type, unsigned char id, unsigned short size)
{
mType = type;
mId = id;
mSize= size;
}
Message(const Message &o)
{
mType = o.mType;
mId = o.mId;
mSize = o.mSize;
}
char getType() const {return mType;};
unsigned char getId() const {return mId;};
unsigned short getSize() const {return mSize;};
protected:
char mType;
unsigned char mId;
unsigned short mSize;
};
class JoinMessage : public Message
{
public:
JoinMessage () : Message ('j', 0, sizeof (JoinMessage))
{
team = TEAM_SPECTATOR;
}
JoinMessage (unsigned char id) : Message ('j', id, sizeof (JoinMessage)){}
JoinMessage (const JoinMessage &o) : Message (o)
{
team = o.team;
setName(o.getName());
}
void setName(std::string newName)
{
if (newName.length() > MAX_PLAYER_NAME_LENGHT)
newName = newName.substr(0, MAX_PLAYER_NAME_LENGHT);
memset(name, 0, MAX_PLAYER_NAME_LENGHT);
for(unsigned int i = 0; i < newName.length(); i++)
name[i] = newName[i];
}
std::string getName() const
{
std::string stringToReturn;
for(unsigned int i = 0; i < MAX_PLAYER_NAME_LENGHT; i++)
{
if (name[i])
stringToReturn.push_back(name[i]);
else
break;
}
return stringToReturn;
}
TeamIdentifier team;
private:
unsigned char name[MAX_PLAYER_NAME_LENGHT];
};
// there are a lot other messages
MessageQueue
MessageQueue::~MessageQueue()
{
boost::mutex::scoped_lock lock (queueMutex);
while(messageQueue.size() > 0)
{
// the crash is non-reproducible
// works 90% of the time
delete messageQueue.front (); // <- Debug Assertion Failed … _BLOCK_TYPE_IS_VALID
messageQueue.pop_front();
}
}
void MessageQueue::enqueMessage (Message* message)
{
{
boost::mutex::scoped_lock lock (queueMutex);
messageQueue.push_back(message);
}
}
Message* MessageQueue::dequeMessage ()
{
boost::mutex::scoped_lock lock (queueMutex);
if (messageQueue.size() == 0)
return nullptr;
Message* message = messageQueue.front ();
messageQueue.pop_front();
return message;
}
MessageSystem
template <class MessageType>
void broadcast (MessageType &message)
{
MessageType *internMessage = new MessageType(message);
boost::mutex::scoped_lock lock (mRecipientMapMutex);
std::map <boost::asio::ip::udp::endpoint, MessageQueue *>::iterator it;
for (it = mRecipientMap.begin ();
it != mRecipientMap.end ();
it++)
{
it->second->enqueMessage(internMessage);
}
}
template <class MessageType>
void post (MessageType &message, boost::asio::ip::udp::endpoint &recipient)
{
MessageType *internMessage = new MessageType(message);
std::map <boost::asio::ip::udp::endpoint, MessageQueue* >::iterator it;
MessageQueue *messageQueue = NULL;
{
boost::mutex::scoped_lock lock (mRecipientMapMutex);
it = mRecipientMap.find (recipient);
if (it != mRecipientMap.end())
messageQueue = it->second;
if(messageQueue)
messageQueue->enqueMessage (internMessage);
}
}
void MessageSystem::sendThreadMain ()
{
// copy endpoints to vecotr so it can be
// deleted from map while iterating
std::vector<udp::endpoint> endpoints;
{
boost::mutex::scoped_lock lock (mRecipientMapMutex);
std::map <udp::endpoint, MessageQueue *>::iterator mapIt = mRecipientMap.begin ();
while (mapIt != mRecipientMap.end())
{
endpoints.push_back(mapIt->first);
mapIt++;
}
}
std::vector<udp::endpoint>::iterator endpointIt = endpoints.begin();
while (endpointIt != endpoints.end())
{
char sendBuffer[PACKET_SIZE];
int sendBufferPosition = 0;
{
boost::mutex::scoped_lock lock (mRecipientMapMutex);
MessageQueue *messageQueue = mRecipientMap[*endpointIt];
if (messageQueue == nullptr)
{
mRecipientMap.erase(*endpointIt);
endpointIt++;
continue;
}
while (Message *message = messageQueue->dequeMessage ())
{
if (sendBufferPosition + message->getSize() > PACKET_SIZE)
{
// put message back and send it later
messageQueue->enqueMessage (message);
break;
}
// copy message into buffer
std::memcpy (
&sendBuffer [sendBufferPosition], message, message->getSize());
sendBufferPosition += message->getSize();
// deleting this message causes a crash if 2 or more
// recipients are registered within MessageSystem
//delete message; <- RANDOM CRASH elsewhere in the program
}
}
.... // more code down here that seems not related to the error
Today I figured it out on my own. It was #1 of the 4 possibilities mentioned in the Question.
deleting objects more then once (by saving multiple pointers to the exact same object)
Here is my Solution in MessageQueue:
template <class MessageType>
void broadcast (MessageType &message)
{
// I was creating 1 new Message right here but I need 1 new Message
// in EVERY MessageQueue so i moved the next line ...
// MessageType *internMessage = new MessageType(message);
boost::mutex::scoped_lock lock (mRecipientMapMutex);
std::map <boost::asio::ip::udp::endpoint, MessageQueue *>::iterator it;
for (it = mRecipientMap.begin ();
it != mRecipientMap.end ();
it++)
{
// ... down here. Now every queue contains its own copy of the Message
MessageType *internMessage = new MessageType(message);
it->second->enqueMessage(internMessage);
}
}
It might be a simple problem of wrong order. You are doing:
while(messageQueue.size() > 0)
{
delete messageQueue.front();
messageQueue.pop_front();
}
Maybe deleting the message after popping it, instead of before, would do the trick:
while(messageQueue.size() > 0)
{
Message* pFront = messageQueue.front();
messageQueue.pop_front();
delete pFront;
}
Anyway, I am not confident at all on this solution, since deleting the object pointed by pFront should have no effect on the queue itself, which just stores pointers. But you can try.
Well, I faced similar problem,
the following code
Message* message = messageQueue.front ();
messageQueue.pop_front();
return message;
The code that produced error with me was:
Point *p = q.LookFor(&q, &pts[5], &Dist);
cout ...
delete p;
It seems that the function delete the pointer it creates in the runtime, so you're not allowed to delete it "again"
so I replaced it with
Point p = *(q.LookFor(&q, &pts[5], &Dist));
and it's gone.
I have created a live continuous mjpeg stream. A crude illustration is like this
....[image (jpeg)]->[text "content-length"]->[image (jpeg)]->[text "content-length"]->....
As you can see I receive data from gstreamer media pipe line which contains image and my own injected text
(Note: Although I am using Gstreamer, my question is only related to C++ principles.)
In order to parse this real-time data, I am trying to receive and push it into the queue. Subsequently I plan to parse the data for the word "content-length" after queue contains a certain number of packets.
My code looks like the following:
void clear( std::queue<char> &q )
{
std::queue<char> empty;
std::swap( q, empty );
}
static GstFlowReturn new_buffer (GstAppSink *app_sink, gpointer user_data)
{
GstBuffer* buffer = gst_app_sink_pull_buffer(app_sink);
//create queue
std::queue<char> q;
g_print("The input buffer contents are\n");
gint i=0;
for(i=0; buffer->data[i];i++)
{
//g_print("\n%d",i);
q.push(buffer->data[i]);
}
//g_print("\nsize of inbuf is %d\n",GST_BUFFER_SIZE(buffer));
g_print("\n");
gst_buffer_unref(buffer);
//#####################
//parsing method here???
//#####################
clear(q);
return GST_FLOW_OK;
}
I have used circular queues/ ring buffer in C/C++ before. Is that the best option? Or is the C++ STL queues would be more appropriate in this scenario like above?
I ended up using ringbuffer class
In header file declare
//queue size
enum { rb_size = 5 }; // ---->element1 -> element2 -> .... -> elementN -> gap ->
// ^ |
// | |
// <--------------------<------------------<-------------V
typedef struct
{
char * data[rb_size];
int head, tail;
} ring_buffer_struct;
namespace myspace{
class ring_buffer{
private:
protected:
public:
//========= constructor ============
ring_buffer()
{
//If necessary initialization can happen here.
}
//========== destructor =============
virtual ~ring_buffer()
{
}
//===================================
virtual void rb_start(ring_buffer_struct *b);
virtual bool rb_empty(ring_buffer_struct const *b);
virtual char * rb_front(ring_buffer_struct const *b);
virtual char * rb_rear(ring_buffer_struct const *b);
virtual void rb_pop_front(ring_buffer_struct *b);
virtual ring_buffer_struct* rb_push_back(ring_buffer_struct *b);
}; //end of class
}
In cpp file
//start
void myspace::ring_buffer::rb_start(ring_buffer_struct *b)
{
b->head = 0; b->tail = 0;
}
//clear
bool myspace::ring_buffer::rb_empty(ring_buffer_struct const *b)
{
return b->head == b->tail;
}
//front element
char * myspace::ring_buffer::rb_front(ring_buffer_struct const *b)
{
return b->data[b->head]; //data gets popped
}
//rear element
char * myspace::ring_buffer::rb_rear(ring_buffer_struct const *b)
{
return b->data[b->tail]; //data gets pushed
}
//pop out front element
void myspace::ring_buffer::rb_pop_front(ring_buffer_struct *b)
{
if(b->head < b->tail)
{
++b->head;
}
if(b->head > b->tail)
{
b->head = 0;
}
}
//push in rear element
ring_buffer_struct* myspace::ring_buffer::rb_push_back(ring_buffer_struct *b)
{
int new_tail = b->tail;
if (++new_tail >= rb_size)
{ //beginning of the queue
new_tail = 0;
}
if (new_tail != b->head)
{
//middle of the queue
b->tail = new_tail;
}
if (new_tail <= b->head)
{
b->tail = 0;
}
return b;
}
And to use in the main()
...
char element1[10] = "abcdefghi";
char element2[10] = "bcdefghij";
char element3[10] = "cdefghijk";
ring_buffer_struct rb;
myspace::ring_buffer q;
q.rb_empty(&rb); //make sure empty
q.rb_start(&rb); //start - initialize
//initialize
uint16_t i;
for(i=0;i<rb_size;i++)
{
rb.data[rb.tail] = (char *)"000000000";
q.rb_push_back(&rb);
}
rb.data[rb.tail] = element1;
q.rb_push_back(&rb);
q.rb_pop_front(&rb); //now parse
rb.data[rb.tail] = element2;
q.rb_push_back(&rb);
q.rb_pop_front(&rb); //now parse
...
For parsing: I looked at this post
Simple string parsing with C++
Off topic suggestion:
When using the swap trick to clear out an STL container, don't call std::swap explicitly, as you may end up not getting a better-optimized version. The better way is:
void clear( std::queue<char> &q )
{
std::queue<char> empty;
using std::swap;
swap( q, empty );
}
This allows the compiler to choose a specialized version of swap that's optimized for the type of container you're using. You could also try q.swap(empty);, but I'm not sure all STL implementations offer that.
I stumbled about a method which seems to be present in all data objects like QList, QQueue, QHash...
I even investigated so far I can see the source code of it, which is
inline void setSharable(bool sharable) {
if (!sharable) detach(); d->sharable = sharable;
}
in qlist.h (lines 117).
But what effect does it have on the QList, QQueue, QHash... ? And is it in any way related to threading (which sounds reasonable)?
Thanks for any answer, and please only answer if you got actual knowledge.
No one could say more clear:
http://qt.nokia.com/doc/4.6/implicit-sharing.html
It is common practice to realize containers this way.
The sharable state you're asking about has nothing to do with mutlithreading. It is instead an implementation detail of copy-on-write data classes (even single-threaded ones) that hand out references to internal state.
Consider a class String that is implemented using CoW (for illustration purposes, this class isn't usable in threaded contexts, because accesses to d->refcount aren't synchronised, it also doesn't ensure that the internal char arrary ends in '\0', and might as well eat your grandmother; you have been warned):
struct StringRep {
StringRep()
: capacity(0), size(0), refcount(0), sharable(true), data(0) {}
~StringRep() { delete[] data; }
size_t capacity, size, refcount;
bool sharable; // later...
char * data;
};
class String {
StringRep * d;
public:
String() : d(new StringRep) { ++d->refcount; }
~String() { if (--d->refcount <= 0) delete d; }
explicit String(const char * s)
: d(new StringRep)
{
++d->refcount;
d->size = d->capacity = strlen(s);
d->data = new char[d->size];
memcpy(d->data, s, d->size);
}
String(const String &other)
: d(other.d)
{
++d->refcount;
}
void swap(String &other) { std::swap(d, other.d); }
String &operator=(const String &other) {
String(other).swap(*this); // copy-swap trick
return *this;
}
And a sample function each for mutating and const methods:
void detach() {
if (d->refcount == 1)
return;
StringRep * newRep = new StringRep(*d);
++newRep->refcount;
newRep->data = new char[d->size];
memcpy(newRep->data, d->data, d->size);
--d->refcount;
d = newRep;
}
void resize(size_t newSize) {
if (newSize == d->size)
return;
detach(); // mutator methods need to detach
if (newSize < d->size) {
d->size = newSize;
} else if (newSize > d->size) {
char * newData = new char[newSize];
memcpy(newData, d->data, d->size);
delete[] d->data;
d->data = newData;
}
}
char operator[](size_t idx) const {
// no detach() here, we're in a const method
return d->data[idx];
}
};
So far so good. But what if we want to provide a mutable operator[]?
char & operator[](size_t idx) {
detach(); // make sure we're not changing all the copies
// in case the returned reference is written to
return d->data[idx];
}
This naïve implementation has a flaw. Consider the following scenario:
String s1("Hello World!");
char & W = s1[7]; // hold reference to the W
assert( W == 'W' );
const String s1(s2); // Shallow copy, but s1, s2 should now
// act independently
W = 'w'; // modify s1 _only_ (or so we think)
assert( W == 'w' ); // ok
assert( s1[7] == 'w' ); // ok
assert( s2[7] == 'W' ); // boom! s2[7] == 'w' instead!
To prevent this, String has to mark itself non-sharable when it hands out a reference to internal data, so that any copy that is taken from it is always deep. So, we need to adjust detach() and char & operator[] like this:
void detach() {
if (d->refcount == 1 && /*new*/ d->sharable)
return;
// rest as above
}
char & operator[](size_t idx) {
detach();
d->shareable = false; // new
return d->data[idx];
}
When to reset the shareable state back to true again? A common technique is to say that references to internal state are invalidated when calling a non-const method, so that's where shareable is reset back to true. Since every non-const function calls detach(), we can reset shareable there, so that detach() finally becomes:
void detach() {
if (d->refcount == 1 && d->sharable) {
d->sharable = true; // new
return;
}
d->sharable = true; // new
StringRep * newRep = new StringRep(*d);
++newRep->refcount;
newRep->data = new char[d->size+1];
memcpy(newRep->data, d->data, d->size+1);
--d->refcount;
d = newRep;
}