Two pcap_compile() on one device at same time? - c++

I have two threads and each one has packet capture from the same deviсe at the same time but the program crashes when the second thread reaches the pcap_compile() function. Also each thread has their own variables and don't use global. It seems that they get the same handle of the device, therefore the program crashes. Why do I need two threads? Because I want to seperate packets on the sent and on the recived by specified pcap filter. So how do I solve this? Or is it better to use one thread and sort manually the sent and the received packets by using the address from tcp header?

pcap_compile is not thread safe. You must surround all calls to it that may be encountered by separate threads with a critical section/mutex to prevent errors because of non thread-safe state within the parser that compiles the expression (for the gory details, it uses YACC to create code for parsing the expression and the code generated for that is eminently not thread safe).
You need to explicitly open the device once per thread that you're planning on using for the capture, if you reuse the same device handle across multiple threads then it will simply not do what you're asking for. You should open the pcap handle within the thread that you're planning on using it, so each thread that's planning on doing capture should do it's own pcap_open.
to guard the call to pcap_compile with a Critical Section, you could create a simple wrapper (C++ wrapper of windows critical section):
class lock_interface {
public:
virtual void lock() = 0;
virtual void unlock() = 0;
};
class cs : public lock_interface {
CRITICAL_SECTION crit;
public:
cs() { InitializeCriticalSection(&crit); }
~cs() { DeleteCriticalSection(&crit); }
virtual void lock() {
EnterCriticalSection(&crit);
}
virtual void unlock() {
LeaveCriticalSection(&crit);
}
private:
cs(const locker &);
cs &operator=(const cs &);
};
class locker {
lock_interface &m_ref;
public:
locker(lock_interface &ref) : m_ref(ref) { m_ref.lock(); }
~locker() { m_ref.unlock(); }
private:
locker(const locker &);
locker &operator=(const locker &);
};
static cs section;
int
wrapped_pcap_compile(pcap_t *p, struct bpf_program *fp, const char *str, int optimize, bpf_u_int32 netmask)
{
locker locked(section);
pcap_compile(p, fp, str, optimize, netmask);
}

If you are using C++11, you can have something like:
int thread_safe_pcap_compile_nopcap(int snap_len, int link_type,
struct bpf_program *fp, char const *str,
int optimize, bpf_u_int32 netmask) {
static std::mutex mtx;
std::lock_guard<std::mutex> lock(mtx);
return pcap_compile_nopcap(snap_len, link_type, fp, str, optimize, netmask);
}
It is similar for pcap_compile function.

Related

Thread safety of boost::unordered_map<int, struct> and shared_mutex

I’m trying to parse the ts stream data coming from sockets with 4 threads. I’ve decided to use boost shared mutex to manage connections and data receiving. But I’m totally newbie in c++ and I’m not sure if I’ll do it right with tread safety. I’m using boost unordered_map<int, TsStreams>, when a new user is connecting, I’m locking the mutex with a unique lock and adding a user to the map, when this user is disconnecting, I’m locking the mutex with a unique lock and remove him from the map. TsStreams structure contains the vector and some additional variables while the user sending the data, I’m using the shared lock to get the user’s TsStreams reference from the map, add new data to the vector and modify additional variables. Is modifying TsStreams in that way thread-safe or not?
class Demuxer {
public:
Demuxer();
typedef signal<void (int, TsStream)> PacketSignal;
void onUserConnected(User);
void onUserDisconnected(int);
void onUserData(Data);
void addPacketSignal(const PacketSignal::slot_type& slot);
private:
mutable PacketSignal packetSignal;
void onPacketReady(int, TsStream);
TsDemuxer tsDemuxer;
boost::unordered_map<int, TsStreams> usersData;
boost::shared_mutex mtx_;
};
#include "Demuxer.h"
Demuxer::Demuxer() {
tsDemuxer.addPacketSignal(boost::bind(&Demuxer::onPacketReady, this, _1, _2));
}
void Demuxer::onUserConnected(User user){
boost::unique_lock<boost::shared_mutex> lock(mtx_);
if(usersData.count(user.socket)){
usersData.erase(user.socket);
}
TsStreams streams;
streams.video.isVideo = true;
usersData.insert(std::make_pair(user.socket, streams));
}
void Demuxer::onUserDisconnected(int socket){
boost::unique_lock<boost::shared_mutex> lock(mtx_);
if(usersData.count(socket)){
usersData.erase(socket);
}
}
void Demuxer::onUserData(Data data) {
boost::shared_lock<boost::shared_mutex> lock(mtx_);
if(!usersData.count(data.socket)){
return;
}
tsDemuxer.parsePacket(data.socket, std::ref(usersData.at(data.socket)), (uint8_t *) data.buffer, data.length);
}
void Demuxer::onPacketReady(int socket, TsStream data) {
packetSignal(socket, data);
}
void Demuxer::addPacketSignal(const PacketSignal::slot_type& slot){
packetSignal.connect(slot);
}
struct TsStreams{
TsStreams() = default;
TsStreams(const TsStreams &p1) {}
TsStream video;
TsStream audio;
};
struct TsStream
{
TsStream() = default;
TsStream(const TsStream &p1) {}
boost::recursive_mutex mtx_; // to make sure to have the queue, it may not be necessary
uint64_t PTS = 0;
uint64_t DTS = 0;
std::vector<char> buffer;
uint32_t bytesDataLength = 0;
bool isVideo = false;
};
class TsDemuxer {
public:
typedef signal<void (int, TsStream)> PacketSignal;
void parsePacket(int socket, TsStreams &streams, uint8_t *data, int size);
connection addPacketSignal(const PacketSignal::slot_type& slot);
private:
PacketSignal packetSignal;
void parseTSPacket(int socket, TsStream &stream, uint8_t *data, int size);
void parseAdaptationField(BitReader &bitReader);
void parseStream(int socket, TsStream &stream, BitReader &bitReader, uint32_t payload_unit_start_indicator);
void parsePES(TsStream &stream, BitReader &bitReader);
int64_t parseTSTimestamp(BitReader &bitReader);
};
void TsDemuxer::parsePacket(int socket, TsStreams &streams, uint8_t *data, int size) {
//some parsing
if(video){
streams.video.mtx_.lock();
parseTSPacket(socket, streams.video, (uint8_t *)buf, 188);
}else{
streams.audio.mtx_.lock();
parseTSPacket(socket, streams.audio, (uint8_t *)buf, 188);
}
}
void TsDemuxer::parseTSPacket(int socket, TsStream &stream, uint8_t *data, int size)
{
//some more parsing
parseStream(socket, stream, bitReader, payload_unit_start_indicator);
}
void TsDemuxer::parseStream(int socket, TsStream &stream, BitReader &bitReader, uint32_t payload_unit_start_indicator) {
if(payload_unit_start_indicator)
{
if(!stream.buffer.empty()){
packetSignal(socket, stream);
stream.buffer = vector<char>();
stream.bytesDataLength = 0;
}
parsePES(stream, bitReader);
}
size_t payloadSizeBytes = bitReader.numBitsLeft() / 8;
copy(bitReader.getBitReaderData(), bitReader.getBitReaderData()+payloadSizeBytes,back_inserter(stream.buffer));
stream.mtx_.unlock();
}
The demuxer looks correct to me. There are a few inefficiencies though:
You don't need to count before you erase. Just erase. If an element is not present, this will do nothing. That saves you one lookup. Likewise, don't use count followed by at. Use find (see below for the use).
You may want to move as much work as possible out of the critical section. Foe example in onUserConnected you could create the TsStreams object before acquiring the lock.
Note that changing an unordered map will never invalidate pointers or references to elements in the map unless they are erased. That means in onUserData you don't have to hold the lock on the map while parsing the packet.
That is, assuming you don't call onUserData for the same user from two different threads. You could prevent this by introducing a second lock the TsStream object. Likewise, you should guard against erasing the element while another thread may still parse the last packet. I would use a shared_ptr for this. Something like this:
class Demuxer {
...
boost::unordered_map<int, boost::shared_ptr<TsStreams> > usersData;
boost::shared_mutex mtx_;
};
void Demuxer::onUserData(Data data) {
boost::shared_lock<boost::shared_mutex> maplock(mtx_);
auto found = usersData.find(data.socket);
if(found == usersData.end())
return;
boost::shared_ptr<TsStreams> stream = found->second;
boost::unique_lock<boost::recursive_mutex> datalock(stream->mtx_);
maplock.unlock();
tsDemuxer.parsePacket(data.socket, *stream, (uint8_t *) data.buffer, data.length);
}
If you reduce the time the Demuxer lock is taken with this approach, you should probably replace that shared mutex with a normal one. shared mutexes have much higher overhead and are not worth it for such short critical sections.
The TsDemuxer looks a bit wonky:
In TsDemuxer::parsePacket you never unlock the mutex. Shouldn't that be a unique_lock? Likewise, in parseStream the unlock seems unpaired. In general, using a unique_lock object is always the way to go compared to manual locking and unlocking. If anything, lock and unlock the unique_lock, not the mutex.
Remarks unrelated to multithreading
stream.buffer.clear() is more efficient thanstream.buffer = vector<char>() because this will reuse the buffer memory instead of deallocating it completely.
As others have noted, these parts of boost are now part of the standard library. Replace boost:: with std:: and enable a recent C++ standard like C++14 or 17 and you are fine. At worst you have to replace shared_mutex with shared_timed_mutex.
In Demuxer, you pass the User and Data objects by value. Are you sure those shouldn't be const references?

How can I implement a C++ Reader-Writer lock using a single unlock method, which can be called be a reader or writer?

I'm working on a project, which requires the use of specific OS abstractions and I need to implement a reader-writer lock using their semaphore and mutex. I currently, have a setup in the format:
class ReadWriteLock
{
public:
ReadWriteLock(uint32_t maxReaders);
~ReadWriteLock();
uint32_t GetMaxReaders() const;
eResult GetReadLock(int32_t timeout);
eResult GetWriteLock(int32_t timeout);
eResult Unlock();
private:
uint32_t m_MaxReaders;
Mutex* m_WriterMutex;
Semaphore* m_ReaderSemaphore;
};
In this implementation I need to use this Unlock method to either unlock the writer and release all reader semaphore slots, or to simply unleash a reader semaphore slot, however, I am struggling as I cannot think of an implementation, which will be work in all cases. How can I make this work in the given setup? I know it is possible as POSIX was able to implement a universal unlock method in their implementation, but I cannot find any indication of how that was done, so would appreciate any information people can share.
Note that I cannot use C++11 or other OS primitives.
Well, define two functions UnlockRead and UnlockWrite.
I believe you do not need both accesses (Write/Read) at the same time in the same place. So what I am proposing is to have two other classes for locking access:
class ReadWriteAccess
{
public:
ReadWriteAccess(uint32_t maxReaders);
~ReadWriteAccess();
uint32_t GetMaxReaders() const;
uint32_t GetMaxReaders() const;
eResult GetReadLock(int32_t timeout);
eResult GetWriteLock(int32_t timeout);
eResult UnlockWrite();
eResult UnlockRead();
private:
uint32_t m_MaxReaders;
Mutex* m_WriterMutex;
Semaphore* m_ReaderSemaphore;
};
And have separate classes for read and write lock and use RAII to be always on safe side:
class ReadLock
{
public:
ReadLock(ReadWriteAccess& access, int32_t timeout) : access(access)
{
result = access.GetReadLock(timeout);
}
eResult getResult() const { return result; }
~ReadLock()
{
if (result)
access.UnlockRead();
}
private:
ReadWriteAccess& access;
eResult result;
};
and use like this:
T someResource;
ReadWriteAccess someResourceGuard;
void someFunction()
{
ReadLock lock(someResourceGuard);
if (lock.getResult())
cout << someResource; // it is safe to read something from resource
}
Of course, the very similar implementation you can easily write by yourself for WriteLock
Since OP insisted in comments to have "one" Unlock - please consider the drawbacks:
Assume it is implemented some kind of stack of last calls to Lock functions:
class ReadWriteLock
{
public:
ReadWriteLock(uint32_t maxReaders);
~ReadWriteLock();
uint32_t GetMaxReaders() const;
eResult GetReadLock(int32_t timeout)
{
eResult result = GetReadLockImpl(timestamp);
if (result)
lockStack.push(READ);
}
eResult GetWriteLock(int32_t timeout)
{
eResult result = GetWriteLockImpl(timestamp);
if (result)
lockStack.push(WRITE);
}
eResult Unlock()
{
LastLockMode lockMode = lockStack.top();
lockStack.pop();
if (lockMode == READ)
UnlockReadImpl();
else
UnlockWriteImpl();
}
private:
uint32_t m_MaxReaders;
Mutex* m_WriterMutex;
Semaphore* m_ReaderSemaphore;
enum Mode { READ, WRITE };
std::stack<Mode> lockStack;
};
But the above would work only in one-thread application. And one-thread application never need any locks.
So - you have to have multi-thread stack - like:
template <typename Value>
class MultiThreadStack
{
public:
void push(Value)
{
stackPerThread[getThreadId()].push(value);
}
Value top()
{
return stackPerThread[getThreadId()].top();
}
void pop()
{
stackPerThread[getThreadId()].pop();
}
private:
ThreadId getThreadId() { return /* your system way to get thread id*/; }
std::map<ThreadId, std::stack<Value>> stackPerThread;
};
So use this MultiThreadStack not std::stack in ReadWriteLock.
But, the std::map above would need ReadWriteLock to lock access to it from multuple threads - so, well, either you know all your threads before you start using this stuff (preregistration) or you end up in the same problem as described here. So my advice - if you can - change your design.
When acquiring the lock successfully the type is known: either you have many readers running or only one writer, you cannot have both readers and writers running with a validly acquired lock.
So it suffices to store the current lock mode when a lock call succeeds and all following unlock calls (potentially many in case reading permit was provided, only one indeed if writing lock was requested) will be of that mode.

Ensure that a thread doesn't lock a mutex twice?

Say I have a thread running a member method like runController in the example below:
class SomeClass {
public:
SomeClass() {
// Start controller thread
mControllerThread = std::thread(&SomeClass::runController, this)
}
~SomeClass() {
// Stop controller thread
mIsControllerThreadInterrupted = true;
// wait for thread to die.
std::unique_lock<std:::mutex> lk(mControllerThreadAlive);
}
// Both controller and external client threads might call this
void modifyObject() {
std::unique_lock<std::mutex> lock(mObjectMutex);
mObject.doSomeModification();
}
//...
private:
std::mutex mObjectMutex;
Object mObject;
std::thread mControllerThread;
std::atomic<bool> mIsControllerInterrupted;
std::mutex mControllerThreadAlive;
void runController() {
std::unique_lock<std::mutex> aliveLock(mControllerThreadAlive);
while(!mIsControllerInterruped) {
// Say I need to synchronize on mObject for all of these calls
std::unique_lock<std::mutex> lock(mObjectMutex);
someMethodA();
modifyObject(); // but calling modifyObject will then lock mutex twice
someMethodC();
}
}
//...
};
And some (or all) of the subroutines in runController need to modify data that is shared between threads and guarded by a mutex. Some (or all) of them, might also be called by other threads that need to modify this shared data.
With all the glory of C++11 at my disposal, how can I ensure that no thread ever locks a mutex twice?
Right now, I'm passing unique_lock references into the methods as parameters as below. But this seems clunky, difficult to maintain, potentially disastrous, etc...
void modifyObject(std::unique_lock<std::mutex>& objectLock) {
// We don't even know if this lock manages the right mutex...
// so let's waste some time checking that.
if(objectLock.mutex() != &mObjectMutex)
throw std::logic_error();
// Lock mutex if not locked by this thread
bool wasObjectLockOwned = objectLock.owns_lock();
if(!wasObjectLockOwned)
objectLock.lock();
mObject.doSomeModification();
// restore previous lock state
if(!wasObjectLockOwned)
objectLock.unlock();
}
Thanks!
There are several ways to avoid this kind of programming error. I recommend doing it on a class design level:
separate between public and private member functions,
only public member functions lock the mutex,
and public member functions are never called by other member functions.
If a function is needed both internally and externally, create two variants of the function, and delegate from one to the other:
public:
// intended to be used from the outside
int foobar(int x, int y)
{
std::unique_lock<std::mutex> lock(mControllerThreadAlive);
return _foobar(x, y);
}
private:
// intended to be used from other (public or private) member functions
int _foobar(int x, int y)
{
// ... code that requires locking
}

Is possible to get a thread-locking mechanism in C++ with a std::atomic_flag?

Using MS Visual C++2012
A class has a member of type std::atomic_flag
class A {
public:
...
std::atomic_flag lockFlag;
A () { std::atomic_flag_clear (&lockFlag); }
};
There is an object of type A
A object;
who can be accessed by two (Boost) threads
void thr1(A* objPtr) { ... }
void thr2(A* objPtr) { ... }
The idea is wait the thread if the object is being accessed by the other thread.
The question is: do it is possible construct such mechanism with an atomic_flag object? Not to say that for the moment, I want some lightweight that a boost::mutex.
By the way the process involved in one of the threads is very long query to a dBase who get many rows, and I only need suspend it in a certain zone of code where the collision occurs (when processing each row) and I can't wait the entire thread to finish join().
I've tryed in each thread some as:
thr1 (A* objPtr) {
...
while (std::atomic_flag_test_and_set_explicit (&objPtr->lockFlag, std::memory_order_acquire)) {
boost::this_thread::sleep(boost::posix_time::millisec(100));
}
... /* Zone to portect */
std::atomic_flag_clear_explicit (&objPtr->lockFlag, std::memory_order_release);
... /* the process continues */
}
But with no success, because the second thread hangs. In fact, I don't completely understand the mechanism involved in the atomic_flag_test_and_set_explicit function. Neither if such function returns inmediately or can delay until the flag can be locked.
Also it is a mistery to me how to get a lock mechanism with such a function who always set the value, and return the previous value. with no option to only read the actual setting.
Any suggestion are welcome.
By the way the process involved in one of the threads is very long query to a dBase who get many rows, and I only need suspend it in a certain zone of code where the collision occurs (when processing each row) and I can't wait the entire thread to finish join().
Such a zone is known as the critical section. The simplest way to work with a critical section is to lock by mutual exclusion.
The mutex solution suggested is indeed the way to go, unless you can prove that this is a hotspot and the lock contention is a performance problem. Lock-free programming using just atomic and intrinsics is enormously complex and cannot be recommended at this level.
Here's a simple example showing how you could do this (live on http://liveworkspace.org/code/6af945eda5132a5221db823fa6bde49a):
#include <iostream>
#include <thread>
#include <mutex>
struct A
{
std::mutex mux;
int x;
A() : x(0) {}
};
void threadf(A* data)
{
for(int i=0; i<10; ++i)
{
std::lock_guard<std::mutex> lock(data->mux);
data->x++;
}
}
int main(int argc, const char *argv[])
{
A instance;
auto t1 = std::thread(threadf, &instance);
auto t2 = std::thread(threadf, &instance);
t1.join();
t2.join();
std::cout << instance.x << std::endl;
return 0;
}
It looks like you're trying to write a spinlock. Yes, you can do that with std::atomic_flag, but you are better off using std::mutex instead. Don't use atomics unless you really know what you're doing.
To actually answer the question asked: Yes, you can use std::atomic_flag to create a thread locking object called a spinlock.
#include <atomic>
class atomic_lock
{
public:
atomic_lock()
: lock_( ATOMIC_FLAG_INIT )
{}
void lock()
{
while ( lock_.test_and_set() ) { } // Spin until the lock is acquired.
}
void unlock()
{
lock_.clear();
}
private:
std::atomic_flag lock_;
};

waiting on a condition variable in a helper function that's called from the function that acquires the lock

I'm new to the boost threads library.
I have a situation where I acquire a scoped_lock in one function and need to wait on it in a callee.
The code is on the lines of:
class HavingMutex
{
public:
...
private:
static boost::mutex m;
static boost::condition_variable *c;
static void a();
static void b();
static void d();
}
void HavingMutex::a()
{
boost::mutex::scoped_lock lock(m);
...
b() //Need to pass lock here. Dunno how !
}
void HavingMutex::b(lock)
{
if (some condition)
d(lock) // Need to pass lock here. How ?
}
void HavingMutex::d(//Need to get lock here)
{
c->wait(lock); //Need to pass lock here (doesn't allow direct passing of mutex m)
}
Basically, in function d(), I need to access the scoped lock I acquired in a() so that I can wait on it. (Some other thread will notify).
Any help appreiciated. Thanks !
Have you tried simple reference? According to boost 1.41 documentation at http://www.boost.org/doc/libs/1_41_0/doc/html/thread/synchronization.html it should be all that is required.
...
void HavingMutex::a()
{
boost::mutex::scoped_lock lock(m);
...
b(lock);
}
void HavingMutex::b(boost::mutex::scoped_lock &lock)
{
if (some condition) // consider while (!some_condition) here
d(lock);
}
void HavingMutex::d(boost::mutex::scoped_lock &lock)
{
c->wait(lock);
}
void HavingMutex::notify()
{
// set_condition;
c->notify_one();
}
Also in boost example they have while cycle around wait. Wait could be interrupted in some cases by system itself, not really you got the lock.
I suggest you reconsider having all methods static with static members also. Instead create them as normal members, and create one global object.