Libcurl - CURLSHOPT_LOCKFUNC - how to use it? - c++

Please tell me the nuances of using the option CURLSHOPT_LOCKFUNC ?
I found an example here.
I can't quite understand, is there an array of mutexes used to block access to data?
--Here is this part of the code from the example:
static pthread_mutex_t lockarray[NUM_LOCKS]; Is it an array of mutexes ?
//.....
static void lock_cb(CURL *handle, curl_lock_data data,
curl_lock_access access, void *userptr)
{
(void)access;
(void)userptr;
(void)handle;
pthread_mutex_lock(&lockarray[data]);
}
That is, from this section of code - I can conclude that different mutexes are used to lock CURLSHOPT_LOCKFUNC ?
I don't understand how it is? Isn't there more than one mutex needed to synchronize threads to access data?
And this part of the code is also not clear:
pthread_mutex_lock(&lockarray[data]);
What kind of variable is data ?
The documentation says: https://curl.se/libcurl/c/CURLSHOPT_LOCKFUNC.html
The data argument tells what kind of data libcurl wants to lock. Make
sure that the callback uses a different lock for each kind of data.
What means: "what kind of data libcurl wants to lock" ??
I can't understand.

Yes, the code is using an array of mutexes.
curl_lock_data is an enum that is defined in curl.h and specifies the different types of data that curl uses locks for:
/* Different data locks for a single share */
typedef enum {
CURL_LOCK_DATA_NONE = 0,
/* CURL_LOCK_DATA_SHARE is used internaly to say that
* the locking is just made to change the internal state of the share
* itself.
*/
CURL_LOCK_DATA_SHARE,
CURL_LOCK_DATA_COOKIE,
CURL_LOCK_DATA_DNS,
CURL_LOCK_DATA_SSL_SESSION,
CURL_LOCK_DATA_CONNECT,
CURL_LOCK_DATA_LAST
} curl_lock_data;
Since the values are sequential, the code is using them as indexes into the array.
So, for example, when multiple threads are accessing shared cookie data, they will lock/unlock the CURL_LOCK_DATA_COOKIE mutex.
curl_lock_access is another enum defined in curl.h, which specifies why a given lock is being obtained:
/* Different lock access types */
typedef enum {
CURL_LOCK_ACCESS_NONE = 0, /* unspecified action */
CURL_LOCK_ACCESS_SHARED = 1, /* for read perhaps */
CURL_LOCK_ACCESS_SINGLE = 2, /* for write perhaps */
CURL_LOCK_ACCESS_LAST /* never use */
} curl_lock_access;
The difference between these reasons doesn't matter for a mutex, which has only a notion of exclusive access, meaning only 1 thread at a time can hold the lock. That is why the code is not using the access parameter.
But, other types of locks, for instance a RW (reader/writer) lock, do have a concept of shared access (ie, multiple threads can hold the lock at the same time) vs exclusive access. When multiple threads want to access the same shared data for just reading only, it makes sense for them to not block each other. Only when a thread wants to modify the shared data should other threads then be blocked from accessing that data, for reading or writing, until the modification is finished.
For example:
const int NUM_LOCKS = CURL_LOCK_DATA_LAST + 1;
static pthread_rwlock_t lockarray[NUM_LOCKS];
...
static void lock_cb(CURL *handle, curl_lock_data data,
curl_lock_access access, void *userptr)
{
(void)handle;
(void)userptr;
switch (access) {
case CURL_LOCK_ACCESS_SHARED:
pthread_rwlock_rdlock(&lockarray[data]);
break;
case CURL_LOCK_ACCESS_ SINGLE:
pthread_rwlock_wrlock(&lockarray[data]);
break;
}
}
static void unlock_cb(CURL *handle, curl_lock_data data,
curl_lock_access access, void *userptr)
{
(void)handle;
(void)access;
(void)userptr;
pthread_rwlock_unlock(&lockarray[data]);
}

Related

Is using volatile on shared memory safe?

Lets suppose following:
I have two processes on Linux / Mac OS.
I have mmap on shared memory (or in a file).
Then in both processes I have following:
struct Data{
volatile int reload = 0; // using int because is more standard
// more things in the future...
};
void *mmap_memory = mmap(...);
Data *data = static_cast<Data *>(mmap_memory); // suppose size is sufficient and all OK
Then in one of the processes I do:
//...
data->reload = 1;
//...
And in the other I do:
while(...){
do_some_work();
//...
if (data->reload == 1)
do_reload();
}
Will this be thread / inter process safe?
Idea is from here:
https://embeddedartistry.com/blog/2019/03/11/improve-volatile-usage-with-volatile_load-and-volatile_store/
Note:
This can not be safe with std::atomic<>, since it does not "promise" anything about shared memory. Also constructing/destructing from two different processes is not clear at all.
Will this be thread / inter process safe?
No.
From your own link:
One problematic and common assumption is that volatile is equivalent to “atomic”. This is not the case. All the volatile keyword denotes is that the variable may be modified externally, and thus reads/writes cannot be optimized.
Your code needs atomic access to the value. if (data->reload == 1) won't work if it reads some partial/intermediate value from data->reload.
And nevermind what happens if multiple threads do read 1 from data->reload - your posted code doesn't handle that at all.
Also see Why is volatile not considered useful in multithreaded C or C++ programming?

Synchronizing method calls on shared object from multiple threads

I am thinking about how to implement a class that will contain private data that will be eventually be modified by multiple threads through method calls. For synchronization (using the Windows API), I am planning on using a CRITICAL_SECTION object since all the threads will spawn from the same process.
Given the following design, I have a few questions.
template <typename T> class Shareable
{
private:
const LPCRITICAL_SECTION sync; //Can be read and used by multiple threads
T *data;
public:
Shareable(LPCRITICAL_SECTION cs, unsigned elems) : sync{cs}, data{new T[elems]} { }
~Shareable() { delete[] data; }
void sharedModify(unsigned index, T &datum) //<-- Can this be validly called
//by multiple threads with synchronization being implicit?
{
EnterCriticalSection(sync);
/*
The critical section of code involving reads & writes to 'data'
*/
LeaveCriticalSection(sync);
}
};
// Somewhere else ...
DWORD WINAPI ThreadProc(LPVOID lpParameter)
{
Shareable<ActualType> *ptr = static_cast<Shareable<ActualType>*>(lpParameter);
T copyable = /* initialization */;
ptr->sharedModify(validIndex, copyable); //<-- OK, synchronized?
return 0;
}
The way I see it, the API calls will be conducted in the context of the current thread. That is, I assume this is the same as if I had acquired the critical section object from the pointer and called the API from within ThreadProc(). However, I am worried that if the object is created and placed in the main/initial thread, there will be something funky about the API calls.
When sharedModify() is called on the same object concurrently,
from multiple threads, will the synchronization be implicit, in the
way I described it above?
Should I instead get a pointer to the
critical section object and use that instead?
Is there some other
synchronization mechanism that is better suited to this scenario?
When sharedModify() is called on the same object concurrently, from multiple threads, will the synchronization be implicit, in the way I described it above?
It's not implicit, it's explicit. There's only only CRITICAL_SECTION and only one thread can hold it at a time.
Should I instead get a pointer to the critical section object and use that instead?
No. There's no reason to use a pointer here.
Is there some other synchronization mechanism that is better suited to this scenario?
It's hard to say without seeing more code, but this is definitely the "default" solution. It's like a singly-linked list -- you learn it first, it always works, but it's not always the best choice.
When sharedModify() is called on the same object concurrently, from multiple threads, will the synchronization be implicit, in the way I described it above?
Implicit from the caller's perspective, yes.
Should I instead get a pointer to the critical section object and use that instead?
No. In fact, I would suggest giving the Sharable object ownership of its own critical section instead of accepting one from the outside (and embrace RAII concepts to write safer code), eg:
template <typename T>
class Shareable
{
private:
CRITICAL_SECTION sync;
std::vector<T> data;
struct SyncLocker
{
CRITICAL_SECTION &sync;
SyncLocker(CRITICAL_SECTION &cs) : sync(cs) { EnterCriticalSection(&sync); }
~SyncLocker() { LeaveCriticalSection(&sync); }
}
public:
Shareable(unsigned elems) : data(elems)
{
InitializeCriticalSection(&sync);
}
Shareable(const Shareable&) = delete;
Shareable(Shareable&&) = delete;
~Shareable()
{
{
SyncLocker lock(sync);
data.clear();
}
DeleteCriticalSection(&sync);
}
void sharedModify(unsigned index, const T &datum)
{
SyncLocker lock(sync);
data[index] = datum;
}
Shareable& operator=(const Shareable&) = delete;
Shareable& operator=(Shareable&&) = delete;
};
Is there some other synchronization mechanism that is better suited to this scenario?
That depends. Will multiple threads be accessing the same index at the same time? If not, then there is not really a need for the critical section at all. One thread can safely access one index while another thread accesses a different index.
If multiple threads need to access the same index at the same time, a critical section might still not be the best choice. Locking the entire array might be a big bottleneck if you only need to lock portions of the array at a time. Things like the Interlocked API, or Slim Read/Write locks, might make more sense. It really depends on your thread designs and what you are actually trying to protect.

Setting all TLS (thread local storage) variables to a new, single value in C++

I have a class Foo with the following thread-specific static member:
__declspec(thread) static bool s_IsAllAboutThatBass;
In the implementation file it is initialized like so:
__declspec(thread) bool Foo::s_IsAllAboutThatBass = true;
So far so good. Now, any thread can flip this bool willy nilly as they deem fit. Then the problem: at some point I want each thread to reset that bool to its initial true value.
How can I slam all instances of the TLS to true from a central thread?
I've thought of ways I could do this with synchronization primitives I know about, like critical sections, read/write sections, or events, but nothing fits the bill. In my real use cases I am unable to block any of the other threads for any significant length of time.
Any help is appreciated. Thank you!
Edit: Plan A
One idea is to use a generation token, or cookie that is read by all threads and written to by the central thread. Each thread can then have a TLS for the last generation viewed by that thread when grabbing s_isAllAboutThatBass via some accessor. When the thread local cookie differs from the shared cookie, we increment the thread local one and update s_isAllAboutThatBass to true.
Here is a light weighted implementation of "Plan A" with C++11 Standard atomic variable and thread_local-specifier. (If your compiler doesn't support them, please replace to vendor specific facilities.)
#include <atomic>
struct Foo {
static std::atomic<unsigned> s_TokenGeneration;
static thread_local unsigned s_LocalToken;
static thread_local bool s_LocalState;
// for central thread
void signalResetIsAllAboutThatBass() {
++s_TokenGeneration;
}
// accessor for other threads
void setIsAllAboutThatBass(bool b) {
unsigned currToken = s_TokenGeneration;
s_LocalToken = currToken;
s_LocalState = b;
}
bool getIsAllAboutThatBass() const {
unsigned currToken = s_TokenGeneration;
if (s_LocalToken < currToken) {
// reset thread-local token & state
s_LocalToken = currToken;
s_LocalState = true;
}
return s_LocalState;
}
};
std::atomic<unsigned> Foo::s_TokenGeneration;
thread_local unsigned Foo::s_LocalToken = 0u;
thread_local bool Foo::s_LocalState = true;
The simplest answer is: you can't. The reason that it's called thread local storage is because only its thread can access it. Which, by definition, means that some other "central thread" can't get to it. That's what it's all about, by definition.
Now, depending on how your hardware and compiler platform implements TLS, there might be a trick around it, if your implemention of TLS works by mapping TLS variables to different virtual memory addresses. Typically, what happens is that one CPU register is thread-specific, it's set to point to different memory addresses, and all TLS variables are accessed as relative addresses.
If that is the case, you could, perhaps, derive some thread-safe mechanism by which each thread takes a pointer to its TLS variable, and puts it into a non-TLS container, that your "central thread" can get to.
And, of course, you must keep all of that in sync with your threads, and clean things up after each thread terminates.
You'll have to figure out whether this is the case on your platform with a trivial test: declare a TLS variable, then compare its pointer address in two different threads. If it's different, you might be able to work around it, in this fashion. Technically, this kind of pointer comparison is non-portable, and implementation defined, but by this time you're already far into implemention-specific behavior.
But if the addresses are the same, it means that your implementation uses virtual memory addressing to implement TLS. Only the executing thread has access to its TLS variable, period, and there is no practical means by which any "central thread" could look at other threads' TLS variables. It's enforced by your operating system kernel. The "central thread" must cooperate which each thread, and make arrangements to access the thread's TLS variables using typical means of interthread communications.
The cookie approach would work fine, and you don't need to use a TLS slot to implement it, just a local variable inside your thread procedure. To handle the case where the cookie changes value between the time that the thread is created and the time that it starts running (there is a small delay), you would have to pass the current cookie value as an input parameter for the thread creation, then your thread procedure can initialize its local variable to that value before it starts checking the live cookie for changes.
intptr_t g_cookie = 1;
pthread_rwlock_t g_lock;
void* thread_proc(void *arg)
{
intptr_t cookie = (intptr_t)arg;
while (keepRunningUntilSomeCondition)
{
pthread_rwlock_rdlock(&g_lock);
if (cookie != g_cookie)
{
cookie = g_cookie;
s_IsAllAboutThatBass = true;
}
pthread_rwlock_unlock(&g_lock);
//...
}
pthread_exit(NULL);
}
void createThread()
{
...
pthread_t thread;
pthread_create(&thread, NULL, &thread_proc, (void*)g_cookie);
...
}
void signalThreads()
{
pthread_rwlock_wrlock(&g_lock);
++g_cookie;
pthread_rwlock_unlock(&g_lock);
}
int main()
{
pthread_rwlock_init(&g_lock, NULL);
// use createThread() and signalThreads() as needed...
pthread_rwlock_destroy(&g_lock);
return 0;
}

Multithreading using pthread in C++ with shared variables

I'm new to threading (and C/C++ for that matter), and I'm attempting to use multiple threads to access shared variables.
In the main, I've created a variable char inputarray[100];
Thread 1: This thread will be reading data from stdin in 2 byte bursts, and appending them to the inputarray. (input by feeding a file in)
Thread 2: This thread will be reading data 1 byte at a time, performing a calculation, and putting its data into an output array.
Thread 3: This thread will be outputting data from the output array in 2 byte bursts. (stdout)
I've attempted the input part and got it working by passing a struct, but would like to do it without using a struct, but it has been giving me problems.
If I can get input down, I'm sure I'll be able to use a similar strategy to complete output. Any help would be greatly appreciated.
Below is a rough template for the input thread.
#include <stdio.h>
#include <pthread.h>
using namespace std;
void* input(void* arg) {
char reading[3];
fread(reading,1,2,stdin);
//append to char inputarray[]..???
}
int main() {
char inputarray[100];
pthread_t t1;
pthread_create(&t1, NULL, &input, &inputarray);
void *result;
pthread_join(t1,&result);
return 0;
}
Several issues:
I think array on stack is very bad choice for shared variable, because it has a fixed size and it's not clear from Thread 2 and 3 where to put new elements or where to read elements from. I would propose to use std::vector or std::deque instead.
Initially your container is empty. Then Thread 2 pushes some elements to it.
Thread 3 is polling (or waiting on condition variable) container, and once it found new elements - print them
You have to synchronize access to shared variable with mutex (consider pthread mutex, std::mutex or boost::mutex). You might also want to use condition variable to notify Thread 3 about new elements in the queue. But for initial implementation it's not needed.
Do you really have to use pthread primitives? Normally it's much easier and safer (i.e. exception safety) to use std::thread, std::mutex (if you have modern compiler), or boost::thread, boost::mutex otherwise.
You are on the correct track:
As a note the pthreads libraries are C libs so you need to declare the callbacks as C functions:
extern "C" void* input(void* arg);
Personally I would pass the address of the first element:
pthread_create(&t1, NULL, &input, &inputarray[0]);
This then makes your code look like this:
void* input(void* arg) {
try
{
char* inputarray = (char*)arg;
size_t inputLocation = 0;
// Need to make sure you don't over run the buffer etc...
while(!finished())
{
fread(&inputarray[inputLocation],1,2,stdin);
inputLocation += 2;
}
}
catch(...){} // Must not let exceptions escape a thread.
return NULL;
}
The trouble with this style is that you are putting the responsibility for coordination into each individual thread. The writer thread has to check for end the reader thread has to check there is data available etc. All this needs coordination so now you need some shared mutexes and condition variables.
A better choice is to move that responsibility into the object the does the communication. So I would create a class that has the basic operations needed for communication then make its methods do the appropriate checks.
class Buffer
{
public:
void write(......); //
void read(.....); //
private:
// All the synchronization and make sure the two threads
// behave nicely inside the object.
};
int main()
{
pthread_t threads[3];
std::pair<Buffer, Buffer> comms;
// comms.first inputToRead
// comms.second processesToOutput
pthread_create(&threads[0], NULL, &readInput, &comms.first); // Input
pthread_create(&threads[1], NULL, &procInput, &comms); // Processing
pthread_create(&threads[2], NULL, &genOutput, &comms.second); // Output
void *result;
pthread_join(threads[0],&result);
pthread_join(threads[1],&result);
pthread_join(threads[2],&result);
}
As a side note:
Unless there is something very strange about your processing of data. This would probably be faster written as a single threaded application.

Thread safe container

There is some exemplary class of container in pseudo code:
class Container
{
public:
Container(){}
~Container(){}
void add(data new)
{
// addition of data
}
data get(size_t which)
{
// returning some data
}
void remove(size_t which)
{
// delete specified object
}
private:
data d;
};
How this container can be made thread safe? I heard about mutexes - where these mutexes should be placed? Should mutex be static for a class or maybe in global scope? What is good library for this task in C++?
First of all mutexes should not be static for a class as long as you going to use more than one instance. There is many cases where you should or shouldn't use use them. So without seeing your code it's hard to say. Just remember, they are used to synchronise access to shared data. So it's wise to place them inside methods that modify or rely on object's state. In your case I would use one mutex to protect whole object and lock all three methods. Like:
class Container
{
public:
Container(){}
~Container(){}
void add(data new)
{
lock_guard<Mutex> lock(mutex);
// addition of data
}
data get(size_t which)
{
lock_guard<Mutex> lock(mutex);
// getting copy of value
// return that value
}
void remove(size_t which)
{
lock_guard<Mutex> lock(mutex);
// delete specified object
}
private:
data d;
Mutex mutex;
};
Intel Thread Building Blocks (TBB) provides a bunch of thread-safe container implementations for C++. It has been open sourced, you can download it from: http://threadingbuildingblocks.org/ver.php?fid=174 .
First: sharing mutable state between threads is hard. You should be using a library that has been audited and debugged.
Now that it is said, there are two different functional issue:
you want a container to provide safe atomic operations
you want a container to provide safe multiple operations
The idea of multiple operations is that multiple accesses to the same container must be executed successively, under the control of a single entity. They require the caller to "hold" the mutex for the duration of the transaction so that only it changes the state.
1. Atomic operations
This one appears simple:
add a mutex to the object
at the start of each method grab a mutex with a RAII lock
Unfortunately it's also plain wrong.
The issue is re-entrancy. It is likely that some methods will call other methods on the same object. If those once again attempt to grab the mutex, you get a dead lock.
It is possible to use re-entrant mutexes. They are a bit slower, but allow the same thread to lock a given mutex as much as it wants. The number of unlocks should match the number of locks, so once again, RAII.
Another approach is to use dispatching methods:
class C {
public:
void foo() { Lock lock(_mutex); foo_impl(); }]
private:
void foo_impl() { /* do something */; foo_impl(); }
};
The public methods are simple forwarders to private work-methods and simply lock. Then one just have to ensure that private methods never take the mutex...
Of course there are risks of accidentally calling a locking method from a work-method, in which case you deadlock. Read on to avoid this ;)
2. Multiple operations
The only way to achieve this is to have the caller hold the mutex.
The general method is simple:
add a mutex to the container
provide a handle on this method
cross your fingers that the caller will never forget to hold the mutex while accessing the class
I personally prefer a much saner approach.
First, I create a "bundle of data", which simply represents the class data (+ a mutex), and then I provide a Proxy, in charge of grabbing the mutex. The data is locked so that the proxy only may access the state.
class ContainerData {
protected:
friend class ContainerProxy;
Mutex _mutex;
void foo();
void bar();
private:
// some data
};
class ContainerProxy {
public:
ContainerProxy(ContainerData& data): _data(data), _lock(data._mutex) {}
void foo() { data.foo(); }
void bar() { foo(); data.bar(); }
};
Note that it is perfectly safe for the Proxy to call its own methods. The mutex will be released automatically by the destructor.
The mutex can still be reentrant if multiple Proxies are desired. But really, when multiple proxies are involved, it generally turns into a mess. In debug mode, it's also possible to add a "check" that the mutex is not already held by this thread (and assert if it is).
3. Reminder
Using locks is error-prone. Deadlocks are a common cause of error and occur as soon as you have two mutexes (or one and re-entrancy). When possible, prefer using higher level alternatives.
Add mutex as an instance variable of class. Initialize it in constructor, and lock it at the very begining of every method, including destructor, and unlock at the end of method. Adding global mutex for all instances of class (static member or just in gloabl scope) may be a performance penalty.
The is also a very nice collection of lock-free containers (including maps) by Max Khiszinsky
LibCDS1 Concurrent Data Structures
Here is the documentation page:
http://libcds.sourceforge.net/doc/index.html
It can be kind of intimidating to get started, because it is fully generic and requires you register a chosen garbage collection strategy and initialize that. Of course, the threading library is configurable and you need to initialize that as well :)
See the following links for some getting started info:
initialization of CDS and the threading manager
http://sourceforge.net/projects/libcds/forums/forum/1034512/topic/4600301/
the unit tests ((cd build && ./build.sh ----debug-test for debug build)
Here is base template for 'main':
#include <cds/threading/model.h> // threading manager
#include <cds/gc/hzp/hzp.h> // Hazard Pointer GC
int main()
{
// Initialize \p CDS library
cds::Initialize();
// Initialize Garbage collector(s) that you use
cds::gc::hzp::GarbageCollector::Construct();
// Attach main thread
// Note: it is needed if main thread can access to libcds containers
cds::threading::Manager::attachThread();
// Do some useful work
...
// Finish main thread - detaches internal control structures
cds::threading::Manager::detachThread();
// Terminate GCs
cds::gc::hzp::GarbageCollector::Destruct();
// Terminate \p CDS library
cds::Terminate();
}
Don't forget to attach any additional threads you are using:
#include <cds/threading/model.h>
int myThreadFunc(void *)
{
// initialize libcds thread control structures
cds::threading::Manager::attachThread();
// Now, you can work with GCs and libcds containers
....
// Finish working thread
cds::threading::Manager::detachThread();
}
1 (not to be confuse with Google's compact datastructures library)