How to use "pessimistic" Couchbase locking with the C API? - c++

This is incredibly frustrating... I'm using the Couchbase v3 C API, and I ran into a problem that would be solved perfectly by using their "pessimistic" locking. After much effort I believe I've finally figured out how to lock records using it (with lcb_cmdget_locktime(), which presumably takes a parameter in microseconds), but I can't figure out how to unlock the record once I'm done with it, other than by letting it time out, which isn't an acceptable solution.
The documentation provides one example, here, but for some unknown reason it's in Javascript instead of C (!!!), and the concepts don't map to the C API.
Does anyone know how to unlock a pessimistic lock, or have any example C/C++ code using that API? Barring that, does anyone know where to find the source code for any of the non-C APIs that use the C one, since I should be able to work it out from there? (I haven't been able to locate any of those either, not sure they're open-source.)

Thank you for question
Time in seconds, note that the server might reset time to default, if it larger than maximum time (both durations are configurable). The following command will help to discover effective values for the feature.
$ cbstats -u Administrator -p password localhost all | grep ep_getl
ep_getl_default_timeout: 15
ep_getl_max_timeout: 30
To lock the key, one must use get operation with and set lock time using lcb_cmdget_locktime and capture CAS value in case of successful lock, for instance like this
struct my_result {
lcb_STATUS status{LCB_SUCCESS};
uint64_t cas{0};
};
static void get_callback(lcb_INSTANCE *instance, lcb_CALLBACK_TYPE, const lcb_RESPGET *resp)
{
my_result *res = nullptr;
lcb_respget_cookie(resp, (void **)&res);
res->status = lcb_respget_status(resp);
if (res->status == LCB_SUCCESS) {
lcb_respget_cas(resp, &res->cas);
}
}
It is good idea to put get with lock code into loop
uint64_t locked_cas{0};
int retries = 3;
while (retries > 0) {
std::string document_id{"foo"};
my_result result{};
lcb_CMDGET* cmd = nullptr;
lcb_cmdget_create(&cmd);
lcb_cmdget_key(cmd, document_id.c_str(), document_id.size());
lcb_cmdget_locktime(cmd, 5);
lcb_get(instance, &result, cmd);
lcb_cmdget_destroy(cmd);
lcb_wait(instance, LCB_WAIT_DEFAULT);
if (result.rc == LCB_SUCCESS) {
locked_cas = result.cas;
break;
} else if (result.rc == LCB_ERR_DOCUMENT_LOCKED || result.rc == LCB_ERR_TEMPORARY_FAILURE) {
std::this_thread::sleep_for(std::chrono::milliseconds(100));
--retries;
continue;
} else {
std::cerr << "Unexpected issue during get with lock: " << lcb_strerror_short(result.rc) << "\n";
break;
}
}
Once the key got locked, the update operation might be performed, but keep in mind that it MUST use locked_cas, otherwise mutation operation will fail.
std::string document_id{"foo"};
std::string new_value{"new value"};
lcb_CMDSTORE* cmd = nullptr;
lcb_cmdstore_create(&cmd, LCB_STORE_REPLACE);
lcb_cmdstore_key(cmd, document_id.c_str(), document_id.size());
lcb_cmdstore_value(cmd, new_value.c_str(), new_value.size());
lcb_cmdstore_cas(cmd, locked_cas);
lcb_store(instance, nullptr, cmd);
lcb_cmdstore_destroy(cmd);
lcb_wait(instance, LCB_WAIT_DEFAULT);
To unlock key you also need locked_cas, or just wait until server will unlock the document automatically.
std::string document_id{"foo"};
lcb_CMDUNLOCK *cmd = nullptr;
lcb_cmdunlock_create(&cmd);
lcb_cmdunlock_key(cmd, document_id.c_str(), document_id.size());
lcb_cmdunlock_cas(cmd, locked_cas);
lcb_unlock(instance, nullptr, cmd);
lcb_cmdunlock_destroy(cmd);
lcb_wait(instance, LCB_WAIT_DEFAULT);

Related

How to read ZeroMQ return values from .recv() and .send() methods in c++?

I'm trying to write a c++ class for communicating between two computers via ZeroMQ.
To be able to handle errors I am trying to read the return values of the .recv()- and .send()- methods but I get the following error
error: cannot convert 'zmq::send_result_t' {aka 'zmq::detail::trivial_optional<unsigned int>'} to 'int' in assignment
ret = msocket.send(reply, zmq::send_flags::none);
The code looks like this:
Publisher::Publisher(dataHandler & mdatahandler) :datahandler(mdatahandler)
{
// construct a REP (reply) socket and bind to interface
socketState.bind("tcp://*:5555");
//socketAngles.bind("tcp://*:5556");
//socketCurrents.bind("tcp://*:5557");
}
Publisher::~Publisher()
{
socketState.close();
//socketAngles.close();
//socketCurrents.close();
}
std::string Publisher::transfer(zmq::socket_t& msocket, std::string replyString,
int receiveFlag = 0)
{
zmq::send_result_t ret = 0;
if (receiveFlag)
{
zmq::message_t receivedData;
ret = msocket.recv(receivedData, zmq::recv_flags::none);
if (verbose)
{
std::cout << "Received " << receivedData.to_string() << std::endl;
}
return receivedData.to_string();
}
zmq::message_t reply{ replyString.cbegin(), replyString.cend() };
// send the reply to the client
ret = msocket.send(reply, zmq::send_flags::none);
if (ret == -1)
{
std::cout << zmq_strerror(errno) << std::endl;
}
}
the socket is defined as
zmq::context_t context{ 1 };
zmq::socket_t socketState{ context, ZMQ_REP };
How can I reliably catch errors and is there a better way of handling errors if they occur?
Edit:I added the zmq::send_result_t but how can I do anything with it? I can't compare it to anything and I can't print it either.
zmq::recv_result_t is based on trivial_optional<unsigned int>.
trivial_optional<T> is a class template that encapsulates that it may or may not contain a value. Instances of type trivial_optional<unsigned int> are interrogated with bool trivial_optional<unsigned int>::has_value() to see if there is a value.
If there is a value it is extracted using T trivial_optional<unsigned int>::operator*() or T trivial_optional<unsigned int>::value().
zmq::recv_result_t ret(msocket.recv(receivedData, zmq::recv_flags::none));
if (ret.has_value() && (EAGAIN == ret.value()))
{
// msocket had nothing to read and recv() timed out
....
}
Q : "How can I reliably catch errors and is there a better way of handling errors if they occur?"
This can be split into two questions:
Part A: "How can I reliably catch errors"
First understand the language tools. There are exceptions related tools and best-practices and other Do-s and Don't-s. Obey them.
Part B: "a better way of handling errors"
The best way of handling errors is by avoiding them completely - this does not save the Planet ( you can read about Ms. Margaret Hamilton ( she saved lives and national pride on doing this correctly for the Apollo Guidance Computer software ) and her genuine methodology, that saves unavoidable principally colliding cases ).
The next, a lot weaker strategy is to design architectures ( then code ), that thoroughly inspects the state of the system ( return values, RTT-times and other factors ), so as to be continuously ready to handle Exception, as it happens and in full-context with the state of the system ( not to find yourself surprised as standing as uninformed as a blind person in the middle of the crossroads, once the Exception was thrown ... and it will be thrown, at some later time, so be prepared a-priori, not panicking ex-post, resorting to but chaotic ad-hoc options )
Solution :
Step 1) Understand and master the language tools.
Step 2) Understand and master the ZeroMQ tools ( No REP can ever start with .send() )
Step 3) Understand and master the published ZeroMQ API, there are all details on details needed for successful Exception handling details ( preventive error-state indications - hidden gems in { EINVAL | ETERM | ENOTSOCK | EINTR | ... } error-states, explained for each and every API call method, in due context for each one such method.
If still not convinced, at least read the fabulous Pieter Hintjens' book "Code Connected, Volume 1", there one will get the roots of what the Zen-of-Zero is all about.

Implementing/replacement of volatile std::string

As already noted here using volatile std::string isn't a good idea.
I'm developing an application on FreeRTOS and I need to have a string which is moved between tasks. There is one task which receives commands through UART and can be asked by other tasks to get the response on a specified command. I want to use std::move on a string to make the application optimal.
Is there a neat and fast replacement of volatile std::string or do I have to implement a class with volatile field on my own? Maybe this approach is bad and I should use another structure to handle moving around responses on the commands?
EDIT: Here's some code.
I get the single bytes of commands through interrupt. The commands are human readable commands terminated with \r.
void rx_interrupt(char c)
{
if(c == '\r')
{
c == '\0')
BaseType_t higher_prior_task_woken = pdFALSE;
vTaskNotifyGiveFromISR(rx_task_handle, &higher_prior_task_woken);
portYIELD_FROM_ISR(higher_prior_task_woken);
}
rx_buf.push_byte(c);
}
rx_buf is a circular buffer which allows to pop whole commands as std::string.
Then the rx_task:
for (;;)
{
auto notif_num = ulTaskNotifyTake(pdTRUE, portMAX_DELAY);
while (notif_num--)
{
auto comm = rx_buf.pop_command();
if (comm.length() == 0)
continue;
if (is_unsolicited_command(comm))
handle_unsolicited_command(std::move(comm));
if (is_awaited_command(comm))
handle_awaited_command(std::move(comm));
}
}
The rx_task is needed because I must firstly check whether an asynchronous event occurred which is indicated by an unsolicited command.
The received (awaited) commands may be long so I want to move them.
void handle_awaited_command(std::string &&cmd)
{
os_lockguard guard(var_mux);
if (!awaiting)
return;
awaited_command = std::move(cmd); // Problematic line
xSemaphoreGive(cmd_received_sem);
awaited_cmd_handled = true;
}
Finally any of the other task may await a command:
std::string get_command()
{
os_lockguard guard_glob(mux);
{
os_lockguard guard(var_mux);
awaiting = true;
}
xSemaphoreTake(cmd_received_sem, timeout);
{
os_lockguard guard(var_mux);
if(awaited_cmd_handled)
return std::move(awaited_command); // Problematic line
else
return std::string("");
}
}
The thing is that the definitions looks like that:
volatile bool awaiting;
volatile bool awaited_cmd_handled;
volatile std::string awaited_command;
So I have here a volatile std::string.

Object pointer randomly points to 0x00000

I'm running a Visual C++ MFC application in release mode. I'm compiling everything using Visual Studio 2010.
My app runs a mini CNC mill through USB VCP communication.
I have a XML file that stores the app's settings.
My problem is this: ocassionaly (and this is repeatable) the pointer to the tinyxml2::XMLDocument I'm using gets set to 0x000.
Info:
Occasionally, the XML file get written to while the mill is running.
Before the error happens, the mill I'm running siezes for almost 30 seconds.
I'm using mutex locks to make sure the xmldoc doesn't get written to file twice at once.
The mutex locks are working, and the mutex error never occurs. I know the mutex code isn't perfect, but that isn't the issue. Honest.
I never write to the xmldoc pointer except when the parent class is booting up.
And then, all of a sudden, the xmlDoc pointer gets set to zero.
Any thoughts anyone?
Here is my saving code, although the problem may lie elsewhere:
void XMLSettings::SaveToXML()
{
HANDLE g_Mutex = CreateMutex( NULL, TRUE, "XMLSavingMutex");
DWORD wait_success = WaitForSingleObject( g_Mutex, 30000L);
if(wait_success == WAIT_OBJECT_0){
CIsoProApp* pApp = (CIsoProApp*)AfxGetApp();
if(PathFileExists(pApp->DrivePath + "IsoPro\\temp.xml"))
{
DeleteFile(pApp->DrivePath + "IsoPro\\temp.xml");
}
if(0==&xmlDoc)
{
OutputDebugString("xmlDoc == NULL");
}
int errorcode = xmlDoc->SaveFile(pApp->DrivePath + "IsoPro\\temp.xml");
if(errorcode != 0)
{
OutputDebugString("xmlDoc == errorcode");
}
if(0==&xmlDoc)
{
OutputDebugString("xmlDoc == NULL2");
}
if(0==xmlDoc)
{
OutputDebugString("xmlDoc == NULL");
}
if(PathFileExists(pApp->DrivePath + "IsoPro\\Settings.xml"))
{
DeleteFile(pApp->DrivePath + "IsoPro\\Settings.xml");
}
MoveFile(pApp->DrivePath + "IsoPro\\temp.xml",pApp->DrivePath + "IsoPro\\Settings.xml");
ReleaseMutex(g_Mutex);
}
else
{
int errorInt = GetLastError();
CString error;
error.Format("%d",errorInt);
if(errorInt != ERROR_ALREADY_EXISTS)
{
AfxMessageBox("XMLSavingMutex Error. WaitSuccess = " + wait_success);
AfxMessageBox("XMLSavingMutex Error. GetLastError = " + error);
}
}
CloseHandle(g_Mutex);
}
Since it seems that you are creating a Mutex each time SaveToXML is called, you should change your call to
HANDLE g_Mutex = CreateMutex( NULL, FALSE, "XMLSavingMutex");
Doing this will create a named mutex that allows the implementation to dictate who the owner is; other threads will receive the same mutex.
From the doc:
Two or more processes can call CreateMutex to create the same named mutex. The first process actually creates the mutex, and subsequent processes with sufficient access rights simply open a handle to the existing mutex. This enables multiple processes to get handles of the same mutex, while relieving the user of the responsibility of ensuring that the creating process is started first. When using this technique, you should set the bInitialOwner flag to FALSE; otherwise, it can be difficult to be certain which process has initial ownership.
(Credit to WhozCraig for pointing out named mutexes)
It appears that I was accessing the xml getter while writing the xml to a file. I put a single mutex lock in place for all xml actions and things seem to be functioning properly. Thanks to everyone for their help. I'll be in touch with more info if it becomes available.

How to edit SIM800l library to ensure that a call is established

I use SIM800l to make calls with arduino UNO with AT commands. By using this library I make calls with gprsTest.callUp(number) function. The problem is that it returns true even the number is wrong or there is no credit.
It is clear on this part code from GPRS_Shield_Arduino.cpp library why it is happening. It doesnt check the return of ATDnumberhere;
bool GPRS::callUp(char *number)
{
//char cmd[24];
if(!sim900_check_with_cmd("AT+COLP=1\r\n","OK\r\n",CMD)) {
return false;
}
delay(1000);
//HACERR quitar SPRINTF para ahorar memoria ???
//sprintf(cmd,"ATD%s;\r\n", number);
//sim900_send_cmd(cmd);
sim900_send_cmd("ATD");
sim900_send_cmd(number);
sim900_send_cmd(";\r\n");
return true;
}
The return of ATDnumberhere; on software serial communication is:
If number is wrong
ERROR
If there is no credit
`MO CONNECTED //instant response
+COLP: "003069XXXXXXXX",129,"",0,"" // after 3 sec
OK`
If it is calling and no answer
MO RING //instant response, it is ringing
NO ANSWER // after some sec
If it is calling and hang up
MO RING //instant response
NO CARRIER // after some sec
If the receiver has not carrier
ATD6985952400;
NO CARRIER
If it is calling , answer and hang up
MO RING
MO CONNECTED
+COLP: "69XXXXXXXX",129,"",0,""
OK
NO CARRIER
The question is how to use different returns for every case by this function gprsTest.callUp(number) , or at least how to return true if it is ringing ?
This library code seems better than the worst I have seen at first glance, but it still have some issues. The most severe is its Final result code handling.
The sim900_check_with_cmd function is conceptually almost there, however only checking for OK is in no way acceptable. It should check for every single possible final result code the modem might send.
From your output examples you have the following final result codes
OK
ERROR
NO CARRIER
NO ANSWER
but there exists a few more as well. You can look at the code for atinout for an example of a is_final_result_code function (you can also compare to isFinalResponseError and isFinalResponseSuccess1 in ST-Ericsson's U300 RIL).
The unconditional return true; at the end of GPRS::callUp is an error, but it might be deliberate due to lack of ideas for implementing a better API so that the calling client could check the intermediate result codes. But that is such a wrong way to do it.
The library really should do all the stateful command line invocation and final result code parsing with no exceptions. Just doing parts of that in the library and leaving some of it up to the client is just bad design.
When clients want to inspect or act on intermediate result codes or information text that comes between the command line and the final result code, the correct way to do it is to let the library "deframe" everything it receives from the modem into individual complete lines, and for everything that is not a final result code provide this to the client through a callback function.
The following is from an unfinished update to my atinout program:
bool send_commandline(
const char *cmdline,
const char *prefix,
void (*handler)(const char *response_line, void *ptr),
void *ptr,
FILE *modem)
{
int res;
char response_line[1024];
DEBUG(DEBUG_MODEM_WRITE, ">%s\n", cmdline);
res = fputs(cmdline, modem);
if (res < 0) {
error(ERR "failed to send '%s' to modem (res = %d)", cmdline, res);
return false;
}
/*
* Adding a tiny delay here to avoid losing input data which
* sometimes happens when immediately jumping into reading
* responses from the modem.
*/
sleep_milliseconds(200);
do {
const char *line;
line = fgets(response_line, (int)sizeof(response_line), modem);
if (line == NULL) {
error(ERR "EOF from modem");
return false;
}
DEBUG(DEBUG_MODEM_READ, "<%s\n", line);
if (prefix[0] == '\0') {
handler(response_line, ptr);
} else if (STARTS_WITH(response_line, prefix)) {
handler(response_line + strlen(prefix) + strlen(" "), ptr);
}
} while (! is_final_result(response_line));
return strcmp(response_line, "OK\r\n") == 0;
}
You can use that as a basis for implementing proper handling. If you want to
get error responses out of the function, add an additional callback argument and change to
success = strcmp(response_line, "OK\r\n") == 0;
if (!success) {
error_handler(response_line, ptr);
}
return success;
Tip: Read all of chapter 5 in the V.250 specification, it will teach you almost everything you need to know about command lines, result codes and response handling. Like for instance that a command line should also be terminated with \r only, not \r\n-
1 Note that CONNECT is not a final result code, it is an intermediate result code, so the name isFinalResponseSuccess is strictly speaking not 100% correct.

Libcurl returns timeout error when used from multiple threads, but not from a single thread

I'm using libcurl with C++. I have made a thread-safe class for downloading webpages. Each call to static download method creates "easy" handle, performs the job and frees the handle. When I use it from the single thread - everything's fine. But when I spawn several threads to download several pages in parallel I sometimes (not fro every download, but quite often) get error saying "timeout". I have a reasonably high timeout configured (5 sec connection timeout and 25 sec global timeout).
Any ideas as to what might be the cause and how to fix it?
P. S. It happens on both Windows and Linux.
Here's the code of the method in question:
void CHttpDownloaderLibcurl::downloaderThread( const CUrl& url, CThreadSafeQueue<CHtmlPage>& q)
{
CHtmlPage page (url);
CURL* handle = curl_easy_init();
if (!handle)
{
assert(handle);
return;
}
int curlErr = setCurlOptions(handle, url, (void*)onCurlDownloadCallback, (void*)&page.byteArray());
if (CURLE_OK != curlErr)
{
assert("Error setting options" == (char*)curlErr);
return;
}
curlErr = curl_easy_perform(handle);
page._info = getInfo(handle);
curl_easy_cleanup(handle);
if (CURLE_OK != curlErr)
{
if (curlErr == CURLE_OPERATION_TIMEDOUT)
{
CLogger() << "Curl timeout!";
}
else
CLogger() << url.urlString() << ": Error performing download = " << curlErr;
return;
}
q.push(page);
}