I am writing a multi-threaded application in C++ using Boost threads (pthread). The application spawns 100 threads and each thread does the following task (I am writing a code snippet that will be running in each thread):
try {
driver = get_driver_instance();
con = driver->connect(SettingsClass.HostName, \
SettingsClass.UserName,SettingsClass.Password);
// SettingsClass is a global static class whose members
// (HostName, UserName, Password, etc) are initialized once
// before *any* thread is created.
con->setSchema("MyDatabase");
driver->threadInit();
string dbQuery = "select A, B, C from XYZTable where D=?";
prepStmt = con->prepareStatement(dbQuery);
prepStmt->setInt(1, 1);
rSet = prepStmt->executeQuery();
/* Do Something With rSet, the result set */
delete rSet;
delete prepStmt;
if (con != NULL && !con->isClosed()) {
con -> close();
driver->threadEnd();
delete con;
}
catch (SQLException &e)
{
/* Log Exception */
}
On running the process (the app, as earlier mentioned, i.e. with 100 such threads), I attach gdb midway and observe that more than 40% of the threads have hanged in the read() call. All the backtraces have mysql library functions (vio_read(), etc) and none are from my code as my code does not perform any I/O.
Could anyone point out why is this issue arising. Should I check my code / network or MySQL server configuration? Have I used the C++ connector library properly?
Related
So I'm trying to adapt a CPP library for ESP-IDF/Arduino BLE control into ESP-IDF v5.0 but I'm getting some errors regarding Semaphore from FreeRTOS when waiting. This is most likerly due to some APIs changing between ESP-IDF 3.0 to 5.0 but since Im just learning FreeRTOS I got a bit lost with the error.
Some logs from the program before the crash:
I (2376) BLEClient: >> connect(a6:c0:82:01:17:72)
I (2386) : 119660
I (2386) BLEDevice: add conn_id: 0, GATT role: client
I (2386) FreeRTOS: Semaphore taking: name: RegEvt (0x3ffdb0d4), owner: <N/A> for connect
I (2396) FreeRTOS: Semaphore taken: name: RegEvt (0x3ffdb0d4), owner: connect
I (2406) : 119628
assert failed: xQueueGenericSend queue.c:837 (pxQueue->pcHead != ((void *)0) || pxQueue->u.xSemaphore.xMutexHolder == ((void *)0) || pxQueue->u.xSemaphore.xMutexHolder == xTaskGetCurrentTaskHandle())
The traceback:
Backtrace: 0x40081d1e:0x3ffdacb0 0x40090699:0x3ffdacd0 0x4009687d:0x3ffdacf0 0x40090e62:0x3ffdae10 0x400dc597:0x3ffdae50 0x400d94d9:0x3ffdae90 0x400da145:0x3ffdaf10 0x400e0226:0x3ffdaf60 0x400e0922:0x3ffdaf80 0x400f73fd:0x3ffdafc0 0x400f95ab:0x3ffdafe0
0x40081d1e: panic_abort at C:/Espressif/frameworks/esp-idf-v5.0/components/esp_system/panic.c:412
0x40090699: esp_system_abort at C:/Espressif/frameworks/esp-idf-v5.0/components/esp_system/esp_system.c:135
0x4009687d: __assert_func at C:/Espressif/frameworks/esp-idf-v5.0/components/newlib/assert.c:78
0x40090e62: xQueueGenericSend at C:/Espressif/frameworks/esp-idf-v5.0/components/freertos/FreeRTOS-Kernel/queue.c:828 (discriminator 2)
0x400dc597: FreeRTOS::Semaphore::give() at C:/Users/amng8/Sync/PROJECTS/PillCPP/components/BLE/FreeRTOS.cpp:107
0x400d94d9: BLEClient::gattClientEventHandler(esp_gattc_cb_event_t, unsigned char, esp_ble_gattc_cb_param_t*) at C:/Users/amng8/Sync/PROJECTS/PillCPP/components/BLE/BLEClient.cpp:233
0x400da145: BLEDevice::gattClientEventHandler(esp_gattc_cb_event_t, unsigned char, esp_ble_gattc_cb_param_t*) at C:/Users/amng8/Sync/PROJECTS/PillCPP/components/BLE/BLEDevice.cpp:163
0x400e0226: btc_gattc_cb_to_app at C:/Espressif/frameworks/esp-idf-v5.0/components/bt/host/bluedroid/btc/profile/std/gatt/btc_gattc.c:24
0x400e0922: btc_gattc_cb_handler at C:/Espressif/frameworks/esp-idf-v5.0/components/bt/host/bluedroid/btc/profile/std/gatt/btc_gattc.c:1002
0x400f73fd: btc_thread_handler at C:/Espressif/frameworks/esp-idf-v5.0/components/bt/common/btc/core/btc_task.c:207
ELF file SHA256: b8542ed11e07aba6
0x400f95ab: osi_thread_run at C:/Espressif/frameworks/esp-idf-v5.0/components/bt/common/osi/thread.c:165
The code in question is from the file BLEClient.cpp function bool BLEClient::connect(BLEAddress address, esp_ble_addr_type_t type):
/**
* #brief Connect to the partner (BLE Server).
* #param [in] address The address of the partner.
* #return True on success.
*/
bool BLEClient::connect(BLEAddress address, esp_ble_addr_type_t type) {
ESP_LOGI(LOG_TAG, ">> connect(%s)", address.toString().c_str());
// We need the connection handle that we get from registering the application. We register the app
// and then block on its completion. When the event has arrived, we will have the handle.
m_appId = BLEDevice::m_appId++;
BLEDevice::addPeerDevice(this, true, m_appId);
m_semaphoreRegEvt.take("connect");
// clearServices(); // we dont need to delete services since every client is unique?
esp_err_t errRc = ::esp_ble_gattc_app_register(m_appId);
if (errRc != ESP_OK) {
ESP_LOGE(LOG_TAG, "esp_ble_gattc_app_register: rc=%d %s", errRc, GeneralUtils::errorToString(errRc));
return false;
}
m_semaphoreRegEvt.wait("connect");
m_peerAddress = address;
// Perform the open connection request against the target BLE Server.
m_semaphoreOpenEvt.take("connect");
errRc = ::esp_ble_gattc_open(
m_gattc_if,
*getPeerAddress().getNative(), // address
type, // Note: This was added on 2018-04-03 when the latest ESP-IDF was detected to have changed the signature.
1 // direct connection <-- maybe needs to be changed in case of direct indirect connection???
);
if (errRc != ESP_OK) {
ESP_LOGE(LOG_TAG, "esp_ble_gattc_open: rc=%d %s", errRc, GeneralUtils::errorToString(errRc));
return false;
}
uint32_t rc = m_semaphoreOpenEvt.wait("connect"); // Wait for the connection to complete.
ESP_LOGD(LOG_TAG, "<< connect(), rc=%d", rc==ESP_GATT_OK);
return rc == ESP_GATT_OK;
} // connect
and what seems to be triggering the crash is the .wait("connect") command.
Once again I'm new to FreeRTOS and Semaphore and not sure what's useful or not to get help so if you need any more information please let me know.
I've been trying to use the mysql++ library in my application (windows x64 based) but I can't seem to connect to my sql server.
Some information:
I used this code to connect to the server:
mysqlpp::Connection conn(db, 0, user, pass, 3306);
this definitely has the right data in it.
and then, my sql server is the standard service from the MySQL install. And I'm pretty sure I used the standard settings. I can connect to it using the MySql Workbench and I edited some new tables and such but my own program doesn't seem to connect.
I read the documentation and I can't find anything specific that might suggest something why I can't connect.
Oh, so many issues, so little time...
Have you checked that your program has permissions to access the database?
Does your program have the correct privileges?
Is your host name correct?
What errors are you getting?
What exception is thrown?
When you use the debugger, what line is the error on?
Here's my method:
sql::Connection * const
Manager ::
get_db_connection(void) const
{
//-------------------------------------------------------------------------
// Use only one connection until proven that more connections will make
// the program more efficient or have a beneficial impact on the user.
// Thus the change in returning sql::Connection * rather than a smart pointer.
// A smart pointer will delete its contents.
//-------------------------------------------------------------------------
static const char host_text[] = "tcp://127.0.0.1:3306/";
static std::string host_name;
if (!m_connection_initialized)
{
host_name = host_text;
initialize_db_driver();
host_name += m_dataset_info.m_dsn_name;
try
{
m_p_connection = m_p_sql_driver->connect(host_name.c_str(),
m_dataset_info.m_user_name.c_str(),
m_dataset_info.m_password.c_str());
}
catch (sql::SQLException &e)
{
/*
The MySQL Connector/C++ throws three different exceptions:
- sql::MethodNotImplementedException (derived from sql::SQLException)
- sql::InvalidArgumentException (derived from sql::SQLException)
- sql::SQLException (derived from std::runtime_error)
*/
wxString wx_text = wxT("# ERR: SQLException in ");
wx_text += wxT(__FILE__);
wxLogDebug(wx_text);
wx_text.Printf(wxT("# ERR: (%s) on line %d"),
__FUNCTION__,
__LINE__);
wxLogDebug(wx_text);
wx_text.Printf(wxT("# ERR: %s (MySQL error code: %d, SQLState: %s)"),
e.what(),
e.getErrorCode(),
e.getSQLState());
wxLogDebug(wx_text);
wxLogDebug(wxT("Verify that mysqlcppconn.dll is in the PATH or in the working directory."));
// throw Manager_Connection_Not_Initialized();
m_connection_initialized = false;
}
catch (...)
{
std::cout << "Unhandled database SQL exception\n" << flush;
m_connection_initialized = false;
}
m_connection_initialized = true;
}
return m_p_connection;
}
I have this code in a big project that makes a connection to a MySQL database, and does not work:
boost::shared_ptr<sql::Connection> connection;
sql::Driver *driver = get_driver_instance();
assert(driver != 0);
std::string server = "servname", user = "pietro", password = "abc";
try
{
connection.reset(driver->connect(server, user, password));
assert(connection != 0);
if(connection->isClosed() == false) // <-- segmentation fault
{
}
}
I get a segmentation fault where indicated (all parameters are valid).
However, this same code works in a test project.
Going into the sql::Connection::isClosed() member function with a debugger, I obtain no information about the possible cause; here is where I get:
mysql-connector-c++-1.0.5/driver/mysql_connection.cpp - line 430
/* {{{ MySQL_Connection::checkClosed() -I- */
void
MySQL_Connection::checkClosed()
{
CPP_ENTER_WL(intern->logger, "MySQL_Connection::checkClosed");
if (!intern->is_valid) {
throw sql::SQLException("Connection has been closed");
}
}
This checkClosed() function is successfully executed seven times from connection.reset() just before. The value of the "intern" pointer does not change and is not null at this stage.
When I check if the connection is closed, the checkClosed() function is run again. Now the "intern" pointer value is 0x8, a location I cannot access.
Here I get a SIGSEGV, segmentation fault.
Let me know if you would like the disassembled code...
Platform:
MySQL 5.x
MySQL Connector/C++ 1.0.5
Linux - OpenSuse 11.4
P.S.:
I noticed that all the shared_ptr's member functions work as expected:
connection.get(); // = 0x8fb4a0
connection.use_count(); // = 1
connection.unique(); // = 1
while all the calls done on the pointed object cause the segmentation fault (SIGABRT):
connection->getClientInfo();
connection->isClosed();
either your connection deleted, or you have memory corruption. use valgrind to find answer
I started to program client/server applications in J2ME recently.Now I'm working with c++ builder 2010 indy components (e.g. TidTTCPServer) and J2ME. My application is designed to restart the kerio winroute firewall service from a remote machine.
My server application is written in c++ builder 2010, I've put a TidTCTServer component into a form which binded to 127.0.0.1:4500. That's listening on port 4500 in local machine.
Then i've added a listbox that i need to add every upcoming packets converted to UnicodeString.
//void __fastcall TForm1::servExecute(TIdContext *AContext)
UnicodeString s;
UnicodeString txt;
txt=Trim(AContext->Connection->IOHandler->ReadLn());
otvet->Items->Add(txt);
otvet->ItemIndex=otvet->Items->Count-1;
if (txt=="1") {
AContext->Connection->IOHandler->WriteLn("Suhrob");
AContext->Connection->Disconnect();
}
if (txt=="2") {
AContext->Connection->IOHandler->WriteLn("Shodi");
AContext->Connection->Disconnect();
}
//----------------------------------------------------------------------------
// void __fastcall TForm1::servConnect(TIdContext *AContext)
++counter;
status->Panels->Items[0]->Text="Connections:" + IntToStr(counter);
status->Panels->Items[1]->Text="Connected to " + AContext->Connection->Socket->Binding->PeerIP + ":" + AContext->Connection->Socket->Binding->PeerPort;
and my client side code looks smth like this:
else if (command == send) {
// write pre-action user code here
InputStream is=null;
OutputStream os=null;
SocketConnection client=null;
ServerSocketConnection server=null;
try {
server = (ServerSocketConnection) Connector.open("socket://"+IP.getString()+":"+PORT.getString());
// wait for a connection
client = (SocketConnection) Connector.open("socket://"+IP.getString()+":"+PORT.getString());
// set application-specific options on the socket. Call setSocketOption to set other options
client.setSocketOption(SocketConnection.DELAY, 0);
client.setSocketOption(SocketConnection.KEEPALIVE, 0);
is = client.openInputStream();
os = client.openOutputStream();
// send something to server
os.write("texttosend".getBytes());
// read server response
int c = 0;
while((c = is.read()) != -1) {
// do something with the response
System.out.println((char)c);
}
// close streams and connection
}
catch( ConnectionNotFoundException error )
{
Alert alert = new Alert(
"Error", "Not responding!", null, null);
alert.setTimeout(Alert.FOREVER);
alert.setType(AlertType.ERROR);
switchDisplayable(alert, list);
}
catch (IOException e)
{
Alert alert = new Alert("ERror", e.toString(), null, null);
alert.setTimeout(Alert.FOREVER);
alert.setType(AlertType.ERROR);
switchDisplayable(alert, list);
e.printStackTrace();
}
finally {
if (is != null) {
try {
is.close();
} catch (Exception ex) {
System.out.println("Failed to close is!");
}
try {
os.close();
} catch (Exception ex) {
System.out.println("Failed to close os!");
}
}
if (server != null) {
try {
server.close();
} catch (Exception ex) {
System.out.println("Failed to close server!");
}
}
if (client != null) {
try {
client.close();
} catch (Exception ex) {
System.out.println("Failed to close client!");
}
}
}
my client application gets connected with the server but when i try to send data such as
os.write("texttosend".getBytes());
I cannot get text data on the server using. That's I am not getting sent packets in the server from client.
txt=Trim(AContext->Connection->IOHandler->ReadLn());
Guys, where am I wrong? is the way i'm doing is ok?
Or do I need to use StreamConnection instead of SocketConnection?
And when i use telnet to send data it works cool, strings will be added to listbox
telnet 127.0.0.1 4500
texttosend
23
asf
Any help is appreciated !!!
Thanks in advance!
The main problem is that you are using ReadLn() on the server end. ReadLn() does not exit until a data terminator is encountered (a LF line break character is the default terminator) or if a reading timeout occurs (Indy uses infinite timeouts by default). Your J2ME code is not sending any data terminator, so there is nothing to tell ReadLn() when to stop reading. The reason it works with Telnet is because it does send line break characters.
The other problem with your code is that TIdTCPServer is a multi-threaded component, but your code is updating the UI components in a thread-unsafe manner. You MUST synchronize with the main thread, such as by using Indy's TIdSync and/or TIdNotify classes, in order to update your UI safely from inside of the server's event handlers.
Yes, flush method is necessary to call after sending bytes, but ..... finally....
then i tried to include my connection code in a new thread that implements Runnable worked perfectly. Now I've found where I was wrong!!!!!!!!!!!!!!
That's guys you need to include above code in the following block.
Thread t= new Thread(this);
t.start();
public void run()
{
//here paste the code
}
Try OutputStream.flush()?
If not, try writing to a known working server, instead of one you've created yourself (something like writing "HELO" to an SMTP server), this will help you figure out which end the error is at.
I'm not sure if this is a known issue that I am running into, but I couldn't find a good search string that would give me any useful results.
Anyway, here's the basic rundown:
we've got a relatively simple application that takes data from a source (DB or file) and streams that data over TCP to connected clients as new data comes in. its a relatively low number of clients; i would say at max 10 clients per server, so we have the following rough design:
client: connect to server, set to read (with timeout set to higher than the server heartbeat message frequency). It blocks on read.
server: one listening thread that accepts connections and then spawns a writer thread to read from the data source and write to the client. The writer thread is also detached(using boost::thread so just call the .detach() function). It blocks on writes indefinetly, but does check errno for errors before writing. We start the servers using a single perl script and calling "fork" for each server process.
The problem(s):
at seemingly random times, the client will shutdown with a "connection terminated (SUCCESFUL)" indicating that the remote server shutdown the socket on purpose. However, when this happens the SERVER application ALSO closes, without any errors or anything. it just crashes.
Now, to further the problem, we have multiple instances of the server app being started by a startup script running different files and different ports. When ONE of the servers crashes like this, ALL the servers crash out.
Both the server and client using the same "Connection" library created in-house. It's mostly a C++ wrapper for the C socket calls.
here's some rough code for the write and read function in the Connection libary:
int connectionTimeout_read = 60 * 60 * 1000;
int Socket::readUntil(char* buf, int amount) const
{
int readyFds = epoll_wait(epfd,epEvents,1,connectionTimeout_read);
if(readyFds < 0)
{
status = convertFlagToStatus(errno);
return 0;
}
if(readyFds == 0)
{
status = CONNECTION_TIMEOUT;
return 0;
}
int fd = epEvents[0].data.fd;
if( fd != socket)
{
status = CONNECTION_INCORRECT_SOCKET;
return 0;
}
int rec = recv(fd,buf,amount,MSG_WAITALL);
if(rec == 0)
status = CONNECTION_CLOSED;
else if(rec < 0)
status = convertFlagToStatus(errno);
else
status = CONNECTION_NORMAL;
lastReadBytes = rec;
return rec;
}
int Socket::write(const void* buf, int size) const
{
int readyFds = epoll_wait(epfd,epEvents,1,-1);
if(readyFds < 0)
{
status = convertFlagToStatus(errno);
return 0;
}
if(readyFds == 0)
{
status = CONNECTION_TERMINATED;
return 0;
}
int fd = epEvents[0].data.fd;
if(fd != socket)
{
status = CONNECTION_INCORRECT_SOCKET;
return 0;
}
if(epEvents[0].events != EPOLLOUT)
{
status = CONNECTION_CLOSED;
return 0;
}
int bytesWrote = ::send(socket, buf, size,0);
if(bytesWrote < 0)
status = convertFlagToStatus(errno);
lastWriteBytes = bytesWrote;
return bytesWrote;
}
Any help solving this mystery bug would be great! at the VERY least, I would like it to NOT crash out the server even if the client crashes (which is really strange for me, since there is no two-way communication).
Also, for reference, here is the server listening code:
while(server.getStatus() == connection::CONNECTION_NORMAL)
{
connection::Socket s = server.listen();
if(s.getStatus() != connection::CONNECTION_NORMAL)
{
fprintf(stdout,"failed to accept a socket. error: %s\n",connection::getStatusString(s.getStatus()));
}
DATASOURCE* dataSource;
dataSource = open_datasource(XXXX); /* edited */ if(dataSource == NULL)
{
fprintf(stdout,"FATAL ERROR. DATASOURCE NOT FOUND\n");
return;
}
boost::thread fileSender(Sender(s,dataSource));
fileSender.detach();
}
...And also here is the spawned child sending thread:
::signal(SIGPIPE,SIG_IGN);
//const int headerNeeds = 29;
const int BUFFERSIZE = 2000;
char buf[BUFFERSIZE];
bool running = true;
while(running)
{
memset(buf,'\0',BUFFERSIZE*sizeof(char));
unsigned int readBytes = 0;
while((readBytes = read_datasource(buf,sizeof(unsigned char),BUFFERSIZE,dataSource)) == 0)
{
boost::this_thread::sleep(boost::posix_time::milliseconds(1000));
}
socket.write(buf,readBytes);
if(socket.getStatus() != connection::CONNECTION_NORMAL)
running = false;
}
fprintf(stdout,"socket error: %s\n",connection::getStatusString(socket.getStatus()));
socket.close();
fprintf(stdout,"sender exiting...\n");
Any insights would be welcome! Thanks in advance.
You've probably got everything backwards... when the server crashes, the OS will close all sockets. So the server crash happens first and causes the client to get the disconnect message (FIN flag in a TCP segment, actually), the crash is not a result of the socket closing.
Since you have multiple server processes crashing at the same time, I'd look at resources they share, and also any scheduled tasks that all servers would try to execute at the same time.
EDIT: You don't have a single client connecting to multiple servers, do you? Note that TCP connections are always bidirectional, so the server process does get feedback if a client disconnects. Some internet providers have even been caught generating RST packets on connections that fail some test for suspicious traffic.
Write a signal handler. Make sure it uses only raw I/O functions to log problems (open, write, close, not fwrite, not printf).
Check return values. Check for negative return value from write on a socket, but check all return values.
Thanks for all the comments and suggestions.
After looking through the code and adding the signal handling as Ben suggested, the applications themselves are far more stable. Thank you for all your input.
The original problem, however, was due to a rogue script that one of the admins was running as root that would randomly kill certain processes on the server-side machine (i won't get into what it was trying to do in reality; safe to say it was buggy).
Lesson learned: check the environment.
Thank you all for the advice.