I know there are lots of similar questions here, also there are lots of results, when I google it, but none of them answers my question. I read this, this, this and this but none of them works for me. I don't talk about any locks, I don't wanna do this using MySQL c++ connector, just the C API.
Also, what is very important here: I do this on LINUX. Why I mention this? Because in the documentation for mysql_options:
MYSQL_OPT_READ_TIMEOUT - ...This option works only for
TCP/IP connections and, prior to MySQL 5.0.25, only for Windows.
MYSQL_OPT_WRITE_TIMEOUT- ... This option works only for
TCP/IP connections and, prior to MySQL 5.0.25, only for Windows
So, is there any way to set a query timeout for versions, prior 5.0.25?
My MySQL version:
[root#xxx kiril]# mysql --version
mysql Ver 14.12 Distrib 5.0.22, for redhat-linux-gnu (i686) using readline 5.0
EDIT: At least, is there any way to cancel a query? I can start a timer as different thread, but when it expires.. can I cancel the query somehow?
Okay, I found a solution.. Thanks to Will and PRR( my co-worker ).
I cannot start a new thread on each query, as this is a real-time application, that is supposed to process 1000+ messages per second..(anyway, thanks to R.. for the idea).
Also, it was not possible to terminate the connection through the library, nor to cancel/kill the query, as the problem was in the DB server..
And here's a brute-force solution, but still much better that _EXIT( FAILURE ): Here's the related question: "How to force closing socket on Linux?" - so, I just closed the socket using a system call.
Important NOTE: (thanks Will) - It turned out, that our MySQL library wrapper has s "fail-safe" flag, so that on closed socket (or other critical error), it tries to "solve" the problem, so it reopens the socket, by itself, in my case. So, I just turned off this option and everything is fine now - the execute is terminated because of an exception - this is the "softest" way to do this.
This should be done through another thread, of course - a timer, for example.
EDIT: The timeouts are really working for versions after 5.0.25. But, at least on RHEL4 and RHEL5, the timeouts are tripled for some reason! For example, if some of the timeouts is set to 20sec, the real timeout is ~60sec..
Also, another important thing is, that these timeouts(as any other options) MUST be set after mysql_init and before mysql_connect or mysql_real_connect.
I suppose you could implement a timeout for the C function call (as described in this thread C++: How to implement a timeout for an arbitrary function call?), but you would need to think carefully about what kind of state you would leave the DB in - presumably these are just for reads of the database, not inserts/updates.
I have never tryed to do that, but i've been reading and i think that could means that MYSQL_OPT_WRITE_TIMEOUT and MYSQL_OPT_READ_TIMEOUT are only for windows prior MySQL version 5.0.25 but now sould be working for every TCP/IP connection. Take a look here
Regards
EDIT: I would try to update my mysql server to a newer version and try if it works.
If you don't mind using threads, you could start the query from a new thread and have the main thread perform a short pthread_cond_timedwait for the new thread to set a condition variable that it make the connection. Then you can let the thread linger around until the actual mysql call times out. Just make sure it's detached so its resources get freed when it finally does time out. This solution isn't pretty, but it should at least work.
Related
I have a client/server program written in C++. I want to check the client response (an attribute of a C++ object) through a command send by the server, with a timeout if no response.
I am waiting for an expected value during some seconds. If the expected value is not observed, I need to return with a timeout. I was thinking about a thread and a poll to check the expected value in an specific time interval.
I wonder if C++11/14 features - std::promise, std::future, std::condition_variable or something else - can do it more easily for this case. The inconvenient i see about it is that i have to notice each changing value with a notify.
Well, i need some advice.
None of the C++ language features you mentioned can help in your scenario, because they are intended for interaction within a single running program - which may be multi-threaded, but not separated into two completely independent processes.
However, the networking library you are using (on the server side) might possibly have convenience facilities for doing this.
I realize this is a general and somewhat vague answer, but your question was also not very specific.
How to wait for a value with timeout
Within a process, one would typically use a condition variable.
I want to check the client response ... through a command send by the server
There is no standard way to communicate between processes in C++ (unless you count interaction with filesystem). As such, there is also no standard way to enforce a timeout on such communication.
Before you can know how to implement the timeout, you must figure out how you are going to communicate between the client and the server. That choice will be affected by what system you are targeting, so you should first figure that out.
If you are on a Linux environment you can try rpcgen and play with .x flies but you’ll have to study it a bit. Not sure for Windows env. Also you can use Dbus which is more intuitive.
[edit] Dbus or probably libdbus for you is an IPC cross platform toolkit or library that can fit your need. RPCGEN is an old tool that does the same thing but more complicated. I don’t have a snippet, I apologize but you can search for “qt dbus example”.
About the first requirement, server waits for a response with a timeout.
Have you tried select() or poll(). They can help us to monitor the socket connection between server and client in a period.
Or we can use signal() and alarm(), to check the response after a few seconds.
In Bekerley API, combine setsockopt() with SO_RCVTIMEO, SO_SNDTIMEO can also set the timeout for the request.
I'm not sure about the library you are implementing, but I hope it has any similar functions.
The second requirement, you are waiting for expected value for a duration.
I think condition variable is a good solution for this.
Why not using boost::thread with a timed_join?
boost::thread server_thread(::server_checker_method, arg1, arg2, arg3);
if (server_thread.timed_join(boost::posix_time::milliseconds(1000))) // wait for 1s
{
// Expected value found in the server in less than 1s
}
else
{
// Checking expected value took more than 1s, timeout !!!
}
You can put your checking mechanism in the server_checker_method and return if the expected values are OK. Otherwise, iterate over the loop until the timeout reaches.
I have a problem with an sqlite3 db which remains locked/unaccessible after a certain access.
Behaviour occurs so far on Ubuntu 10.4 and on custom (OpenEmbedded) Linux.
The sqlite version is 3.7.7.1). Db is a local file.
One C++-applications accesses the db periodically (5s). Each time several insert statements are done wrapped in a deferred transaction. This happens in one thread only. The connection to the db is held over the whole lifetime of the application. The statements used are also persistent and reused via sqlite3_reset. sqlite_threadsafe is set to 1 (serialized), journaling is set to WAL.
Then I open in parellel the sqlite db with the sqlite command line tool. I enter BEGIN IMMEDIATE;, wait >5s, and commit with END;.
after this the db access of the application fails: the BEGIN TRANSACTION returns return code 1 ("SQL error or missing database"). If I execute an ROLLBACK TRANSACTION right before the begin, just to be sure there is not already an active transaction, it fails with return code 5 ("The database file is locked").
Has anyone an idea how to approach this problem or has an idea what may cause it?
EDIT: There is a workaround: If the described error occures, I close and reopen the db connection. This fixes the problem, but I'm currently at a loss at to why this is so.
Sqlite is a server less database. As far as I know it does not support concurrent access from multiple source by design. You are trying to access the same backing file from both your application and the command tool - so you attempt to perform concurrent access. This is why it is failing.
SQLite connections should only be used from a single thread, as among other things they contain mutexes that are used to ensure correct concurrent access. (Be aware that SQLite also only ever supports a single updating thread at once anyway, and with no concurrent reads at the time; that's a limitation of being a server-less DB.)
Luckily, SQLite connections are relatively cheap when they're not doing anything and the cost of things like cached prepared statements is actually fairly small; open up as many as you need.
[EDIT]:
Moreover, this would explain closing and reopening the connection works: it builds the connection in the new thread (and finalizes all the locks etc. in the old one).
I am using boost::asio::ip::udp::socket to communicate. I use socket.receive_from(...) to receive messages from clients. This is working fine for now, but I want to be able to shut down my server. Right now I am calling receive_from in a while-loop, which depends on a bool condition which I can set. However, this is pretty useless if I cannot force the thread to exit receive_from at regular intervals or at a certain call.
Is this even possible? I have tried googling, but found no clear answer. I have tried using socket.cancel() but this seems to have no effect.
Am I using the socket in the correct way?
There's no good way to do what you want using the synchronous receive_from method. You should use the asynchronous async_receive_from method if you desire timeouts and cancelability. There's a ticket on the Boost.Asio trac website describing this.
I answered a similar question recently that you might find useful as well.
For THIS reason, I want to try something new - close the socket using some system call.
The situation in two words - can't set query timeout of the mysql library (the C API, refer to the link for more info), so I want to try closing the socket to see how the library will react. Probably this is not a good idea, but still wanna try it.
Here's what I've done - there's another started thread - a timer. So, after a specific timeout (let's say 10 second), if there's no response, I want to close the socket. The MYSQL struct has member net, that is also a struct, and holds the fd. But when I try to do this:
shutdown( m_pOwner->m_ptrDBConnection->m_mysql.net.fd, SHUT_RDWR );
close( m_pOwner->m_ptrDBConnection->m_mysql.net.fd );
nothing happens. The returned values from shutdown and close are 0, but the socket is still opened (because after 60sec waiting, there's a returned result from the DB, that means that the mysql client is still waiting for response from the DB.
Any ideas?
Thanks
EDIT - Yes, there's a running transaction, while I'm trying to close the socket. But this is the actual problem - I cannot terminate the query, nor to close the connection, nothing, and I don't wanna wait the whole timeout, which is 20min and 30 sec, or something like this. That's why I'm looking for a brute-force.. :/
Just a shot in the dark, but make sure you cancel/terminate any running transactions. I'm not familiar with the MySQL C API, but I would imagine there is a way to check if there are any active connections/queries. You may not be able to close the socket simply because there are still things running, and they need to be brought to some "resolved" state, be that either committed or rolled back. I would begin there and see what happens. You really don't want to shutdown the socket "brute force" style if you have anything pending anyway because your data would not be in a reliable "state" afterwards - you would not know what transactions succeeded and which ones did not, although I would imagine that MySQL would rollback any pending transactions if the connection failed abruptly.
EDIT:
From what I have found via Googling "MySQL stopping runaway query", the consensus seems to be to ask MySQL to terminate the thread of the runaway/long-running query using
KILL thread-id
I would imagine that the thread ID is available to you in the MySQL data structure that contains the socket. You may want to try this, although IIRC to do so requires super user priviledges.
EDIT #2:
Apparently MySQL provides a fail-safe mechanism that will restart a closed connection, so forcefully shutting down the socket will not actually terminate the query. Once you close it, MySQL will open another and attempt to complete the query. Turning this off will allow you to close the socket and cause the query to terminate.
The comments below show how the answer was found, and the thought process involved therein.
It looks like you are running into an issue with the TCP wait timer, meaning it will close eventually. [Long story short] it is sort of unavoidable. There was another discussion on this.
close vs shutdown socket?
As far as I know, If shutdown() and close() both return 0 there's no doubt you had successfully closed a socket. The fact is that you could have closed the wrong fd. Or the server could not react properly to a correct shutdown (if so, this could be considered a bug of the server: no reason to still wait for data incoming). I'd keep looking for a supported way to do this.
HI Folks,
I have a threaded C++ program where up to three threads are calling select on a three separate socket descriptors waiting for data to become available. Each thread handles one socket and adds it to the readfds with a timeout of 300 seconds. After select returns if there is data available I'm calling recv to read it.
Is there anything that I need to be aware of with winsock and threads because for some reason after a number of hours the select calls all seem to be not timing out. Can a multi threaded program select from a number of threads without issue?
I know that I should have one thread listening to all three sockets however that would be a large change for this app and I'm only looking to apply a bug fix.
cheers,
Martin.
Without some code, I can only make a guess. The most common reason for select timeouts to not work properly is that the struct timeval argument is not reset in every iteration of the loop. Some (all?) select implementations update the timeout parameter to reflect the amount of time spent blocking. However, this usually results in select not blocking.
Are you sure that select is the problem and you do not have something else going on like stack corruption, a memory leak, etc.?
Edit: I have used select from multiple threads on different sockets before under Windows without problems. I forgot to mention this in the initial post.
I'm assuming that it is ok to use the select on differnet threads because the following link says it is grand.
http://tangentsoft.net/wskfaq/intermediate.html#threadsafety