I'm using Qt4.8 and I want to inject network faults into pre-existing code that uses QNetworkAccessManager. However, this class - and particularly the related QNetworkReply seem to have some strange semantics. I created a subclass of QNetworkAccessManager as well as one of QNetworkReply. My QTestNetworkAccessManager returns either an object created by the base QNetworkAccessManager, or the next test reply in the list:
QNetworkReply *QTestNetworkAccessManager
::createRequest( Operation op,
const QNetworkRequest &req,
QIODevice * outgoingData )
{
// QList<QTestNetworkReply*> m_replies;
if (m_replies.isEmpty())
{
return QNetworkAccessManager::
createRequest( op, req, outgoingData );
}
QTestNetworkReply* pReply = m_replies.takeFirst();
pReply->setUrl(req.url()); // lie about URLs
// this sets a timer that fires finished, etc.
pReply->queueFinished();
return pReply;
}
In the simple, trivial case of a single request, my test code seems to work, but two problems show up when I put it in a bigger app doing lots of network traffic.
First is QNetworkAccessManager seems to have private knowledge of the QNetworkReply objects it's creating. (based on inspecting the Qt source). For example it creates various flavors of QHttpReplyImpl and various other Impl objects and hooks them up in various ways, so my test objects don't seem to be good citizens in that environment. This seems to cause problems like error() signals not propagating correctly and my memory space is experiencing stack corruption.
(I'm not posting the entire QTextNetworkReply object because it's pretty simple and my question isn't really to troubleshoot my specific code.)
My question is: Has anyone done this kind of fault injection with QNetworkAccessManager and been able to simulate various HTTP faults.
One thought was to use the HttpProxy hooks to create a proxy server that actually injects data at the socket level so that it traverses the entire QHttp* object parser, but that seems like a lot of trouble.
Is there really no easier way to inject faults into QNetworkAccessManager so it acts like it is talking to a live server?
Thanks!
P.S. I am familiar with https://blogs.kde.org/2010/08/28/implementing-reusable-custom-qnetworkreply which is why I went down the road I'm on of subclassing QNAM, but it doesn't seem to handle the error semantics correctly and the gitorious link to the code is broken.
Related
Writing a cross platform app using Qt (including Windows with MinGW). For reading data from SSL socket, I am creating a separate thread. This thread is there for historical reason, because earlier the app was written using C socket/ssl/crypto libraries. Now all these are being replaced with Qt Network library.
For blocking thread, waitForReadyRead(milliseconds) seems a better choice. Now according to Qt hierarchy:
QIODevice
|
QAbstractSocket
|
QTcpSocket
|
QSslSocket
Documentation of QAbscractSocket::waitForReadyRead() suggests:
Note: This function may fail randomly on Windows. Consider using the event loop and the readyRead() signal if your software will run on Windows.
But the similar warning is Not mentioned in the QIODevice::waitForReadyRead().
Question: Is QSslSocket::waitForReadyRead() consistently usable for all the platforms?
Why am I not using readyRead() signal?
For some strange reason, if I slot some method with readyRead() then it's not getting called. Moreover, the QSslSocket::write() also doesn't work, which works otherwise with above approach. Due to complexity of my code, I am unable to present it here.
To your question: yes you can use QSslSocket::waitForReadyRead() but on Widows it can timeout even when the data came to the socket. So if timeout occurs you have to check if it is timeout or the method failed. The check is simple just if QAbstractSocket::bytesAvailable() > 0 then data are ready for read otherwise it's timeout.
This approach is ok when you use small timeout and your application isn't sensitive on delay (e.g. communication between temperature sensor and cloud with temperature history). But if any unnecessary delay is not acceptable for you then you should use the signal/slot interface.
For more information you can look at the bug report on the Qt's bug tracker.
according to your question.
The implementation of QIODevice does nothing but return false. So there is no need for the hint of sometimes failing.
The implementation of QAbstractSocket calls something called "nativeSelect" internally, which is then directed to the corresponding method, depending on the OS you are running on. For Windows the select implementation sometimes seem to return a negative false.
But this should not harm you, because you should get the hint for available data from your next call to waitForReadyRead().
QSslSocket's waitForReadyRead() internaly uses QAbstactSocket's implementation appart from some SSL checks.
Regarding your problem with the signals and slots.
A mistake, I made when I was new to Qt was, that I tried to signals before I started a MainLoop by calling QApplication::exec() or something else.
The signal slot mechanism does not work without a run loop.
Hope you can get some hints from this.
Regards
Problem might be use resources.
When you will use waitForReady* you creating constraint one socket per one thread (otherwise you will have strange bugs).
Now question is how many sockets do you have? If it depends it on run time data, you may do not know that.
Some embedded systems have limit on number of treads what can impact your application and IMO this is only limit which can impact such implementation.
This part of your question:
Why am I not using readyRead() signal? For some strange reason, if I
slot some method with readyRead() then it's not getting called.
Moreover, the QSslSocket::write() also doesn't work, which works
otherwise with above approach. Due to complexity of my code, I am
unable to present it here.
Looks suspicious.
I never seen someone had similar problem. Maybe some parts of your code is blocking an event loop?
Though it's not an exact answer to the Qn, I am posting a possible implementation of waitForReadyRead(), which can be used under a local event loop.
class MySslSocket : QSslSocket
{
Q_OBJECT
public:
virtual
bool
waitForReadyRead (int milliseconds) override final
{
QEventLoop eventLoop;
QTimer timer;
connect(this, SIGNAL(readyRead()), &eventLoop, SLOT(quit()));
connect(&timer, SIGNAL(timeout()), &eventLoop, SLOT(quit()));
timer.setSingleShot(true);
timer.start(milliseconds);
eventLoop.exec();
return timer.isActive();
}
};
This can be either use exclusively for Windows, or it can be used in general for all platforms.
I want to invoke CefV8Context::Eval function and get the returned value in browser process's UI thread. But the CEF3 C++ API Docs states that V8 handles can only be accessed from the thread on which they are created. Valid threads for creating a V8 handle include the render process main thread (TID_RENDERER) and WebWorker threads. Is that means I should use the inter-process communication (CefProcessMessage) to invoke that method and get the return value? If so, how to do this in synchronous mode?
Short answer: CefFrame::ExecuteJavaScript for simple requests will work. For more complex ones, you have to give up one level of synchronousness or use a custom message loop.
What I understand you want to do is to execute some Javascript code as part of your native App's UI Thread. There are two possibilities:
It's generic JS code, doesn't really access any variables or functions in your JS, and as such has not context. This means Cef can just spin up a new V8 context and execute your code - see CefFrame::ExecuteJavaScript(). To quote the examples on CEF's JS Integration link:
CefRefPtr browser = ...;
CefRefPtr frame = browser->GetMainFrame();
frame->ExecuteJavaScript("alert('ExecuteJavaScript works!');",
frame->GetURL(), 0);
It's JS code with a context. In this case, read on.
Yes - CEF is designed such that only the RenderProcess has access to the V8 engine, you'll have to use a CefProcessMessage to head over to the Browser and do the evaluation there. You sound like you already know how to do that. I'll link an answer of mine for others who don't and may stumble upon this later: Background process on the native function at Chromium Embedded Framework
The CEFProcessMessage from Browser to Render processes is one place where the request has to be synchronized.
So after your send your logic over to the render process, you'll need to do the actual execution of the javascript code. That, thankfully, is quite easy - the same JS integration link goes on to say:
Native code can execute JS functions by using the ExecuteFunction()
and ExecuteFunctionWithContext() methods
The best part - the execution seems to be synchronous (I say seems to, since I can't find concrete docs on this). The usage in the examples illustrates this:
if (callback_func_->ExecuteFunctionWithContext(callback_context_, NULL, args, retval, exception, false)) {
if (exception.get()) {
// Execution threw an exception.
} else {
// Execution succeeded.
}
}
You'll notice that the second line assumes that the first has finished execution and that the results of said execution are available to it. So, The CefV8Value::ExecuteFunction() call is by nature synchronous.
So the question boils down to - How do I post a CefProcessMessage from Browser to Renderer process synchronously?. Unfortunately, the class itself is not set up to do that. What's more, the IPC Wiki Page explicitly disallows it:
Some messages should be synchronous from the renderer's perspective.
This happens mostly when there is a WebKit call to us that is supposed
to return something, but that we must do in the browser. Examples of
this type of messages are spell-checking and getting the cookies for
JavaScript. Synchronous browser-to-renderer IPC is disallowed to
prevent blocking the user-interface on a potentially flaky renderer.
Is this such a big deal? Well, I don't really know since I've not come across this need - to me, it's ok since the Browser's message loop will keep spinning away waiting for something to do, and receive nothing till your renderer sends a process message with the results of JS. The only way the browser gets something else to do is when some interaction happens, which can't since the renderer is blocking.
If you really definitely need synchronousness, I'd recommend that you use your custom MessageLoop which calls CefDoMessageLoopWork() on every iteration. That way, you can set a flag to suspend loop work until your message is received from renderer. Note that CefDoMessageLoopWork() and CefRunMessageLoop() are mutually exclusive and cannot work with each other - you either manage the loop yourself, or let CEF do it for you.
That was long, and covers most of what you might want to do - hope it helps!
Hi I have a QNetworkAccessManager which I use to send request to get image data from server. This call is asynchronous. I do multiple calls with it. Each call is done by opening a new instance of QNetworkAccessManager So when a specific condition occurs I want to stop the QNetworkAccessManager from receiving the replies from its network requests. Is there any way to do it? Thanks.
Don't use a new QNetworkAccessManager for each request but share the manager. It's usually fine to have just one. Of course one can have multiple if the application design suggests it - but e.g. managing multiple of them in a single controlling object is usually unnecessary. Just have one manager with the same lifetime as the object controlling the network requests.
To cancel running operations, keep the QNetworkReply* pointers QNetworkAccessManager::get/put/post return and call abort() when your condition occurs.
Connect to the finished() signal to remove them from the bookkeeping (as otherwise you would end up with dangling pointers).
If that becomes too complicated, think about using the command pattern. In this answer I describe why I find it particularly useful in this context.
I'm attempting to create a library whose API will be used in the following way:
WebService *service = new WebService( username, password );
User *user = service->getAuthenticatedUser();
UserAssets *assets = user->assets();
// And so on
Neither the authenticated user, nor their assets, will be downloaded when the WebServer instance is created, rather they will only be retrieved if explicitly requested.
Whenever I've had to retrieve data from the network before using Qt, I've followed the standard pattern of connection the finished() signal of the QNetworkReply to the appropriate slot and using that slot to handle the returned data.
My problem here is that pattern does not seem to accommodate my desired use-case. I would like the users of my library (other developers) to be able to use a single line to request and receive the data they desire, but the 'Qt way' seems, at least from my point of view, to require them to initiate the request on one line, and then connect some signal to some other slot to get the data back, which is not the experience I would like them to have.
I'm new to network programming, both in general and with Qt, but I've used libraries written in Python, communicating with the exact same service, that are able to achieve this, so it does seem to be possible.
Is it possible to perform the full lifecycle of a HTTP request with Qt with a single function call?
Your best bet is probably to use a QEventLoop. This would allow you to 1) initiate the HTTP connection and, from your caller's perspective, 2) effectively block until you get a response.
For example:
QNetworkReply wait for finished
As already other have mentioned you could use QEventLoop to wait for finished() or error() signals, and the quitting event loop. This solution while working, have some serious disadvantages.
If it takes longer to download given address, then you might be stuck in your event loop for quite a while. The event loop is processing events nicely, so your app doesn't frezze, but there are some quirks connected to it anyway. Imagine that user is waiting for load, and then presses another button, to load something else. Then you will have multiple loop-in-loop, and first file will have to wait for the second to finish downloading.
Doing things in single call suggest to many programmers, that this will happen at one instant. But your function is processing events internally, so this might not hold. Imagine a code like
// some pointer accessible to many functions/methods (eg. member, global)
MyData* myData=0;
Then a code calling your function:
if (myData){
QNetworkReply* reply = getMyWobsite(whatever);
myData->modify(reply);
}
Seems fine, but what if some other slot happens to call
myData=0;
If this slot will be executed while waiting for request, application will crash. If you decide to use QEventLoop in your function, be sure to mention it in function documentation, so programmers using it will be able to avoid such problems.
If you are not using qt for anything else, you might even consider some alternative libraries (eg. libcurl) that might have what you need already implemented.
I'm building a Qt application and using the QNetworkAccessManager to manager my http requests. Due to the asyncronous nature of QNetworkAccessManager you have to bind a slot to recieve the QNetworkReply when it's done.
I'm new to multithreaded design so I'm not sure how to handle this. I will have 3 seperate types of network replies which need to be parsed in the bound slot and then passed to the correct handler. One will extract a link, the 2nd will extract a picture, and the third will parse a post request's reply to verify that it was successful. All of this takes place within a QWidget subclass.
So, I'm wondering how this is normally handled. As I see it, based purely on intuition as I've done little reading on this specific subject, I would think there are two ways to handle this. One would be to rebind the network manager's finished() signal depending on the call and the other would be to use some sort of state flags and check those to see what the reply is expected to be. What is the preferred method, not necessarily between these two, that's just all I could think of myself, someone more experienced may well have a better solution.
Now, I'm also fairly new to c++, so if the later is the better way what's the best way to handle flags in this case? Would I use a bitset, an enum/flag setting function, or something else? Thanks guys!
If you know the type of reply you're going to get as a result of doing specific request you can connect void QNetworkReply::finished () signal to suitable handler.