I'm trying to write a simple GUI application using Qt framework.
The purpose of this app is to retrieve data from my isp and parse them for presentation.
How do i authenticate my user/password with the webserver and retrieve the html page in question?
Are there any utility libs that make this task trivial?
I figure i need to interact with the server php script and simulate a form input somehow.
Am i on the right track?
You're on the right track, I suggest taking a look at curl.
That should make it alot easier.
edit: Hm, thought it did more than just file-transfer.
Otherwise here's a load more of interesting lib's
The way to authenticate depends completely on the authentication method used by the server. If it's some form to log in you need to retrieve that and send the correct data to the forms action target (usually as POST request). You could do this by constructing your request using QHttpRequestHeader and then simply sending it to the server. If you even know about the form you might even not need to retrieve the login page. If the website uses HTTP authentication you should be able using QAuthenticator.
Related
I am fairly new in web development and I decided to handle a user's availability to send a POST request to server. However, I do not know even whether it is possible or not but when a user close my Django site without using logout button (For example close the browser or tab), in order to understand the user is offline or online, I want to send a request to server. As a result, when the server does not get an answer from the user for a while, it automatically logout the user.
Can you tell me is it a good way to handle a user's availability and first of all is it a realistic solution? If it is, can you suggest me a document or example that helps me please.
I agree to to the answer of #Mounir. That's not related to django, if you want to know when a user is "disconnected"(close the tab or window) you need to use javascript, in concrete you need to use Sockets.
I recommend you this ones:
http://socket.io/
https://github.com/centrifugal/centrifugo
I'm using centrifugo for one project right now. In concrete, for chat and inbox messaging. It's very simple to make it run with Django.
Greetings
For logging out user you can use the Session expiration, but for all other staff you want to achieve I don't see any thin really related to Django itself, everything depend on your requirements and is more Browser/javascript related than Django.
I frequently need to access to the router to do some simple task: restart, switch on/off some options... And the repeated process really annoying me :(
I want to write a batch script/program which do it for me.
I can't flash the router with the DD-WRT firmware to access it via console. The policy don't allow it.
Then I come up with an idea that use Qt write a small program which will interact with the browser. Access the router IP, fill the password, browse to the option page...
However, AFAIK, I can only do such thing with a web-driver like selenium, which is unavailable with Qt.
So is there any solution? I really like the idea use Qt or a script to solve the problem.
Any ideas are appreciated :)
It is possible to just send form response and don't load the page in the QWebFrame. You can use some tool (firebug, wireshark) to catch the request/response when you click "save" and learn what to send to your router everytime you want this configuration. It will be simple HTTP GET or POST request. That request can be done with curl, wget or your simple QT application using QNetworkRequest or wethewer.
It will be more tricky if it needs authentification other then HTTP basic auth. You will need two requests
first one - authenticate and save all cookies
second one - set parameters and add all cookies to response
I have done a few mobile apps using django as my backend, and now I am working on a chrome extension. I want my users to be able to send POST requests up to the server using the app/extension but is there a way to do it without first asking the server for a CSRF token? It just saves the app from making two requests every time we want to submit data. For example, I want to update my profile on my social media app or update a wallet from a chrome extension. It would be nice to open up the profile view input the data and push it to the server. It's less sleek if I have to open the profile, then wait for it to grab a token from the server and then I can submit the data. Is there another way to do this? Or am I stuck making multiple requests every time I want to submit data?
Also, a little clarification, CSRF prevents sites from submitting forms with user's data. But what is to stop me from making a site that uses ajax or something to grab the real site and steal the CSRF token and then paste that into my cross site request form? I feel like there is a loophole here. I know that I am not quite understanding this all the way.
You can, and should, make any API endpoint CSRF exempt.
Django offers the csrf_exempt decorator for exactly this, see https://docs.djangoproject.com/en/dev/ref/contrib/csrf/#django.views.decorators.csrf.csrf_exempt.
Also CSRF is intended to prevent unintended actions being performed via GET request forgeries. It is not intended to make it impossible for an automated system to submit forms, there are captchas for that.
As for what prevents you from using AJAX to grab the whole site and extract the token is something called the Same-Origin Policy. This is implemented by the browser and prevents any AJAX call from returning data when the target of the AJAX call is a different domain without the correct headers set. (I'm not entirely sure what sandboxing is applied to chrome extensions concerning this). As such it will, or at least should, fail to get data via AJAX for normal websites, e.g. a profile page. If you want to interact with third party websites you should look into whether or not they offer an API.
As the title implies,
I need to fetch data from certain website which need logins to use.
The login procedure might need cookies, or sessions.
Do I need QtWebkit, or can I get away with just QNetworkAccessManager?
I have no experience at both, and will start learning as I go.
So please save me a bit of time of comparing both ^^
Thank you in advance,
Evan
Edit: Having read some related answers,
I'll add some clarifications:
The website in concern does not have an API. So I will need to scrape web elements for the data myself.
Can I do that with just QNetworkAccessManager?
No, in most cases you don't need a full simulated web browser. In most cases, just performing the same web requests like a web browser would do is enough.
Try to record the web requests in your browser, using a plugin like "HTTP Live Headers" or "Firebug" in Firefox. I think Chrome provides a similar tool out of the box. These tools record the GET and POST requests done by the website when you send a form in the webpage.
Another option is to inspect the HTML code of the login page. Find the <form> tag and its fields. Put them together in a GET / POST request in your application to simulate the same form.
Remember that some pages use randomized "tokens" in their forms, some set the tokens as cookies. In such cases, you need to request the login page itself in your application first (before sending the filled in form). Both QWebView and QNetworkAccessManager have cookie support.
To sum things up, I think QWebView provides a far more elegant way to simulate user interaction with a web page. The manual way is, however, more "lightweight", as you don't need Webkit and your application might be faster (because only the HTML page is loaded, without any linked resources like images, CSS, javascript files).
QWebView as class name states is a view, so it views something (in this case web pages). If you don't need to display loaded page, then you don't need a view. QNetworkAccessManager may do the work, but you need some knowledge about HTTP protocol, and also anything about target site: how does it hande logins, what type of request you have to send to login etc.
I was just wondering if it is possible and if so what the best way to create a web-page that is only accessible from a custom iPhone application? For example, if you tried to access the webpage from the iPhone's built in browser, or any other browser it would display an error page but when accessed from a custom built application it would be fully functional.
One idea that has come up is to change the User-Agent string in the embedded browser inside the application to something custom. I'm not sure if this is viable though.
I hope this makes sense.
Thanks in advance.
-Ben
Any and all request headers can and will be spoofed. Authentication is the only plausible solution.
Changing the User-Agent string is a good method. I haven't tried it personally, but you should be able to alter the NSURLRequest object and change the user-agent before the request is made.
You could also use other custom data in the HTTP request to allow/block visits. You could add a query string to the URL or include some unique POST data.
Note this isn't a real security measure as anyone could fake any part of the HTTP request to gain access. Someone could easily read the HTTP traffic generated from your app and use that to figure out how to access the site with any browser.