Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 6 years ago.
Improve this question
How do I interact with websites in C++?
For example, a website has a dropbox, text area and a button, and I want my program to fill text inside the text area, choose an option from the dropbox, and make the button fire its event("clicking" it).
How can I achieve something like that?
thanks!
First you have to understand that on the server there is no text box or button.
These are constructs that are built by the browser to display to you.
The browser will then take user input into the text box and interprets the clicks on the button. What happens (usually) when the button is clicked is that the browser sends an HTML "POST" request to the server. The browser builds the post request based on what the user has done in the UI.
Example:
Server Sends to browser:
<html>
<head><meta http-equiv="Content-Type" content="text/html;charset=utf-8"><title>TestDoc</title></head>
<body>
<form action="http://website.com/form.html" method="post">
<div>
<textarea name="userinfo" rows="2" cols="30">Some Text</textarea>
<input type="submit">
</div>
</form>
</body>
</html>
Your browser interprets this and displays a text box and submit button. When you press the submit button the browser builds an HTTP post command that is sent back to the browser. It connects to the website at http://website.com/form.html (see form tag in code snippet above) and sends the content of the text area (tagged with the value userinfo).
You can manually do the same.
But you need to understand what values the website is expecting and build the appropriate command based on what the website is expecting. To do this, the easiest way is to use libCurl. The documentation for this package explains in detail how to build a post request.
Here is a post example:
Take a look al libcurl. It's a C library, but you could definitely use it from C++ to achieve what you want.
If you need just to do some casual WebSite interaction I would suggest you to take a look at languages that are more suitable (and easier to use) than C++ for that task. Python (with Mechanize library), Ruby, PHP, Perl...
Even Java and C# have native libraries to deal with stuff like this.
Okay, it really depends on a few things. You may be overcomplicating your problem by thinking that you need to get the program to fill in the form as though it was a human. Instead, why don't you skip that step?
When you submit a form, your browser sends a HTTP POST (usually) request across the Internet with your form details in the header. If you know what the form is going to be beforehand, you can simply get the program to send POST requests to the server as though somebody had submitted a form.
If you don't know what the form will be, then you need to send a GET request to the server to retrieve the page and then somehow analyse the page to extract the information needed to fill the form details (that's your problem) and then send the POST request with the details as above.
This can all be done with the libcurl library. It allows you to send HTTP requests through a simple interface.
Now, if your problem actually requires you to make a program that acts as a human and really manipulates the browser to fill in the form, then you need to learn about simulating key presses and mouse clicks, which is likely going to be platform dependent.
If libcurl is not enough for your needs you might want to take a look at Qt, specifically the QtWebKit module. It basically incorporates a complete browser engine which also Google Chrome uses. Using that engine you can even execute your own JavaScript code in the context of the Website and simulate for instance a login .
Related
I apologize for the beginner question but I don't know where else to turn.
I have the following code in my website:
<script>
FB.Event.subscribe('edge.create', function(href, widget) {
alert('You just liked the page!');
});
</script>
On the developer page, it states that it will record likes in the database. Well, how do I view them?
The button shows up fine and everything is working properly but I just don't know how to view them.
Thank you.
The code you have posted is for notifying your page (in Javascript obviously) that the user viewing the page has 'liked' it. This is not actually needed in order for the Like functionality to work.
You need to have the Like button on the page already for someone to Like the page (eg. by using the tag with the necessary Javascript SDK loaded - http://developers.facebook.com/docs/reference/plugins/like/ - though it sounds like you already have that done).
As for viewing the 'likes', your Like button will tell you how many people have liked it (if you have that option set), however you cannot directly view who actually Liked your page as Facebook do not allow that (privacy concerns)
The script that you posted can be modify to perform other actions, for example you could make an AJAX call to your own server to record information of the 'like' - eg. if the users are logged in, then you would know who it was that liked it.
As the title implies,
I need to fetch data from certain website which need logins to use.
The login procedure might need cookies, or sessions.
Do I need QtWebkit, or can I get away with just QNetworkAccessManager?
I have no experience at both, and will start learning as I go.
So please save me a bit of time of comparing both ^^
Thank you in advance,
Evan
Edit: Having read some related answers,
I'll add some clarifications:
The website in concern does not have an API. So I will need to scrape web elements for the data myself.
Can I do that with just QNetworkAccessManager?
No, in most cases you don't need a full simulated web browser. In most cases, just performing the same web requests like a web browser would do is enough.
Try to record the web requests in your browser, using a plugin like "HTTP Live Headers" or "Firebug" in Firefox. I think Chrome provides a similar tool out of the box. These tools record the GET and POST requests done by the website when you send a form in the webpage.
Another option is to inspect the HTML code of the login page. Find the <form> tag and its fields. Put them together in a GET / POST request in your application to simulate the same form.
Remember that some pages use randomized "tokens" in their forms, some set the tokens as cookies. In such cases, you need to request the login page itself in your application first (before sending the filled in form). Both QWebView and QNetworkAccessManager have cookie support.
To sum things up, I think QWebView provides a far more elegant way to simulate user interaction with a web page. The manual way is, however, more "lightweight", as you don't need Webkit and your application might be faster (because only the HTML page is loaded, without any linked resources like images, CSS, javascript files).
QWebView as class name states is a view, so it views something (in this case web pages). If you don't need to display loaded page, then you don't need a view. QNetworkAccessManager may do the work, but you need some knowledge about HTTP protocol, and also anything about target site: how does it hande logins, what type of request you have to send to login etc.
We're developing a tablet app, and my coworker had a rogue thought:
What if you put a manifest attribute on the html tag of a ColdFusion page? Would it
still work if the user couldn't get to the server?
I think it's lunacy, but I wanted to hear what you guys had to say.
The client side is not aware of how is generated the page. He didn't even know if the page is generated. So, the page is cached like another page.
If you are not connected, the could handle this case with some javascript and alert the user you can't communicate with the server.
You could simply disable the application or work with localStorage etc...
My C++ application is using a web browser (IE) control. I need to detect when the user clicks a button on a specific web page (using the element ID of that button).
How can I do this? I already have an event sink implemented, but I do not know how to catch mouse click on DOM elements.
The event you are looking for is probably related to:
onclick event
But whether this is accessible from the MS IE API from C++ via EventSinks, I don't know.
Other Round-About Way to Get Click Information to Your C++ Code
I am not an expert on it, but companies like UserZoom, analyze the information by inserting Javascript via a plug-in to the web browser. Here is a quote from their FAQ of how they collect the click-streams:
UserZoom FAQ
Data Collection & Tracking
What type of information is UserZoom
capable of tracking during the tasks?
With the plug-in version or non plug-in with JavaScript tracking code,
UserZoom can track participants’ navigation paths as well as where
participants have clicked on pages throughout the tasks (heatmaps).
With the non plug-in version, navigation paths and participant clicks
cannot be captured.
Now with this knowledge, go find a javascript library that can get the clicks and the document object model info.
Google Search: "how to get where the user clicked element in javascript"
How to get the target element when clicked?
StackOverflow: JavaScript: Get clicked element
Then you need to insert that bit of javascript into the webpages into the browser the user is using.
Adding Javascript with an add-on/extension/plugin in Firefox and Chrome is relatively easy, I've heard, but for IE, I think you have your work cut out for you to create the add-on in IE. After much searching I found this:
Inject HTML and JavaScript into an existing page with BHO using MS Visual Studio 2010 and C#
And lastly, get that information out of the javascript and into your code using JSON or AJAX and setting up a local webserver to receive it:
Google Search: receive ajax c++
Jquery Ajax Calling Functions
AJAX and the C++ Programmer
Hopefully that information gives you a starting point. Good luck.
We have a page which is dynamically generated after a few queries in the database. There are some links that when they are clicked by the user, update some information on the database but they change nothing on the webpage (or the display a discrete message).
How could we stay on the same page without re-rendering it?
Ideally, the corresponding view.py would process the queries and update the database but not the webpage.
You can send and receive your own XMLHttpRequest, but it is too much of works to do and IE will create a lot of problems.
Have you ever heard about jQuery? I strongly recommend you take a look at it and learn how to send and receive Ajax request using it.
You need to make an AJAX call back to the server with the user's actions, and process it on the server. You probably want a different view to process the AJAX request -- you could do it with the same view, but it would be somewhat silly to. The response from that view contains data (probably as JSON) or HTML, which you display on the page with javascript. Check out jquery -- it's great for the client side.
You could accomplish this with plain Javascript and AJAX. When the user clicks on a link, use XMLHttpRequest to call view.py to process the queries.
eg. For the link: <a href="#" onclick=submitdb(); >Click me!</a>
For a tutorial on implementing AJAX (XMLHttpRequest) with Javascript, have a look here:
http://www.quirksmode.org/js/xmlhttp.html