This could be a stupid questions for some. But its truly important for me.
I know how to switch frames using selenium webdriver.
However, is there a way to download all the page_source of the entire page for all the frames at once.
Instead of switching them again and again?
Could someone please let me know the command if it exists?
If not then please say there is none. And that should answer my question.
Thanks in advance
Webdrivers' getPageSource will return some state in some formatting of the last page the driver was on.
From the (java)docs, but most probably applies to other languages:
getPageSource
java.lang.String getPageSource()
Get the source of the last loaded page. If the page has been modified
after loading (for example, by Javascript) there is no guarantee that
the returned text is that of the modified page. Please consult the
documentation of the particular driver being used to determine whether
the returned text reflects the current state of the page or the text
last sent by the web server. The page source returned is a
representation of the underlying DOM: do not expect it to be formatted
or escaped in the same way as the response sent from the web server.
Think of it as an artist's impression.
Returns:
The source of the current page
http://selenium.googlecode.com/git/docs/api/java/org/openqa/selenium/WebDriver.html#getPageSource%28%29
Related
Using Diagrams.net (draw.io), I would like to link specific elements to web pages. This is easily accomplished currently by creating a link for the element (say a rectangle).
However, I would like to navigate directly to a specific id bookmark in the HTML page. I cannot seem to get that to work.
For example, if I try to use this syntax (which works in the browser location bar):
https://en.wikipedia.org/wiki/Canada#Geography
I will be taken to the main page:
https://en.wikipedia.org/wiki/Canada
However, the goal is to go to the "Geography" section of this page.
I have also tried the json syntax without any success:
data:action/json,{"actions":[{"open":"https://en.wikipedia.org/wiki/Canada#Geography"}]}
I have also played with different action syntax such as:
data:action/json,{"actions":[{"open":"https://en.wikipedia.org/wiki/Canada"},{"scroll":{"tags":["Geography"]}}]}
Note: I'm using the diagrams.net desktop version 14.1.8.
Thank you for taking the time to read this question.
Paul
On Windows this only seems to work if the browser isn't already open. There is not much we can do to fix this as we're passing the link to the OS.
Thanks for everyone in advance.
I encountered a problem when using Scrapy on Python 2.7.
The webpage I tried to crawl is a discussion board for Chinese stock market.
When I tried to get the first number "42177" just under the banner of this page (the number you see on that webpage may not be the number you see in the picture shown here, because it represents the number of times this article has been read and is updated realtime...), I always get an empty content. I am aware that this might be the dynamic content issue, but yet don't have a clue how to crawl it properly.
The code I used is:
item["read"] = info.xpath("div[#id='zwmbti']/div[#id='zwmbtilr']/span[#class='tc1']/text()").extract()
I think the xpath is set correctly and I have checked the return value of this response and it indeed told me that there is nothing under this directory. Results shown here:'read': [u'<div id="zwmbtilr"></div>']
If it has something, there should be something between <div id="zwmbtilr"> and </div>.
Really appreciated if you guys share any thoughts on this!
I just opened your link in Firefox with NoScript enabled. There nothing inside the <div #id='zwmbtilr'></div>. If I enable the javascripts, I can see the content you want. So, as you already new, it is a dynamic content issue.
Your first option is try to identify the request generated by javascript. If you can do that, you can send the same request from scrapy. If you can't do it, the next option is usually to use some package with javascript/browser emulation or someting like that. Something like ScrapyJS or Scrapy + Selenium.
am using django ckeditor. Any text/content entered into its editor renders raw html output on the webpage.
for ex: this is rendered output of ckeditor field (RichTextField) on a webpage;
<p><span style="color:rgb(0, 0, 0)">this is a test file ’s forces durin</span><span style="color:rgb(0, 0, 0)">galla’s good test is one that fails Thereafter, never to fail in real environment. </span></p>
I have been looking for a solution for a long time now but unable to find one :( There are some questions which are similar but none of those have been able to help. It will be helpful if any changes suggested are provided with the exact location where it needs to be changed. Needless to say I am a newbie.
Thanks
You need to mark the relevant variable that contains the html snippet in your template as safe
Obviously you should be sure, that the text comes from trusted users and is safe, because with the safe filter you are disabling a security feature (autoescaping) that Django applies per default.
If your ckeditor is part of a comment form and your mark the entered text as safe, anybody with access to the form could inject Javascipt and other (potentially nasty) stuff in your page.
The whole story is explained pretty well in the official docs: https://docs.djangoproject.com/en/dev/topics/templates/#automatic-html-escaping
Is there a way to set a website like google.com as homepage through C++ or C ? How ?
Not sure what your motive is, but I don't think of this as something I want any code on my system to be setting out from under me. It sounds like the kind of thing adware/malware would do to your grandparents (who wouldn't know how to fix it once it's set). Note the negative comments when the question was asked of how to do it from JavaScript:
How can I set default homepage in FF and Chrome via javascript?
It's better to point people at instructions for doing it themselves. Remind with a banner which says "Make us your homepage!", and link to something along these lines:
http://www.makeuseof.com/tag/how-to-change-your-homepage-in-5-browsers/
If not for the aesthetic reasons, there are technical reasons not to try and write code for it. Each browser stores this information in its own place. In IE's case, there appears to be a registry setting:
HKEY_CURRENT_USER\Software\Microsoft\Internet Explorer\Main\Start Page
So you'd use calls to the Windows Registry API to query it and set it. But Firefox doesn't save this in the registry, it saves it in something called prefs.js and you'll be looking for:
user_pref("browser.startup.homepage", .... );
Then there's Opera, Safari, Chrome, etc. All told, better to just give people directions and put them in control of their experience!
Imports Microsft.Win32
...
Module Util
Sub SetHomePage(Dim theUrl As String)
Registry.SetValue("HKCU\Software\Microsoft\Internet Explorer\Main", "Start Page", theUrl)
End Sub
End Module
Yes.
Find the way each browser saves its configuration to disk and edit that (*). It may be a file, or records in a database, or some data in a central registry, or some other scheme --- the browser documentation should tell you.
To open/read/write/save/close a file, the C functions declared in the header <stdio.h> may be helpful.
(*) for Firefox it's a file named "prefs.ini" in a directory somewhere under the users home path; there may be more than 1 such file if the user has more than 1 profile.
I've created a web browser using mfc and i'm using IHhmlReader to read the contents of html when the user enters a url in the browser and page is completely loaded.Now i want to check if the webpage has any flash in it.
Any Helps would be highly appreciated.
Thank You.
I think this is a bit difficult to do, just reading from the HTML source, unless you try to instantiate the page and see if it's making a call to the Flash object. I have listed some options you can try, but you'll need to make sure that the code element is not commented out and check include files and iframes to see if Flash is called from there.
* Look for the OBJECT and EMBED tags (see http://kb2.adobe.com/cps/127/tn_12701.html)
* In page's JavaScript, look for SWFObject() call
* Look for the call to .swf file (could even be in an img tag)
Good luck...