chromedriver can't click when running a script, but can in shell - python-2.7

I have a problem in general with clicking in Chromedriver when the code is being ran by Python. This code is used in the script:
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.support.ui import WebDriverWait
driver.get("https://www.marktplaats.nl/")
cook_button = WebDriverWait(driver, 15).until(EC.element_to_be_clickable((By.XPATH, "//form[#method='post']/input[#type='submit']"))).click()
It just times out giving "NoSuchElementException". But if I put those lines manually in the Shell, it clicks like normal. For what it's worth, I'm using the latest 2.40 Chromedriver and Chrome v67. Running it headless doesn't make any difference.
EDIT
The program actually breaks after on the third command when it tries to find an element that doesn't exist because the click wasn't completed
driver.get(master_link) # get the first page
wait_by_class("search-results-table")
page_2_el = driver.find_element_by_xpath("//span[#id='pagination-pages']/a[contains(#data-ga-track-event, 'gination')]")
So, page_2_el command gives this exception, but only because the click before wasn't completed successfully to remove the warning about cookies.And I'm sure the xpath search is good because it runs with geckodriver in Firefox, but won't do it here with Chromedriver.
EDIT2 See a video of the bug here https://streamable.com/tv7w4 Notice how it flinches a bit, see when it writes on the console "before click" and "after click"
SOLUTION
Replaced
cook_button = WebDriverWait(driver, 15).until(EC.element_to_be_clickable((By.XPATH, "//form[#method='post']/input[#type='submit']"))).click()
With
N_click_attempts = 0
while 1:
if N_click_attempts == 10:
print "Something is wrong. "
break
print "Try to click."
N_click_attempts = N_click_attempts+1
try:
cook_button = WebDriverWait(driver, 15).until(EC.element_to_be_clickable((By.XPATH, "//form[#method='post']/input[#type='submit']"))).click()
time.sleep(2.0)
except:
time.sleep(2.0)
break
It seems that the click is now completed. I have other clicks in the script and they work fine with element.click(), this one was problematic for some reason.

Your path is correct, but I would suggest a smaller one:
//form/input[2]
And about NoSuchElementException - you can try to add a pause, to wait until element loads and becomes 'visible' for selenium. Like this:
import time
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.support.ui import WebDriverWait
driver.get("https://www.marktplaats.nl/")
cook_button = WebDriverWait(driver, 15).until(EC.element_to_be_clickable((By.XPATH, "//form[#method='post']/input[#type='submit']"))).click()
time.sleep(5) # wait 5 seconds until DOM will reload
According edit in the question I would suggest to add time.sleep(5) after clicking on the button. And for the same reason, because after clicking the whole DOM reloads and selenium should wait until reload will be done. On my computer it takes about 2-3 seconds to full reload the DOM.

Related

Selenium - Click button after completing file conversion on webpage

I want to wait for a page to finish converting a json file, then automatically download it. The following python code works.
import time
from selenium import webdriver
chrome = webdriver.Chrome()
chrome.get('https://json-csv.com/')
load_data = chrome.find_element_by_id('fileupload')
load_data.send_keys('C:\\path_to_file')
load_data.submit()
# Wait arbitrary duration before downloading result
time.sleep(10)
get_results = chrome.find_element_by_id('download-link')
get_results.click()
chrome.quit()
However, every time I run the script, I need to wait 10 seconds, which is more than enough for the page to finish converting the file. This is not time efficient. The page may finish loading the new file in 5 seconds.
How can I click the download button the moment the file is done converting?
What I've tried
I've read a solution to a similar problem, but it threw an error: ElementNotVisibleException: Message: element not visible.
Also tried following the documentation example:
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
...
wait = WebDriverWait(chrome, 10)
get_result = wait.until(EC.element_to_be_clickable((By.ID, 'download-link')))
get_result.click()
This downloads some nonsense .tmp file instead.
You need to make a small change in the approach as follows:
Rather waiting for the WebElement as By.ID, 'download-link' with clause element_to_be_clickable, I would suggest you to try to wait for the WebElement as By.ID, 'convert-another' with clause element_to_be_clickable and then click on the DOWNLOAD link as follows:
wait = WebDriverWait(chrome, 10)
wait.until(EC.element_to_be_clickable((By.ID, 'convert-another')))
chrome.find_element_by_css_selector("a#download-link.btn-lg.btn-success").click()
chrome.quit()
Your code is ok. The exception is because you call load_data.submit() after the load_data.send_keys('C:\\path_to_file').
Remove this line:
chrome.get('https://json-csv.com/')
load_data = chrome.find_element_by_id('fileupload')
load_data.send_keys('C:\\path_to_file')
wait = WebDriverWait(chrome, 10)
get_result = wait.until(EC.element_to_be_clickable((By.ID, 'download-link')))
get_result.click()

selenium webdriver tab not switching

selenium webdriver tab swithching not working .
code is :
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
driver = webdriver.Chrome()
driver.get('https://www.google.com')
driver.implicitly_wait(2)
driver.find_element_by_tag_name('body').send_keys(Keys.CONTROL+'t')
driver.switch_to.window(driver.window_handles[-1])
driver.get('http://www.rediff.com')
driver.find_element_by_tag_name('body').send_keys(Keys.CONTROL+'t')
driver.switch_to.window(driver.window_handles[-1])
driver.get('http://www.stackoverflow.com')
driver.switch_to.window(driver.window_handles[0])
In the last line if i change the index from [0] to [1] or [2] there is no change.
Use Keys to have the browser go back to the tab you want. If you want to jump to tab 1 use: driver.find_element_by_tag_name('body').send_keys(Keys.CONTROL + '1')

Element not found in cache - Selenium (Python)

I just wrote a simple webscraping script to give me all the episode links on a particular site's page. The script was working fine, but, now it's broke. I didn't change anything.
Try this URL (For scraping ) :- http://www.crunchyroll.com/tabi-machi-late-show
Now, the script works mid-way and gives me an error stating, ' Element not found in the cache - perhaps the page has changed since it was looked up'
I looked it up on internet and people said about using the 'implicit wait' command at certain places. I did that, still no luck.
UPDATE : I tried this script in a demote desktop and it's working there without any problems.
Here's my script :-
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
import os
import time
from subprocess import Popen
#------------------------------------------------
try:
Link = raw_input("Please enter your Link : ")
if not Link:
raise ValueError('Please Enter A Link To The Anime Page. This Application Will now Exit in 5 Seconds.')
except ValueError as e:
print(e)
time.sleep(5)
exit()
print 'Analyzing the Page. Hold on a minute.'
driver = webdriver.Firefox()
driver.get(Link)
assert "Crunchyroll" in driver.title
driver.implicitly_wait(5) # <-- I tried removing this lines as well. No luck.
elem = driver.find_elements_by_xpath("//*[#href]")
driver.implicitly_wait(10) # <-- I tried removing this lines as well. No luck.
text_file = open("BatchLink.txt", "w")
print 'Fetching The Links, please wait.'
for elem in elem:
x = elem.get_attribute("href")
#print x
text_file.write(x+'\n')
print 'Links have been fetched. Just doing the final cleaning now.'
text_file.close()
CleanFile = open("queue.txt", "w")
with open('BatchLink.txt') as f:
mylist = f.read().splitlines()
#print mylist
with open('BatchLink.txt', 'r') as inF:
for line in inF:
if 'episode' in line:
CleanFile.write(line)
print 'Please Check the file named queue.txt'
CleanFile.close()
os.remove('BatchLink.txt')
driver.close()
Here's a screenshot of the error (might be of some help) :
http://i.imgur.com/SaANlsg.png
Ok i didn't work with python but know the problem
you have variable that you init -> elem = driver.find_elements_by_xpath("//*[#href]")
after that you doing some things with it in loop
before you finishing the loop try to init this variable again
elem = driver.find_elements_by_xpath("//*[#href]")
The thing is that the DOM is changes and you loosing the element collection.

(Python,Selenium) How to minimize firefox window while running

how to minimize a firefox window by using selenium and python
i tried with
try:
body=None
body = driver.find_element_by_tag_name("body")
body.send_keys("{%+" "+N}")
print "entered keys"
except NoSuchElementException:
print "item body is not exists"
code:2
------
body.send_keys(Keys.CONTROL+Keys.ESCAPE+'D')
code:3
------
body.send_keys("{%" "n}")
Nothing worked for me i want to minimize my firefox window while running or after invoking
or run in invisible mode which has no focus
The following code should help:
from selenium import webdriver
from selenium.webdriver.common.action_chains import ActionChains
from selenium.webdriver.common.keys import Keys
driver = webdriver.Firefox()
driver.get("http://www.google.com")
actionChain = ActionChains(driver).key_down(Keys.ALT)
actionChain.send_keys(Keys.SPACE)
actionChain.send_keys("n")
actionChain.perform()

ipython showing gibberish when debugging Django with Netbeans

I'm using Netbeans for coding Django. When I insert:
import ipdb; ipdb.set_trace()
Flow execution gets stopped but it shows gibberish, such as:
[1;32m/path/to/file/models.py[0m(344)[0;36macceptBid[1;34m()[0m
[1;32m 343 [1;33m [1;32mimport[0m [1;37mipdb[0m[1;33m;[0m [1;37mipdb[0m[1;33m.[0m[1;37mset_trace[0m[1;33m([0m[1;33m)[0m[1;33m[0m[0m
[0m[1;32m--> 344 [1;33m [1;32mreturn[0m [1;37mself[0m[1;33m.[0m[1;37msenderId[0m[1;33m([0m[1;33m)[0m [1;33m==[0m [1;37muser_obj[0m[1;33m.[0m[1;37mid[0m[1;33m[0m[0m
[0m[1;32m 345 [1;33m[1;33m[0m[0m
[0m
I can use next, skip and everything from pdb. But I can not see where I'm in the code, which forces me to use pdb instead of ipdb.
for me worked fine with just commenting the line and add a pass sentence in ipdb/__main__.py
from IPython.utils import io
def update_stdout():
# setup stdout to ensure output is available with nose
#THIS IS THE LINE TO COMMENT #########################
#io.stdout = sys.stdout = sys.__stdout__
#REMEMBER TO ADD pass
pass
else:
from IPython.Debugger import Pdb, BdbQuit_excepthook
from IPython.Shell import IPShell
from IPython import ipapi
These are ANSI escape codes, which are used for the text colours in ipdb's output. For some reason, the terminal you're debugging in is not accepting the codes and is printing them as text. You may be able to find a setting in NetBeans to either change what the terminal is reporting itself as, or what it actually accepts.
This is the problem:
https://github.com/gotcha/ipdb/issues/31
and you can solve it by commenting out a line in ipdb/_ _ main _ _.py
What I have done to be able to use ipdb with Django Netbeans, is disable coloring output in ipdb. There are several ways to do this. If you have installed ipdb through easy_install you can edit the code in __init__.py leaving it like:
import sys
from IPython.Debugger import Pdb
from IPython.Shell import IPShell
from IPython import ipapi
shell = IPShell(argv=[''])
def set_trace():
Pdb("NoColor").set_trace(sys._getframe().f_back)
Also you can create yourself a hook to import ipdb without colors. I hope this helps :)