I have a PyGtk app with an embedded WebKit.WebView. Users login to our service through the embedded browser and they can make payments in PayPal page.
This app is used by different users in the same session, so the problem is that PayPal bakes cookies and remembers the email of the last payment.
I just want to complete the payment and delete PayPal-related cookies from my WebView but I cannot find any documentation about removing cookies.
To re-create the webView is not an option because my web page is quite heavy to load.
Any help would be highly appreciated
I figured out by myself.
First I added a custom cookiejar. This allows me to handle the cookies
from gi.repository import Soup
cookiejar = Soup.CookieJarText.new("cookies/biscottini.txt", False)
cookiejar.set_accept_policy(Soup.CookieJarAcceptPolicy.ALWAYS)
session = gi.repository.WebKit.get_default_session()
session.add_feature(cookiejar)
Then I found the Soup class structure and wrote down this simple snippet to clear the cookies (there's also a filter to delete only PayPal cookies)
def clearCookies(deleteAll=False):
for cookie in cookiejar.all_cookies():
if deleteAll or ('paypal' in cookie.get_domain().lower()):
cookiejar.delete_cookie(cookie)
I now have a problem though: after clearing PayPal's cookies if I try to make a payment, I get the error: "Message Corrupt"
Related
I have a server side application that uses cookies for session management. The browser has some script that sends an ajax request to add information to the session. This is working well and in production.
The business wants to be able to insert this application in other companies' websites via iframes. ie myapp.com is in an iframe in otherbusiness.com and when the user clicks a button in the application in the iframe launched from myapp.com, it sends a request with a cookie that contains the session id to update the user's session on the myapp.com server.
For the browser to be able to send a cookie, 3rd party cookies needs to be enabled by setting the cookie options of SameSite=None and Secure. This works for all browsers except Safari.
Safari no longer accepts 3rd party cookies.
The only solution I can come up with is to use session ids in the URL but this is a little cumbersome.
Can anyone suggest a better option or perhaps a good implementation of session ids in the url?
I used hidden html fields to pass the session id and expiration.
My server side code checks for a cookie if it cannot find it, looks for the session id and expiration in the hidden fields.
This avoids security issues with passing the id in the url. It is a little clumsy to implement but it works.
I want to crawl a website which has a strong security protocol and want to crawl data as fast as possible. Thus I thought I need a multi-login-cookie, multi-user-agent, and multi-proxy crawler.
I have tens of usernames and passwords and I can login using each one and get all the cookies. To hide the identity of my crawler I thought I should also replace the user-agent setting and my IP. I have found many user agents and proxies.
I learned that the cookie is needed each time I send a request to the server, and that cookie should be of the same identity and contain the information of the previous request and the corresponding response. I've gained the knowledge of how to pass it through requests without logging in from this answer. And I know two ways to login in, one outside the scrapy(by passing the cookie to the cookiesmiddleware in the middleware.py file:
from cookies import cookies # script written to login some accounts and return the cookies
import random
class CookiesMiddleware(object):
def process_request(self, request, spider):
cookie = random.choice(cookies)
request.cookies = cookie
) and another inside it.
What's more in the middleware.py file I passed the user agents randomly in the same as for cookies to the scrapy requests.
My question is: if I pass the cookies randomly as aforementioned, will one spider get the same cookie each time it sends a request? If not the server side will detect me as a bot and block me. What's worse, the same applies to the user-agents and proxies. How to bond each trinity(login cookie, user-agent and proxy) starting from the login, extending the aforesaid answer both in the horizontal and vertical dimension?
To be more precise, should I pass the login cookie in the form of {cookies= user1_cookie} or { meta={'cookiejar': user1_cookie},? And should I pass the user agent and proxy in the meta parameter?
Thanks. Please kindly point me in the right direction, and any suggestions will be highly received and appreciated.
Seems like you are looking for cookiejar. It will allow you to store multiple cookie sessions in single spider session.
Using middleware for random cookies is a bad idea since cookies in most cases store your whole browsing sessions.
How is it possible to save the session id or the cookies I need to stay logged in at a website after the login, when I'm using HtmlUnitDriver from Selenium 2.0.
Enabling the cookies at first:
webClient.getCookieManager().setCookiesEnabled(true);//enable cookies
Check out the Cookies it has some more info and extra code you might want to check out
I created a Google site page with 5 links on it. Is it possible to create on my site a script or something that stores in a cookie the link on which the user has clicked, and then the next time he will connect to the page, he will be automatically redirected to the link he clicked on ? For information, the user connect to the site with his Google email account.
How can I do that please?
Thank you very much in advance for your help
While it is possible to read cookies and redirect using JavaScript inside a Google Page (using widgets), browsers will not allow you to set cookies for a completely different domain for obvious security reasons.
Related:
How to set a cookie for another domain
Cross-Domain Cookies
What's your favorite cross domain cookie sharing approach?
You could theoretically try and send an AJAX request from the Google Page with a "where should I direct this user to?" and expect a URL or a null.
See:
CORS $.ajax session cookies (access-control-allow-credentials & withCredentials=true)
Cross domain POST request is not sending cookie Ajax Jquery
But overall, your task is not as straightforward as it may seem. The browser will, fortunately, not play along.
I am trying to use rasterize.js from the phantomjs' exmaple folder to capture a web page into PDF. The problem I am having is that the web page requires user log in. Then I saw there is a cookie option for Phantomjs. The web site is Django powered just in case it matters.
So I logged into the site manfully, opened the firebug, and typed document.cookie in firebug. I got the cookie settings and put into my cookies.txt.
[http://localhost:7000]
csrftoken: f3da886168fae33b840e7f6c93240dff
sessionid: 27e90c3214b0ec94dadc739665724708
django_language: en
Then I used the rasterize.js to create the pdf like below:
phantomjs --cookies-file=cookies.txt examples/rasterize.js http://localhost:7000/reports /tmp/report.pdf
But the result is still the log in page.
Have you tried automating the login process in PhantomJS, making sure to include cookies param? This way phantomjs can authenticate a user just like a browser would, using cookies.
Another option would be to inspect the requests using a proxy like fiddler, compare the request made from firefox that works and the request made from phantomjs that is not authenticated. The information in the header should tell you what you are missing.
You can add cookies by using the function phantomjs provides:
http://phantomjs.org/api/webpage/method/add-cookie.html