I try to get cookie information with CookieManager in Android WebView. However, all it does is providing a key value pair of cookies, no additional information.
I also tried this post and use connection.getHeaderFields.get("Set-Cookie"); to get all cookies received from server. But from what I can tell, I don't see all cookies that I am supposed to receive from server. This can also be found by some cookies in CookieManger receive a new value but not show in Set-Cookie field.
Ideally, I would like to intercept every response and inspect their cookies with all information from server such as name, value, max-age, secure, etc. Any suggestion how to do this with Android WebView (WebViewClient)?
Related
What I need:
I need to get all the currently applicable cookies for a page state. For example, I navigate to a site in the US, the browser has a lot of cookies populated by that page. I then decline all cookies on that site. Afterwards the browser reports much fewer cookies for that page. I need to be able to get that list of applicable cookies for the page as displayed in Chrome's View site information pop-out dialog left of the URL.
What I've tried:
I'm currently using Puppeteer, though I've also tried using Playwright. Both can utilize the Chrome Devtools Protocol (CDP) to gather information about a browser session and other related HTTP data.
I have a use case where I need to see the current state of cookies as it pertains to a web page. CDP provides two methods to accomplish this. Network.getAllCookies and Network.getCookies.
Network.getCookies returns cookies for a given URL, or else it provides all the cookies for the URL of the current page when no URL is provided. The problem with this is it's not a full picture. Network.getCookies does not include 3rd party cookies unless that domain is specifically requested. But there's no way for me to know all the 3rd party cookies for a given page.
If I switch it up and use Network.getAllCookies, that has too much information. It returns with all the cookies currently known to the browser session, regardless of applicability to the current page. For example, if I visit site A, then nav to site B, then I call Network.getAllCookies, I get all of the cookies from both pages since they are all in the same browser context. But I only want the 1st and 3rd party cookies for the current page I'm on.
In a headful browser it is possible to see all the applicable 1st and 3rd party cookies are for a site. If you click on the lock icon (secure site) to the left of the url, there is an option to view cookies. This is the most complete and accurate view of cookies I've come to know. Somehow the browser knows what 3rd party cookies originated from the current page and requests made by the current page and therefore displays them there. But how? And how can I get that list through CDP?
Note that document.cookie is useless as it does not include httpOnly cookies, and to my knowledge also doesn't return detailed cookie information (such as sameSite, secure, expiry, etc.)
I've tried using the CDP session directly attached to both of the above mentioned libraries. I've tried monitoring HTTP requests using Puppeteer and gathering set-cookie data from response headers, but CDP is unreliable when it comes to providing all the response header set-cookie information.
TLDR:
I'm expecting to programmatically retrieve only the relevant cookies for a page visit up to that point. If cookies become irrelevant in that page session, I don't want to receive them any more.
Network.getAllCookies returns too many cookies as it gives the whole browser context and not just the page relevant cookies.
Network.getCookies only returns cookies by a url.
document.cookie only returns application cookies.
How can I get all the 1st and 3rd party cookies that pertain to my current page?
I want this exact list:
I have a server side application that uses cookies for session management. The browser has some script that sends an ajax request to add information to the session. This is working well and in production.
The business wants to be able to insert this application in other companies' websites via iframes. ie myapp.com is in an iframe in otherbusiness.com and when the user clicks a button in the application in the iframe launched from myapp.com, it sends a request with a cookie that contains the session id to update the user's session on the myapp.com server.
For the browser to be able to send a cookie, 3rd party cookies needs to be enabled by setting the cookie options of SameSite=None and Secure. This works for all browsers except Safari.
Safari no longer accepts 3rd party cookies.
The only solution I can come up with is to use session ids in the URL but this is a little cumbersome.
Can anyone suggest a better option or perhaps a good implementation of session ids in the url?
I used hidden html fields to pass the session id and expiration.
My server side code checks for a cookie if it cannot find it, looks for the session id and expiration in the hidden fields.
This avoids security issues with passing the id in the url. It is a little clumsy to implement but it works.
In my application I have filter that sets cookie:
HttpContext.Response.Cookies.Append("myCookie", "value")
When I print cookie value in my view:
HttpContext.Request.Cookies["myCookie"];
it is not there. If I refresh page cookie is set. This happens because using Response.Cookies.Append does not update Request.Cookies collection. Is there way to view current cookies with changes made during request?
In my web application I have IAsyncActionFilter, that updates cookie, however corrected value is visible only after page is refreshed, and I would like to finish current request with new value. I know that it will be set by browser when response finishes, but I already know that I have new value for that cookie, and I would like to propagate this value to views reading cookies.
No. This is how cookies work. They are sourced from the client. In other words, the cookie is set by the client after it receives the response from the server with the Set-Cookie header. It's then only after the client makes another request, sending the cookie back, that it exists server-side. It's not clear what you're trying to achieve ultimately here, but you need to force a new request after setting a cookie, to access that cookie, even if that's simply returning a redirect to the same page.
I'm setting a cookie in a response from my web service. The set-cookie header is coming through, and I can see the cookie in the network tab in Chrome, but the cookie isn't being stored. It doesn't show up in the resources->cookies tab, and the cookie isn't sent with subsequent requests. Nothing shows up in the JS console. I've also tried leaving the domain field off the cookie, but it still isn't stored.
Is there a way to debug the browser to understand why the cookie was rejected from being stored?
Turns out it had to do with the way I was making the request. I expected fetch() to work the same way as XHR requests. Setting credentials: 'include' on my fetch call resolved the problem. See 5.6.14 of the fetch spec
I don't really understand when session is created and per what entity it is created (per ip, per browser, per logged in user). I see in documentation that sessions by default is created per visitor - but what is visitor (browser or ip)?
What are HTTP sessions?
To display a webpage your browser sends an HTTP request to the server, the server sends back an HTTP response. Each time you click a link on website a new HTTP transacation takes place, i.e. it is not a connection that is persistant over time (like a phone call). Your communication with a website consists of many monolitic HTTP transactions (tens or hundres of phonecalls, each phonecall being a few words).
So how can the server remember information about a user, for instance that a user is logged in (ip addresses are not reliable)? The first time you visit a website, the server creates a random string, and in the HTTP response it asks the browser to create a so called HTTP cookie with that value. A cookie is really just a name (of the cookie) and a value. If you go to a simple session-enabled Django site, the server will ask your browser to set a cookie named 'sessionid' with such a random generated value.
The subsequent times your browser will make HTTP requests to that domain, it will include the cookie in the HTTP request.
The server saves these session ids (for django the default is to save in the database) and it saves them together with so called session variables. So based on the session id sent along with an HTTP request it can dig out previously set session variables as well as modify or add session variables. If you delete your cookies (ctrl+shift+delete in Firefox), you will realize that no website remembers you anymore (Gmail, Facebook, Django sites, etc.) and you have to log in again. Most browsers will allow you to disable cookies in general or for specific sites (for privacy reasons) but this means that you can not log into those websites.
Per browser, per window, per tab, per ip?
It is not possible to log into different GMail accounts within the same browser, not even from different windows. But it is possible to log in to one account with Firefox and another with Chrome. So the answer is: per browser. However, it is not always that simple. You can use different profiles in Firefox, and each can keep different cookies and thus you can log into different accounts simultaneously. There are also Firefox plugins for keeping multiple sessions, e.g. MultiFox.
The session all depends on which session cookie your browser sends in it's HTTP request.
Play around
To get the full understanding of what is going on, I recommend installing the FireBug and FireCookie plugins for Firefox. The above screenshots are taken from FireBug's net panel. FireCookie will give you an overview of when and which cookies are set when you visit a site, and will let you regulate which cookies are allowed.
If there is a server side error, and you have DEBUG=True, then the Django error message will show you information about the HTTP request, including the cookies sent
It's browser (not IP). A session is basically data stored on your server that is identified by a session id sent as a cookie to the browser. The browser will send the cookie back containing the session id on all subsequent requests either until the browser is closed or the cookie expires (depending on the expires value that is sent with the cookie header, which you can control from Django with set_expiry).
The server can also expire sessions by basically ignoring the (unexpired) cookie that the browser sends and requiring a new session to be started.
There is a great description on how sessions work here.