Analyzing HTTP headers in Firebug - cookies

I was just trying to analyse all the HTTP header fields in Firefox plugin - Firebug. First I logged out from Stack Overflow and then cleared all the cookies from my browser.
Then I went to the Stack Overflow's home page. I mean while saw the HTTP request and response header fields. This is what I saw:
Response Headers
Via 1.0 proxy_server
Content-Length 135
Date Mon, 05 Mar 2012 06:01:33 GMT
Content-Type application/json
Cache-Control private
X-Cache MISS from sampark.ncb.ernet.in
Request Headers
Host stackoverflow.com
User-Agent Mozilla/5.0 (Windows NT 6.0; rv:12.0a2) Gecko/20120303 Firefox/12.0a2
Accept application/json, text/javascript, */*; q=0.01
Accept-Language en-us,en;q=0.5
Accept-Encoding gzip, deflate
Proxy-Connection keep-alive
X-Requested-With XMLHttpRequest
Referer http://stackoverflow.com/
Cookie __qca=P0-383120279-1330927291125; __utma=140029553.974890682.1330927291.1330927291.1330927291.1; __utmb=140029553.1.10.1330927291; __utmc=140029553; __utmz=140029553.1330927291.1.1.utmcsr=(direct)|utmccn=(direct)|utmcmd=(none); gauthed=1
There is a cookie included in the request header. But as I said I have removed all the cookies from my browser. How is the cookie included in the request? What is actually happening here?
I did as Andy Davies told. I first cleared all the cookies, restarted Firefox and then went to www.stackoverflow.com. Firebug shows this:
GET http://stackoverflow.com/tags/ios/subscriber-info?_=1331946084371
The headers for the above request contained:
Cache-Control private
Content-Encoding gzip
Content-Length 390
Content-Type text/html; charset=utf-8
Date Sat, 17 Mar 2012 01:01:19 GMT
Vary Accept-Encoding<Br>
Request Headers
Accept text/html, */*; q=0.01
Accept-Encoding gzip, deflate
Accept-Language en-us,en;q=0.5
Connection keep-alive
Cookie __utma=140029553.1336172974.1331946082.1331946082.1331946082.1; __utmb=140029553.1.10.1331946082; __utmc=140029553; __utmz=140029553.1331946082.1.1.utmcsr=(direct)|utmccn=(direct)|utmcmd=(none); __qca=P0-115511794-1331946081644; gauthed=1
Host stackoverflow.com
Referer http://stackoverflow.com/
User-Agent Mozilla/5.0 (Windows NT 6.0; rv:12.0a2) Gecko/20120303 Firefox/12.0a2
X-Requested-With XMLHttpRequest
If this is not the first request, then why is it not showing the first request?

Did you restart the browser after clearing the cookies as, from my memory, some browsers don't clear the cookies for any sites currently open?
The snippet you've posted looks like a JSON response part way through loading the page, so it is not the initial request for the HTML page.
When the HTML page would have been requested again, the Google Analytics cookie (which is what you've got above) would have been resent, so any subsequent components on the page will also get the cookie too.

Related

Odd Behaviour of Icon Caching

Consider this view that generates an ico image:
from django.http import HttpResponse
from app.somewhere import Favicon
# View URL: `/<str:colour>.ico`
def favicon( request, colour ):
response = HttpResponse(
Favicon.render( colour ),
status=200
)
response['Content-Type'] = 'image/x-icon'
response['Cache-Control'] = 'public, max-age=31536000'
return response
Favicon.render() returns a valid byte stream, do not pay any attention on that.
Here is a link element in head of my HTML document:
<link rel=icon href=/7f9fa4.ico>
Now comes the question: why each time I reload the page, my browser, Chromium 73, makes a request to /7f9fa4.ico, instead of retrieving the icon from cache? If I will open /7f9fa4.ico in a new tab, first time request to the server would be sent, further my browser will retrieve an image from cache; now tell me what's wrong with the browser-caching system.
Here is a request (cookies and preferences are omitted):
GET /7f9fa4.ico HTTP/1.1
Host: localhost:8000
Connection: keep-alive
Pragma: no-cache
Cache-Control: no-cache
User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/73.0.3683.86 Safari/537.36
Referer: http://localhost:8000/
And these are response headers:
HTTP/1.1 200 OK
Date: Mon, 03 Jun 2019 07:03:58 GMT
Server: WSGIServer/0.2 CPython/3.6.8
Content-Type: image/x-icon
Cache-Control: public, max-age=31536000
X-Frame-Options: SAMEORIGIN
Content-Length: 196
Console output (if it somehow could help):
[05/Jun/2019 09:17:42] "GET /7f9fa4.ico HTTP/1.1" 200 196
Also, if I will remove link element from head, browser will make requests to /favicon.ico (which in my case just mirrors /ffffff.ico) each time I reload the page with the same effect.
What you may find is that this request is being made to validate the cached content. I noticed that the request you sent to the server has Cache-Control: no-cache and Pragma: no-cache.
https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Cache-Control#Cacheability
no-cache
Forces caches to submit the request to the origin server for validation before releasing a cached copy.
So it forces caches to submit the request for validation.
https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Pragma#Directives
no-cache
Same as Cache-Control: no-cache. Forces caches to submit the request to the origin server for validation before releasing a cached copy.
These state that the browser would be expected to send a request to your server for "validation" before it uses the cache icon.

cookies not writtent even when I receive set-cookie response header

Browser is ignoring reponse cookies (csrftoken + seesionid). document.cookie() returns empty string and chrome developer tool shows this site has no cookies
How to solve this ?
FRONT : Angular 2 (localhost:4200)
Back : Django/DRF (localhost:8000)
Login route : [post] /login
Response header :
Access-Control-Allow-Credentials:true
Access-Control-Allow-Origin:http://localhost:4200
Allow:POST, OPTIONS
Date:Wed, 05 Apr 2017 07:38:24 GMT
Server:WSGIServer/0.2 CPython/3.5.2
Set-Cookie:sessionid=d5v1mri12bniyvyqqt55ar8mfl9mr2jk; expires=Wed, 19-Apr-2017 07:38:24 GMT; HttpOnly; Max-Age=1209600; Path=/
Set-Cookie:csrftoken=5PcTF8aQ1O79gdrylZcGchnmKyRy6zwS3kL2jR5dY2CMdjPfEYyhkoJjOzsDZuvj; expires=Wed, 04-Apr-2018 07:38:24 GMT; Max-Age=31449600; Path=/
Vary:Accept, Cookie, Origin
X-Frame-Options:SAMEORIGIN
Request header :
Accept:application/json, text/plain, */*
Accept-Encoding:gzip, deflate, br
Accept-Language:en-US,en;q=0.8,fr;q=0.6
Connection:keep-alive
Content-Length:51
content-type:application/json
Cookie:sessionid=d5v1mri12bniyvyqqt55ar8mfl9mr2jk; csrftoken=sml3uocRIeiB3KfHSnNkJXBJn3QAFN3p7lLtdvhrALgUwoVnfNjGM5PIy2L3UHls
Host:127.0.0.1:8000
Origin:http://localhost:4200
Referer:http://localhost:4200/
User-Agent:Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Ubuntu Chromium/56.0.2924.76 Chrome/56.0.2924.76 Safari/537.36
Cookies are not shared among different ports of a host (localhost in your case). The browser shows you content from localhost:4200, but the cookies are set to localhost:8000. They are correctly set and used, as can be seen in your second listing titled "Request header". So to see them in the browser, you would have to open a URL from localhost:8000. Even then the sessionid would not be listed in document.cookie(), because it's marked as "HttpOnly" (which means not available to JavaScript).
If you are use angular cli you could set proxy.
ng serve --proxy-config proxy.conf.json
sample of proxy.conf.json
{
"/api": {
"target": "http://localhost:3000",
"secure": false
}
}
Or local development use
https://chrome.google.com/webstore/detail/allow-control-allow-origi/nlfbmbojpeacfghkpbjhddihlkkiljbi

Read Request Payload in ColdFusion

Trying out Content Security Policy and using the Report Only method and the data is sending a POST (See edits, supposed to be POST, but Request Headers say GET) to my page which I then use to email myself. In the browser developer tools I can see the data in the Request Payload, but I don't know how to access this scope. The FORM scope is empty and the GetHttpRequestData() doesn't show a Payload key.
How do I access the data in the Request Payload in ColdFusion?
EDIT: In the dev tools I can see in the network tab a POST to the page with 4 sections: General, Response Headers, Request Headers and Request Payload. I need to access the Request Payload data.
Request Headers:
GET /campuses/content-report/ HTTP/1.1
Host: domain.com
Connection: keep-alive
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/43.0.2357.130 Safari/537.36
Accept: */*
Referer: http://domain.com/page/
Accept-Encoding: gzip, deflate, sdch
Accept-Language: en-US,en;q=0.8
Cookie: **[Removed]**
Request Payload:
{
"csp-report":{
"document-uri":"http://domain.com/page/",
"referrer":"http://domain.com/",
"violated-directive":"default-src 'self' 'sha256-L2Tc50iUaBz2udc-dnkwO-FKzrsl5cLnzkFu5LgX5ao=' https://stats.g.doubleclick.net http://netdna.bootstrapcdn.com http://www.google-analytics.com https://www.google.com https://fonts.gstatic.com https://ajax.googleapis.com https://cdn.jsdelivr.net https://maxcdn.bootstrapcdn.com http://www.etutoring.org",
"effective-directive":"img-src",
"original-policy":"default-src 'self' 'sha256-L2Tc50iUaBz2udc-dnkwO-FKzrsl5cLnzkFu5LgX5ao=' https://stats.g.doubleclick.net http://netdna.bootstrapcdn.com http://www.google-analytics.com https://www.google.com https://fonts.gstatic.com https://ajax.googleapis.com https://cdn.jsdelivr.net https://maxcdn.bootstrapcdn.com;style-src 'self' 'unsafe-inline' https://fonts.googleapis.com https://maxcdn.bootstrapcdn.com;script-src 'self' https://www.google-analytics.com https://ajax.googleapis.com https://cdn.jsdelivr.net 'nonce-3E89EBDB49F712C7D90D1B39E348BBBF';report-uri /campuses/content-report;",
"blocked-uri":"https://www.google-analytics.com",
"status-code":200
}
}
GetHttpRequestData contains Request Headers but not Request Payload. I did notice that the website I read said the Security policy would send the data via POST but the Headers say GET. Would that cause the issue? Should they be in the URL scope? Not at my machine at the moment.
So I was looking in the right spot, GetHttpRequestData but what I didn't realize was I setup the url to not have a trailing slash and my IIS Rewrite rules was redirecting to the / version of the page and the header data was being lost in the process.

play framework cookies cors

now I am using play framework 2.3, how to share cookie on cors (cross domain)
i have play application on port localhost:9000 and i want to share cookies to my client running on localhost:5000
client cannt get the cookies
i try with this in play framework app:
response().setCookie(AUTH_TOKEN, authToken); //with setting on application.conf (session.domain="http://localhost:5000") cookies sent but cannt read by javascipt /failed
response().setCookie(AUTH_TOKEN, authToken,10000,"localhost:5000","/"); //fail
response().setCookie(AUTH_TOKEN, authToken,10000,"127.0.0.1:5000","/*"); //fail
response().setCookie(AUTH_TOKEN, authToken,10000,"127.0.0.1:5000","/*"); //fail
and this is response on chrome :
Remote Address:127.0.0.1:9000
Request URL:http://localhost:9000/common/login
Request Method:POST
Status Code:200 OK
Request Headersview source
Accept:application/json, text/javascript, */*; q=0.01
Accept-Encoding:gzip,deflate,sdch
Accept-Language:en-US,en;q=0.8,id;q=0.6,ms;q=0.4
Connection:keep-alive
Content-Length:33
Content-Type:application/x-www-form-urlencoded; charset=UTF-8
Host:localhost:9000
Origin:http://localhost:5000
Referer:http://localhost:5000/index-login.html
User-Agent:Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/35.0.1916.114 Safari/537.36
Form Dataview sourceview URL encoded
username:adilramdan
password:1234
Response Headersview source
Access-Control-Allow-Headers:X-Requested-With, Content-Type, X-AUTH-TOKEN
Access-Control-Allow-Origin:*
Content-Length:52
Content-Type:application/json; charset=utf-8
Set-Cookie:authToken=8cd7d5cc-600a-42a0-ab79-d7ff2b4f71b0; Expires=Tue, 15 Jul 2014 02:30:09 GMT; Path=/
<------COOKIES IS AVAILABLE ON RESPONSE SERVER BUT JAVA SCRIPT CLIENT SAY NO COOKIES FOUND
how the right way?
any one can help me?
You may need to use the Access-Control-Allow-Credentials header. If you are using AJAX/JQuery you must set
withCredentials = true
In your request. The server must also respond with header
Access-Control-Allow-Credentials: true
Note that if you do this you must specify a value for Access-Control-Allow-Origin. The browser will not accept Access-Control-Allow-Credentials header while Access-Control-Allow-Origin is a wildcard.
See the resources here:
https://developer.mozilla.org/en-US/docs/Web/HTTP/Access_control_CORS#Access-Control-Allow-Credentials
http://www.html5rocks.com/en/tutorials/cors/

Django CSRF failure on ajax post requests on Opera only

Ok so AJAX POST requests work fine in Mozilla and Chromium, but fail in Opera. I get the standard CSRF error (403). I tried different versions of Opera and they failed in every one I tried. Btw, I'm using the jquery/django snippet that sets X-CSRFToken in the header, as verified in the "bad Opera request" below.
I made a view in a different project that was very simple and ajax post requests worked fine in Opera. I looked at the request details and see differences. The good request doesn't set any weird X-Opera-Info and other opera params even thought I'm using the same browser. If this is the issue, is there a way to remove those extra params? Or does anyone have any other advice or ideas on what the issue may be? I know it's not my view function because I tried just returning an HttpResponse immediately and even that gets 403'd. Thanks a million guys.
####################
OPERA GOOD REQUEST
##############
Request details
POST /test HTTP/1.1
User-Agent: Opera/9.80 (X11; Linux i686; U; en) Presto/2.7.62 Version/11.00
Host: 127.0.0.1:8000
Accept-Language: en-US,en;q=0.9
Accept-Charset: iso-8859-1, utf-8, utf-16, *;q=0.1
Accept-Encoding: deflate, gzip, x-gzip, identity, *;q=0
Referer: http://127.0.0.1:8000/test
Cookie: csrftoken=1c6441404c991f7ae3b6d7d49f91f280
Cookie2: $Version=1
Connection: Keep-Alive, TE
TE: deflate, gzip, chunked, identity, trailers
Content-Length: 6
Content-Type: application/x-www-form-urlencoded
Accept: */*
X-CSRFToken: 1c6441404c991f7ae3b6d7d49f91f280
X-Requested-With: XMLHttpRequest
Content-Transfer-Encoding: binary
###################
OPERA BAD REQUEST
####################
Request details
POST http://facebook.example.com/remove-person HTTP/1.1
User-Agent: Opera/9.80 (X11; Linux i686; U; en) Presto/2.7.62 Version/11.00
Host: facebook.example.com
Accept-Language: en-US,en;q=0.9
Accept-Charset: iso-8859-1, utf-8, utf-16, *;q=0.1
Accept-Encoding: deflate, gzip, x-gzip, identity, *;q=0
Referer: http://facebook.example.com/
Cookie: signed_request=5-f0_7pZLILrp6MLocsdMoNYAaZr-wCnU2cPbLC1bZg.eyJhbGdvcml0aG0iOiJITUFDLVNIQTI1NiIsImV4cGlyZXMiOjEzMDg4MTYwMDAsImlzc3VlZF9hdCI6MTMwODgxMTQyNywib2F1dGhfdG9rZW4iOiIyMjMyNDY5NDEwMjc3MTR8Mi5BUURVRGM2ZFFLSElnN1h3LjM2MDAuMTMwODgxNjAwMC4xLTU0MDIwMjZ8b2QtX1diNTh3aG1wTnNHYUh4cTNtOVBpWkswIiwidXNlciI6eyJjb3VudHJ5IjoidXMiLCJsb2NhbGUiOiJlbl9VUyIsImFnZSI6eyJtaW4iOjIxfX0sInVzZXJfaWQiOiI1NDAyMDI2In0; csrftoken=d4cdc6a75ed264d295a410dd98982c42; fbs_223246941027714="access_token=223246941027714%7C2.AQBlhzavZjzd8c7J.3600.1308819600.1-5402026%7CdsD6VESpGJb3m0EdD1mhFZtDI24&base_domain=example.com&expires=1308819600&secret=QaTNS988wl0FU6A0LG9qDQ__&session_key=2.AQBlhzavZjzd8c7J.3600.1308819600.1-5402026&sig=61e7e13091501f35793d3cda8c20835b&uid=5402026"
Cookie2: $Version=1
Connection: Keep-Alive, TE
TE: deflate, gzip, chunked, identity, trailers
Content-Length: 14
X-Opera-Info: ID=448, p=4, f=15, sw=1440, sh=900
X-Opera-ID: e79c37b56a58510d26b56882453bddb6d2c2dae858129139113f6346ea23ca6b
X-Opera-Host: r18-02:12420
X-OA: 1322 b5834cb13259fbd50b87b576b5e8b9a8bcc1384478c2ea79cc65614dc1b67c27
X-OB: evenes
Content-Type: application/x-www-form-urlencoded
Accept: */*
X-CSRFToken: d4cdc6a75ed264d295a410dd98982c42
X-Requested-With: XMLHttpRequest
Content-Transfer-Encoding: binary
I recently ran into a similar problem. I was trying to do, for the first time, a post from AJAX on a view where Django wasn't sending a CSRF cookie. For reasons I can't explain, this was working on all browsers I tried except Opera.
This scenario is described in the Django docs, and they suggest using the ensure_csrf_cookie decorator.
Another thing you can do if it is too burdensome to wrap all the potential views with that decorator is to add something like this to your base template:
<script>
var csrf_token = "{{ csrf_token }}";
</script>
And then whenever you do an AJAX post, always include the key pair csrfmiddlewaretoken: csrf_token with your POST data.
Once I did the above, my posts with Opera started working.
Does it work if you disable Opera Turbo? These extra headers seem to be Turbo-related and perhaps this makes some kind of difference.
X-CSRFToken: {{ csrf_token }}}
value should be send from client in AJAX request HTTP headers to be recognized by Django