The block of code for your reference is given below:
String hostname = "Hostname Value";
URI uri = new URI(hostname + "/solr/add/story/" + story.getId() + ".html");
final HTTPConnection con = new HTTPConnection(uri);
con.setAllowUserInteraction(false);
final HTTPResponse response = con.Get(uri.getPathAndQuery());
Here, while accessing the response i am getting below exception:
[ WARN] [com.thestreet.cms.integration.solr.SolrService] 12/02/2013 22:52:54-Unable
update front end search engine index with story 10446446
java.net.ProtocolException: Bad Set-Cookie header: FV=OID-|PID-|MID-|PUC-|DATE-
529D5595; path=/; expires=Tue, 01-Jan-2035 00:00:00 GMT; domain=.thestreet.com;,
BRIS=C0.A8.41.91|529D55951FB74EF; path=/; expires=Tue, 01-Jan-2035 00:00:00 GMT;
domain=.thestreet.com;,
RGIS=-1386042773,192.168.65.145,BA42A8C0,1076F795713A21E010941898- 0-1386042773-;
path=/; expires=Tue, 01-Jan-2035 00:00:00 GMT; domain=.thestreet.com;,
JSESSIONID=8A8A377CF937F6184D3F4774CC6F4CBA; Version=1; Path="/"; HttpOnly
No '=' found for token starting at position 432
at HTTPClient.Cookie.parse(Cookie.java:243)
at HTTPClient.CookieModule.handleCookie(CookieModule.java:454)
at HTTPClient.CookieModule.responsePhase1Handler(CookieModule.java:403)
at HTTPClient.HTTPResponse.handleResponse(HTTPResponse.java:724)
at HTTPClient.HTTPResponse.getStatusCode(HTTPResponse.java:190)
at com.thestreet.cms.integration.solr.SolrService$1.run(SolrService.java:450)
at java.lang.Thread.run(Thread.java:722)
This seems to be caused by Httponly bit in the cookie header since its not in the form of key-value pair. Is there any way to avoid a cookie header or cookie check while reading the response? Please help.
Thanks in advance.
The problem is your http request/response contains "HttpOnly" in its header. It appears to be that this value is not supported by application servers anymore. In order to fix this I wrote a workaround where "HttpOnly" is removed from the response on a server side.
String header = resp.getHeader("Set-Cookie");
if (header != null && header.endsWith("HttpOnly")) {
resp.setHeader("Set-Cookie", header.substring(0, header.length() - 8));
}
But the best solution is to remove "HttpOnly" from the header on the http client side.
Related
Environment: Worklight 6.2
HTTP Adapter
Response header from backend contains cookies with key (WC_PERSISTENT)
adapter.xml has
<connectionPolicy xsi:type="http:HTTPConnectionPolicyType"
cookiePolicy="IGNORE_COOKIES" maxRedirects="20">
But responseHeaders contains duplicate keys, it seems Worklight server is adding this key.
WC_PERSISTENT cookie is used to track session management in websphere commerce and session timeout is managed in WAS console.
I am new to Websphere, how can I remove the duplicate cookie or not let Worklight add it in responseHeader?
Sample cookie:
"arr": [
"WC_PERSISTENT=pLeLrvmT3fn9TbLo7qEiK1Q5Ss0%3d%0a%3b2015%2d09%2d23+04%3a39%3a39%2e966%5f1442983179966%2d465081%5f0; Expires=Thu, 22 Sep 2016 04:39:39 GMT; Path=\/; Domain=.xyz.com",
"COSTEXCLVAT=disabled; Path=\/; Domain=.xyz.com",
"XYZStats.trailingTag=XYZStats.member,private|; Path=\/; Domain=.xyz.com",
"WC_SESSION_ESTABLISHED=true; Path=\/; Domain=.xyz.com",
"WC_PERSISTENT=kIPpdZQy9v8DUCpCLMHSg6zpkYM%3d%0a%3b2015%2d09%2d23+04%3a39%3a40%2e057%5f1442983179966%2d465081%5f7; Expires=Thu, 22 Sep 2016 04:39:39 GMT; Path=\/; Domain=.xyz.com",
"WC_AUTHENTICATION_1632733901=1632733901%2cuIeVQv0MRMpXTg32wGaT9zj1QVM%3d; Path=\/; Domain=.xyz.com; Secure",
"WC_ACTIVEPOINTER=%2d20%2c7; Path=\/; Domain=.xyz.com",
"WC_USERACTIVITY_16327111111=163211111%2c7%2c0%2cnull%2cnull%2cnull%2cnull%2cnull%2cnull%2cnull%2cbUjUqwUJf8g6D2mrgAw%2fkNzqKJpY%2fs4MXGXY53%2fpMwMxcerOUca5SRGOS%2fmVYKga2zpr9CpT57L9%0akK%2fF3nRmN47DSCdUKGitR4vnxvU%2brBuoX%2f0vLXPmVYLT1XNkgPfYhXPpAm33UHHDpgqqArBnZg%3d%3d; Path=\/; Domain=.xyz.com"
]
But if we request back-end from curl we get only 1 instance of WC_PERSISTENT
From a Worklight perspective it does not duplicate these cookies. It does sound like the cookies come from WebSphere Commerce ("WC"). I do suggest to contact the WebSphere Commerce support team.
Also note that setting the cookiePolicy of the HTTP adapter as "ignoreCookies" does not mean that cookies will not be added - again, these cookies are coming from the response from your backend/application server, not Worklight. Read more about an HTTP adapter's cookiePolicy parameter here: What does the cookiePolicy adapter parameter do in Worklight 6.1+?
I have decorated a Django view with cache_control as follows:
#cache_control(
private=True,
max_age=5 * 60, # 5 minutes
)
def my_view(req):
…
When I try it with the local test server, it works as expected: subsequent page views in Chrome use the cached resource and don't make a request. When deployed in production, though, Chrome seems to ignore the Cache-control header and makes a new request every time I hit that page.
Here's the full list of headers that the production server responds with:
Cache-Control:private, max-age=300
Connection:close
Content-Encoding:gzip
Content-Length:13135
Content-Type:text/html; charset=utf-8
Date:Wed, 22 Jan 2014 20:39:29 GMT
P3P:CP="IDC CURa ADMa OUR IND PHY ONL COM STA"
Server:nginx/1.4.1
Set-Cookie:csrftoken=87y26bT5uPmyA9wt51N7m4blyqBH5nSo; expires=Wed, 21-Jan-2015 20:39:29 GMT; Max-Age=31449600; Path=/
Vary:Cookie,Accept-Encoding
What could be going wrong? Any ideas? Thanks in advance!
Got it: it was a combination of Google Analytics' cookie and the Vary:Cookie header (set by Django's SessionMiddleware). Analytics' cookie changes with each request, but since ga.js doesn't load when working on localhost, the problem only showed up in production.
I am trying to detect if my application has set a cookie that holds an "alert message" for the user on the next page, where the Javascript displays it if detected.
In my vcl_fetch(), I need to detect if the specific cookie value "alert_message" appears anywhere in the Set-Cookie header (presumably in the VCL variable beresp.http.Set-Cookie). If detected, then I do not want to cache that next page (since Varnish strips the Set-Cookie header by default, which would obliterate the alert message before it makes it back to the browser).
So here is my simple test:
if(beresp.http.Set-Cookie ~ "alert_message") {
set req.http.no_cache = 1;
}
Strangely, it fails to evaluate to true.
So I throw the variable into the Server header to see what it looks like:
set beresp.http.Server = " MyApp Varnish implementation - test reading set-cookie: "+beresp.http.Set-Cookie;
But for some reason this only displays the FIRST Set-Cookie line in the response headers.
Here are the relevant response headers:
Server: MyApp Varnish implementation - test reading cookie: elstats_session=7d7279kjmsnkel31lre3s0vu24; expires=Wed, 10-Oct-2012 00:03:32 GMT; path=/; HttpOnly
Set-Cookie:app_session=7d7279kjmsnkel31lre3s0vu24; expires=Wed, 10-Oct-2012 00:03:32 GMT; path=/; HttpOnly
Set-Cookie:alert_message=Too+many+results.; expires=Tue, 09-Oct-2012 20:13:32 GMT; path=/; domain=.my.app.com
Set-Cookie:alert_key=flash_error; expires=Tue, 09-Oct-2012 20:13:32 GMT; path=/; domain=.my.app.com
Vary:Accept-Encoding
How do I read and run string detection on ALL Set-Cookie header lines?
You can resolve it with header.get function from vmod header (Varnish version >= 3)
For example, I have simple PHP script and more than one Set-Cookie in it:
<?php
setcookie ("Foo", "test", time() + 3600);
setcookie ("Bar", "test", time() + 3600);
setcookie ("TestCookie", "test", time() + 3600);
?>
By default, only first Set-Cookie header will be parsed with ' if(beresp.http.Set-Cookie ~ "somedata" '.
Of course, we can use std.collect procedure from vmod std (already comes with Varnish 3 and not requires compilation) to collapse all our Set-Cookie headers to one, but it will break cookies - Bar and TestCookie will not set.
header.get avoids this defect: it will check all headers for regex match:
if (header.get(beresp.http.set-cookie,"TestCookie=") ~ "TestCookie")
{
set beresp.http.cookie-test = 1;
return(hit_for_pass);
}
So, with it I got in response headers on first and next requests:
cookie-test:1
Set-Cookie:Foo=test; expires=Tue, 09-Oct-2012 22:33:37 GMT
Set-Cookie:Bar=test; expires=Tue, 09-Oct-2012 22:33:37 GMT
Set-Cookie:TestCookie=test; expires=Tue, 09-Oct-2012 22:33:37 GMT
X-Cache:MISS
If I comment out setcookie for cookie TestCookie, then I will got HITs on next requests.
I'm writing a networked game in C++ using Winsocks 2.2, using Visual Studio 2010, and decided it would be a good idea to use my web-server to store a list of active servers for the game. When a server starts up, it will register itself with my web server, on exit un-register; and the server itself will attempt to clean the list when someone accesses the server list depending (this behavior I'm still working on designing to not involve too much work on the server; but I figured when the game server tries to add itself, my php file will use fsockopen to detect if it can actually access the server from an outside network, if not, the server won't get added until it can properly setup port forwarding or somehow resolve the issue).
Okay, so after some research, I figured out how to get something from the server using a TCP connection from formatting a specialized message for the HTTP server. Here's what I have:
if(FAIL == Connection::Get_Connection(&m_Connection, networkSettings.ServerListAddress, 80))
{
return FAIL;
}
m_Connection.SendMsg("GET /servers.php HTTP/1.1\r\nHost: cyclotron.leetnightshade.com\r\nUser-Agent: CycloTron\r\n\r\n");
I'm expecting back properly formatted data, which I'm not exactly getting. Here's what I'm getting:
2f
Server Count:1
129.21.138.1,40000,Depth of Hell
0
Here's another output of some garbage with all the header information:
HTTP/1.1 200 OK
Date: Tue, 12 Apr 2011 23:23:11 GMT
Server: Apache
X-Powered-By: PHP/5.2.17
Expires: Thu, 19 Nov 1981 08:52:00 GMT
Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
Pragma: no-cache
Set-Cookie: PHPSESSID=8254688ee345202bd177d57e4ba339b2; path=/
Set-Cookie: PHPSESSID=73eae89f61e7268f433af9bdfe299173; path=/
Set-Cookie: PHPSESSID=8fb5d6fd9f1023bb00290b4daa3c7952; path=/
Connection: close
Transfer-Encoding: chunked
Content-Type: text; charset=us-ascii
e
Server Count:1
21
129.21.138.1,40000,Depth of Hell
0
This is what my output is supposed to look like, and I do get this on occasion, but not all the time:
HTTP/1.1 200 OK
Date: Tue, 12 Apr 2011 23:32:13 GMT
Server: Apache
X-Powered-By: PHP/5.2.17
Expires: Thu, 19 Nov 1981 08:52:00 GMT
Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
Pragma: no-cache
Set-Cookie: PHPSESSID=a3c88c2d96d45c6f6d3b029e095c429a; path=/
Set-Cookie: PHPSESSID=bf19734ff60813d6d0a5ba944410356a; path=/
Set-Cookie: PHPSESSID=c36a2d9e12c81d4a19a7f41dc5522b4e; path=/
Content-Length: 47
Connection: close
Content-Type: text; charset=us-ascii
Server Count:1
129.21.138.1,40000,Depth of Hell
I don't think this matters too much, but this is my PHP code on the web server:
$num = mysql_num_rows($result);
echo 'Server Count:'.$num;
while ($row = mysql_fetch_assoc($result))
{
// TODO: check date of entry, if it's really old, remove it.
echo PHP_EOL.$row['address'].','.$row['port'].','.$row['displayName'];
}
And here's some of the code involving receiving the string (yes it's a little bare bones at the moment, and I realize I could use a cstring function to look for the two new lines so I don't have to do the string copy, I'm just trying to stick to using strings to make things easier):
memset(m_MsgBuffer, 0, sizeof (char) * M_BufferSize);
m_Received = recv(m_Connection.M_Socket, m_MsgBuffer, M_BufferSize, 0);
m_MsgBuffer[m_Received] = '\0';
string str = string(m_MsgBuffer);
size_t index = str.find("\r\n\r\n");
str.erase(0,index);
std::cout << "Received message: " << str << std::endl;
So, do any of you have an idea where this garbage data is coming from?
EDIT: After looking at the correct header information, the one with garbage has "Transfer-Encoding: chunked" and doens't have "content-length." ...what is going on?
The so-called "garbage" is in fact the chunked data from the server. An HTTP/1.1 server is free to send data back in chunked format, if it prefers, and the HTTP/1.1 spec is pretty clear: "All HTTP/1.1 applications MUST be able to receive and decode the "chunked" transfer-coding".
The details of chunked encoding are described in the HTTP/1.1 spec:
http://www.w3.org/Protocols/rfc2616/rfc2616-sec3.html#sec3.6.1
If you are writing an HTTP client, you need to read the HTTP spec...
How do I set a P3P compact privacy policy from Django so that IE accepts cookies from my site when the security settings are on HIGH - i.e. no cookies accepted unless there's a Compact Privacy Policy.
Cheers
Guy
Middleware is the preferred way to do things like this on an "every request" basis. For instance, here is a simple bit of middleware to add the same (example) P3P header to every response Django generates:
In settings.py:
P3P_COMPACT = 'policyref="http://www.example.com/p3p.xml", CP="NON DSP COR CURa TIA"'
MIDDLEWARE_CLASSES += ('myapp.middleware.P3PHeaderMiddleware',)
In myapp/middleware.py:
from django.conf import settings
class P3PHeaderMiddleware(object):
def process_response(self, request, response):
response['P3P'] = getattr(settings, 'P3P_COMPACT', None)
return response
You could also get a similar effect in a single view by setting the P3P header in the response:
def my_view(request):
response = render_to_response('my_template.html')
response['P3P'] = 'CP="NON DSP COR CURa TIA"'
return response
To expand on the topic a little bit, cookies and headers such as the P3P header are both sent at the same time, as part of the response; in fact, under the hood, cookies are set with another response header. You can see the cookie header using curl:
$ curl --head http://www.google.com/
HTTP/1.1 200 OK
Date: Wed, 13 Jan 2010 00:04:59 GMT
Expires: -1
Cache-Control: private, max-age=0
Content-Type: text/html; charset=ISO-8859-1
Set-Cookie: PREF=ID=d2c09762c479f94e:TM=1263341099:LM=1263341099:S=oJby3NpU4RsRfuYa; expires=Fri, 13-Jan-2012 00:04:59 GMT; path=/; domain=.google.com
Set-Cookie: NID=30=kdKrd5e-u6Xs7cUe3p4eaNDtv6SO88uBL5v6_M1XMTSRmkh7okxrWLOm-l_uZdN37PxQIe4dBlekFFVCpTFXGyIDlUrz1hEwhgVLvXfIik_VeVWGmWzKbA5qu_Zq0sOi; expires=Thu, 15-Jul-2010 00:04:59 GMT; path=/; domain=.google.com; HttpOnly
Server: gws
X-XSS-Protection: 0
Transfer-Encoding: chunked
I don't know terribly much about p3p but I did a little digging and found this:
http://www.w3.org/TR/P3P11/#Well_Known_Location
You put the file at /w3c/p3p.xml
It looks as though p3p policies are similar to robots.txt files.
Additionally you can set p3p headers on all your pages if the robots.txt method isn't the way you want to go. That's a side-note, however, since you want the compact version which I'm assuming is the p3p.xml file.
Hope this helps get you on the right track.