I want to parse some information from the User-Agent: HTTP header. The problem is that I'm getting two User-Agent: HTTP headers in the same HTTP Request:
CONNECT www.facebook.com:443 HTTP/1.1
Host: www.facebook.com
Proxy-Connection: keep-alive
User-Agent: Mozilla/5.0 (http://iim.com/a.jph) AppleWebKit/536.6 (KHTML, like Gecko) Chrome/20.0.1092.
CONNECT www.facebook.com:443 HTTP/1.1
Host: www.facebook.com
Proxy-Connection: keep-alive
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/536.6 (KHTML, like Gecko) Chrome/20.0.1092.
CONNECT www.facebook.com:443 HTTP/1.1
I want the regex to match the non-http portion e.g Windows NT 6.1; WOW64. The flow analyzer software I'm using java regex engine.
My attempts
User-Agent:\s+.*?\((.*?)\)
Its matching both; I want to skip http portion of it.
Use a negative lookahead to prevent the match of http:
User-Agent:\s+.*?\((?!http)(.*?)\)
Though you might want to change the .*? to negated classes:
User-Agent:[^(]+\((?!http)([^)]+)\)
Related
After deploying a blazor server side project to an Amazon Web Services environment via AWS Elastic Beanstalk, the website experiences frequent disconnects that I do not understand.
These disconnects do not happen locally when testing.
Errors:
[2020-04-30T16:29:18.326Z] Error: Connection disconnected with error 'Error'.
and
Failed to load resource: the server responded with a status of 504 ()
The request causing the 504 errors has a header like so:
Request URL: https://mywebserver/_blazor?id=UOPQELxzuEcaGbpNUQA01Q&_=1588264098305
Request Method: GET
Status Code: 504
Remote Address: 3.11.236.203:443
Referrer Policy: no-referrer-when-downgrade
content-length: 550
content-type: text/html
date: Thu, 30 Apr 2020 16:29:20 GMT
server: awselb/2.0
status: 504
:authority: mywebserver
:method: GET
:path: /_blazor?id=UOPQELxzuEcaGbpNUQA01Q&_=1588264098305
:scheme: https
accept: */*
accept-encoding: gzip, deflate, br
accept-language: en-GB,en-US;q=0.9,en;q=0.8
content-type: text/plain;charset=UTF-8
sec-fetch-dest: empty
sec-fetch-mode: cors
sec-fetch-site: same-origin
user-agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/81.0.4044.129 Safari/537.36
x-requested-with: XMLHttpRequest
This issue seems remarkably similar to a Github issue posted here, that is currently being investigated by Microsoft, although this issue is not AWS-related: https://github.com/dotnet/aspnetcore/issues/19094
Any help would be vastly appreciated!
i managed to fix this in the end.
Apparently the issue was caused by the use of LongPolling. LongPolling issues requests and waits 100 seconds before cancelling the request. By setting the timeout to be greater than the default of 60 seconds in the Elastic Load Balancer in my AWS settings, the disconnects stopped. 110 seconds was recommended as a safe timeout value.
Issue on Asp.NET github that I opened for those interested: https://github.com/dotnet/aspnetcore/issues/21369
I have configured SAML Web SSO for my application.
Today I came across the following scenario:
User tries to access a page of a Service Provider (SP) who is not authenticated and is redirected to the SAML login page of WSO2 to enter his credentials using Local Basic Authenticator.
User opens a second tab with the same Service Provider before authenticating on the first tab.
User logs in successfully on any of the two tabs
Refreshing the tab that is still open on the Login page doesn't login the user, instead displays the login form.
From the http_access_logs I am seeing the following:
When the user tries to authenticate:
[20/Feb/2020:17:02:39 +0000] GET /samlsso?tenantDomain=domain.com&SAMLRequest=abcdef.....&RelayState=myRelayState HTTP/1.1 302 - - Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.116 Safari/537.36 0.018
[20/Feb/2020:17:02:39 +0000] GET /authenticationendpoint/login.do?RelayState=myRelayState&.........parameters_ommited..... HTTP/1.1 200 2953 - Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.116 Safari/537.36 0.003
[20/Feb/2020:17:02:40 +0000] GET /logincontext?sessionDataKey=e3a4e5c3-60dd-4466-8586-9dae01a4a8d8&.....parameters_ommited.... Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.116 Safari/537.36 0.000
This is the flow of the requests when directed from the SP.
However when the user is refreshing the login page, even when he has authenticated on another tab, only the latter two requests are received.
[20/Feb/2020:17:14:26 +0000] GET /authenticationendpoint/login.do?RelayState=myRelayState&.....parameters_ommited..... Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.116 Safari/537.36 0.004
[20/Feb/2020:17:14:27 +0000] GET /logincontext?sessionDataKey=e3a4e5c3-60dd-4466-8586-9dae01a4a8d8&.....parameters_ommited...... Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.116 Safari/537.36 0.007
From my understanding the reason is that the user is not directed to the /samlsso on refresh, it asks for the login form directly.
Is the expected functionality that I would like something that the WSO2 server can handle, and if so how?
Good morning,
I have to convert a string in a url string with hex code.
I found many webapps that makes this but i need to find a web service call like:
https://urlencode.com/<string>
and it response with the string encoded.
example:
call: https://urlencode.com/Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.79 Safari/537.36 Edge/14.14393
response: Mozilla%2F5.0%20(Windows%20NT%2010.0%3B%20Win64%3B%20x64)%20AppleWebKit%2F537.36%20(KHTML%2C%20like%20Gecko)%20Chrome%2F51.0.2704.79%20Safari%2F537.36%20Edge%2F14.14393
do you know something or i have to make the code manually?
I have written the MJPEG streaming server on HTTP. My HTTP server receives the request and sends the MJPEG. So far it is working fine. I am using C++ and POCO library.
The problem that I don't know when to stop the streaming. On stopping the browsing from browser (firefox) it sends same request as when requesting the stream. I used fiddler and following is the HTTP request while requesting and stopping the browser.
Requesting the MJPEG from browser (firefox):
GET http://mymjpegserver.com?vms=dvtel&deviceid=1&rgb=true&server=10.8.10.80&userid=admin&password= HTTP/1.1
Host: 10.1.39.7:8081
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:37.0) Gecko/20100101 Firefox/37.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate
Connection: keep-alive
Stopping the streaming from browser (firefox) by clicking on cross "x" button
GET http://mymjpegserver.com?vms=dvtel&deviceid=1&rgb=true&server=10.8.10.80&userid=admin&password= HTTP/1.1
Host: 10.1.39.7:8081
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:37.0) Gecko/20100101 Firefox/37.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate
Connection: keep-alive
And then after few moments when ever I switch browser tabs or window it sends following request
GET http://mymjpegserver.com?vms=dvtel&deviceid=1&rgb=true&server=10.8.10.80&userid=admin&password= HTTP/1.1
Host: 10.1.39.7:8081
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:37.0) Gecko/20100101 Firefox/37.0
Accept: image/png,image/*;q=0.8,*/*;q=0.5
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate
Connection: keep-alive
That is only Accept value is changed to "image/png...". But, I can not use this Accept value to differentiate the termination, since this request is only sent when the Tab/Window is switched.
And on closing the Tab, nothing sent to the Http Server.
My question is, how on Web Server I can check the request to stop the streaming?
I try to realize WebSocket protocol at MS Http Server API under win server 2008 (haven't HTTP_SEND_RESPONSE_FLAG_OPAQUE flag).
HTTP_RESPONSE response={0};
const char upgrade_val[]="Websocket";
response.Headers.KnownHeaders[HttpHeaderUpgrade].RawValueLength=strlen(upgrade_val);
response.Headers.KnownHeaders[HttpHeaderUpgrade].pRawValue =upgrade_val;
const char connection_val[]="Upgrade";
response.Headers.KnownHeaders[HttpHeaderConnection].RawValueLength=strlen(connection_val);
response.Headers.KnownHeaders[HttpHeaderConnection].pRawValue =connection_val;
HTTP_UNKNOWN_HEADER unknown[1];
response.Headers.UnknownHeaderCount=1;
response.Headers.pUnknownHeaders =unknown;
const char accept_name[]="Sec-WebSocket-Accept";
unknown[0].NameLength =_countof(accept_name)-1;
unknown[0].pName =accept_name;
unknown[0].RawValueLength=HANDSHAKE_KEY_LENGTH;
unknown[0].pRawValue =base64_key;
response.Version.MajorVersion=1;
response.Version.MinorVersion=1;
response.StatusCode =HTTP_SWITCHING_PROTOCOLS;
const char reason[] ="Switching Protocols";
response.ReasonLength =strlen(reason);
response.pReason =reason;
HttpSendHttpResponse(iocp,RequestId,HTTP_SEND_RESPONSE_FLAG_MORE_DATA,&raw_response,NULL,NULL,NULL,NULL,NULL,NULL);
Browser send header (from Fiddler)
GET http://server.host/ HTTP/1.1
Upgrade: websocket
Connection: Upgrade
Host: server.host
Origin: null
Pragma: no-cache
Cache-Control: no-cache
Sec-WebSocket-Key: V86c1TFOwWfZqhS42C0arA==
Sec-WebSocket-Version: 13
Sec-WebSocket-Extensions: x-webkit-deflate-frame
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/27.0.1453.94 Safari/537.36
server's response (from Fiddler)
HTTP/1.1 101 Switching Protocols
Upgrade: Websocket
Server: Microsoft-HTTPAPI/2.0
Sec-WebSocket-Accept: 3io2SU7uJIeFlwy0+OFJUDNrA44=
Date: Tue, 28 May 2013 09:27:59 GMT
EndTime: 13:27:59.006
ReceivedBytes: 0
SentBytes: 0
and browser shows 1016 error.
Assuming your Sec-WebSocket-Accept header value is calculated correctly, and assuming your response variable is supposed to be named raw_response instead, then your response is missing the required Connection: Upgrade header despite your code assigning a value for it.
HTTP_SEND_RESPONSE_FLAG_OPAQUE (Windows 8 and later) is required for WebSockets.
WebSocket servers on Windows Server (2011-12-20)
As it stands right now, Server 2008/R2 boxes cannot host WebSockets.
At least, not whilst sharing ports 80 and 443 with IIS web server
Re: [hybi] WebSocket protocol as it stands (2010-11-06)
we need to tweak http.sys to recognize what is really a non-HTTP
request, as an HTTP request