I'm re-configuring an existing SOAP web service to run behind an Azure Application Gateway with Web Application Firewall.
The SOAP web service is written in C# and runs in Azure as a web role within an Azure Cloud Service. It supports both SOAP 1.1 and SOAP 1.2 at present.
I have configured an Azure Application Gateway, with the Web Application Firewall enabled, to run in front of this service. The WAF is using the OWASP 3.0 rule set.
Sending test requests from SoapUI, it appears that the Application Gateway WAF is allowing SOAP 1.1 requests through, but is blocking SOAP 1.2 requests (returning a 403 error). I can't find any reference to why this might be happening in the documentation or anything else. I know it's the WAF, because disabling it allows the SOAP 1.2 requests through.
The HTTP headers for the (working) SOAP 1.1 request look like this (service and namespace URLs removed):
POST http://{serviceURL}/{service}.asmx HTTP/1.1
Accept-Encoding: gzip,deflate
Content-Type: text/xml;charset=UTF-8
SOAPAction: "http://{namespaceURL}/{method}"
Content-Length: 3672
Host: {serviceURL}
Connection: Keep-Alive
User-Agent: Apache-HttpClient/4.1.1 (java 1.5)
The HTTP header for the (not working) SOAP 1.2 request looks like this:
POST http:/{serviceURL}/{service}.asmx HTTP/1.1
Accept-Encoding: gzip,deflate
Content-Type: application/soap+xml;charset=UTF-8;action="http://{namespaceURL}/{method}"
Content-Length: 3652
Host: {serviceURL}
Connection: Keep-Alive
User-Agent: Apache-HttpClient/4.1.1 (java 1.5)
Looking at the WAF logs, I think that the issue is the change to the Content-Type, which based on my (not in-depth) understanding of SOAP 1.2, is correct.
Any thoughts appreciated. It seems like SOAP remains in wide enough use that the Azure Appliction Gateway / WAF should support it.
Related
In our GKE Kubernetes cluster we have a deployment running a pod with a third-party unmodifiable legacy application exposing an HTTP service (port :80) protected with based authentication. The cluster is exposed to the outside Internet through a GCP load balancer that takes care of terminating HTTPS connections.
When, from the Internet, I try to access the service using CURL, it works perfectly, I got an HTTP status 200.
curl -vvv 'https://username:password#portale-ambrogio-test.sigemi.cloud/admin'
* Trying 34.107.254.133...
* Connected to portale-ambrogio-test.sigemi.cloud (34.107.254.133) port 443 (#0)
* found 129 certificates in /etc/ssl/certs/ca-certificates.crt
* found 516 certificates in /etc/ssl/certs
* ALPN, offering http/1.1
* SSL connection using TLS1.2 / ECDHE_RSA_AES_128_GCM_SHA256
* server certificate verification OK
* server certificate status verification SKIPPED
* common name: *.sigemi.cloud (matched)
* server certificate expiration date OK
* server certificate activation date OK
* certificate public key: RSA
* certificate version: #3
* subject: CN=*.sigemi.cloud
* start date: Tue, 24 May 2022 00:00:00 GMT
* expire date: Wed, 24 May 2023 23:59:59 GMT
* issuer: C=GB,ST=Greater Manchester,L=Salford,O=Sectigo Limited,CN=Sectigo RSA Domain Validation Secure Server CA
* compression: NULL
* ALPN, server accepted to use http/1.1
* Server auth using Basic with user 'username'
> GET /admin HTTP/1.1
> Host: portale-ambrogio-test.sigemi.cloud
> Authorization: Basic aR.......................==
> User-Agent: curl/7.47.0
> Accept: */*
>
< HTTP/1.1 200 OK
< Ntrip-Version: Ntrip/2.0
< Access-Control-Allow-Origin: *
< Server: NTRIP BKG Caster/2.0.39
< Content-Type: text/html
< Date: Tue, 06 Sep 2022 15:53:09 GMT
< Via: 1.1 google
< Transfer-Encoding: chunked
< Alt-Svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000
When I access the same resource through the browser, it fails instead.
Do you know why that could happen? I think it is something related to some security policy implemented by the browser but what?
[EDIT] Now with the browser I'm explicitly typing username and password in the authentication pop up to avoid the problem pointed out by #john-hanley
The problem was caused by few facts.
Our web application was not fully compliant with any of the HTTP specs (1.1, 2 and 3) regarding the name of the HTTP headers. In particular HTTP/1.1 states that header names should be considered case insensitive, while HTTP/2 and HTTP/3 states that headers should be lower case. Our application expected header names to be capitalized, so it always looked for the "Authentication" header.
When modern browsers connect to an HTTP web server, they usually use HTTP/1.1 and in this case, they tend to capitalize header names, so browsers actually send a header which name is spelled "Authorization".
However when modern browsers connect to HTTPS, they upgrade to HTTP/2 or HTTP/3 id supported by the web server. If they upgrade to such protocols, they obey it, so the send the authorization header in lower case, so "authorization".
Our reverse proxy is provided by GCP, that indeed supports HTTP/3. So, when the browser connects to our GCP reverse proxy it switches to HTTP/3 and start sending lower case headers.
The reverse proxy the forward the request to our application in lower case.
And because of 1) the app is unable to authenticate.
Hope this report can help, because I spend a whole day trying to figure it out (and I actually learnt about HTTP/2 and HTTP/3).
The solution was obviously to fix our application and made it fully HTTP compliant.
The CLI curl is converting username:password from the URI into Authorization: Basic ....
Chrome stopped supporting URI credentials in the address bar after version 52. Internally, this is still supported via JavaScript.
I came across a situation where I have to invoke a SOAP webservice (deployed in another server) from one of the production server manually and check whether everything is fine.
as these are all live servers there are no network tools like wget, curl and nc are available. I tried checking for a solution is google but no luck.
As a workaround I can write a java client socket and invoke the service but even that is not allowed in here.
telnet is there but am not sure how to make it work for my case.
Is there any other way to invoke remote services without these tools?
After trying few hours finally, I was able to invoke SOAP service with telnet as below
first open a TCP connection to the remote server as below.
$> telnet hostname portname
Once it is connected, frame a request as one of the below methods and paste on the screen and press enter key two times.
There are two ways we can call a service.
Method 1: instead of mentioning endpoint path in POST header, we can give it in SOAPAction header.
POST / HTTP/1.1
Accept-Encoding: gzip,deflate
Content-Type: text/xml;charset=UTF-8
SOAPAction: "<endpoint URL from WSDL>"
Content-Length: <number of bytes you are sending in body section>
Host: <hostname>:<port>
Proxy-Connection: Keep-Alive
User-Agent: Apache-HttpClient/4.1.1 (java 1.5)
<SOAP Request payload>
Method 2: mentioning endpoint path in the request header itself, so we can give empty value in "" SOAPAction header (it means request path itself is the Endpoint path).
POST /soap/server HTTP/1.1
Accept-Encoding: gzip,deflate
Content-Type: text/xml;charset=UTF-8
SOAPAction: ""
Content-Length: <payload size>
Host: hostname:port
Proxy-Connection: Keep-Alive
User-Agent: Apache-HttpClient/4.1.1 (java 1.5)
<SOAP Request payload>
Response: Once the call invoked successfully, the response will be printed as below
HTTP/1.1 200 OK
Content-Type: text/xml;charset=UTF-8
Content-Length: <response payload size>
<SOAP response payload>
For more information on SOAPHeader check this link
Note: Make sure the length of the request payload is correct before sending it.
APIM gateway is setting Host header as host:port to call backend, like this:
GET /api/category HTTP/1.1 Accept-Language:
en-US,en;q=0.8,pt;q=0.6 token:
6785ea7b-#######-#######-93f06834660a Accept-Encoding: gzip,
deflate, sdch X-Forwarded-Server: server01 X-Forwarded-For:
172.XX.XXX.XX User-Agent: Mozilla/5.0 (X11; Linux x86_64) Postman-Token: ece4261e-d610-655c-f06c-f24a99f007c8 Accept:
/ X-Forwarded-Host: server01 Cache-Control: no-cache
Host: api.empresa.net:80 Connection: Keep-Alive
This causes problems with Web Application Firewall, how can I change this?
Enviroment: RedHat Linux 6.7, APIM 1.9.1
Thanks
I know your pain. We were running into the same problem with our load balancer. I believe there is a fix coming from WSO2 for this. In the meantime, you will need to create a custom "in" sequence under "/_system/governance/apimgt/customsequences/in" with the following content:
<sequence xmlns="http://ws.apache.org/ns/synapse" name="remove_port_In">
<property name="REQUEST_HOST_HEADER" value="api.empresa.net" scope="axis2"/>
</sequence>
You can then associate this with your API as an inbound custom sequence through the publisher application prior to publishing.
As per the protocol specification,
The Host request-header field specifies the Internet host and port
number of the resource being requested, as obtained from the original
URI given by the user or referring resource.
Host = "Host" ":" host [ ":" port ] ; Section 3.2.2
A "host" without any trailing port information implies the default
port for the service requested (e.g., "80" for an HTTP URL). For
example, a request on the origin server for
http://www.w3.org/pub/WWW/ would properly include:
GET /pub/WWW/ HTTP/1.1
Host: www.w3.org
Here is my set up and dilema
We have created an ipad application that uses an HTML wrapper to display information to the user. It is a telerik App builder solution running an HTML 5 site with JS and connect to our web API side for fetching data.
The server side is a WEB API 2 running on a web project, no Katana, just a the same modified template from the SPA sample file.
The way the user authenticates on the IPAD is by issuing a /token request. In fact here is the
actual fiddler traffic when the user request a token
POST http://development01/MobileAPITest/Token HTTP/1.1
Host: development01
Connection: keep-alive
Accept-Encoding: gzip, deflate
Content-Type: application/x-www-form-urlencoded; charset=UTF-8
Accept-Language: en-us
Accept: */*
Origin: file://
Content-Length: 57
Connection: keep-alive
User-Agent: Mozilla/5.0 (iPad; CPU OS 7_1_2 like Mac OS X) AppleWebKit/537.51.2 (KHTML, like Gecko) Mobile/11D257 (391865488)
grant_type=password&username=johndoe&password=pass1234
So. How can i set up a test system to test this at the http level?
I assume you want to test the Web Api itself?
You shouldn't have a need to test the controllers via HTTP protocol, the framework has been tested to work up until the point of giving your method a request object and a deserialized version of the incoming model.
Setup a test project for the site and test the controller functionality by directly issuing commands to the controller methods. This relieves you of the burden of having a HTTP client mixed with the testing.
If you necessary want to write tests that include calling the API via HTTP then I suggest you either use a HTTP client to call your API from the test project.
A good step-by-step unit testing tutorial for ASP.NET Web Api 2 can be found here:
http://www.asp.net/web-api/overview/testing-and-debugging/unit-testing-with-aspnet-web-api
And for testing with routing:
http://www.asp.net/web-api/overview/testing-and-debugging/unit-testing-controllers-in-web-api
I'm writing Qt client for ASP.NET web service with FORMS based authentication.
The service consists of 3 methods:
Login(user,pass)
Helloworld() - this method returns info about authenticated user.
Logout()
Every thing working fine on the dot.net client with CookieContainer.
The problem begins with HelloWorld() methods. it returns null because I can't access server session.
I'm doing the following:
from the response of Login() request I'm getting the cookies which are sent to client:
QNetworkAccessManager *manager = http.networkAccessManager();
cookie = manager->cookieJar();
When sending the second soaprequest for HelloWorld method I adding these cookies
to QtSoapHttpTransport http:
http.networkAccessManager()->setCookieJar(cookie)
but the request which is going from server is empty.
I moved further with my investigation and monitored HTTP traffic coming to server from Qt client and .NET client.
The HTTP Header for both SOAP requests are different:
This is Request coming from .NET client
POST /test/service1.asmx HTTP/1.1
User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; MS Web Services Client Protocol 2.0.50727.3053)
VsDebuggerCausalityData: uIDPoyhZznNkbItPkJSR3EA+zEIAAAAAUkpe7URduE6nmhnT8f uQeqCQBMlX0zxCm65yW4ZPBkUACQAA
Content-Type: text/xml; charset=utf-8
SOAPAction: "http://localhost/test/HelloWorld"
Host: localhost:8080
Cookie: MyAuthenCookie=DC7620DA79E080FECA37AC6866BF2690D57 B37443506F0D7EEA9DF209827360894D80D37E1B121D73EE44 766BDAEE16BA3FB0E8B95ADB1252AB00A76706930ACDC87CF9 F26744B7E9E3EB7FBB3812997
Content-Length: 291
Expect: 100-continue
and this is Request coming from Qt SOAP client:
POST /test/Service1.asmx HTTP/1.1
Content-Type: text/xml;charset=utf-8
SOAPAction: http://localhost/test/HelloWorld
Content-Length: 350
Cookie: MyAuthenCookie=9AFB2B22EE78D19DFD52BD2193A3D71627C F7303C15E4354E43CC2F31AECBDFFAD09176AA45F33B35C3C3 73891F1FE994580E8EE70FD4D01507670743138E74E152CFF4 EB3C37D90D3A7A0E272A804C3
Connection: Keep-Alive
Accept-Encoding: gzip
User-Agent: Mozilla/5.0
Host: localhost:8080
Does any body have any idea what might be the problem and hoe to solve it?
How can I modify Headers for HTTP POST request in QtSoapHttpTransport object in order to make it identical to .NET request?
Thank you in advance,
Danny.
Are you running IIS or the ASP.NET Development server?
I was able to recreate a similar problem where everything worked fine using ASP.NET Development server but under IIS the session was null.
One thing to look for is when you invoke the session-enabled-service you should see the ASP.NET_SessionId being set in the response headers
Set-Cookie: ASP.NET_SessionId=5vxqwy45waoqma45lbbozj45; path=/; HttpOnly
Set-Cookie: MyAuthenCookie=510969D70201B358F8B0BBEEE7E79316B7ABCCC74312B0BD678DA4BE90E5C51CD6E7CDCA486DDB41BCBF489DB7280B3B979FD70B78D7F63B03C33431ADDAFDCA; expires=Mon, 07-Dec-2009 06:41:04 GMT; path=/; HttpOnly
To get sessions working under IIS, I had to add the following to web.config under system.web:
<sessionState mode="InProc" cookieless="false" timeout="20" />
<httpModules>
<add name="Session" type="System.Web.SessionState.SessionStateModule"
</httpModules>
This was under Windows 2003.