SocketTimeoutException after connecting via proxy with authentication - web-services

I wrote code to communicate with a non-public webservice and everything seemed to work even when connecting via proxy. The problem started after changing to proxy with authorization. I get the following warning:
WARN [HttpMethodDirector] Required credentials not available for BASIC <any realm>#destination.url.com:80
WARN [HttpMethodDirector] Preemptive authentication requested but no default credentials available
After which an exception is thrown: java.net.SocketTimeoutException: Read timed out
"destination.url.com" is the destination address, so I guess I went past the proxy.
I read somewhere on stackoverflow that this warning is usually thrown when passing in a username/password when don't need to. I don't know if it's true. Using HttpProxy to connect to a host with preemtive authentication
Also here some guys point the same problem, but reproducible by running through a non-authenticated proxy to an authenticated remote repository. My situation is the opposite. The destination doesn't require authorization (the authorization data are put into the body of SOAP messages).
https://www.jfrog.com/jira/browse/RTFACT-4147
Any ideas how to help my situation? Here is some of my code:
MyStub stub = new MyStub();
Options Options = stub._getServiceClient().getOptions();
Options.setProperty(org.apache.axis2.Constants.Configuration.CHARACTER_SET_ENCODING,"utf-8");
Options.setProperty(org.apache.axis2.Constants.Configuration.ENABLE_SWA,org.apache.axis2.Constants.VALUE_FALSE);
HttpTransportProperties.ProxyProperties proxyProperties = new HttpTransportProperties.ProxyProperties();
proxyProperties.setProxyName("proxyAddress");
proxyProperties.setProxyPort("proxyPort");
proxyProperties.setUserName("proxyUsername");
proxyProperties.setPassWord("proxyPassword");
Options.setProperty(org.apache.axis2.transport.http.HTTPConstants.PROXY,proxyProperties);

It happens that the warning has nothing to do with the exception. Exception is thrown because the webservice doesn't support chunked messages. Simply adding this solves the problem:
options.setProperty(org.apache.axis2.transport.http.HTTPConstants.CHUNKED, Boolean.FALSE);
The warning itself is strange, but there is a way to remove it, by simply adding any login and password. The destination doesn't require any, but the warning vanishes.

Related

Getting CORS when calling `https://login.tapkey.com/connect/token`

When I attempt to make a call to https://login.tapkey.com/connect/token using a code returned in the authentication stage from the client I get a CORS error, I however I'm able to make that call from the BackenEnd.
CORS applies only to browser clients. That's the nature of CORS restrictions and what they should prevent.
Make sure that you setup the correct hostname in the client configuration in the developer section.

HTTP 407 Proxy Authentication Required while accessing Amazon S3

I have tried everything but I cant seem to fix this issue that is happening for only one client behind a corporate proxy/firewall. Our Silverlight application connects to Amazon S3 for downloading/Uploading some documents. On one client and one client only it returns a 407 error and after that the application fails to save anything.
Inner Exception:
System.ServiceModel.ProtocolException: [UnexpectedHttpResponseCode]
Arguments: 407,Proxy Authentication Required
We had something similar at a different client but there was more of a CORS issue. to resolve this I used cloud-front to fake a sub-domain that then accesses the S3 bucket and it solved the issue. I was hoping it would fix it with this client as well but it didnt.
I have tried adding this code to web.config as suggested by a lot of answers
<system.net>
<defaultProxy useDefaultCredentials="true" >
</defaultProxy>
</system.net>
I have read articles about passing a proxy headers with basis authentication using username and password but I am not sure how this would help us. The Proxy server is used by client and any authentication it requires is outside our domain.
**Additional Information**
The Silverlight code references 2 services. One is our wcf service that retrieves all the data for the application. One is The Amazon S3 service that uses the amazon Soap api, the endpoint for which is at http://s3.amazonaws.com/doc/2006-03-01/AmazonS3.wsdl?
If I go into our app and only use part of the system that dont make any calls to the Amazon S3 api the application works fine. As soon as I go to a part of the system that makes a call to the S3, the problem starts. funny enough the call to S3 goes fine and I can retrieve the doc fine but then any calls to our wcf service return 407.
Any ideas?
**Update 2**
Based on comments from Elliot Nelson I check the stack we were using for making http requests in our application. Turns out we are using client http for both http and https requests by default. Here is the code we have in the App.xaml constructor
public App()
{
Startup += Application_Startup;
UnhandledException += Application_UnhandledException;
InitializeComponent();
WebRequest.RegisterPrefix("http://", WebRequestCreator.ClientHttp);
WebRequest.RegisterPrefix("https://", WebRequestCreator.ClientHttp);
}
Now, to understand the differences between clienthttp and browserhttp and when to use them. Also, the potential impacts/issues of switching to browserhttp.
**Update 3**
Is there a way to request browsers to run your in-browser Silverlight application in trusted mode and would it help bypass this issue?
(Answer #2)
So, most likely (for corporate environments like this network), almost nothing can be done without whatever custom proxy settings are set in IE, usually pushed by corporate policy. To take advantage of these proxy settings, you want to use WebRequestCreator.BrowserHttp, which automatically uses the browser's default settings when making requests.
There's a table of the differences between these two clients available in the Microsoft docs. I'm guessing you were using something (maybe setting custom headers or reading the raw response body) that wasn't supported in BrowserHttp.
For security reasons, you can't "ask" the browser what its proxy settings are and use them, so this is a tricky situation. You can specify Browser vs Client handling by domain, or even for a specific request (the same page above describes how); you may be able in this case to get away with just using ClientHttp for your service calls and BrowserHttp for your S3 calls, and avoid the problem altogether!
For next steps, I'd try that approach; if it doesn't work, I'd try switching wholesale to BrowserHttp just to see if it bypasses the proxy issue (there's almost no chance the application will actually work, since you're probably using ClientHttp-only options).
Long term, you may want to consider making changes to your services so they are usable by a BrowserHttp-only application (this would require you to be pretty basic in your requests/responses, but using only BrowserHttp would be a guarantee you'd work in pretty much any corp network).
Running in trusted mode is probably a group policy thing which would require their AD admins to approve / whitelist your app.
I think the underlying issue you are facing is that the proxy requires NTLM authentication and for whatever reason the browser declines to provide your app with that context.
One way to prove that it's an NTLM auth issue is to test with curl - get it to make a req through the proxy, then it should be a bit easier to code to. EG the following curl will get you through 99% of Windows corporate proxies (assuming the proxy is at proxy-host.corp:3128):
C:\> curl.exe -v --proxy proxy-host:3128 --proxy-user : --proxy-ntlm https://www.google.com
NOTE The --proxy-user : tells curl to use the current user session to perform the NTLM challenge.
So if you can get the client to run that, you can at least identify that NTLM works, then it's a just a matter of getting the app to perform the NTLM challenge using the default credentials (which may or may not be provided by the browser session)
Since you described this as a silverlight application, I'm going to assume you can't use classic browser-proxy troubleshooting like "move browser to public network" or "try a different browser", to isolate the problem.
You should try to isolate the proxy server, and have the customer use the required proxy-auth.
The application is making request, but it might be intercepted by a transparent proxy, or the result might be coming from what you consider a web server.
In the early days, the 401 error was pretty strictly associated with web-auth, and 407 was for proxy-auth.
Architecturally, the separation is a convenience, a web server can have both web server, proxy, and reverse-proxy behaviors.
What happens is your customer's environment is making a web connection to the destination, but it receives a HTTP 407 status from some host, probably their network, or sometimes the provider. Almost certainly the request is received not forwarded. The HTTP client your application lives in needs to provide the credentials that host requires. Companies have environments that are complex enough where often your customer will say this is the first time they have heard of this (some proxy-auth is also dynamic or destination specific).
Also, in some corporate environments, the operator will allow temporary or permanent white-listing from the proxy-auth service. You should see if they can do this, even temporarily, to confirm there aren't going to be other problems.
In the end, it sounds like your application might not robustly support proxy-auth, or the proxy-auth type they use in their environment.

API Console Issue

I've been using WSO2 API Manager 1.9.1 for the past month on a static IP and we liked it enough to put it on Azure behind a full qualified domain name. As we are still only using for internal purposes, we shut the VM down during off hours to save money. Our Azure setup does not guarantee the same IP address each time the VM restarts. The FQDN allows us to always reach https://api.mydomain.com regardless of what happens with the VM IP.
I updated the appropriate config files to the FQDN and everything seems to be working well. However! The one issue I have and cannot seem to resolve is calling APIs from the API consoloe. No matter what I do, I get a response as below
Response Body
no content
Response Code
0
Response Headers
{
"error": "no response from server"
}
Mysteriously, I can successfully make the same calls from command line or SOAPUI. So it's something unique about the API Console. I can't seem to find anything useful in the logs or googling. I do see a recurring error but it's not very clear or even complete (seems to cut off).
[2015-11-17 21:33:21,768] ERROR - AsyncDataPublisher Reconnection failed for
Happy to provide further inputs / info. Any suggestions on root cause or where to look is appreciated. Thanks in advance for your help!
Edit#1 - adding screenshots from chrome
The API Console may not be giving you response due to following issues
If you are using https, you have to type the gateway url in browser and accept it before invoke the API from the API Console (This case there is no signed certificate in the gateway)
CORS issue which may due to your domain is not in access allow origins response of Options call
If you create a API which having https backend. You have to import endpoint SSL certificate to client-trustore.jks

Jetty Webservice - https protocol based address is not supported

I am using jetty version 7.5.1 .
My webservice works fine with a "http://..." endpoint, but when I change it to "https://..." things go wrong.
Endpoint e = Endpoint.create(webservice);
e.publish("https://localhost:" + serverPort + "/ws/mywebservice);
I get the following error message:
"https protocol based address is not supported".
I've tried using an SslChannelConnector, a SelectChannelConnector and the combination of both.
Connector connector = new SelectChannelConnector();
connector.setPort(59180);
SslContextFactory factory = new SslContextFactory();
factory.setKeyStore("keystore");
factory.setKeyStorePassword("password");
factory.setKeyManagerPassword("password");
factory.setTrustStore("keystore");
factory.setTrustStorePassword("password");
SslSelectChannelConnector sslConnector = new SslSelectChannelConnector(factory);
sslConnector.setPort(443);
sslConnector.setMaxIdleTime(30000);
server.setConnectors(new Connector[]{connector, sslConnector});
I also tried modifying the port in the publish path. But without success.
Could it be that something went wrong with the creation of my keystore file?
Even I put the wrong password though, it does show a different error message, explaining that my password is wrong.
My options are running out. Any ideas?
EDIT: More information:
Servlets work fine with HTTPS now. But the webservices are not. Am I maybe publishing it the wrong way ?
I found several threads on various forums with similar problems. But never found a solution. I would like to write down my solution for future victims:
The publish method only accepts the http protocol. Even if you are publishing for https, this should still be "http://...". On the other hand, you should use the port of your SSL connector.
Endpoint e = Endpoint.create(webservice);
e.publish("http://localhost:443/ws/mywebservice);
Use any other protocol and you will always get the "xxx protocol based address is not supported" exception. See source code.
Note 1: The webservice already works fine at this point. However there is a point of discussion: The generated wsdl file (at https://localhost:443/ws/mywebservice?wsdl) will reference the http://... path. You could argue if the wsdl file is a requirement or just documentation.
Correcting a hostname in a WSDL file is not that hard, but replacing the protocol is harder. The easiest solution is probably to just edit the wsdl file and host the file, which is not very "dynamic" of course.
Alternatively, I solved it by creating a WsdlServlet which replaces the address. On the other hand, it does feel bad to create an entire class just to fix 1 character. :)
Note 2: Another bug in this jetty release, is the authentication. It's impossible to offer the webservice without any authentication. The best thing you can get, after turning off all possible authentication: you will still have to use 'preemptive authentication' and enter a random username and password.

How to get/debug request message when calling a Web Service

I have an application that calls a Https web service (as it seems created with java, not sure though). I get an error as response:
"Error on verifying message against security policy Error code:1000"
Now I don't exactly understand the error code and currently cannot find any responsible to answer me correctly. I don't ask for the error ofcourse cause this could be something about certificates, security from server etc.
Though I would like to catch the request client call I make, and see the whole envelope message to compare with a couple of samples I have so I might catch something.
How can I do this....I remember there is a tool that u can do such things when debugging a WCF service call, can this tool be used in this situation? Can someone rember me the name of the tool :)
I created the client using Add Service Reference, from VS 2010 and it created some custom bindings. On these bindings it created this a tag with an attribute decompressionEnabled="true" but I deleted because VS was complaining attribute is not allowed!!!
The documentation I have for these services says about authentication credential inside the message transport object that serialized in the request (requestObject) but refers to another couple of password and username properties I cannot seem to find them. Tried to add the in client.ClientCredentials.UserName.UserName and Password properties, but I get a read only error there (strange not always).
They also mention in the specifications about Connect with SOAP Security Extensions (WS-Security) which I don't understand if me, the client, has to do something from it's side, aren't these supposed to extract in the config file when generated?
Any hints and tips are welcome.
Thank you.