We are facing the following problem. What we have:
1) "Our" ASMX web-services, hosted both on 80 and 8080 ports
2) "Their" unsupported unknown solution which works with our web-services. That is, it is working as is and almost can not be modified. I don't know how it is implemented. It looks like WWF hosted on IIS, but I can't say for sure. But it is 100% .NET solution as ours.
So far this solution calls our services through WCF client proxies using endpoints configured in settings. They are standard WCF-configuration:
K4 default
The problem with these endpoints is that they are using default HTTP port 80. We have load balancing, but even round-robin balancing is implemented on port 8080 (so that for port 80 all requests go to 1 specific server but not to 1 random of 3 nodes for every request). So we asked to change endpoint ports in configuraiotn file and test it is working. And this is really working on test server k4-dv.
However when we tried to change ports on production server, everything stopped working. Error that is seen in logs is following:
Metadata error
followed by this:
HTTP 503 error
And some other errors, but I can't say whether they are relevant to each other:
Message contract
Timeout
So my question is fairly obvious: what is the reason of this error and what can be changed, taking in account, that test server works with both ports? What settings should we check?
The main problem is that I don't know reasons of this error and did not manage to find the solution. And there are no people from "their side" who develops this solution. We have only "support" than can check something, change settings, but in this case they can only wait a suggestion from us to fix problem with 8080 port. They don't know reasons either.
Related
I am repeating a question that I posted at https://forums.aws.amazon.com/thread.jspa?threadID=275855&tstart=0
to reach out more people.
Hi,
I am trying to deploy a REST service in AWS. The current architecture is:
Domain name (Route 53) -> Load Balancer -> Single EC2 instance (bound to an Elastic IP). And I use TLS/SSL certificate issued by a Certificate Manager.
The instance is Ubuntu 16.04 machine, and the service is implemented with (bare) Vert.X (==no proxy server).
However, 504 Error (gateway timeout) occurs after a few different requests (each of which takes <1s) in a series, and then it does not respond. The requests do not reach the server instance after a few requests. I checked that it happens in the same way when I access both the domain name and the load balancer directly. I have confirmed that the exact same scenario is working with direct URL.
I run up a dummy server returning "hello world" and it's working okay with the load balancer. The problem should be caused by something no coherent between the load balancer and the server code, but I can't get where to start.
I have checked several threads complaining the 504 errors, and followed some of the instructions, but they do not work. Especially I set keep-alive option in Vert.x and set the idle time longer than the balancer's. As the delays are not longer than the idel time with the direct communication, I believe it is not the problem anyway. I have checked the Security Groups also and confirmed the right ports are open. (The first few requests are working, so it must not be the problem also.)
Does any of you have a sense where I should start looking at? Even better, know the source of the problem?
Thanks in advance.
EDIT: I just found the issue in some of the code. I've answered myself below. Thanks for reading!
Found the issue in my code. Some of the APIs (implemented by my colleague...) was not flushing the buffer of HTTP responses in the server.
In Vert.X Java, it was resp.end().
It was somehow working with direct access probably the buffer was flushed at some point, but that flush seems not caught by the load balancer.
Hope nobody experiences this, but in case...
I am experimenting with following setup.
Clone/copy (but not redirect) all incoming HTTP requests from port 80 to another port say 8080 on same machine. I have a simple NGINX + Lua based WAF which is listening on 8080. Essentially, I am running two instances of webservers here, one which is serving real requests and other one working on cloned traffic for detection purpose. I don't care about being able to block the malicious requests so I dont care about being inline.
I want to use WAF only for detection purpose i.e. it should analyze all incoming requests, raise alert and drop the request after that. This will not hamper anything from users point of view since port 80 is serving real requests.
How can I clone traffic this way and just discard it after analysis is done ? Is this feasible ? If yes, please suggest any tools which can clone traffic with minimal performance hit.
2.
Have a look : https://github.com/buger/gor
Example instructions are straightforward. Additional logging or certain forwards you could possibly add as well
In the current Nginx version, there is an ngx_http_mirror_module, which retranslates requests to another endpoint and ignores responses. See also this answer
I'm writing a very simple server-client application in C++, using WinSocks.
When I set client's parameters as: resolveHost ("google.pl") for server's IP and 80 for port, I receive a html (with GET header).
When I run the server first (with port 10000) and then client (port 10000 IP inet_addr("127.0.0.1") it seems to works (I receive something).
But when I change the server's IP in client to inet_addr("188.246.158.171") (my IP for now, I don't have static IP, I've checked my actual IP with some website), it for sure DOES NOT works.
My computer is behind the router - can it be the problem?
If so, can I solve it in the way that don't requite any configuration to router (port forwarding etc.). Also, I don't want to make anything router's model-specific, so I intentionally haven't posted the model of my router ;)
Why I don't want to change configuration?
Because my application will be for "simple people". And simple people won't try to make configuration, and I won't require them to do so.
Also, many applications (on-line games for example) works fine with my router without any configuration - my application cannot be exception (by the example of other applications on my computer, I know it's possible and common to achieve it).
I use Visual Studio 2012, works on Windows 7 x64.
If you run a listening server behind a router, you must configure port forwarding rules on the router if you want outside clients to reach the server through the router. You cannot avoid that. However, if the router supports uPnP (Universal Plug and Play), your server code can configure port forwarding rules programmably, such as with Microsoft's IUPnPNAT interface. Otherwise, you have to use the router's own configuration software (usually an HTTP interface running on the router itself).
You also have to configure ports/permissions on a local firewall, if installed. Some firewalls have APIs for that, such as Microsoft's Firewall.
Online games and other peer-to-peer apps employ several NAT Traversal techniques to get around NATs, firewalls, etc. Hole Punching, Role Reversal, etc. Do some searches, information is readily available.
I have made a simple Jetty server using this tutorial here, http://www.seas.upenn.edu/~cis550/jetty.html
Now that I have the initial steps completed, I can succesfully connect to my server by entering localhost:8080/hello or 127.0.0.1:8080/hello in my browser. Everything is going great up until this point. The servlet is working properly.
I am now interested in connecting to the server using "Machine B". I want to build a server that can be connected by any browser, anywhere.
My IP is XX.173.XXX.216, parts hidden for obvious privacy reasons. On my "Machine B" I try and enter XX.173.XXX.216:8080/hello into the browser URL. The website cannot be found. From experience I questioned my port mapping and opened my 8080 port.
After a few google searches and a while of reading, I give up. Does anyone know a possible solution to my problem? Are there any further steps I need to do in the setup of the Jetty server to make it accessible from external machines?
It is usually not hard getting from the inside of a firewall and out. It is much trickier to get from the outside and in (just have a look at the hoops Skype jumps through to be able to get two computers each behind a firewall to talk to each other directly).
First of all you need to ensure that Jetty listens to the right network devices.
Identify the IP-number of the computer running Jetty (not 127.0.0.1) and see if you can connect to "http://that-ip-number:8080" from the machine itself. If this doesn't work, fix your jetty configuration. Then put another computer on the same network and connect to "http://that-ip-number:8080" too. This MUST work before you go any further. If this doesn't work, fix your firewall configuration on the computer running Jetty.
When the above works,then you can start thinking about exposing the port through your router. The router most likely has a web interface where you can configure this, but the easiest way to do that is to use a uPNP tool (if your router supports that) which can redirect an external port to a specific host and port on your internal network. I use "Port Map" under OS X which has a very simple interface.
Good luck
Afternoon all,
We have a group of four BizTalk servers: two orchestration hosts and two adapter hosts. We have a number of orchestrations exposed as web services, and for the purposes of this question, it is important to note that these web services are hosted on the adapter servers, and run under the BizTalkServerIsolatedHost host instance.
This morning, we started seeing odd errors on both of the adapter servers when SOAP calls came into the web services, like this:
The Messaging Engine failed to
register the adapter for “SOAP” for
the receive location blahblahblah.
Please verify that the receive
location exists, and that the isolated
adapter runs under an account that has
access to the BizTalk databases.
We restarted IIS on both servers, which fixed the errors on ONE server, but the other server continued to fail. The errors continued after a reboot as well.
After chasing our tails for a while, we eventually discovered that the BizTalkServerIsolatedHost host instance on the still-failing server was gone. Just... gone. These applications have been in production for months. Everything had been working swimmingly through the morning, until this just happened.
I don't want to muddy the waters, because I think the problems are unrelated, but in the interest of providing enough information, this problem exactly coincided with a problem in our load-balancing network hardware. The load balancer, which provides a single URL to consumers, and round-robins between the two adapter servers, just stopped working. This problem has not been resolved, so I don't know what happened, but it certainly made troubleshooting more interesting...
So, I have two questions:
Has anyone seen this before, where a host instance disappears?
We cannot find anything in the event viewer or anywhere else that says the host instance was deleted. Is this logged somewhere?
Thanks,
Jason