I have configured Jetty as follows.
# ps aux | more
USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
jetty 1 7.0 9.8 9931680 2421304 ? Ssl 12:20 4:31 java -Djava.io.tmpdir=/tmp/jetty -server -XX:+UseConcMarkSweepGC -XX:+UseParNewGC -Djava.security.egd=file:/dev/./urandom -Dhttp.proxyHost=XXX -Dhttp.proxyPort=XXX -Dhttps.proxyHost=XXX -Dhttps.proxyPort=XXX -Dhttp.nonProxyHosts=localhost\|127.*\|[::1]\|10.*\|172.16.*\|172.17.*\|172.18.*\|172.19.*\|172.20.*\|172.21.*\|172.22.*\|172.23.*\|XXX -Xms1g -Xmx3g -Xdebug -Xrunjdwp:transport=dt_socket,server=y,suspend=n,address=XXX -jar /usr/local/jetty/start.jar
I have the feeling that the no-proxy variable (http.nonProxyHosts) is not read out and the HTTP connect proxy is always used.
Since I cannot change a third-party software so easily myself, I wonder how I call some/certain local resources. I can only change config. How?
The JVM system properties for controlling URL networking in your command line ...
-Dhttp.proxyHost=XXX
-Dhttp.proxyPort=XXX
-Dhttps.proxyHost=XXX
-Dhttps.proxyPort=XXX
-Dhttp.nonProxyHosts=localhost\|127.*\|[::1]\|10.*\|172.16.*\|172.17.*\|172.18.*\|172.19.*\|172.20.*\|172.21.*\|172.22.*\|172.23.*\|XXX
... exist for outgoing connections made by any component in your JVM that uses the JVM specific HTTP Client behaviors.
In other words, the built in classes in the JVM.
java.net.HttpURLConnection (if using java.net.URL originated connections)
java.net.http.HttpClient (if you are using Java 9+)
If you are using an HTTP Client that doesn't use one of the 2 above classes, support for these system properties is vanishingly small.
Jetty's own org.eclipse.jetty.client.HttpClient intentionally does not support the JVM system properties (as they cast far too wide of a net for practical use)
Many REST libraries also have dedicated HTTP Clients that don't use the JVM URL based HTTP connections, and also do not support these system properties.
Jetty Server itself does not have or use these concepts for incoming connections to the Jetty Server.
These concepts only exist for outgoing connections.
In the meantime I found out that the backslashes "\" are too much. After I removed them and only pipes "|" are left in the nonProxyHosts variable, everything works. :)
Related
I'm creating a Django web application that'll assist in barebone server deployments, where a bare bone server will PXE boot to a custom LiveCD to send a cURL command to register itself to a DRF REST API.
When Django receives the POST request it'll start a Go app remotely that'll find the bare bone server based on entries in the REST API then start configuring the server. What would be the best way to identify/introduce the bare bone server to my Go server?
My thought is either to use a parser parameter to identify the server then Go will pull the bare bone server info from the REST API or add a Boolean field in the REST API and the Go app will look for entries that are TRUE then flip it to FALSE when it starts setting up the bare bone server.
Would that be the best way to get this done or is there a better way?
Actually, PXELINUX comes with an identification mechanism based on the systems MAC dress and the configuration can be customized accordingly. Since you need to do accounting of your bare metal servers anyway (port security anyone? ;) ), you should know the MAC dresses of all the interfaces on your bare metal servers anyway.
Your directory usually looks like this (path prefix may be different).
/srv/pxe/pxelinux.cfg/default
Now what happens is that your system starts up, sends a DHCP Request and gets an offer containing the DHCP options "next-server" and "filename". When the system selects said offer, it will connect to the "next-server" and request "filename", usually pxelinux.0. Here is your first potential hook: Write a tftp server which deals with the request and registers your system.
Now pxelinux.0 is executed , it will read the above config file. But here is the thing: Say the Mac address of the system is 23:67:33:5a:cc:e8, and the file
/srv/pxe/pxelinux.cfg/23-67-33-5a-cc-e8
exists, this will be read instead. Which is your second hook: the request will be logged by tftp.
Regardless of wether the default or a system specific config file is used, basically we are talking of GRUB config file. Assuming you use Kickstart to install the system, it will look something like this
default linux
prompt 0
timeout 1
label linux
kernel /images/yourdistro/vmlinuz
ipappend 2
append initrd=/images/yourdistro/initrd.img console=ttyS0,115200
Now, here is the thing: you have several possibilities to execute a custom program on boot:
Append the path to your executable to the append parameter. By convention, the Kernel will send all parameters it does not know to pid 1. Though I have not tested wether systemd adheres to the convention and simply executes a parameter it does not know in turn, I assume as much.
cron. Most cron implementations nowadays support the #boot time definition.
the init system, be it either systemd or openrc or good ol' SYSV init.
Last but not least, how to configure the machine. I strongly suggest against reinventing the wheel. I had quite similar requirements in a (closed source) project. We used kickstart to do the basic system installation and simply shot a curl command after reboot to Ansible Tower, triggering the more detailed configuration. Since we had a DHCP server with the MAC, an IP reserved for said MAC and a hostname readily configured (dnsmasq, caugh, caugh), that was not much of a problem. Basically, all we had to do manually is to register the MAC address and assign an IP and a hostname, then fire up the machine.
Background: I've got an C++/Qt-based application that communicates with servers on the user's LAN. It uses non-blocking TCP and UDP sockets, and the networking is implemented via calls to the BSD sockets API (i.e. socket()/send()/recv()/select()/etc). It all works well.
The other day, just for fun, I decided to recompile the application using emscripten, so that it could run as a WebAssembly app inside a web browser.
This worked surprisingly well -- within an hour or two, I had my app up and running inside Google Chrome. However, the app's usefulness in this configuration is severely limited by the fact that it isn't able to connect to any servers -- presumably this is because it is running in a restricted/sandboxed environment.
If I wanted to pursue this line of development beyond the clever-hack-demo stage and try to make it useful, I would need to find a way for my program to discover and connect to servers on the user's LAN.
My question is: is that functionality at all possible for a Emscripten/WebAssembly-based app to perform? If so, what steps would I need to take? (i.e. would it require upgrading the LAN's servers to handle WebSocket-based connections? Would it require adding some sort of proxy server to run on the web server that the web page was served from? Is UDP even a thing in a web-app context? Are there other hoops that would also have to be jumped through?)
Is there a way to get the metrics for Jetty server, for example, the queue length, processing time, etc.
I looked at the QueuedThreadPool class, and tracked the the calling chain for getQueueSize, but I didn't find it got exposed.
Thanks.
"Queue Length" is a very vague term and can apply to many many different things.
As for seeing the metrics for Jetty, enable JMX, restart your server, and take a look.
Goto your ${jetty.base} and add the jmx module.
$ cd /path/to/mybase
$ java -jar /path/to/jetty-dist/start.jar --add-to-start=jmx
Once you have restarted, use the jmc command that comes with your JDK (or if you are using an ancient JDK use jconsole) and attach to the running Jetty process to see the various things exposed by Jetty for monitoring.
My application runs in Windows and is implemented using C++/Qt.
The application will invoke another application deployed in the Linux server which in turn will invoke some third party tools. The Linux server application will send some status updates based on the running of third party tools. Usually the third party application will run for hours and the updates will be sent at various stages. The Linux server may also has to send some files in addition to the status updates and the Windows client will also send some files required for the running of those third party tools.
I planned to implement this in libssh2 since file transfers can be done and applications can be executed as well using libssh2_channel_exec(). Updates can be sent and received through non-blocking socket transfers. Also the transfers must be secured and they are password authenticated, so I thought SSH will conform my requirements.
I also looked into Qpid of apache which implements the AMQP. The messaging seems to be a more appropriate one for my status updates since the updates are less frequent. But I am not so sure about the secured connection, password authentication and also the application invocation.
So, which one can I choose between these two? Or is there any other better option available? I am not quite used to network programming so any pointers, links regarding this are welcome..
Have you considered some web-based solutions like XML-RPC, REST, SOAP or other? Note that you can either have constant network connection and stream updates or just make your client ask for update as often as it needs.
Also, I think that building solution based on some of these protocols will give you easier coding - no need for some low-level solutions when you have great libraries. As for security part, I would consider SSL that is part of HTTPS protocol to be secure enough. Of course you can also do it hybrid style, for example SSH tunel to secure server and use SSH key authorization.
But if you are sure youwant SSH or AMQP then use first one - I think it has better security. Also, try not using username/passowrd. Instead use mentioned above keys.
Start with SSH, and then consider layering other protocols on top. You can use SSH port forwarding to create a VPN connection to a server, and maybe that will make it easier to use something like AMQP or 0MQ.
Is there any way in C++ on windows to monitor a program and redirect any outgoing requests it makes on a specific port? I have a simple C++ http proxy and want it to be able to automatically redirect all browser requests on port 80 through itself.
The simple way to do it is to create a Windows kernel hook to trap socket requests and reroute them to your proxy.
Some useful documentation on this is:
http://www.internals.com/articles/apispy/apispy.htm
If you're using Windows Vista or better, consider Windows Filtering Platform (WFP):
http://www.microsoft.com/whdc/device/network/wfp.mspx
Also consider looking at Detours (commercial) and EasyHook (free). They significantly simplify the process of writing hooks and redirecting API calls (both Win32 and Application).
The program would have to be run with administrative privileges in kernel mode of the host OS.
While I don't have extensive experience with windows kernel hooks, in BSD and linux its trivial to install a kernel module that over-writes the system calls for creating sockets and could easily redirect all sockets to a proxy socket of choice.
If you mean [any destination port] to [one port] then you will have to rely on special drivers. The problem with windows is the inability to natively block [drop] packets. For example a common solution is winpcap. However, while you can monitor traffic, you cannot stop the traffic or modify it in a useful way.
On windows the only solution I've seen would be to use some open TUN/TAP adapter. With that, you would be able to modify every packet that leaves your system.
If you know beforehand the destination port you will be using then it gets rather simple. Simply write a passthrough c++ socket program that will only change the destination port.
If you want to redirect browser requests then you can simply edit the settings in your browser.