NXLog TCP Module Not Working - amazon-web-services

I am trying to use NXLog to forward logs from a file on one computer to a file on another computer. I am new to NXLog but feel like I now have a good handle on it after consulting the documentation. Below is my configuration file for first the "from" computer and then the "to" computer. However, it does not seem to work.
I think the problems lies in one or more of the following:
NXLog config file
AWS Windows ec2 box ip/port setup
Network setup
Here what I have done:
"From" computer NXLog config file:
define ROOT C:\Program Files (x86)\nxlog
Moduledir %ROOT%\modules
CacheDir %ROOT%\data
Pidfile %ROOT%\data\nxlog.pid
SpoolDir %ROOT%\data
LogFile %ROOT%\data\nxlog.log
<Extension _syslog>
Module xm_syslog
</Extension>
<Input file>
Module im_file
File "C:\Logs\AllLogItems.txt"
InputType LineBased
</Input>
<Output tcp>
Module om_tcp
Host 52.91.47.52
Port 3389
OutputType LineBased
</Output>
<Route 1>
Path file => tcp
</Route>
"To" computer NXLog config file:
define ROOT C:\Program Files (x86)\nxlog
Moduledir %ROOT%\modules
CacheDir %ROOT%\data
Pidfile %ROOT%\data\nxlog.pid
SpoolDir %ROOT%\data
LogFile %ROOT%\data\nxlog.log
<Extension _syslog>
Module xm_syslog
</Extension>
<Input in>
Module im_tcp
InputType LineBased
</Input>
<Output out>
Module om_file
File "C:\Users\Administrator\logs\\nxlog_test.txt"
CreateDir FALSE
Truncate FALSE
OutputType LineBased
</Output>
<Route 1>
Path in => out
</Route>
For the "to" computer, according to the documentation, if no Host is specified, localhost is the default.
When I start the NXLog service on the "from" computer this is what the log looks like:
2016-01-21 19:37:05 INFO nxlog-ce-2.9.1504 started
2016-01-21 19:37:05 INFO connecting to 52.91.47.52:3389
And then when I run my application which will write logs(which will then cause nxlog to react), the nxlog log looks like this:
2016-01-21 19:37:05 INFO nxlog-ce-2.9.1504 started
2016-01-21 19:37:05 INFO connecting to 52.91.47.52:3389
2016-01-21 19:40:25 INFO reconnecting in 1 seconds
2016-01-21 19:40:26 INFO connecting to 52.91.47.52:3389
When I start NXLog on the "to" computer, every seems to be fine as the nxlog log file looks like this:
2016-01-22 01:05:04 INFO nxlog-ce-2.9.1504 started
Now, the "to" computer is a AWS ec2 Windows box. The port I'm using in the nxlog (3389) is the RDP port assigned to the instance:
My thought was that even though the nxlog log didn't show an error, that this port was causing issues as is may be in use. Hence, I tried to open some more tcp ports (5000-5010) as shown in the picture. However, when I try any of these ports (5000-5010) in the nxlog config file, I get this error:
2016-01-21 19:50:32 INFO nxlog-ce-2.9.1504 started
2016-01-21 19:50:32 INFO connecting to 52.91.47.52:5005
2016-01-21 19:50:53 INFO reconnecting in 1 seconds
2016-01-21 19:50:53 ERROR couldn't connect to tcp socket on 52.91.47.52:5005; A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond.
I kind of expected this as when I go to a site like yougetsignal.com, it still says that ports 5000-5010 are closed. So it might be a problem with this.
I've tried to debug this for days and have ran out of options. Thanks for your help.

Decided this was a question more appropriate on ServerFault and so I posted it there are received a working answer. It can be found here

Related

Connecting to Tensorboard Logs on a Remote Server: The Connection was Reset

When trying to connect to tensorboard logs on a remote server and entering the address http://localhost:16006/ in chrome and firefox I get this message in the command line "channel 3: open failed: connect failed: Connection refused" multiple times and this message on the browser, "The Connection was Reset".
I ssh into the server like this: ssh -L 16006:127.0.0.1:6006 username#machine and then go one level up the log folder and write: tensorboard --logdir logs --port 16006
I tired:
tensorboard --logdir logs --port 16006 --bind-all
and also
tensorboard --logdir logs --host localhost, also
tensorboard --logdir logs --host 127.0.0.1
None of the above has wroked. I tried running the line above from another environment which didn't help. I went to the office and tried connecting to the logs from the server machine directly and it worked.
It used to work before when accessing remotely. Do you know what the problem is? any hint would be immensely appreciated.
I am having the same problem, but I think you should designate --port 6006 since it looks like you are forwarding remote port 6006 to the local port 16006.
Even so, since 6006 is the default port, the other commands should have worked, but you would have to go to http://127.0.0.1:16006 on your local machine, rather than the link it provides.
Some more in depth explanations can be found here how to run tensorboard on a remote server and how to see tensorboard over ssh
Even following this advice though I am still getting a 'channel 3: open failed: connect failed: Connection failed' error

Artifactory OSS 6.5.2 - can't connect to the UI from servers on the network

I have recently installed Artifactory OSS 6.5.2 on a remote server in our network which runs on windows server 2012.
I can enter the UI locally (the machine running the Artifactory instance) through any of the browsers with this address:
"http://{local-ip}:8081/artifactory/webapp/#/"
When I try entering the UI from one of the machines on the network I get a "This site can’t be reached" message after multiple attempts to connect.
The request.log at {ARTIFACTORY_HOME}\logs\request.log shows that the request got through and succeeded:
"REQUEST|{remote-ip}|anonymous|GET|/webapp/|HTTP/1.1|200|0"
The same is showed for requests coming from the server running the Artifactory instance:
"REQUEST|{local-ip}|anonymous|GET|/webapp/|HTTP/1.1|200|0"
However, in contrary to the previous request from a remote machine, the initial request is followed by more requests:
"REQUEST|{local-ip}|anonymous|GET|/ui/auth/screen/footer|HTTP/1.1|200|0
REQUEST|{local-ip}|anonymous|GET|/ui/treebrowser/repoOrder|HTTP/1.1|200|0
REQUEST|{local-ip}|anonymous|GET|/ui/onboarding/initStatus|HTTP/1.1|200|0
REQUEST|{local-ip}|anonymous|GET|/ui/auth/current|HTTP/1.1|200|0"
I thought maybe there is an automatic redirection that uses 'localhost' instead of the ip or hostname so I tried changing the {ARTIFACTORY_HOME}\tomcat\conf\server.xml:
<Service name="Catalina">
<Connector port="8081" sendReasonPhrase="true" relaxedPathChars='[]' relaxedQueryChars='[]'/>
<!-- Must be at least the value of artifactory.access.client.max.connections -->
<Connector port="8040" sendReasonPhrase="true" maxThreads="50"/>
<!-- This is the optional AJP connector -->
<Connector port="8019" protocol="AJP/1.3" sendReasonPhrase="true"/>
<Engine name="Catalina" defaultHost="localhost">
<Host **name="localhost" -> name="{hostname}** appBase="webapps" startStopThreads="2"/>
</Engine>
</Service>
But then the Artifactory failed to initialize:
"[art-init] [INFO ] (o.a.s.a.ArtifactoryAccessClientConfigStore:643) -
Using Access Server URL: http://localhost:8040/access (bundled)
source: detected
[art-init] [INFO ] (o.a.s.a.AccessServiceImpl:308) - Waiting for
access server...
[art-init] [WARN ] (o.j.a.c.AccessClientHttpException:41) -
Unrecognized ErrorsModel by Access. Original message: Failed on
executing /api/v1/system/ping, with response: Not Found"
I did not set any proxies or reverse proxies as I don't think it's related, but I may be mistaken as I don't have a lot of experience with web services.
Any ideas or suggestions?
Thnx,
Tom.
I was deploying artifactory 6 via helm, then upgraded to 6.8.2 and ran into this.
had to
cd $ARTIFACTORY_HOME && chown -R artifactory:artifactory .
artifactory itself, on startup, seemed not to be able to deploy the access.war and then maybe also was not able to read the credentials it needed to hit this /access context health check "ping" api endpoint.

Datastax Agent (Cassandra) Opscenter setup issue

I've setup opscenter on one of cassandra cluster nodes. After installation, when setting up my cluster, I tried installation of datastax agent on all the cluster nodes via UI, but it failed. So, I had to install the agents manually.
After manually installing the agents, the node in which opscenter is installed is able to connect, but not the other nodes. It still says, "2 agents failed to connect". What could be the issue?
PS : My cassandra cluster is setup on AWS in ubuntu
My agent.log file looks like this
ERROR [os-metrics-9] 2015-07-27 07:04:43,390 Long os-stats collector failed: Cannot run program "iostat": error=2, No such file or directory
ERROR [os-metrics-7] 2015-07-27 07:04:43,391 Long os-stats collector failed: Cannot run program "iostat": error=2, No such file or directory
ERROR [os-metrics-8] 2015-07-27 07:04:53,391 Long os-stats collector failed: Cannot run program "iostat": error=2, No such file or directory
ERROR [os-metrics-3] 2015-07-27 07:04:53,392 Long os-stats collector failed: Cannot run program "iostat": error=2, No such file or directory
ERROR [StompConnection receiver] 2015-07-27 07:05:02,946 failed connecting to **.**.**.**:61620:java.net.ConnectException: Connection timed out
You have to set the stomp_interface in the address.yaml like
stomp_interface: <ip-address>
After agent restart it should be connected.
As your agent have been able to connect from the same box where opscenter is installed, so it sounds like :
You might have not configured your firewall properly. If you please try by disabling firewall on all your boxes.
You may have multiple interfaces and C* installation picked up an undesired interface. So run ifconfig or ip command on all of your instances and check with C* yaml.
About iostat failure message : You have not install sysstat pkg. Seems, you have not install dependencies as part of DSE install.
The agents uses iostat to collect some information from disks. If it cant find it you will get that error but it just means those metrics will be missing some os metrics (likely a lot of disk and cpu metrics will be missing)
These are some useful configurations that you should keep in mind when starting the agent manually in the conf/address.yaml file:
###A name for the node to use as a label throughout OpsCenter.
alias:
###Reachable IP address of the opscenterd machine. The connection made will be on stomp_port. Internal IP in this case
stomp_interface:
###Port for the agent's HTTP service (default: 61621).
#api_port: 61621
###The stomp_port used by opscenterd. == Must match with the 'incoming_port' in opscenter.conf
stomp_port: 61620
###The IP used to identify the node.
local_interface: 100.73.158.44
###The IP that the agent HTTP server listens on.
agent_rpc_interface:
###Host used to connect to local JMX server.
jmx_host: 100.73.158.44
###Whether or not to use SSL communication between the agent and opscenterd.
use_ssl: 1
To solve the "Cannot run program 'iostat'" error, do this:
sudo apt-get install sysstat

Installing and Viewing Neo4j on Existing AWS EC2 Instance

I'm trying to install the enterprise edition of neo4j on an existing EC2 (Amazon linux) instance. So far I've
wget "link to enterprise"
untar the file
renamed and moved the folder to NEO4J_HOME
then went into the config files for neo4j.properties to make the following changes:
# Enable shell server so that remote clients can connect via Neo4j shell.
remote_shell_enabled=true
# The network interface IP the shell will listen on (use 0.0.0 for all interfaces)
remote_shell_host=127.0.0.1
# The port the shell will listen on, default is 1337
remote_shell_port=1337
EDITED Christophe Willemsen pointed out that for my original error, I had forgotten to restart the server at that point but I was still unable to access the web server while it was running. So to make it more clear, I've edited the remaining post:
I went to neo4j-server.properties and uncommented:
org.neo4j.server.webserver.address=0.0.0.0
And start the server
NEO4J_HOME/bin/neo4j start
WARNING: Max 1024 open files allowed, minimum of 40 000 recommended. See the Neo4j manual.
Using additional JVM arguments: -server -XX:+DisableExplicitGC -Dorg.neo4j.server.properties=conf/neo4j-server.properties -Djava.util.logging.config.file=conf/logging.properties -Dlog4j.configuration=file:conf/log4j.properties -XX:+UseConcMarkSweepGC -XX:+CMSClassUnloadingEnabled -XX:-OmitStackTraceInFastThrow
Starting Neo4j Server...WARNING: not changing user
process [28557]... waiting for server to be ready..... OK.
http://localhost:7474/ is ready.
checking the status:
NEO4J_HOME/bin/neo4j status
Neo4j Server is running at pid 28557
I can run the shell but the when I go to localhost 7474 I still can not connect
Any help would be appreciative. The only tutorial or help I've found assumed I was starting from scratch with a new instance. If someone could provide some instructions for installing or fix my configuration that would be great.
Thanks!
You have to edit neo4j-server.properties and uncomment the line with:
org.neo4j.server.webserver.address=0.0.0.0
So that the db listens on an external interface not just localhost, and you have to open the port (7474) in your firewall rules.
Make sure to secure access to the db though:
http://neo4j.com/docs/stable/security-server.html

WSO2 Identity Server startup error - Error in initializing thrift transport

I just installed WSO IS and receive this error when starting the server.
ERROR {org.wso2.carbon.identity.entitlement.internal.EntitlementServiceComponent}
- Failed to initialize Entitlement Service
ERROR {org.wso2.carbon.identity.entitlement.internal.EntitlementServiceComponent}
- Error in initializing thrift transport
org.apache.thrift.transport.TTransportException: Could not bind to port 10500
I've verified nothing is using the 10500 port.
In order to test WSO2 locally on my development workstation, I hard-coded a hostname via the hosts file and modified the HostName and MgtHostName entities in ./repository/conf/carbon.xml. One caveat is that my workstation has a dynamic IP address. When the IP does occasionally change, leaving the hostname pointing to an unresponsive IP, starting WSO2 IS fails with the exact same error.
Repairing the host entry and restarting the WSO2 Identity Server resolves it.
Not sure but you can try this:
1.Kill the java process if windows system or in Linux check if another process running
already which was not shut down properly by searching using ps -ef | grep java.
2.Restart the IS server.