We are using hawtio to have a fancy and nice web interface for seeing JMX MBeans and Camel Route in our project. However, we have noticed that after weeks running Hawtio stops working and we are getting a Jetty error when trying to access it.
We are using hawtio in standalone mode, version 1.2.0/offline. Also I guess it worth to mention that our Camel routes are pretty heavy and consume many resources (not sure if that impacts hawtio). When trying to access we get this:
HTTP ERROR 404
Problem accessing /ourContextPath/. Reason:
Not Found
Powered by Jetty://
It seems like there is no active resource for our context path and I something went wrong like a thread stopped working or something.
Does anybody have any idea how to solve this or how to find what's causing this? Also, is this a known bug fixed in the latest version (1.2.1)?
Jetty needs a work/temp directory to operate.
Default behavior is to use whatever java.io.tmpdir points to.
However, on many unix installations, this points to /tmp, and that directory is often cleaned out by other processes.
To fix, either specify a java.io.tmpdir to be somewhere other than /tmp
$ java -Djava.io.tmpdir=/var/run/jetty -jar start.jar
or create a ${jetty.base}/work/ directory (if running Jetty 9.1+)
or create a ${jetty.home}/work/ directory (if running versions of Jetty prior to 9.1)
See the answer at Jetty: Starts in C:\Temp for more details on how this work/temp directory operates and is configured.
Related
I had installed ColdFusion 2018 recently and with the installation less than a month old (and my understanding of the technology even less), my Cold Fusion service has stopped working. I have tried a number of things and have referred to a number of articles and out of many such errors where the service is not being accessible, some of them were able to get it resolved. However, some other obscure reason that may be causing this error have been untouched and unknown.
Whenever, I try to restart the service, I get an error as shown below:
Windows could not start the ColdFusion 8 Application Server on Local Computer. For more information, review the System Event Log. If this is a non-Microsoft service, contact the service vendor, and refer to server-specific code error 2.”
Without much understanding, I started to google it out. Looking into every one of these posts, I tried
Configure JRE and try to relaunch the service by looking at "JAVA_HOME" variable and JVM.config
Run the batch files in every possible combination to find if anything clicks
Check if the present JAVA version works and is compatible with Coldfusion version installed
Fiddling with the "SessionStorage" var in neo-runtime.xml file as some suggested
and few other tricks coupled with a numerous service restart attempts and a few machine reboots as well.
A service that renders Cold Fusion pages should be shut down abruptly. To add to agony, the CF Admin also depends on the service and hence does not work.
Any pointers to any potential solutions?
I have a problem with a web service that is installed and started with a .msi that is created with the WiX toolset.
The service can be installed and started on all the machines I tested so far (shown as running in the Services Manager) but on some machines it is not reachable (for example via a browser) and not shown in the list of listening ports on that machine (displayed with 'netstat -a').
I am trying to figure out what's going wrong but I am not really familiar with web service development and configuration. It's a third party service, thus I don't know how it works internally.
A good starting point for me would be to find out, what exactly happens when a service is installed and started during the execution of the .msi-file.
Maybe I could try to tackle the problem on a lower level then.
Below is my code in the ServiceInstall-Element:
<ServiceInstall
Id="ServiceID"
Type="ownProcess"
Vital="yes"
Name="ServiceName"
DisplayName="ServiceDisplayName"
Description="Lorem Ipsum"
Start="auto"
Account="LocalSystem"
ErrorControl="normal"
Interactive="no"
Arguments="action=run">
</ServiceInstall>
The argument is important - without it, the service won't start or run.
Maybe someone else encounterd the same or a similar problem and can help me out.
Thanks already in advance - each hint is appreciated.
EDIT I (15.04.18):
As it might be a problem with the specific service, I will add some further information here:
It's a third party software called CryptoLicensing:
http://www.ssware.com/cryptolicensing/cryptolicensing_net.htm
Part of this software is that specific program, that serves as a License Server and does the license registration, for example in a customer's network.
The service can be run as a Windows application or installed and run as a Windows service. In both cases it should be listening on a (pre-)specified port on the installed machine.
Whenever I start the .exe as an application, everything works as intended. The service is reachable (for example with the browser) and can be accessed from other machines in the network.
When the .exe is installed and started as a service, it does not work as intended on every machine. For example if I install and start the service on my laptop, it is shown as running in the Services Manager, but is not reachable on its assigned URL (not even on the localhost) nor is the specific port displayed in the active listening ports, for example with 'netstat -a'.
The service itself starts without any error messages and does not log any errors or exceptions as it seems to be running without any problems.
I contacted the vendor, but sometimes he doesn't reply quickly and he is not very specific in his replies.
Before asking the question I assumed that it was a problem with the Windows user rights and the WiX installer but during the discussion here I had the feeling that it might a problem with the service itself.
I hope this 'new' piece of information helps in isolating and location the problem.
Thanks to everyone who helped so far!
Hopefully not stating the obvious here, but WiX doesn't do much except populate the ServiceInstall table in the MSI file, so this is about why Windows Installer won't start the service. ServiceInstall table:
https://msdn.microsoft.com/en-us/library/windows/desktop/aa371637(v=vs.85).aspx
Also, this isn't really about ServiceInstall - it's probably about the ServiceControl element in your WiX source, but it's not clear whether that's how you're starting it or if you're starting it manually later on. That does make a difference. What is the error message and where are you getting it, and is it a 1920 or 1921 error (in the context of ServiceControl).
The main reason a service will start on one system but not another is missing dependencies. If your service is C++ based (the post doesn't say) then there are probably dependencies on C runtimes, UCRT runtimes, MFC or ATL runtimes and so on.
First: are you sure this service is intended to run as LocalSystem? (MSDN, SO).
Second: did you check the event logs in detail for anything obvious? If the service is good you should find a hint at least. Something to start with. I find that I sometimes miss the actual logs in the event viewer because it is so "crowded". My take on it: empty the log and stop and restart the service.
Something locking / blocking: If the service installs and runs OK I would suspect other factors such as firewalls (hardware & software), security software in general (anti-virus, malware scanners), network configuration issues (proxies, WINS, DNS and all the complexities involved in networking). Is the service trying to reach an UNC path?
Diverse Machines: What are the target machines? Are they virtual, are they physical, are they test machines, are they operative SOE machines in corporate networks? Are they the same OS version and edition?
Further Ideas: It is not quite related, but maybe skim this list of suggestions for debugging from another answer (I am not sure why it was down-voted, I think it is an OK list to inspire debugging ideas): Windows Application Startup Error Exception code: 0xe0434352 (maybe just skim the bolded words for ideas - Recommended).
sc.exe: And finally, perhaps check the sc.exe tool (Service Control) and see if it can provide you with some useful information for debugging.
sc.exe in the context of killing hung services (sample use).
sc.exe from MSDN
Some further links:
Windows Services Frequently Asked Questions (FAQ). Content seems to be up to date - at face value at least. These guys claim to be experts on services. I have no idea who they are.
Essential Tools for Windows Services: SC.EXE
Run Service Control (sc.exe) command on secure port
After almost 20 months we finally (and accidentally) found a solution to the problem! For the few machines, on which the service did not run properly, setting the NoInteractiveServices value in the registry to 0 did the trick. A value of 1 (which is default) means that no service is allowed to run interactively, regardless of whether it has the SERVICE_INTERACTIVE_PROCESS property. More information on Interactive Services.
I am not completely satisfied with the solution, because on all the other machines NoInteractiveServices is set to 1 AND the service runs properly anyway. However, on the machines where the service did not run interactively this solution worked for us. Thus I will accept this as an answer.
If anyone has more information on this issue and can explain why this works, feel free to
add them - I would be very interested!
Please forgive me since I'm rather new to this, but I'll try to explain the situation as best as I can.
The current system is running on ColdFusion 9 and ActiveMQ 5.3.1. We are trying to upgrade the system to ColdFusion 11 and I'm running into some issues. In ColdFusion 9 ActiveMQ appears to run fine on Beta and Production (we also have a local and alpha environment) however in ColdFusion 11 I get the following error: http://puu.sh/gVp1L/d77453416f.png This happens in the /CFIDE/adminapi/eventgateway.cfc:204 which of course is a file that has been abstracted to not be readable.
I have tried the following:
Three different ActiveMQ servers
Two different installs of CF11
Going step by step through the eventgateway api
When I manually try to start the Gateway Instance in CFIDE, it goes to a white page, and nothing shows up in the eventgateway log. The only error I've managed to see was actually in CF9 and it was:
Cannot instantiate class: org.apache.activemq.jndi.ActiveMQInitialContextFactory
I'm assuming that's just a goofy ActiveMQ install locally though. I did some googling on this error and it says that I might be missing a .jar file within ActiveMQ, but when I view the /lib directory I can see the file they claim must be missing. Within CF9, I can actually see errors in the eventgateway log, but like I said, I get a white screen and nothing happens in CF11.
Any ideas?
Thanks,
Nick
This was due to the activemq.jar not being including in the cfusion/lib folder for CF11, but was for CF9.
We are trying to use VisualVM to track down some memory leakage in CF8, however, cannot get the tool to work 100%. Basically, everything comes up, except the Memory sampling. Says that the "JVM is not supported".
However, all the other features work (we can do CPU sampling, just not memory). Found this kind of weird that we can do everything else but the memory stuff, so am wondering if maybe we need to specify another JVM argument to allow this?
Some other info:
We are connecting locally via 127.0.0.1 or localhost.
I installed the Visual GC plugin, and it cannot connect either.
VisualVM and JRUN/CF8 are both using the same Java version (1.6.0_31), however, they are not pulled from the same location (maybe this matters). VisualVM uses the installed JDK, whereas JURN/CF8 uses just the binaries that we copied locally to the CF8 installation folder.
Installed another plugin that shows JVM properties, and it says that the JVM is not "attachable". Don't know what that means, but am just wanting to mention it.
Any help with this would be greatly appreciated. If we can just get that memory sampling, I think we can get on top of our performance issues that have plagued us here recently. Thanks in advance!
EDIT:
Also, just checked, and JRUN is being started under "administrator", whereas I am launching VisualVM under a different user. Maybe this is relevant?
Yes, it is relevant that you are running VisualVM under different user. Memory Sampling uses Attach API, which only works if you are running monitored application and VisualVM as the same user. This is also reason that the JVM properties reports that your application is not attachable. If you run VisualVM as "administrator", it will automatically detect your Coldfusion 8 application and the Memory sampler will work.
I managed to run Django using IIS as webserver (using PyISAPIe) and everything goes well in my test server, mounting Windows 2008 Server R2 64bit.
Then I installed the application on another server with the same configuration and it works fine for the first request. Then when I reload the page, I get a "Service not working" page.
On the event log I see an Application error saying that python26.dll had some problems:
Faulting application name: w3wp.exe
Faulting module name: python26.dll
Exception code: 0x40000015
Faulting application path: C:\Windows\SysWOW64\inetsrv\w3wp.exe
Faulting module path: C:\Windows\system32\python26.dll
Can you give me some hint on how to solve that problem?
UPDATE: "Rapid-Fail Protection" in the Advanced Settings of the Application Pool was set to 5 failures; disabling it, all worked well.
So, now the question is: how can I detect what caused the failures?
UPDATE: I discovered that IIS crashes when there are multiple requests (img, css, js). PyISAPIe is called for each of them, passing them to static server once recognized.
No idea why this happens...
PyISAPIe is not a good choice to run Django on Windows 2008. In this article you can find better solution: Running Django on Windows (with performance tests)
Check the eventlog it should be in there.
You may also find some more detail in the httperror log (C:\Windows\System32\LogFiles\HTTPERR).
I discovered that IIS crashes when there are multiple requests (img,
css, js). PyISAPIe is called for each of them, passing them to static
server once recognized. No idea why this happens...
Do multiple request cause the error on both machines? When there are multiple requests in an ISAPI application, each request runs in its own thread. The Python multi-threading model is GLOBAL - all threads running under that Python process are co-mingled and sharing all global resources, so you must serialize all multi-threaded code running in all applications and processes using your Python engine. This is a serious downside in Python multi-threaded processing and may be the source of your problems. See http://docs.python.org/library/multiprocessing.html and other sources.
But even this only occurs on one machine and not the other, that may still be the cause - it could also depend on many other environmental variables - number of requests, resources of the machine, processors, etc.
Check memory usage on machine (total physical).