We are using API Manager to expose the underlying DSS services. The API Manager starts around 10 java threads in linux for every request/response from the client. But these threads stays there forever and eventually our server runs out of memory.
Shouldn't these threads be cleared off automatically? Do I need to look at some configurations to control this.
Any pointers please.
If you are getting OOM, you can tune the Operating System as well as the JVM. Please refer tuning performance documentation [1].
Related
I'm trying with WSO2 products, and I'm thinking about a scenario where bad code could take up all the CPU time (e.g. dead loop or so). I did try it with WSO2 AS with 2 tenants, A and B. And A's bad code does affect B and B's app will have a very long reponse delay or even stuck. Is there a way to restrict the CPU usage of a tenant? Thanks!
At the moment, you will have to setup your environment in what is known as private jet mode, where each tenant gets its own JVM, if you need total isolation.
In a shared environment, we have stuck thread detection which will ensure that critical threads will not run for more than a specified time period. We have plans for CPU usage limiting on per tenant basis. This would be available in a future release.
My suggestion would be to not run two tenants in one application server. Run two separate processes on the same machine. Better yet, run two separate processes in separate OS-level containers (like a jail or an lxc container). Or separate virtual machines if you can't use containers.
Operating systems give you tools for controlling CPU use - rlimit and nice for processes, and implementation-specific facilities for containers and VMs. Because they're implemented in the OS (or virtual machine manager), they are capable of doing this job correctly and reliably. There's no way an application server can do it anywhere near as well.
In any case, having separate applications share an application server and JVM is a terrible idea which should have been put to death in the '90s. There's just no need for it, and it introduces so many potential headaches.
I have an application running in a Java EE App Server and it needs to call a web service of a partner company.
Using wsimport.exe from my JDK (1.6) I have generated the client classes. I instantiate the service and get the port in order to call the web service.
I noticed that the first call to the web service is slow, and I am led to believe this is because it is validating the WSDL. Subsequent calls are fast.
I could keep the WSDL locally, and apparently that will speed up the first call.
In order to optimise my app, I was thinking I could create a pool of the clients. This has the added advantage that I have some throttling in the app - lets say I have a pool of 5 clients, then at most I will be using memory for 5 clients. If the load increased suddenly on my server, I don't have to worry that an unlimited number of clients would cause an out of memory error. I am assuming, based on past experience, that the web service clients use a lot of memory...
Would you bother with a pool?
How would you get over the first call to the web service being slow?
What is the best way to create that pool, so that I have to do the least amount of programming (i.e. I'd like to use a library / API / whatever, so that I don't have to reinvent the wheel and code some hairy bugs).
The Apache Commons Pool might be exactly what I am after.
It is configurable and seems to have thought of everything.
A colleague of mine suggested that you can use the #WebServiceRef annotation on a field in an EJB. The idea is that the server would inject a reference to a client, from which one can create a port for each thread that calls the EJB.
I assume that injected references come from a pool, although the specification doesn't appear to talk about this. The Javadoc for the annotation explicitly mentions that:
"the injected references are not thread safe"
AKKA with a master/slave setup as shown in the link could work well, albeit a little more complex than the Apache Commons Pool listed in another answer. AKKA also uses an execution pool, with its own threads, which isn't strictly allowed in the Java EE world, although I'd argue that because a well tested framework is in charge of the threads, there is no danger, and it shouldn't interfere with the app servers control of threads anyway as the number of threads being handled by AKKA is minimal.
I have been working with spring web applications using jetty/tomcat app server for around two years now, however the thing that eludes me still is how are multiple requests handled in these servers. I understand that spring is helpful in making singletons, but my understanding is just limited to that.
Can someone point to any good resource that can help me understand how multiple requests are handled.
This can be answered at so many levels I have been staring at it for two days trying to figure out how answer it...so I'll take a kinda high level shot at it.
There is this server port that jetty listens on and some number of acceptor threads whose job it is to get connection objects made between the client and server side. Once you have that connection it flows through the jetty handler architecture doing things like authentication perhaps, or pulling off a session id and attaching a session object to the request. Then it works its way into the servlet handler and the appropriate servlet is found and you start dealing with the servlet-api. At that point you have a thread allocated to your request for all of the time you are in the servlet-api, at least under servlet 2.5. In servlet 3.0 you have some async mechanisms available to you, or you can use jetty-continuations as a way to get async support on servlet 2.5 api.
Anyway, there is a thread pool that the server uses to allocate threads to those connectors which ultimately are the threads spending all their time in the servlet-api. The jetty continuations api and the newer servlet 3.0 support provide mechanisms to release threads back to the primary threadpool so they can spend time on accepting and processing other requests.
There is obviously a lot more going on under the covered related to usage of the nio api's and how jetty efficiently manages all of this stuff, but maybe this is enough to sate your initial question. If not, feel free to peruse the jetty docs (http://www.eclipse.org/jetty/documentation/current) or look to the jetty mailing lists. There has been some discussion on jetty-9 optimizations as it relates to under the covers with http, spdy, and websocket connection handling and processing in the blogs at Webtide (http://webtide.com/blogs).
i am developing a turn based multiplayer game with flex and blazeds.
Problem is that i read that the blazeds can handle only hundereds of concurrent users,but this can be increased by using nio server like jetty 7 and servlet 3.0.
does Tomcat 7 supports nio? and i wonder if i can increase concurrent user count by using tomcat 7and blazeds to a few thousands.
Any clue or help will be appreciated.
Thank you.
Do not worry yet about performance. If your game will be successful you will be able to afford the better technical solution. If not, it will not matter if you can handle 1000 or 1000000 requests.
However, related to your question - you may be able to increase the number of concurrent users by doing server related tunings (like stack size, increase the size of the thread pool).
There a couple of solutions implementing Servlet 3.0 (NIO), but you will have to write your own BlazeDS NIO endpoint - so it does not work out of the box.
Edit:
Using the NIO Jetty connector by can be a good idea...but the first thing which should be done is building and testing a valid performance scenario. For example if you plan to support 10000 connected users and to push 1 msg/sec you need to write stress test for that. After that, you can experiment using various connectors/configurations.
There is one tool created by Adobe which can help you with performance testing - it's located here (take a look at the attachments of Adobe LiveCycle Data Services 3 ES2 Performance Brief.pdf). It contains instructions how to configure/run the stress tool. If you cannot manage to run it let me know
Just to give you an example, on my machine (i7 Q820 8gb ram), using the stress tool I was able to handle 10000 connected users.
I am wanting to write some web services using WCF.
I would like to have a "thread pool" in my web service.
For example, I have nearly 6gb of data I need to manipulate.
I would like the client to call an operation on the webservice and have a new task or thread created. The client is able to call a ListRunningTasks(); and have the webservice return a list of tasks. The client should be able to forcefully kill a task if it is taking too long e.g. KillTask(int taskID); or something. I have previously done some threading, but not inside WCF or a service that doesn't have state. Is this possible? If so, how would one go about implementing such a thing? Any reading, links or suggestions would be great.
Thanks, Mike.
One possible solution:
Implement explicit queues for your outstanding tasks taking into consideration that they take that long (20-30mins as you wrote).
Build a custom component to manage those queues e.g. you might even want capabilities to persist them, resume work when you restart the service etc.
Have explicitly created worker threads that pickup work from those queues.
Implement a WCF service to make your queue manager available to external systems.
Thread pools are more designed to process a high volume of short-running tasks.
You should consider using Windows Workflow Foundation to create such services. A state machine workflow can be exposed as a service in such a way that when method A is called, it will start the workflow (task), after which methods can be called to stop, suspend, or query the running task. WF will handle the state transitions, preventing illegal state changes, and making sure that new tasks are only spun up as necessary.
Note that WF will handle the threading issues for you in an almost transparent manner.