I have one question: how can I close the database connection in wso2 manually when using dblookup mediator?
I saw the dblookup mediator code on github [here] (https://github.com/wso2/wso2-synapse/blob/master/modules/core/src/main/java/org/apache/synapse/mediators/db/DBLookupMediator.java). I saw that the dblookup mediator code uses the "finally" part of the code where all connections are closed, BUT when I check all active sessions to the database, I see that the connections are still open.
It seems the DblookUp Mediator maintains a connection pool. Hence closing the connection may return it back to the pool. If you want to reduce the size of the pool probably you can try tuning the connection pool settings. For example this property maxactive will set max active connections. Take a look at this document.
Also, AFAIK there is no way to manually flush the connections.
Related
Checking out my logs on my App Engine. I get A LOT of
New connection for "<project_id>-central1:<project_name>"
Client closed local connection on /cloudsql/<project_id>-central1:<project_name>/.s.PGSQL.5432
Like happening multiple times a second and just floods my logs.
I was unable to find any information relating to this and maybe this is just a non-issue.
Is there any way to prevent this? (excluding filtering)
Is this inadvertently driving up the cost of operation of opening and closing?
I am using Django on the app engine.
I found this post where it's mentioned that setting -verbose=false will turn off the new/closed connection logs.
I found information about the same error but it wasn't generating a lot of connections. Anyway it was related to the Cloud SQL proxy.
Have you followed the instructions in this guide to configure the PostgreSQL connection to App Engine? I am particularly interested in the ones from "Setting up your local environment".
I did not found any related field in quotas or pricing pages but you can check the billing in the Google Cloud Console: Billing -> Overview -> [PROJECT_ID].
I'm not a django developer but I guess the root of this problem is that django opens a new connection to the database for every request by default.
Source: https://docs.djangoproject.com/en/2.1/ref/databases/
Persistent connections avoid the overhead of re-establishing a connection to the database in each request. They’re controlled by the CONN_MAX_AGE parameter which defines the maximum lifetime of a connection. It can be set independently for each database.
The default value is 0, preserving the historical behavior of closing
the database connection at the end of each request. To enable
persistent connections, set CONN_MAX_AGE to a positive number of
seconds. For unlimited persistent connections, set it to None.
You can try to increase the CONN_MAX_AGE or set it to None and the log messages should disappear.
Changing CONN_MAX_AGE value to None can help, however this may expose your application to bot attacks like exposed me (see the picture below):
Looking for the IP's in abuseIPDB.com I've found a lot of reports of Brute Force/Web App Attack from it.
Maybe setting the variable value to a fixed number may keep your application safe and stop these logs.
Currently we are using wso2 4.1.0 version. we are using soap based services authentication admin, entitlements service for getting policy decision , getting claims values. we are using cxf for our webservice clients. when we make calls with 500 concurrent users from a single machine everything works fine. but when we make it to 1000 concurrent users we are seeing a huge response time for these service calls from wso2 is. can you tell if there is any configuration change that we need to make for tuning. we increased the wso2 is axis client no. of connections per host but we still didn't see any improvement. By the way we are using the default configuration of wso2 is out of the box
Thanks
Kishore
There are some places you can improve the performances.
1.Increase the memory setting. you can find it in the wso2server script file in /bin. can u increase default value and see.. such as
-Xms1024m -Xmx2048m -XX:MaxPermSize=1024m
2.Increase max thread pool size in catalina-server.xml file which can find in /repository/conf/tomcat
maxThreads="250"
3.Increase caching timeout value. entitlement.properties file which can find in /repository/conf/security
4.Please check you are not authenticating for each request. You need to call AuthenticationAdmin 1st time and get the cookie and then use cookie for subsequence requests.
Else could do use some Jconsole or Jprofiler and see what is going wrong? Also according to your environment, this can be the max load that one server can handle, then you need to do horizontal scaling. (adding WSO2 Identity Server more instance in a cluster)
I configured a BAM Server Profile and a stream in order to connect one of my Proxy Services to my BAM, I incorrectly configured the IP adress of my BAM.
When I sent a request to my proxy service, the call failed because the BAM server is not reachable. Does this mean that the exhanges between ESB and BAM is not asynchronous and can imply a failure of my proxy service?
Nicolas, every mediator is synchronous in the chain.
If you want to call the BAM asynchronously, what you need to do is:
1) Use the clone mediator to create an asynchronous thread (Sequential Mediation: NO, Continue Parent: YES). http://wso2.org/project/esb/java/4.0.0/docs/mediators/clone.html
2) In the cloned target do all your async work and place the BAM Agent Mediator.
That way you'll have your mediation working not only safer but also with better performance (since all the transformation you may need while preparing the information to be sent to BAM is happening asynchronously).
No. Although BAM mediator is synchronous as it is a mediator, data sending operation to BAM side is done asynchronously. That means if the BAM server is not reachable due to some reason (e.g., incorrect IP, BAM is not available) ESB still works properly without any problem. Only the message dumping to BAM will fail. That means mediation sequence will function properly with the server not found exception but the message logging will not happen.
During the development time of BAM mediator we considered cloning the entire message and send asynchronously to BAM side but we rejected that idea, as memory cloning will take significant time and processing which will slow down the ESB. But still data sending part (Data Bridge) works asynchronously as mentioned above.
In wso2, VFS running default 5 minutes. I think this is bad for performance. I want VFS running when server is not busy. For example, maybe at 02:00 no body request, then VFS can work. I tried add scheduled task, but it no use. Anyone can tell me how to do this? Thank you very much.
I think your question is related to WSO2 How to set scheduled task? and https://stackoverflow.com/questions/10537190/how-to-vfs-service-exposed-as-a-web-service questions. Seems you want to do the VFS polling on a particular time of the day (scheduled to a non-peak time) or via a web service. But unfortunately I don't think it's possible to plug a CRON expression or a service call to VFS, because of how VFS transport has been implemented. Anyway you can suggest this as a feature request at WSO2 JIRA
What you are suggesting can lead to starvation. If the ESB server is continuously hit with requests, the VFS task will never run. As mentioned in the ESB documentation you can adjust the polling interval by specifying the transport.PollInterval parameter. An example of increasing the polling interval to 30 minutes can look like this in the ESB configuration:
<parameter name="transport.PollInterval">1800000</parameter>
It is easy to display the status of a Windows service or to control it (start/stop) from a GUI application but the question if how about receiving/sending notifications from the service? Like: service telling to the user monitoring it that it needs attention.
Please consider that you can have several controllers started at any time in a multi-user environment.
Do you know an example(open source) for this kind of communication?
Extra points for a platform independent solution :)
If the "user" monitoring the service is a person, how about writing to an event log and then having your enterprise event monitoring system send alerts. Alternatively, have the service log the issue to the event log and then send an email.
If the "user" or client of the service is a separate or multiple applications here are a couple thoughts without fully understadning your problem.
Write to file. Have the client tell the service what file or some agreed token to create a file for that particular client instance and then have the service write to the file. The client can then monitor the file.
Status Ping. Have the client ping the service on a regular basis in a background thread to ask the service how it is doing. If unhealthy, then the client could ping the service for more information. Kind of like getting the LastMessage.
Open Sockets or End Point. Find a way to setup communication back from the service so that the service can call the client. This may be initiated by the client first telling the service where it is so it can call back. I forget the specifics, but WCF should be able to handle this out of the box for you. If you want something more generic you may need to roll your own.
Hope these ideas help.
As an option, use our MsgConnect (free with source) to send messages between services and controllers (crossplatform). Communication between services and controllers is possible both locally (via MMF or socket transport) and remotely (via socket transport).