The thing I want to do is, count how many locks happened during the execution of one JVM application. (I know the lock number may change from run to run, but I just want to get the average number). And I cannot change the application, since it is one benchmark.
I have tried to use JRockit-JDK, but two problems:
-Djrockit.lockprofiling=true does not give me the profile information (link);
does "-Xverbose:locks" print the right information?
The platform I am using is Ubuntu Server.
Any suggestions on this would be great appreciated.
To do this previously I've used AspectJ with a pointcut that detects locking and a counter i.e.
public aspect CountLocks{
private static AtomicInteger locks = new AtomicInteger();
before(Object l) : lock() && args(l) { locks.incrementAndGet(); }
after() : execution(void main(String[])) { System.out.println(locks+" locks"); }
}
But this obviously involves weaving the code, potentially changing its performance characteristics.
Related
I'm using Doobie in a ZIO application, and sometimes I get deadlocks (total freeze of the application). That can happen if I run my app on only one core, or if I reach the number of maximum parallel connections to the database.
My code looks like:
def mkTransactor(cfg: DatabaseConfig): RManaged[Blocking, Transactor[Task]] =
ZIO.runtime[Blocking].toManaged_.flatMap { implicit rt =>
val connectEC = rt.platform.executor.asEC
val transactEC = rt.environment.get.blockingExecutor.asEC
HikariTransactor
.fromHikariConfig[Task](
hikari(cfg),
connectEC,
Blocker.liftExecutionContext(transactEC)
)
.toManaged
}
private def hikari(cfg: DatabaseConfig): HikariConfig = {
val config = new com.zaxxer.hikari.HikariConfig
config.setJdbcUrl(cfg.url)
config.setSchema(cfg.schema)
config.setUsername(cfg.user)
config.setPassword(cfg.pass)
config
}
Alternatively, I set the leak detection parameter on Hikari (config.setLeakDetectionThreshold(10000L)), and I get leak errors which are not due to the time taken to process DB queries.
There is a good explanation in the Doobie documentation about the execution contexts and the expectations for each: https://tpolecat.github.io/doobie/docs/14-Managing-Connections.html#about-transactors
According to the docs, the "execution context for awaiting connection to the database" (connectEC in the question) should be bounded.
ZIO, by default, has only two thread pools:
zio-default-async – Bounded,
zio-default-blocking – Unbounded
So it is quite natural to believe that we should use zio-default-async since it is bounded.
Unfortunately, zio-default-async makes an assumption that its operations never, ever block. This is extremely important because it's the execution context used by the ZIO interpreter (its runtime) to run. If you block on it, you can actually block the evaluation progression of the ZIO program. This happens more often when there's only one core available.
The problem is that the execution context for awaiting DB connection is meant to block, waiting for free space in the Hikari connection pool. So we should not be using zio-default-async for this execution context.
The next question is: does it makes sense to create a new thread pool and corresponding execution context just for connectEC? There is nothing forbidding you to do so, but it is likely not necessary, for three reasons:
You want to avoid creating thread pools, especially since you likely have several already created from your web framework, DB connection pool, scheduler, etc. Each thread pool has its cost. Some examples are:
More to manage for the jvm JVM
Consumes more OS resources
Switching between threads, which that part is expensive in terms of performance
Makes your application runtime more complex to understand(complex thread dumps, etc)
ZIO thread pool ergonomics start to be well optimized for their usage
At the end of the day, you will have to manage your timeout somewhere, and the connection is not the part of the system which is the most likely to have enough information to know how long it should wait: different interactions (ie, in the outer parts of your app, nearer to use points) may require different timeout/retry logic.
All that being said, we found a configuration that works very well in an application running in production:
// zio.interop.catz._ provides a `zioContextShift`
val xa = (for {
// our transaction EC: wait for aquire/release connections, must accept blocking operations
te <- ZIO.access[Blocking](_.get.blockingExecutor.asEC)
} yield {
Transactor.fromDataSource[Task](datasource, te, Blocker.liftExecutionContext(te))
}).provide(ZioRuntime.environment).runNow
def transactTask[T](query: Transactor[Task] => Task[T]): Task[T] = {
query(xa)
}
I made a drawing of how Doobie and ZIO execution context map one other to each other: https://docs.google.com/drawings/d/1aJAkH6VFjX3ENu7gYUDK-qqOf9-AQI971EQ4sqhi2IY
UPDATE: I created a repos with 3 examples of that pattern usage (mixed app, pure app, ZLayer app) here: https://github.com/fanf/test-zio-doobie
Any feedback is welcome.
We have a deadlock situation which occured because of this heavy load on the microservice (Say A) causing multiple requests from different client services (B,C). So these calls from B and C come for the same clientId(key) and are served by different instances of A and they try to update the same clientId data in database at same time causing below error.
CannotAcquireLockException is thrown,
(SQL Error: 60, SQLState: 61000..
ORA-00060: deadlock detected while waiting for resource
We have decided to implement sharding at load balancer(haproxy) level which will ensure same instance of A will always serve the requests from B and C for a specific key(clientId), so we dont have multiple instances processing the request for same key(clientId).
Now we get into the mode of everything in single jvm as we have made sure requests from B and C for a specific clientId always come to same instance of A.
With this its still possible that requests from B and C services come for same clientId with difference in time of nanoseconds. Any then multiple threads will again try to update the same clientId data in database at same time causing same error again.
To improve this we are looking for possible solutions and one solutions is ReentrantReadWriteLock which should take care of this based on the concepts.
We are using spring data jpa and have a save being done which looks like
clientJpaRepository.save(ClientObject);
Now is it possible to use something like below.
public void save(Client clientObject) {
String clientId = clientObject.getClientId();
try {
boolean isLockAcquired = writeLock.tryLock(100, TimeUnit.MILLISECONDS);
if (isLockAcquired) {
clientJpaRepository.save(clientObject);
}
} catch (InterruptedException e) {
log.error("exception occured trying to acquire lock for clientId={}", clientId);
} finally {
writeLock.unlock();
}
}
I am not very sure how its going to deal with the keys. As in i don't want any threads to block if they are wanting to update for different key(clientId 2).
Also, other thing to note is there could be reads happening as part of other API calls for this data from database. They would not be waiting too long hopefully and i hope i don't need to make any changes there for the reads.
Sorry for the long question, Hope i will hear from someone soon.
Thanks.
I am looking for an efficient way to find whether a given application (say app.exe) is single instance or not? I thought of these following sols:
Do CreateProcess() twice and check whether there are two or more instance running of that application? If no, it is single instance application. But, this is not efficient.
Do CreateProcess() and wait for 1-2 sec. If this instance is killed (because there is already an instance running for it), it will be single instance app.
But I am not convinced with both above sol. Is there any other efficient way of doing that in windows?
Please note that I don't to kill or make any modifications to an already running (if any) instance of that application.
Think about it the other way: When you write a program, how do you specify whether it is single-instance or multiple-instance? Is there a way that some other program can get that information out of your program without running it? (Once you answer this question, then you have the answer to your question.)
This problem is not solvable in general because single-instance/multiple-instance-ness is determined at runtime and can be based on runtime conditions. For example, some applications are "sometimes multiple instance, sometimes single": If you run the application to open document X, and then document Y, you will get two instances. But if you open document X, and then document X again, the two instances will fold into one. Other applications may have a configuration switch that lets you select whether they are single-instance or multiple-instance. Or maybe they decide to flip a coin and decide to be single-instance if tails and multiple-instance if heads.
The best way is via using synchronization object called Mutex (Mutually exclusive). You may google it.
I think the following code may help to.
//---------------------------------------------------------------------------
WINAPI _tWinMain(HINSTANCE, HINSTANCE, LPTSTR, int)
{
try
{
HANDLE hMutex=OpenMutex(MUTEX_ALL_ACCESS,0,"SIns");
if (!hMutex) {
//Mutex doesn’t exist. This is the first instance so create the mutex.
//in this case app name is SIns (Single Instance)
hMutex=CreateMutex(0,0,"SIns");
Application->Initialize();
Application->MainFormOnTaskBar = true;
Application->CreateForm(__classid(TfMain), &fMain);
Application->Run();
ReleaseMutex(hMutex);
}
else{
//This is not single. The prev instance is already running
//so informing about it
//remember that if it finds prev instance we're activating it here
//you may do whatsoever here ...... e.g. you may kill process or stuff like this:)
ShowMessage("The program is already running. Switching to ...");
HWND hWnd=FindWindow(0,"SIns");
SetForegroundWindow(hWnd);
}
}
catch (Exception &exception)
{
Application->ShowException(&exception);
}
catch (...)
{
try
{
throw Exception("");
}
catch (Exception &exception)
{
Application->ShowException(&exception);
}
}
return 0;
}
//---------------------------------------------------------------------------
There is no way to do this at all. What happens if the application checks a mutex then makes a messagebox to tell the user an instance is already running and only when the user dismisses it does it kill the application? There are many different ways to ensure mutual exclusion via some shared resource, mutex, shared file, even maybe setting some registry key, the methods are unlimited.
The usual solution is to use some sort of a locking file. Under
traditional Unix, for example, the application will start by creating a
file (which will succeed even if the file exists), then try to create a
link to it (an atomic action); if that fails, the application will
immediately kill itself. Under Windows, the share mode of CreateFile
can be used to the same effect: open a file with share mode 0, and if
that fails, quit. (The Unix solution will leave the lock if the process
crashes, requiring it to be cleaned up manually. The Windows solution
will remove the lock if the system crashes.)
you may use mutexes... I do such check with following code:
bool insureApplicationUniqueness(HANDLE& mutexHandle)
{
mutexHandle=CreateMutexW(NULL,true,UNIQUE_INSTANCE_MUTEX_NAME);
if( mutexHandle&&(ERROR_ALREADY_EXISTS==GetLastError()))
{
CloseHandle(mutexHandle);
return false;
}
return true;
}
but this is for application which source code is yours and which checks is another instance of itself running.
The problem with the notion is that in common environments, there is no explicit static data that determines whether an application is single-instance. You only have behavior to go on, but you cannot fully test behavior.
What if you have an app that is multi-instance, but will fail to open a file that's already open? If you test it twice with the same, valid filename, it would create only a single process, but any other command line argument would cause two processes to exist. Is this a single-instance program?
You could even argue that "single instance" isn't a well-defined catageory of programs for this reason.
I'm trying to do some testing and it requires the Windows system to be up and running for 15 Real-Time minutes before a certain action can ever occur. However, this is very time consuming to HAVE to wait the 15 real-time minutes.
Is there a way to change the value GetTickCount() returns so as to make it appear that the system has been running for 15 real-time minutes?
Edit: There is an app that does something close to what I want, but it doesn't quite seem to work and I have to deal with hexadecimal values instead of straight decimal values: http://ysgyfarnog.co.uk/utilities/AdjustTickCount/
Not directly.
Why not just mock the call, or replace the chunk of code that does the time check with a strategy object?
struct Waiter
{
virtual void Wait() = 0;
virtual ~Waiter() {};
};
struct 15MinWaiter : public Waiter
{
virtual void Wait()
{
//Do something that waits for 15 mins
}
};
struct NothingWaiter : public Waiter
{
virtual void Wait()
{
//Nill
}
};
You could do similar to mock out a call to GetTickCount, but doing this at the higher level of abstraction of whatever is doing the wait is probably better.
For debugging purposes, you can just replace all the calls to GetTickCount() with _GetTickCount(), which can implement to return with GetTickCount() or GetTickCount()+15min, depending whether or not you are debugging.
Why not make it one minute, confirm it works, then change it back to fifteen?
You could do something quite hideous like #define GetTickCount() MyReallyEvilReplacement().
You can use the Application Verifier provided with the Windows SDK to run your app with the "Miscellaneous > TimeRollOver" test. It will fake a tick count which starts at a time that will overflow after a short moment.
Another possibility is to to hibernate / hybrid shutdown / sleep a Windows system, then boot to the BIOS, change the date time to something you require, like add 30 days if you want to test unsigned tick counts. When Windows boots again, it has no way of detecting the appropiate time since the computer really started previously, and thinks it is running for 30 more days. It is important to use sleep / hibernate / hybrid shutdown (the latter being the default since Windows 8), not a full shutdown, as the up time is otherwise reset.
Yet another possibility could be to hook imports of GetTickCount to your own code and let it return arbitrary results.
I am writing a framework for an embedded device which has the ability to run multiple applications. When switching between apps how can I ensure that the state of my current application is cleaned up correctly? For example, say I am running through an intensive loop in one application and a request is made to run a second app while that loop has not yet finished. I cannot delete the object containing the loop until the loop has finished, yet I am unsure how to ensure the looping object is in a state ready to be deleted. Do I need some kind of polling mechanism or event callback which notifies me when it has completed?
Thanks.
Usually if you need to do this type of thing you'll have an OS/RTOS that can handle the multiple tasks (even if the OS is a simple homebrew type thing).
If you don't already have an RTOS, you may want to look into one (there are hundreds available) or look into incorporating something simple like protothreads: http://www.sics.se/~adam/pt/
So you have two threads: one running the kernel and one running the app? You will need to make a function in your kernel say ReadyToYield() that the application can call when it's happy for you to close it down. ReadyToYield() would flag the kernel thread to give it the good news and then sit and wait until the kernel thread decides what to do. It might look something like this:
volatile bool appWaitingOnKernel = false;
volatile bool continueWaitingForKernel;
On the app thread call:
void ReadyToYield(void)
{
continueWaitingForKernel = true;
appWaitingOnKernel = true;
while(continueWaitingForKernel == true);
}
On the kernel thread call:
void CheckForWaitingApp(void)
{
if(appWaitingOnKernel == true)
{
appWaitingOnKernel = false;
if(needToDeleteApp)
DeleteApp();
else
continueWaitingForKernel = false;
}
}
Obviously, the actual implementation here depends on the underlying O/S but this is the gist.
John.
(1) You need to write thread-safe code. This is not specific to embedded systems.
(2) You need to save state away when you do a context switch.