Today I ported my old memory benchmark
from Borland C++ builder 5.0 to BDS2006 Turbo C++ and found out weird thing.
exe from BCB5 runs OK and stable
exe from BDS2006 measure OK only before main Form is started (inside its constructor) and if the benchmark is started again after main form is Activated or even after any VCL component change (for example Caption of main form) then the speed of benchmark thread is strongly affected.
After some research I found out that:
Does not mater if test is inside thread or not.
The process/thread priority,affinity does not affect this either.
Hide of any window (Visibility,Enabled) does not affect this.
call the test form OnIdleEvent does not affect this
does not mater if time is measured by RDTSC or PerformanceCounter
My conclusion is that VCL library runs some code/thread on the background so my questions are:
Is there a way to temporarily pause VCL code/stuff ?
ideal something like Application->Pause(); and Application->Resume(); or just Forms.
what else could cause this behavior and how to avoid it ?
PS.
Test application has no VCL components other than main form. Benchmark is just a few memory transfers by rep stosd with different block sizes (no funny stuff). Source is in this related Q/A. I know BDS2006 is out-dated but I am not looking for upgrade right now so please skip any comments about that they are not help-full at all.
Tested on Windows7 pro x64, 32bit Application
I found out that wndproc in BDS2006::VCL invalidates CACHEs.
I have tried to override wndproc by winapi
for Application->Handle is this easy but it does not stop the processing of messages for Form. When I tried Form1->Handle as window then error 1400 occurs (not valid window handle)
I have tried to override wndproc by VCL
for Application by TApplication events and for Form by override of virtual wndproc member. Message processing stops but their calling sequences remains and the problem is not solved either.
So my conclusion after eliminating every possibility I can think off is that I need to flush CACHE more intensively somehow after setting process/thread for benchmarking.
In DOS I would done it by single instruction but on windows it is more tricky. Well The previous version of memory benchmark used just memory filling which is obviously not enough for BDS2006 exe. I think that instruction CACHE is involved in this problem not data cache so I change it a bit and it finally worked thing out.
Flushing the CPU CACHE:
for (DWORD i=0;i<(128<<20);i+=7)
{
dat[i]+=i;
dat[i]*=i;
dat[i]&=i;
}
Where dat is 128MB allocated memory chunk (or bigger) and must be done after all process/thread priority and affinity changes or all winapi calls prior to benchmarking.
Related
I have been working for 2.5 years on a personal flight sim project on my leisure time, written in C++ and using Opengl on a windows 7 PC.
I recently had to move to windows 10. Hardware is exactly the same. I reinstalled Code::blocks.
It turns out that on first launch of my project after the system start, performance is OK, similar to what I used to see with windows 7. But, the second, third, and all next launches give me lower performance, with significant less fluidity in frame rate compared to the first run, detectable by eye. This never happened with windows 7.
Any time I start my system, first run is fast, next ones are slower.
I had a look at the task manager while doing some runs. The first run is handled by one of the 4 cores of my CPU (iCore5-6500) at approximately 85%. For the next runs, the load is spread accross the 4 cores. During those slower runs on 4 cores, I tried to modify the affinity and direct my program to only one core without significant improvement in performance. The selected core was working at full load, though.
My C++ code doesn't explicitly use any thread function at this stage. From my modest programmer's point of view, there is only one main thread run in the main(). On the task manager, I can see that some 10 to 14 threads are alive when my program runs. I guess (wrongly?) that they are implicitly created by the use of joysticks, track ir or other communication task with GPU...
Could it come from memory not being correctly freed when my program stops? I thought windows would free it properly, even if I forgot some 'delete' after using 'new'.
Has anyone encountered a similar situation? Any explanation coming to your minds?
Any suggestion to better understand these facts? Obviously, my ultimate goal is to have a consistent performance level whatever the number of launches.
trying to upload screenshots of second run as viewed by task manager:
trying to upload screenshots of first run as viewed by task manager:
Well I got a problems when switching to win10 for clients at my work too here few I encountered all because Windows10 has changed up scheduling of processes creating a lot of issues like:
older windowses blockless thread synchronizations techniques not working anymore
well placed Sleep() helps sometimes. Btw. similar problems was encountered when switching from w2k to wxp.
huge slowdowns and frequent freezes for few seconds on older single threaded apps
usually setting affinity to single core solves this. You can do this also in task manager just to check and if helps then you can do this in code too. Here an example on how to do it with winapi:
Cache size estimation on your system?
messed up drivers timings causing zombies processes even total freeze and or BSOD
I deal with USB in my work and its a nightmare sometimes on win10. On top of all this Win10 tends to enforce wrong drivers on devices (like gfx cards, custom USB systems etc ...)
auto freeze close app if it does not respond the wndproc in time
In Windows10 the timeout is much much smaller than in older versions. If the case You can try running in compatibility mode (set in icon properties on desktop) for older windows (however does not work for #1,#2), or change the apps code to speed up response. For example in VCL you can call Process Messages from inside of blocking code to remedy this... Or you can use threads for the heavy lifting ... just be careful with rendering and using winapi as accessing some winapi (any window/visual related stuff) functions from outside main thread causes havoc ...
On top of all this old IDEs (especially for MCUs) don't work properly anymore and new ones are usually much worse (or unusable because of lack of functionality that was present in older versions) to work with so I stayed faith full to Windows7 for developer purposes.
If none of above helps then try to log the times some of your task did need ... it might show you which part of code is the problem. I usually do this using timing graph like this:
both x,y axises are time and each task has its own color and row in graph. the graph is scrolling in time (to the left side in my case) and has changeable time scale. The numbers are showing actual and max (or sliding avg) value ...
This way I can see if some task is not taking too much time or even overlaps its next execution, peaks are also nicely visible and all runs during runtime without any debug tools which might change the behavior of execution.
We have a bigger software running on Win CE6 without problems. The core functionality is implemented in a COM server DLL that provides connection points. The COM client program registers event handlers for the connection points on program startup to get status notifications etc. On program exit it unregisters the handlers by calling the corresponding IConnectionPointImpl::Unadvise methods.
Now, we are porting the program to run on Win EC 7. The new Board Support Package (BSP) for Win EC 7 works well. There are also different versions with different options, created at different times with different sources from Microsoft, but our software always show the same issue.
On program startup, ~10s after launch, IConnectionPointImpl::Unadvise is called unexpectedly on all registered event handlers. We only have one method in our source code that calls IConnectionPointImpl::Unadvise and this is definitely not executed.
The issue appears ~95%, but sometimes the program starts and runs without problems. We cannot use the Debugger because of the size of the program, the performance is very poor.
We guess, that the COM runtime calls the IConnectionPointImpl::Unadvise methods for some reasons. But we have no idea, how to prevent this.
Has anybody observed the same issue? Is there a solution/workaround available? Thanks.
So we finally found how solve this problem.
We remove our dependency on MarshalByReObject and replace it by a proper implementation of ISerializable.
That allow us to load properly inside custom AppDomain our assembly and events are not loose anymore.
But this has a side effect on path where assembly a configuration file are loaded. To solve this we also implement an AppDomain.AssemblyResolve event which allow us to redirect the loading in a proper place.
I hope this can help you ;)
I have used the File New C++ app Wizard in Visual Studio 2015 to create a simple 'Hello World' Window app. When linked with the default stack size, it launches instantly.
When linked with a stack reserve size of 1073741824, the app hangs for tens of minutes before becoming responsive again.
Increasing the stack size gradually from the default will also increase the delay gradually.
In another executable I have used EditBin.exe to change the stack from the 1mb default to the 1gb value above and it also hangs in the same way and becomes responsive later on. Changing it back causes everything to run smoothly again.
I think I have tracked it down to the first WIN32 API call. It does not seem to be the particular call itself but the fact that a new DLL is being loaded and inheriting the stack from the executable. CPU and memory stay idle during this operation. I think it is compounded by the exe being 32 bit and running on 64 bit.
This is a repro for a large application which uses a very big stack so I'd like to figure out what could be happening in general when that first API call is executed. Is memory being zeroed out?
When debugging in Visual Studio, the hang happens here but I think this is misleading.
// Main message loop:
while (GetMessage(&msg, nullptr, 0, 0))
{
Update:
Stack size: Not my code, but the original is a Fortran app which does a lot of calcs for particle physics and that size was set originally. I can't change it without understanding what is going on first.
Release mode: All compiled for release mode, 32 bit.
Memory usage: I don't actually see any memory assigned or CPU activity though. From what I understand, the stack reserve is just a reserve to call upon it's not actively assigned at app start. The app uses the same memory at both stack levels on startup.
Single threaded: Just the example app out of File New.
Loading DLL: In debugging the original Fortran code (of which this is a repro) the issue happened when the DLL was called. Checked with a WinHelp call and a Window Create call. I assumed the DLL was being loaded in process at this point.
Update2:
I understand I need to fix the code, this is part of investigating it. It also happens with this C++ code. What I am after specifically, is what is happening when the WIN API is called and the UI becomes unresponsive? That might provide a clue and a workaround until the 40-50 year old Fortran code is re-worked and upgraded to 64 bit.
I've come across an inexplicable error in SDL 2.0.3 when using hardware rendered graphics. For some reason, around 5 minutes after the program starts my graphical window closes but my console window stays open. There is no error thrown or anything to signify a problem.
When I pause the debugger, the program puts the breakpoint inside of SDL_RenderPresent(). I followed the call stack to a function inside of ntdll.dll called WaitForSingleObject() but I'm not sure what's causing it to hang forever.
Also, this does not happen when I use software rendered graphics. I am running it on an AMD FirePro M5100 FireGL V with the latest drivers installed.
My question is, does anyone know what might cause SDL_RenderPresent() to never return?
From the description seems that there are locks not released by the lower levels of the graphics pipeline.
From the fact that it happens after 5 minutes seems that there is a resource leak somewhere.
All of this is just a wild guess of course, but I'd say that either the application code or the SDL code is leaking resources (handles to textures, vertex buffers and the like) and that part of the code (either at the lower levels of SDL or in the driver) when running out is not behaving nicely (this happens often... in many cases low-resource conditions are not very well tested and handled).
This doesn't happen in software rendering because there resources are basically unlimited. A confirm of this kind of problem would be that when running in software rendering the program works but process memory use keeps growing and growing.
Pay attention also to any code that "catches" any exception/failure and keeps running after that. Writing complex software that works correctly after an abnormal state is extremely difficult (basically impossible beyond trivial cases because exception safety doesn't scale by composition: the only way that doesn't make the complexity explode is to have logical partitioning "walls" and re-initialize whole subsystems).
I have written an app in C++ from Visual Studio 2008, running on Windows 7, that runs fine, using either the debug or release versions, when under the control of the debugger, but when running standalone, using either the debug or release versions, it also runs fine except whenever I click on any unrelated window, say a file explorer window, whereupon the app hangs without any warnings from Windows, I see the little circle thing.
The code is doing something rather computationally intensive, accessing data from a 10Mb global array, and it is well within the 2Gb limit of 32 bit Windows. I have checked for the obvious things, such as uninitialized variables, infinite loops, and the like, I am not allocating any big local arrays, but have found nothing wrong. The code is running directly from the UI thread, blocking, but I do not care as there is nothing else to do till that task completes. Alternatively, I could put this code in its own worker thread communicating back to the UI thread by an interlocked buffer, but this seems redundant. I've tried this on two different machines running Windows 7 and get identical behavior. Is there something about Windows process management that I am overlooking? Is there a way to tell whether there is some sort of memory corruption going on that could cause some other process to affect the app's process?
[Edit1 by spektre] just copied user3481340's code from comment to be readable
I do not think that the computational time, which is about an hour
has anything to do with the problem.
Rather, the windows messaging for the edit box is getting messed-up.
The relevant code is:
int textlen=GetWindowTextLength(Editwin);
int k=strcspn(messagebuf,"\n");
if(k<strlen(messagebuf))textlength=strlen(messagebuf)-k;
else textlength+=k;
SendMessage (Editwin, EM_SETSEL, (WPARAM)textlen, (LPARAM)textlen);
SendMessage (Editwin, EM_REPLACESEL, 0, (LPARAM) messagebuf);
Somehow Windows stops responding to these messages.
Windows7 has some changes (from previous versions of windows)
in process scheduling
which can mess up lock-less multi-thread apps which are working 100% thread safe on older versions of windows. There is not much to do with this (unless add some safety sleeps or locks)
critical sections
I am not 100% convinced that critical sections work properly on Win7SP1 as I have some issues on specific machine setup where they do not work with combination of:
heavy duty USB HS bulk data transfers
OpenGL use
multi-threading with heavy multiple critical sections use
but it could be a hidden bug or messed-up Windows7 installation or even HW error
WindowProc
if your message loop get stuck for a longer time then it should then it will cause the program to be detected as non-responding even if it is not. Usually run in compatibility for XPSP3 helps but sadly not on all machines or all apps :( this is the main reason why many games does not run on Win7
I suspect this is your case and according to your last comment I was right.
Transfer critical processing to some thread instead so the WindowProc is not blocked. If you need an event after computation is done then flag some variable as done and scan for it in some timer so you can respond inside main thread (to avoid winapi calls outside main thread).
32bit Driver+App use on x64 windows (WOW64)
if your app is accessing any driver then on WOW64 you need special driver build to access real hardware instead of emulated by WOW64 !!! If you did not then your App is most likely waiting for real device respond instead is getting emulated one which can be different and cause real hang ups. In that case you need compile your app as x64
Or use some kind of WOW64.x86 -> x64 bridge
Or use driver which can handle it itself (usually link do different dll)