I have a strange CLR OutOfMemoryException in a 64bit C++/CLI process which has mixed native and managed C++ code inside. The process has the following structure:
Native C++ code (Main binary)
Managed C++/CLI code (Main binary)
C# code (External assembly)
Both main binary and external assembly are built for x64 platform. The managed C++/CLI layer (2) serves as an adapter between native C++ code (1) and C# code (3) which is actually a UI layer working with WPF. .NET Framework version is 4.0.
So I have the Main Binary (1 and 2) built with /MACHINE:X64 and /CLR which makes native C++ code (1) to work perfectly with >4Gb of memory consumed by the process, but C++/CLI layer code (2) throws an OutOfMemoryException once I step close to 4Gb. So far it happens in relatively random places but every time I saw it was in Managed-to-Native transitions when C++/CLI code tried to access a native method.
It seems like C++/CLI code inside Main binary doesn't feel itself as a x64-ready code, while the tools tell that the binary is in PE+ 64bit ready format and the native code works well with >4Gb. According to ANTS memory profiler and CLRProfiler the managed heaps are almost empty (several hundred of kb).
What could I miss? Doesn't it look like I'm linking to wrong .NET framework assemblies which are not x64-ready? Where should I look first?
Thanks in advance for any help!
Related
This might be a very specific question. But it might be interesting for some other people, too.
The setting might be quite common: we develop an embedded device using C++ and Linux and have a build system for this. With our device we ship a C# desktop app using a DLL based plugin mechanism. The plugin contains the shared logic used both on the embedded device and the app. To keep processes easy it would be best to create the DLL on our Linux build system (hence I would like to use .Net Core and not "normal" .Net).
As far as my trial and error showed it's not possible - even with .Net Core 3.1. Can you confirm this? The software is for windows only (although it's compiled on Linux) - so theoretically it could possible since .Net Core supports CLI/C++ in Version 3.1 (https://devblogs.microsoft.com/dotnet/announcing-net-core-3-1/).
What I tried:
I compiled pure C# code on Linux to an exe running successfully on windows using the console app dotnet
I successfully created a simple C++/CLI test app on Windows using sample code following this guide: https://devblogs.microsoft.com/cppblog/an-update-on-cpp-cli-and-dotnet-core/
Finally I took the (probably for too naive) approach to replace the test app's .vcxproj with the .csproj of step 1 (.csproj seems to detect files automatically). I got the following error message:
quote CSC : error CS5001: Program does not contain a static 'Main' method suitable for an entry point [/home/vagrant/Projects/cpp_dotnet_on_linux/cs_on_linux.csproj]
Is there something I can improve or is the task just impossible (adding the cpp-files manually to .csproj, ...)? I'm afraid it's just technically impossible because I guess .Net Core is missing a C++ compiler on Linux but I'm not sure.
C++/CLI is not support on Linux.
Some sources:
https://devblogs.microsoft.com/cppblog/the-future-of-cpp-cli-and-dotnet-core-3/ :
We don’t currently have plans for C++/CLI for targeting macOS or Linux. Additionally, compiling with “/clr:pure” and “/clr:safe” won’t be supported for .NET Core.
https://github.com/dotnet/coreclr/issues/659#issuecomment-539742740
C++/CLI on Linux will not be supported and would be very challenging. It would require Microsoft VC++ to support Linux or require Clang or GCC to support C++/CLI. Those are both huge projects with uncertain payoff. It is also unclear if Clang or GCC would ever allow us to upstream our changes. Maintaining an up-to-date fork of a fast moving compiler project forever is very expensive. We have no plans to take on either of those projects.
Setup:
In a C# program i call functions from a .NET C++/CLI dll.
This dll has static linking with a native C++ library. Visual Studio 2017 is the envoirement.
Settings:
Both projects have Debug Information Format "Program Database (/Zi)" set.
The dll project has Generate Debug info "Generate Debug Information (/DEBUG)" set.
Also i have Profile "Yes (/PROFILE)" set for the dll. Not sure what it does.
The .pdb file is generated, placed in the folder of the binary and loaded when starting the program with the Visual Studio 2017 Perfomance Profiler attached.
I see some toplevel calls in the dll, but it seems incomplete and
doesn't reach native C++ code.
I can step into the native C++ code
with debugger.
The debugger tool "Very Sleepy" gives the correct
output.
A colleague said a spectre fix combined with the fact im using Win7 introduced the problem. (https://superuser.com/questions/1287914/visual-studio-2013-profiler-chrashes-and-system-restarts-reboots)
I'm not sure my problem is related to this...? Have i forgotten some setting?
"Sampling" mode profiling does not work in Win7 after the spectre fix (which you discovered in your link). I happen to have a win10 machine with vs2017 on which I am developing a C++/CLI dll (with large portions in native code). On the win10 machine, when I use CPU sampling (starting a C# exe that uses my C++/CLI dll), I get sampling data for the exe and, somehow, for low level calls (like malloc) but not for any of my native code. I am confident this used to work (that is, I was able to get sampling data for my native code). I can step into my native code debugging with a release mode build.
I, too, wonder if this is a known problem or if I'm just missing a setting somewhere. I will continue to try to get this working or get an explanation and, if I found one, I'll post back here.
I know that all assemblies can be decompiled somehow, but C# & VB applications are the easiest to be decompiled into source code using tools like ( .Net Reflector ).
So my question is, if I programmed an application using .Net assemblies and functions with C++, would it be easy to decompile it as if it was a C# or VB application with .Net reflector and such tools?
Ok, if I programmed it without using any function from .Net framework and made UI only what calls .Net assemblies, would be easy to decmpile also ?
My question is similar to this one : Could this C++ project be decompiled with such tools like a .NET Reflector?
but no one has answered him correctly, so can anyone help me ?
I want to use .Net and C++ to make my application compiled into both Native & Managed code!
There is no "C++.Net". There is C++/CLI, which is a C++-like language that can be used to glue native C++ code with the .NET world. The managed code you write in it (ref classes) will be compiled to MSIL. The "native" code will compile to either MSIL or native. If you want to compile some parts to native code you need
#pragma managed(push, off)
void foo() {}
#pragma managed(pop)
in your source. The managed pragma can be used to choose the compilation target per-function. Or you can compile without the /clr flag per-module and set your project to produce a mixed-mode assembly.
Be aware that marshaling the native types to .NET and back can take a serious performance hit on your application - and that happens every time you cross the native-managed boundary. But interoperation between such embedded native code and managed code is much faster than normal p/invoke.
See also this question: C++CLI. Are native parts written in pure C++ but compiled in CLI as fast as pure native C++?
I have a project that requires writing a code for small executable file. I used visual C++ express 2010 IDE to create this file. After I finished writing the code, I tried to copy it to a couple of different PCs. It gives me an error message every time I clicking on this file to execute it. The message states that I have to install (.NET framework). I watched a couple videos on YouTube explaining how overcome this problem by changing the runtime library from multi-threaded Debug DLL (/MDd) to multi-threaded Debug (/Mtd). However; the IDE can’t debug the C++ code because when I create my project by using CLR template!
Is there any way to solve this problem? Can I create a similar program that not requires any further downloading once I using on different PC?
Is learning a different language like JAVA or C# will help creating small programs (like my program) that run on most Window platform machine?
Just use Qt - it runs on Windows, Linux, MacOS, support for Android and iOS is scheduled for this year, plus it supports embedded platforms and some of the more obscure mobile platforms. Also, support for Windows RT was just kickstarted. A complete library with tons of functionality, good documentation and lots of educational resources. It provides tons of tools, from implicitly shared containers through threading, signals and slots, 2D and 3D graphics, widgets, multimedia, sensors... and whatnot...
You can even develop commercial applications under the LGPL license.
Also comes with a pretty good IDE - Qt Creator.
You can develop standard C++ applications or use QML, which is a JavaScript like language for markup and scripting, which is used to build applications from C++ implemented components. You can also extend QML. It is much faster to develop with QML and you still get the advantages of platform native binary under the hood.
Note that you will still need to either ship a few DLLs with your application. Unless of course you use a static build, which requires you to either have your application open source, or purchase a commercial license... which doesn't come cheap...
But still, a few MB of DLLs are far better than the entire .NET framework. A static build will produce executables about 8-9 MB with no external dependencies.
Stick with the C++ standard, avoid Microsoft extensions (managed code), and call only POSIX functions of your OS, then you should be able to write portable programs.
You seem to have created a Managed C++ Project. Instead create an empty Win32 C++ project and then add in your .cpp/.h files. This will limit you to the default libraries available on all PCs with the C++ runtime. If you want to remove that dependency too then statically compile in the runtime using the /MT option. Details # http://msdn.microsoft.com/en-us/library/2kzt1wy3(v=vs.71).aspx
As you move ahead you would need to be conscious of what libraries you take dependencies on and what versions of the OS are those libraries available on or if you need to package them with your bits.
Both Java and C# will help making portable programs. Usually, people will have to install runtime environments for executables written in these two languages, however. These days, C++ is more portable than ever. You can easily run C++ executables in your browser:
https://github.com/kripken/emscripten
http://code.google.com/p/nativeclient/
This makes many of the reasons why Java and C# came about irrelevant.
Open standards like OpenGL also make portable GUI programming easier than ever. Try Qt, if you want to write a simple GUI in C++.
Note: It is possible to run C++ program in any computer without installing anything if you haven't use .NET framework. In your case, there can be two reasons to trigger error in target computer.
New computer doesn't have required run-time assembles.
New computer doesn't have required .NET framework installed.
..........................................
So what to do:
Before start your program you have to design weather are you going to use .NET framework support or not. If you use .NET framework when you develop your program, then you much install same or higher .NET framework in target computer.
If you no need to use .NET component then your target computer should only containing run-time assemblies.
How to get rid of .net framework
right click on the project in solution -> properties -> General -> Common language run time support -> select "No common language run time support".
..........................................
Then what you need is only relevant run-time assemblies be in target computer.
How can run-time assemblies be in new computer:
There are two ways:
Install suitable C++ disputable environment in target computer(if you use VS2008 SP1, C++ RD package should be this. Please consider the solution build architecture also (32 bit/64 bit) before download ).
Deploy run-time assemblies with your solution package. (I like this because user no need to install any third party components)
..........................................
How Deploy assemblies with my project:
for this your all DLL, LIB, EXE should use same run time version.(if not, you face troubling to redirect assemblies by 'manifest' files ).
How to check the run-time version.
open DLL,EXE by visual studio (open->file) -> expand RT_MANIFEST-> double click the file under it ->then assembly dependency details will open. -> copy the data in right column and paste to note pad.
You will see this kind of line there. and ther is the version run-time assemblies your specific DLL or EXE use.
assemblyIdentity type="win32" name="Microsoft.VC90.CRT" version="9.0.21022.8" processorArchitecture="x86".....
After identifying the version of run-time assemblies follow this tutorial and try to run in fresh installed computer.
At last: If you think this bla.. bla.. is so complex and your program is very simple, then you can consider about "run time assemblies statically linking" (try Google). But personally I don't like this method.
good luck!
Native exe!
that means my program can run easily without any requirement?
even if I use .net classes?
You know I want to write a program that is so light and I don't like to use C# or any other .net programing language because all of them need .net-framework 4.5.
Just think a 2.5 MB programm needs a +250 MB .netframework.
New Update - 12/01/2016:
It's almost 4 years ago when I asked this question. As you know Dotnet Native is announced. It's an interesting feature which compile IL into native code.
Compiling Apps with .NET Native
.NET Native is a precompilation technology for building and deploying
Windows apps that is included with Visual Studio 2015. It
automatically compiles the release version of apps that are written in
managed code (C# or Visual Basic) and that target the .NET Framework
and Windows 10 to native code. Typically, apps that target the .NET
Framework are compiled to intermediate language (IL). At run time, the
just-in-time (JIT) compiler translates the IL to native code. In
contrast, .NET Native compiles Windows apps directly to native code.
For developers, this means:
Your apps will provide the superior performance of native code.
You can continue to program in C# or Visual Basic.
You can continue to take advantage of the resources provided by the .NET Framework, including its class library, automatic memory
management and garbage collection, and exception handling.
Last I checked none of the .NET frameworks were 250+ MB! Yes, the offline installer for .NET Framework 3.5 SP1 is 231MB but it contains x86 and x64 versions of .NET 2, 3 and 3.5 sp1.
You should read this http://www.smallestdotnet.com for details on sizes of various versions of the installers.
Now on to your question:
Yes, It is a little annoying to have your clients install a big framework, even 20-40MB does get annoying. With .NET, the advantage is the ease of programming (In my opionion) compared to other Native options.
Your native options are:
MFC - You need only the VS runtimes installed, which is 1-2MB and is usually installed on newer pcs. Also, you can ship your application with the MFC libraries packaged into a dll which is again <2MB
The trade of here is you need to program in C++, the libraries overall are a very thin layer over the native libraries. and people have had harsh opinions about MFC. I've barely just tried it.
Win32 API - This is going all bare bones, and quite difficult, you could use C or C++ but you'd really have to know a lot about the Win32 API and how windows itself works (Stuff like windows messages, hwnds etc) Its not fun, believe me. But during deployment you would not need any external libraries.
There are tons more options, see here:
Native Windows Application Development Options
https://stackoverflow.com/questions/2711599/what-programming-language-should-i-use-to-create-small-native-windows-applicatio
Here are some links on MFC that might help:
Want to learn Windows Programming,some suggestions?
How do I decide whether to use ATL, MFC, Win32 or CLR for a new C++ project?
C++ MFC vs .NET?
https://stackoverflow.com/questions/557114/stick-with-mfc-or-go-to-net
you are able to create native exe by using c++ Win 32 projects.
Alas, nearly everything requires a download runtime library and even if you have one installed, you'll need to download updates for them almost continually. Even Microsoft C++ apps nowadays come with security updates that have to be installed if you've compiled your app with them.
But.. there is a solution of sorts. If you use C++, it has a feature where only the things you use are compiled into the final app. Normally, this would require linking with all the library dlls, but if you statically link with the library, you will end up with a single .exe that is as small as can be, and you will not need any dlls (as all the code contained in the library will be compiled into the .exe).
The benefits are debatable compared to dlls, but as MS has pretty much broken the idea of shared dlls in .NET (ie, you practically have to put all the shared dlls in the same directory as your running app, giving you a nightmare in maintenance if you have these shared dlls spread around all your apps) then there's not much of a difference anymore. Static linking is getting a little bit of a comeback and sounds like its what you want.
For modern C++ development, you'll probably want to take a look at Qt instead of MFC. Its a lot nicer to use and is cross-platform so you can run Qt apps on your Android or Linux platforms as well as Windows.