I have a Fortran code which I try to wrap in a Java/Scala code. My issue is that the state of a variable in fortran is kept, even if I dispose the library between two calls:
Fortran-Code:
subroutine mySub()
implicit none
DOUBLE PRECISION x
COMMON/myCommon/ x
print*,x
x = 99.99
end subroutine mySub
And Java/Scala Code:
trait FortranLibrary extends Library {
// native symbols
def mysub_()
}
def main(args: Array[String]): Unit = {
var INSTANCE: FortranLibrary = Native.synchronizedLibrary(
Native.load("sub.so", classOf[FortranLibrary])
).asInstanceOf[FortranLibrary]
// call native subroutine
INSTANCE.mysub_()
println("------SECOND CALL-----")
// clean library, reload
INSTANCE = null
System.gc()
// make new instance
INSTANCE = Native.synchronizedLibrary(
Native.load(libpath, classOf[FortranLibrary])
).asInstanceOf[FortranLibrary]
// call native subroutine
INSTANCE.mysub_()
}
Prints to the console:
0.000000000000000E+000
------SECOND CALL-----
99.9899978637695
So the previously set x=99.99 is still present in the second call even though the library is disposed as proposed in How to dispose library loaded with JNA , how can avoid this?
EDIT: I'm using intel fortran compiler with -init:zero, so variables should be re-initialized with 0
Under the hood, JNA keeps a reference to a NativeLibrary object once you've loaded it, and does not release it unless dispose() is explicitly called.
The dispose() method is included in the object's finalize() which would be called upon garbage collection, however, this cannot be relied upon to always execute. In the code snippet you've posted, you null the reference and use a single System.gc() call, which is only a suggestion to release the object. A better option, which you've indicated in the comments that you've tried, is to call dispose() or disposeAll() yourself.
However, as also noted in my answer to your linked question, a small time delay is needed to ensure the native library doesn't return the same handle if it's immediately reloaded. JNA source code notes at least a 2ms delay works on OSX, and in your comments it seems you've been successful on your OS with a 10ms delay.
Related
I broke my Visual Studio project (2015, C++) into three pieces:
The main application in a static library
The executable which just has a main function
Unit tests which use the static .lib file so that they can import the required classes and do the unit tests on them.
Before I split it into a lib/exe/tests, the main application was just a standalone executable, and it worked perfectly fine.
Now I can't run the executable with only the main function, nor the unit tests as a certain pointer is always null. The only main difference is that I'm using a raw pointer in this example, but I was using a unique_ptr in my code (currently however I switched to the raw pointer just to make sure this following example is as accurate as possible and it didn't magically compile/run properly with the raw pointer).
The code looks very similar to the following (the extra code has been removed):
// console.hpp
extern Console *console;
The implementation cpp file (only of what is needed):
// console.cpp
Console *console = new Console();
Now in some unrelated function, this code fails due to the console pointer being a nullptr:
// some_other_file.cpp
#include "console.hpp"
// Inside some function...
console->doSomething(); // console is NULL
Again, the code I have worked perfectly fine when it was in one project. Regardless, it all compiles fine with no linking errors even though it has been broken into 3 pieces, but now that pointer is always null.
As a last interesting note, the non-pointer variables that are global and extern do work. Is this something limited to pointers/unique_ptrs?
Is this something fixable by a singleton pattern?
The clue is in this comment: "it appears other code is getting called before the main function that don't let the global variable get initialized."
The code referencing console is probably running as part of the initialization of another global and, in this case, is happening before the initialization of console. You have to be very careful to be sure that you're not depending on the order of the global initializers. You were probably getting lucky before you split the program, and now your luck has run out.
The easiest way to fix it is to use a singleton pattern. Instead of having other parts of the program directly reference the pointer, you have them call a function that returns the pointer, and that function will initialize it the first time. This ensures that it's initialized before it's used.
Here's a common pattern for it:
Console *GetConsole() {
static Console *console = new Console();
return console;
}
Now console is no longer a global variable. Everything that wants to access the console calls GetConsole instead.
The function-local static variable will be initialized the first time the function is called, and after that it just returns the value. If you have multiple threads and an older compiler, you have to do more work to protect against a possible race condition. Modern compilers should do the initialization in a thread-safe way.
I have a working set of TCL script plus C++ extension but I dont know exactly how it works and how was it compiled. I am using gcc and linux Arch.
It works as follows: when we execute the test.tcl script it will pass some values to an object of a class defined into the C++ extension. Using these values the extension using a macro give some result and print some graphics.
In the test.tcl scrip I have:
#!object
use_namespace myClass
proc simulate {} {
uplevel #0 {
set running 1
for {} {$running} { } {
moveBugs
draw .world.canvas
.statusbar configure -text "t:[tstep]"
}
}
}
set toroidal 1
set nx 100
set ny 100
set mv_dist 4
setup $nx $ny $mv_dist $toroidal
addBugs 100
# size of a grid cell in pixels
set scale 5
myClass.scale 5
The object.cc looks like:
#include //some includes here
MyClass myClass;
make_model(myClass); // --> this is a macro!
The Macro "make_model(myClass)" expands as follows:
namespace myClass_ns { DEFINE_MYLIB_LIBRARY; int TCL_obj_myClass
(mylib::TCL_obj_init(myClass),TCL_obj(mylib::null_TCL_obj,
(std::string)"myClass",myClass),1); };
The Class definition is:
class MyClass:
{
public:
int tstep; //timestep - updated each time moveBugs is called
int scale; //no. pixels used to represent bugs
void setup(TCL_args args) {
int nx=args, ny=args, moveDistance=args;
bool toroidal=args;
Space::setup(nx,ny,moveDistance,toroidal);
}
The whole thing creates a cell-grid with some dots (bugs) moving from one cell to another.
My questions are:
How do the class methods and variables get the script values?
How is possible to have c++ code and compile it without a main function?
What is that macro doing there in the extension and how it works??
Thanks
Whenever a command in Tcl is run, it calls a function that implements that command. That function is written in a language like C or C++, and it is passed in the arguments (either as strings or Tcl_Obj* values). A full extension will also include a function to do the library initialisation; the function (which is external, has C linkage, and which has a name like Foo_Init if your library is foo.dll) does basic setting up tasks like registering the implementation functions as commands, and it's explicit because it takes a reference to the interpreter context that is being initialised.
The implementation functions can do pretty much anything they want, but to return a result they use one of the functions Tcl_SetResult, Tcl_SetObjResult, etc. and they have to return an int containing the relevant exception code. The usual useful ones are TCL_OK (for no exception) and TCL_ERROR (for stuff's gone wrong). This is a C API, so C++ exceptions aren't allowed.
It's possible to use C++ instance methods as command implementations, provided there's a binding function in between. In particular, the function has to get the instance pointer by casting a ClientData value (an alias for void* in reality, remember this is mostly a C API) and then invoking the method on that. It's a small amount of code.
Compiling things is just building a DLL that links against the right library (or libraries, as required). While extensions are usually recommended to link against the stub library, it's not necessary when you're just developing and testing on one machine. But if you're linking against the Tcl DLL, you'd better make sure that the code gets loaded into a tclsh that uses that DLL. Stub libraries get rid of that tight binding, providing pretty strong ABI stability, but are little more work to set up; you need to define the right C macro to turn them on and you need to do an extra API call in your initialisation function.
I assume you already know how to compile and link C++ code. I won't tell you how to do it, but there's bound to be other questions here on Stack Overflow if you need assistance.
Using the code? For an extension, it's basically just:
# Dynamically load the DLL and call the init function
load /path/to/your.dll
# Commands are all present, so use them
NewCommand 3
There are some extra steps later on to turn a DLL into a proper Tcl package, abstracting code that uses the DLL away from the fact that it is exactly that DLL and so on, but they're not something to worry about until you've got things working a lot more.
We're working on a relatively large-scale C++ project where we chose at the very beginning to use protobuf as our Lingua Franca for stored and transmitted data.
We had our first problem because of end-of-program memory leaks due to the protobuf generated classes metadata that are stored as static pointer, allocated during the first call to the constructor and never deallocated. We found a nice function provide by Mr. Google to do this clean-up:
google::protobuf::ShutdownProtobufLibrary();
Works fine except there is no symmetric call, so once it's done you can no longer use anything . You have to do that exactly one time in your executable. We did what any lazy developper would have done:
struct LIBPROTOBUF_EXPORT Resource
{
~Resource()
{
google::protobuf::ShutdownProtobufLibrary();
}
};
bool registerShutdownAtExit()
{
static Resource cleaner;
return true;
}
And we added in the protobuf generation of cc files a:
static bool protobufResource = mlv::protobuf::registerShutdownAtExit();
It worked fine for several months.
Then we added the support for dynamically loadable plugins (dlls) in our tool. Some of them using protobuf. Unloading of the plugins worked fine, but when more than one of them used protobuf, we had a nice little crash when unloading the last one.
The reason: the last to unload would destroy the cleaner instance, itself trying to google::protobuf::ShutdownProtobufLibrary(), itself trying to destroy metadata of unloaded types... CRASH.
Long story short: are we condemned to either have lots of "normal" memory leaks or a crash when closing our tool. Does anyone have experienced the same problem and found a better solution? Is my diagnosis bad?
Like johnathon suggested in his comment, use a reference counting scheme, or register your destruction routine with atexit. Such a routine is free-standing, but that could work fine for your case.
Relevant documentation:
MSDN
POSIX
Edit: You're right, it's basically the same thing. Didn't think this through.
Another suggestion: Use a global resource singleton for all protobuf-using plugins. This one has a global destructor, which is only registered when a plugin first uses the protobuf library. Or just set a flag whenver it's used, then call ShutdownProtobufLibrary only if the flag is set.
My boss wants me to write a DLL in C++ (MSVC++2010) which can perform a Volume Shadow Copy which he can call from VB6 (or at a later point other languages) and which can give status updates back while it is not finished. He calls it "events".
I have the feeling that I finally need to learn COM (I'd rather not ...) ... also, a callback function comes to my mind, but it's surely impossible to hand over function pointers from VB6 to C++?
Can someone outline what I have to learn and how this can be accomplished, with or without COM?
EDIT:
to answer a question, the work flow is supposed to be:
VB6 app determines which files to back up
I am given a path and make a temporary volume shadow copy which includes this path and give back a mount point (or similar)
during this step, I regularly tell the VB6 app how far I am
VB6 app makes backup of shadow copy and then deletes shadow copy.
You can pass a pointer to your "display progress" function from the VB app to the C++ DLL app using the AddressOf operator:
Declare Function CallMyDll ...
Sub DisplayStatus(ByVal SomeParameter As Long)
' ...
End SUb
Sub DoSomething()
Call CallMyDll(AddressOf DisplayStatus)
End Sub
Some not so obvious gotchas:
You have to declare your C++ function pointer using the __stdcall calling convention. (Thanks, Alexandre C!)
In your VB callback function, explicitly mark your parameters as by-value using the keyword ByVal. Similarly, in your C++ function pointer, don't mark your parameters as by-reference.
If you want to pass a string to the callback, or retrieve a string from it, you have to take into consideration that VB Strings are not equal to C char*s, C++ std::strings, or Microsoft's CStrings. VB Strings must be mapped to Microsoft's rather obscure BSTR data type.
I forgot a very important thing: Your callback has to be inside a VB Module (i.e., it has to be a "mere function", not a class' or a form's method).
Your boss can call functions exported by a DLL with the Declare statement. That doesn't scale well but is fine for a simple API. Your function should be exported with the extern "C" and __declspec(dllexport) declarators, use the __stdcall calling convention and use only simple argument types.
When it has to provide these status updates?
While setting up VSS? Or while backing up the data and such?
In the latter case VSS just returns a path, which can be used
directly from VB.
But for setup... it might make sense too, because it can be fairly slow,
but I guess you can turn it into a state machine - put all the VSS API calls
into a big switch() and make a function which would call them one by one and
update the state var.
Update: I mean something like this.
Init() and Step() are your functions exported by your dll and called from VB.
Alternatively, you can spawn a thread to do all that (still in the dll) and return
status updates after like Sleep(100) in Step().
int s; // state
int Init( void ) { s=0; }
int Step( void ) {
switch( s ) {
default: break;
case 0:
CoInitialize(0); break;
case 1:
r = CreateVssBackupComponents(&vssc);
if( FAILED(r) ) s=-1;
if( vssc==0 ) s=-2;
break;
case 2:
r = vssc->InitializeForBackup();
if( FAILED(r) ) s=-3;
break;
case 3:
r = vssc->SetBackupState( FALSE, FALSE, VSS_BT_COPY, FALSE );
if( FAILED(r) ) s=-4;
break;
[...]
}
s += (s>=0);
}
I would do the job without COM. Instead, I'd have the VB part send a window handle to the DLL, and the DLL will post a message to the window telling its progress.
You could use COM, but that's kind of in the range of swatting a fly with a sledgehammer.
Take the unix-y route. Create a PROGRAM that performs the copy and outputs a progress indicator to std out. Make the VB app parse this output to grab the completion percentage.
I would do this:
the VB app calls the function in your DLL asking to start the shadow copy. Your Dll starts a thread that performs the shadow copy and returns an ID (a thread id?) back to the VB app
The VB app calls periodically a function "Progress" in your DLL passing the operation ID received before: the progress function returns an integer 0-100 to indicate the progress
This way the VB app can launch several operations in parallel.
The thread that performs the copy should update a "progress" variable every now and then.
Another function in the DLL that stops the copy would be useful too.
I deployed a MATLAB project into a DLL, to be called from C++, and it works just fine. Happy days.
But what happens when the user asks to cancel an operation?
I tried creating a global variable named UserAborted. I initialize it to 0 before running the long function in MATLAB. I also wrote the following two functions:
function AbortIfUserRequested
global UserAborted
if (UserAborted == 1)
error('User Abort');
end
end
function UserAbortLongFunction
global UserAborted
UserAborted = 1;
end
I call upon AbortIfUserRequested in every iteration of the loop in my long function. I also exported UserAbortLongFunction.
I expected that pretty soon after called UserAbortLongFunction, the long function would reach a call to AbortIfUserRequested, and throw an error.
Instead, the long function keeps running until the end, and only then does the value of UserAborted get changed.
All I want to do is abort that long function when the user asks me to! Is there any way to do that?
Try calling the DRAWNOW function in AbortIfUserRequested. Although Matlab is single-threaded (from an API perspective), it does allow for interrupts. I've had success by calling this function with pure M-code where user input (like Ctrl-C) otherwise gets locked out.
Matlab needs to provide callback functions to show execution progress and possibly halt it. A Google search shows lots of people wanting this but no implementation from Mathworks.
Matlab's single-threaded nature might be preventing the update to the global variable's value from propagating while the first function is executing. You could try sticking the abort flag in a Java object, like a HashMap, for a layer of indirection. Since Java objects are passed by reference, an update to its state may be visible immediately, without requiring a change to the Matlab variable itself.
Here's a snippet to do so. (Sorry, I don't have a Matlab Compiler license to test this out in a deployed DLL.)
function AbortIfUserRequested
global SharedState
if SharedState.get('UserAborted')
error('User Abort');
end
end
function UserAbortLongFunction
global SharedState
SharedState.put('UserAborted', 1);
end
function InitUserAbort
global SharedState
SharedState = java.util.Collections.synchronizedMap(java.util.HashMap());
SharedState.put('UserAborted', 0);
end
Matlab app data is also effectively passed by reference. Putting the abort flag in appdata instead of a global variable might work, too. If your library works with a Matlab GUI, you can put the app data on its figure handle instead of the global handle 0. This would be more idiomatic Matlab than the Java object, if it works.
function AbortIfUserRequested
if getappdata(0, 'UserAborted')
error('User Abort');
end
end
function UserAbortLongFunction
setappdata(0, 'UserAborted', 1);
end