I am writing an automatic grader program under linux. There are several graders written in MATLAB, so I want to tie them all together and let students run a program to do an assignment, and have them choose the assignment. I am using a C++ main program, which then has mcc-compiled MATLAB libraries linked to it.
Specifically, my program reads a config file for the names of the various matlab programs, and other information. It then uses that information to present choices to the student. So, If an assignment changes, is added or removed, then all you have to do is change the config file.
The idea is that next, the program invokes the correct matlab library that has been compiled with mcc. But, that means that the libraries have to be recompiled if a grader gets changed. Worse, the whole program must be recompiled if a grader is added or removed. So, I would like one, simple, unchanging matlab library function to call the grader m-files directly. I currently have such a library, that uses eval on a string passed to it from the main program.
The problem is that when I do this, apparently, mcc absorbs the grader m-code into itself; changing the grader m code after compilation has no effect. I would like for this not to happen. It was brought to my attention that Mathworks may not want me to be able to do this, since it could bypass matlab entirely. That is not my intention, and I would be happy with a solution that requires a full matlab install.
My possible solutions are to use a mex file for the main program, or have the main program call a mcc library, which then calls a mex file, which then calls the proper grader. The reason I am hesitant about the first solution is that I'm not sure how many changes I would have to make to my code to make it work; my code is C++, not C, which I think makes things more complicated. The 2nd solution, though, may just be more complicated and ultimately have the same problem.
So, any thoughts on this situation? How should I do this?
You seem to have picked the most complicated way of solving the problem. Here are some alternatives:
Don't use C/C++ at all -- Write a MATLAB program to display the menu of choices (either a GUI for a simple text menu in the MATLAB command window) and then invoke the appropriate MATLAB grading programs.
Write your menu program in C/C++, but invoke MATLAB using a -r argument to run a specific grading program (to speed up the startup times, use the -nodesktop, -nojvm or -nodisplay options as appropriate). However, note that MATLAB will be started anew on each menu selection.
Write your menu program in C/C++ and start MATLAB using the popen command (this sets up a pipe between your C++ program and the MATLAB process). After a menu selection by the user:
your C++ program writes the name of the MATLAB program (and any parameters) to the pipe.
On the MATLAB side, write a MATLAB program to a blocking read on that pipe. When it reads a command, it invokes the appropriate MATLAB function.
You could also use named pipes. See this MATLAB newsgroup thread for more information.
Update: Option #3 above is effectively how the MATLAB engine works, so you are probably better off using that directly.
Don't make this a mex function.
Use a regular m-file that has to be executed in matlab. If you don't want to launch matlab first, write a bat file. I believe -r or -m runs a given command (you will have to cd to the correct directory before running you ml function).
To compile c++ code using mex first install visual studio. Then run (in matlab) mex -setup. Select "locate installed compilers" or some such, and then select your compiler from the list. Now mex will compile c++ code.
Using the MATLAB Engine to Call MATLAB Software from C/C++ and Fortran Programs
Related
I am writing a simulation of some differential equation. My idea was the following:
1. Write the core simulation (moving forward in time, takes a lot of time) in C++
2. Do initialisation and the analysis of the results with a program
like Matlab/Scilab
The reason for (1) is that C++ is faster if implemented correctly.
The reason for (2) is that for me it is easier to make analysis, like plotting etc..., with a program like Matlab.
Is it possible to do it like this, how do I call C++ from Matlab?
Or do you have some suggestions to do it in a different way?
You could certainly do as you suggest. But I suggest instead that you start by developing your entire solution in Matlab and only then, if its performance is genuinely holding your work back, consider translating key elements into C++. This will optimise the use of your time, possibly at the cost of your computer's time. But a computer is a modern donkey without a humane society to intervene when you flog it to death.
As you suggest, well written C++ can be expected to be faster than interpreted Matlab. But ask yourself What is Matlab written in ? For much of its computationally-intensive core functionality Matlab calls libraries written in C++ (or whatever). Your task would be not to write code faster than interpreted Matlab, but faster than C++ (or whatever) written by specialists urged on by a huge market of installed software.
Yes, Matlab has a C/C++ API.
This API permits to:
Write C++ functions which can be invoked from Matlab
Read/Write data from a .mat file
Invoke the Matlab engine from C++
I am working to something similar to what you are trying to do, my approach is:
Import in C++ the input data from a .mat file
Run the simulation
Export the results back in a .mat file
The Matlab API is in C, and I suggest you to write a C++ wrapper for your convenience.
In order to work with Matlab mxArray, I suggest to take a look at the boost::multi_array library.
In particular you can initialize an object of type multi_array_ref from a Matlab mxArray like this:
boost::multi_array_ref<double,2> vec ( mxGetPr (p), boost::extents[10][10], boost::fortran_storage_order() );
This approach made the code much more readable.
You can call your own C, C++, or Fortran subroutines from the MATLAB command line as if they were built-in functions. These programs, called binary MEX-files, are dynamically-linked subroutines that the MATLAB interpreter loads and executes.
You should set compiler, look here Setting up mex to use the Visual Studio 2010 compiler.
All about MEX-files here: http://www.mathworks.com/help/matlab/matlab_external/using-mex-files-to-call-c-c-and-fortran-programs.html.
I have written some mex (c++) code, i have used mxArrays, and few other Matlab functions, i am wondering is it possible to convert it to C++ code easily by including the appropriate header and making some minor changes ?
edit:
By "convert", i mean that i want to compile and run my code without relying (using) on matlab.
As mentioned in the comments above, making this work depends greatly on the nature of the MATLAB functions you are using. Since you have successfully ported much of your MATLAB code to C++, I suggest you continue doing so until you are no longer reliant on the MATLAB libraries to build.
Are there any toolbox functions you are relying on? If so this may be a bigger task than you realize.
Good luck!
You can make executable by using SimulinkCoder - so you can run it without Matlab ... but you still must use Matlab to make executable every time you make change in the code.
Make basic Simulink model with single S-function block in which you specify your mex file.
Use SimulinkCoder (Real Time Workshop) to make executable out of the Simulink model.
I know many have asked this question before, but as far as I can see, there's no clear answer that helps C++ beginners. So, here's my question (or request if you like),
Say I'm writing a C++ code using Xcode or any text editor, and I want to use some of the tools provided in another C++ program. For instance, an executable. So, how can I call that executable file in my code?
Also, can I exploit other functions/objects/classes provided in a C++ program and use them in my C++ code via this calling technique? Or is it just executables that I can call?
I hope someone could provide a clear answer that beginners can absorb.. :p
So, how can I call that executable file in my code?
The easiest way is to use system(). For example, if the executable is called tool, then:
system( "tool" );
However, there are a lot of caveats with this technique. This call just asks the operating system to do something, but each operating system can understand or answer the same command differently.
For example:
system( "pause" );
...will work in Windows, stopping the exectuion, but not in other operating systems. Also, the rules regarding spaces inside the path to the file are different. Finally, even the separator bar can be different ('\' for windows only).
And can I also exploit other functions/objects/classes... from a c++
and use them in my c++ code via this calling technique?
Not really. If you want to use clases or functions created by others, you will have to get the source code for them and compile them with your program. This is probably one of the easiest ways to do it, provided that source code is small enough.
Many times, people creates libraries, which are collections of useful classes and/or functions. If the library is distributed in binary form, then you'll need the dll file (or equivalent for other OS's), and a header file describing the classes and functions provided y the library. This is a rich source of frustration for C++ programmers, since even libraries created with different compilers in the same operating system are potentially incompatible. That's why many times libraries are distributed in source code form, with a list of instructions (a makefile or even worse) to obtain a binary version in a single file, and a header file, as described before.
This is because the C++ standard does not the low level stuff that happens inside a compiler. There are lots of implementation details that were freely left for compiler vendors to do as they wanted, possibly trying to achieve better performance. This unfortunately means that it is difficult to distribute a simple library.
You can call another program easily - this will start an entirely separate copy of the program. See the system() or exec() family of calls.
This is common in unix where there are lots of small programs which take an input stream of text, do something and write the output to the next program. Using these you could sort or search a set of data without having to write any more code.
On windows it's easy to start the default application for a file automatically, so you could write a pdf file and start the default app for viewing a PDF. What is harder on Windows is to control a separate giu program - unless the program has deliberately written to allow remote control (eg with com/ole on windows) then you can't control anything the user does in that program.
I am trying to optimize the interaction between two scripts I have.
Two things I thought of are the c++ program not terminating unless you manually kill it, or generating all info in python before feeding it to c++.
Explanation of the problem:
What the scripts do:
C++ program (not made by me, and I can't program in c++ very well): takes a 7 number array and returns a single number, simple.
Python script (mine, and I can program a bit in python): generates those 7 number arrays, feeds them to the c++ program, waits for an answer and adds it to a list. It then makes the next array.
In theory, this works. However, as it is right now, it opens and closes the c++ program for each call. For one array that is no problem, but I'm trying to upscale to 25k arrays, and in the future to 6+ million arrays. Obviously it is then no longer feasible to open/close it each time, especially since the c++ program first has to load a 130mb VCD file to function.
Two options I thought of myself were to generate all arrays first in python, then feed them to the c++ program and then analyze all results. However, I wouldn't know how to do this with 6M arrays. It is not important however that the results I get back are in the same order as the arrays I feed in.
Second option I thought of was to make the c++ program not quit after each call. I can't program in c++ though so I don't know if this is possible, keeping it 'alive' so you can just feed arrays into it at times and get an answer.
(Note: I cannot program in anything else than python, and want to do this project in python. The c++ program cannot be translated to python for speed reasons.)
Thanks in advance, Max.
Firstly, just to be pedantic, there are no C++ scripts in normal use. C++ compiles, ultimately, to machine code, and the C++ program is properly referred to as a "program" and not a "script".
But to answer your question, you could indeed set up the C++ program to stay in memory, where it listens for connections and sends responses to your Python script. You'd want to study Unix IPC, particularly sockets.
Another way to approach it would be to incorporate what the C++ program does into your Python script, and forget about C++ altogether.
Without the source code or the exact specifications of the Python script and the C++ program, it's difficult to provide more information, but you could modify the C++ code to repeatedly read the array from the standard input and then write the results to standard output.
Then you could use the Python subprocess module to launch the C++ program from your Python script and communicate with it.
Note that simply wrapping a loop around the main() function of the C++ program will not be very helpful, because apparently the main issue is the time the program needs in order to read its data (the VCD that you mentioned).
The loop needs to be strictly around the code that computes the result - which means that you may have to factor everything else out in a way that allows the result computation to be done repeatedly without each run contaminating the next ones.
Okay, your best course of action is probably to write a C/C++ extension to Python that is able to call the C++ code that does the calculation you want. This is not terribly difficult, it will only require a minimal amount of C/C++ coding to make it work. A good explanation of extending Python can be found on the Python page at http://docs.python.org/extending/extending.html
What you in effect do is change your C++ program to be a dynamic library that the Python process can link in and call from the Python script.
If you need a bit of help getting it to work I'm sure we can help you out.
I think the best way is to build C++ extension module for python.
There are lot of ways to do it.
If you have c++ sources you can try SWIG
After that you can use c++ functions/object directly inside python - and manage them by python modules (here processing). It is really simple.
I think you're doing it wrong
What the scripts do: C++ program (not made by me, and I can't program in c++ very well): takes a 7 number array and returns a single number, simple. Python script (mine, and I can program a bit in python): generates those 7 number arrays, feeds them to the c++ program, waits for an answer and adds it to a list. It then makes the next array.
You have this?
python generate_arrays.py | someC++app | python gather_array.py
This allows you to run the three parts in parallel, using every Core of every CPU on the box. The OS makes sure that all three run concurrently.
If you're still not getting 100% CPU Load, you'll have to do something like this.
( python generate_arrays.py --even | someC++app >oneFile ) & ( python generate_arrays.py --odd | someC++app > anotherFile )
python gather_array.py oneFile anotherFile
That will run two copies of python generate_arrays.py and two copies of your magical C++ program.
You'll have to rewrite your generate_arrays.py program so that it takes a command-line option. When the option is --even, you generate 3 million arrays. When the options is --odd you generate the other 3 million arrays.
This (python | c++) & (python | c++) should get to 100% cpu use.
I am planning to write a program that makes calls to cdrecord. (I am a beginner, a beginner trying to "scratch an itch") The program would be written in C++. I have identified that I need to be able to run cdrecord in order for this to work.
cdrecord is written in C. However the documentation on using it is from the command line. The source code includes a main function that powers the command line app, which is the same as the code I wand cdrecord to do.
I am wondering whether I should:
Change main to another name, then include the source file and call it when I need to.
Call the compiled program using the system() command.
Something else.
system() is generally a nice way to go, just be careful not to inject arbitrary untrusted values into the string you execute. For example, if you have a web frontend where with a padsize option defaulted to 0, and someone types in not a number but "0; rm -rf *;", make sure you don't end up calling "cdrecord padsize=0; rm -rf *; ...".
The other thing with system is just that it can be slower starting a second distinct process - this might matter if you're running that program hundreds of times and each time it only has a few milliseconds of work to do, but in your case the overhead of launching is dwarfed by the likely cdrecord run-time.
Using system() will allow you to not worry about cdrecord's code. Personally, I would only include the code in my own program if I had some very pressing issues that require me to include it. I think system() is the way to go.
http://www.cplusplus.com/reference/clibrary/cstdlib/system/
1) Is there any particular reason you would rather include it in your own code, as opposed to just using it as it is?
2) Do you have the rights to change the code and include it in your own program?