MEX-File to Output Pulse in a Loop to a DAQ-Board - c++

i figured I must use a MEX-File to output Digital pulses in a loop (40 kHz) from Matlab to my DAQ-Board, I have some APIs from the DAQ-Board vendor, but I really dont know if they are useful.
It´s a big documentation on the Mathworks website about MEX-File and APIs, that just make me confused.
So I´m asking here if someone can orientate me or showing me an example Code to realise this!!

I wrote a small winsock package using mex functions a while back because Matlab's tcpip stuff was having issues with sending large amounts of data (like images). I don't know much about mex functions other than what I learned to get that package working, and even that was quite some time ago. But, here's some of my notes from before and one of the functions I wrote as an example that hopefully might be some help for you.
Before writing any mex functions, you need to configure Matlab to be able to compile them. You do this by typing "mex -setup" in the matlab command line and following the instructions it gives. I configured it to use the Visual Studio compiler (note that you have to have Visual Studio installed for this option to show up).
After configuring the compiler, you compile your mex functions by typing "mex filename.cpp" in the Matlab command-line. This produces a .mexw32 file (assuming 32-bit) that Matlab uses when you call your mex function.
To write the mex function itself, you write an m-file to declare it and provide comments as well as a cpp file with the actual implementation.
As an example, here's one of the m-files I wrote:
function sendColorImage( socketHandle, colorImage ) %#ok<*INUSD>
%SENDCOLORIMAGE Sends a color image over the given socket
% This function sends a color image over the socket provided. The image
% is really just an MxNx3 matrix. Note that this function sends the
% image data in the order in which Matlab stores it (non-interlaced
% column major order), which is different from most other languages.
% This means the red values for every pixel will be sent first, then the
% green values, then the blue values. Furthermore, the scanlines read
% from the top of the image to the bottom, starting at the left side of
% the image.
%
% socketHande - A handle to the socket over which the image should be
% sent. This handle is returned by the openSocket function when the
% socket is first created.
%
% colorImage - An MxNx3 matrix containing the image data. This matrix
% should be in the same format as a matrix loaded using Matlabs imread
% function.
%
% This is a mex function and is defined in its corresponding .cpp file.
And here's the corresponding cpp file. Note that I just made up my own message format and had corresponding C# code that parsed it back out of the byte stream.
// Instruct the compiler to link with wsock32.lib (in case it isn't specified on the command line)
#pragma comment(lib,"wsock32.lib")
#include "mex.h"
#include <winsock2.h>
#include <cstdio>
#include "protocol.h"
// See the corresponding .m file for documentation on this mex function.
void mexFunction(int nlhs, mxArray* plhs[], int nrhs, const mxArray* prhs[]){
char errorMessage[100];
// Validate input and output arguments
if(nlhs != 0)
mexErrMsgTxt("There are no output arguments for this function.");
if(nrhs != 2)
mexErrMsgTxt("Must have 2 input parameters: the socket handle and the MxNx3 image matrix");
if(!mxIsClass(prhs[0], "uint32"))
mexErrMsgTxt("The first input parameter should be a uint32 containing the socket handle");
if(!mxIsClass(prhs[1], "uint8") || mxGetNumberOfDimensions(prhs[1]) != 3 || mxGetDimensions(prhs[1])[2] != 3)
mexErrMsgTxt("The 2nd input parameter should be an MxNx3 uint8 matrix containing the image");
// Get the socket handle
SOCKET socketHandle = (int)(mxGetPr(prhs[0])[0]);
// Set up the header
int frameWidth = mxGetDimensions(prhs[1])[1];
int frameHeight = mxGetDimensions(prhs[1])[0];
int header[3];
header[0] = COLOR_IMAGE;
header[1] = frameWidth;
header[2] = frameHeight;
// Send the header
int bytesSent;
int totalBytesSent = 0;
while(totalBytesSent < 3*sizeof(int)){
bytesSent = send(socketHandle, ((char*)header) + totalBytesSent, 3*sizeof(int) - totalBytesSent, 0);
if(bytesSent == SOCKET_ERROR){
sprintf(errorMessage, "Error sending image header over the socket: %d", WSAGetLastError());
mexErrMsgTxt(errorMessage);
}
totalBytesSent += bytesSent;
}
// Send the image
totalBytesSent = 0;
int totalBytesToSend = frameWidth * frameHeight * 3;
char* dataPointer = (char*)mxGetData(prhs[1]);
while(totalBytesSent < totalBytesToSend){
bytesSent = send(socketHandle, dataPointer + totalBytesSent, totalBytesToSend - totalBytesSent, 0);
if(bytesSent == SOCKET_ERROR){
sprintf(errorMessage, "Error sending image over the socket: %d", WSAGetLastError());
mexErrMsgTxt(errorMessage);
}
totalBytesSent += bytesSent;
}
}

Related

libjpeg-turbo segmentation fault when writing scanlines to file c++

I have the following code running on windows 10 in QT Creator, I am trying to write rgb formatted data to a jpeg file using the libjpeg-turbo library
#include <stdio.h>
#include <jpeglib.h>
void writeJpeg(const char *filename, std::vector<unsigned char> &image, uint w, uint h) {
struct jpeg_compress_struct cinfo;
struct jpeg_error_mgr jerr;
FILE *outfile;
JSAMPROW row_pointer[1];
int row_stride;
cinfo.err = jpeg_std_error(&jerr);
jpeg_create_compress(&cinfo);
if ((outfile = fopen(filename, "wb")) == NULL) {
fprintf(stderr, "can't open %s\n", filename);
exit(1);
}
jpeg_stdio_dest(&cinfo, outfile);
cinfo.image_width = w;
cinfo.image_height = h;
cinfo.input_components = 3;
cinfo.in_color_space = JCS_RGB;
jpeg_set_defaults(&cinfo);
jpeg_set_quality(&cinfo, 100, true);
jpeg_start_compress(&cinfo, true);
row_stride = w * 3;
JSAMPLE *arr = image.data();
while (cinfo.next_scanline < cinfo.image_height) {
row_pointer[0] = &arr[cinfo.next_scanline * row_stride];
(void)jpeg_write_scanlines(&cinfo, row_pointer, 1);
}
jpeg_finish_compress(&cinfo);
fclose(outfile);
jpeg_destroy_compress(&cinfo);
}
and get SIGSEGV on (void)jpeg_write_scanlines(&cinfo, row_pointer, 1); in this portion of the code
while (cinfo.next_scanline < cinfo.image_height) {
row_pointer[0] = &arr[cinfo.next_scanline * row_stride];
(void)jpeg_write_scanlines(&cinfo, row_pointer, 1);
}
I have tried debugging, but im not entirely sure what the best way to do that is.
I have found out that it crashes after 15 iterations of the loop and my best guess is that I have converted the data poorly from the vector into the pointer array.
Anyways, I cannot figure out how to better allocate the memory or if this is even the problem
Any ideas on what I'm doing wrong plus any tips on how to actually debug this in the future would be greatly appreciated.
EDIT
I changed true to TRUE and printed out every byte in the array like so
while (cinfo.next_scanline < cinfo.image_height) {
row_pointer[0] = &arr[cinfo.next_scanline * row_stride];
QString out = "";
for (int i = 0; i<row_stride; i++)
out += QString::number(row_pointer[0][i]) + " ";
qDebug() << "\n\n==============" << cinfo.next_scanline << "==============\n\n";
qDebug() << out;
(void)jpeg_write_scanlines(&cinfo, row_pointer, 1);
}
and after stepping through its execution, it is able to output all the bytes, yet still crashes on write_jpeg_scanlines
I also noticed some random text being printed out at the end of each scanline
ex:
255 255 255 25520003200!-�U3 on row 12
255 255 255 255eencoded�/�Rf on row 13
231 230 230 255,autoder�)�Pu on row 15, the data where it crashes
not sure if this is just garbage at the end of the pointer, or a symptom of qDebug but it could confirm antons idea of corrupted image data
I see nothing wrong in your code, except C++ bool value true is passed instead of C value TRUE in these calls:
jpeg_set_quality(&cinfo, 100, true);
...
jpeg_start_compress(&cinfo, true);
It may lead to weird crashes sometimes.
Also, the first thing I would try in this case - what if just to output somewhere all the bytes of every row arr[cinfo.next_scanline * row_stride] - does it crash? If it does, possibly you have error in other code preparing the image data.
UPD.: most probably the problem with original code was #include <jpeglib.h> - that should force to search for jpeglib.h in system directories instead of the libjpeg-turbo directories. That could be solved by using #include "jpeglib.h" and specifying the path to libjpeg-turbo include directory to compiler.
Thanks for all of the help, As #AntonMalyshev pointed out, the issue was an incorrect use of libjpeg.
First off I have libjpeg-turbo installed and while it includes a library for libjpeg im sure a better practice would be to just install libjpeg directly. I ended up using #include <turbojpeg.h>.
Secondly, the example code I posted, seems to be only compatible with base libjpeg Using this post and the libjpeg-turbo api I was able to come up with a solution that works.
Thanks, Ethan

pjsip capture and play pcm data

I have some embedded Devices that have no audio device by default. They communicate with each other via a FPGA. So my question is, how do I capture/play back audio from pjsip in pcm in order to send/receive it with the FPGA?
I know that there is pjmedia_mem_player_create() and pjmedia_mem_capture_create() but I can't seem to find any good info towards using these functions.
I tried the following piece of code, but an assertion failed cause one of the function's parameter is "empty".
Error:
pjmedia_mem_capture_create: Assertion `pool && buffer && size && clock_rate && channel_count && samples_per_frame && bits_per_sample && p_port' failed.
Note: I'm mainly using pjsua2 for everything else like registrations, transports etc. Also the default audio is set to null with ep.audDevManager().setNullDev(); as without this, making/receiving a call would simply fail?!
void MyCall::onCallMediaState(OnCallMediaStateParam &prm){
CallInfo ci = getInfo();
pj_caching_pool_init(&cp, &pj_pool_factory_default_policy, 0);
pj_pool_t *pool = pj_pool_create(&cp.factory, "POOLNAME", 2000, 2000, NULL);
void *buffer;
pjmedia_port *prt;
#define CLOCK_RATE 8000
#define CHANELS 1
#define SAMPLES_PER_FRAME 480
#define BITS_PER_SAMPLE 16
pjmedia_mem_capture_create( pool, //Pool
buffer, //Buffer
2000, //Buffer Size
CLOCK_RATE,
CHANELS,
SAMPLES_PER_FRAME,
BITS_PER_SAMPLE,
0, //Options
&prt); //The return port}
UPDATE
The assertion failed cause the buffer variable doesn't have any memory allocated to it. Allocate with twice the amount of samples per frame to have sufficient memory.
buffer = pj_pool_zalloc(pool, 960);
Also a callback needs to be registered with pjmedia_mem_capture_set_eof_cb2() (The two at the end is necessary for PJSIP 2.10 or later) Apparently from there the buffer can be used. Just that my implementation atm doesn't execute the callback.
Looks like I found the solution, I have modified your code and wrote a simple code in C with pjsua API to dump every frame to file. Sorry for mess, I'm not proficient in C:
pjsua_call_info ci;
pjsua_call_get_info(call_id, &ci);
pjsua_conf_port_info cpi;
pjsua_conf_get_port_info(ci.conf_slot, &cpi);
pj_pool_t *pool = pjsua_pool_create("POOLNAME", 2000, 2000);
pjmedia_port *prt;
uint buf_size = cpi.bits_per_sample*cpi.samples_per_frame/8;
void *buffer = pj_pool_zalloc(pool, buf_size);
pjsua_conf_port_id port_id;
pjmedia_mem_capture_create( pool,
buffer,
buf_size,
cpi.clock_rate,
cpi.channel_count,
cpi.samples_per_frame,
cpi.bits_per_sample,
0,
&prt);
pjmedia_mem_capture_set_eof_cb(prt, buffer, dump_incoming_frames);
pjsua_conf_add_port(pool, prt, &port_id);
pjsua_conf_connect(ci.conf_slot, port_id); //connect port with conference
///////dumping frames///
static pj_status_t dump_incoming_frames(pjmedia_port * port, void * usr_data){
pj_size_t buf_size = pjmedia_mem_capture_get_size(port);
char * data = usr_data;
...
fwrite(data,sizeof(data[0]),buf_size,fptr);
...
}
Documenation says pjmedia_mem_capture_set_eof_cb is deprecated but I couldn't make work pjmedia_mem_capture_set_eof_cb2, buf_size is 0 for every call of dump_incoming_frames so just left with deprecated function. I also succeed the same result with creating custom port.
I hope you can modify it easily to your C++/pjsua2 code
UPD:
I have modified the PJSIP and packed audio in-out streaming into proper PJSUA2/Media classes so it can be called from Python. Full code is here.

How to implement Binary (TYPE I) for FTP

I need to set up TYPE command; mode is either ‘I’ (for image), or ‘A’ (for ASCII). To test the image/binary mode, try transferring a jpg/png image file from the FTP server. (create own FTP server)
However, I'm not sure how to do this, some code below. I pretty much somewhat copied the stor/put command - which works
Currently, when i attempt to get an image from the localhost, the image i receive is 0 byes in size.
Also note: bytes = recv(ns, &receive_buffer[n], 1, 0);//receive byte by byte... is in my skeleton/start up code.
...
if(strncmp(receive_buffer,"TYPE",4) == 0){ //attempt
int bytesRecieved1 = 0;
char codeStr[250];
sscanf(receive_buffer,"TYPE %c\r\n",codeStr);
char code = codeStr[0];
char tempBuff[SEGMENT_SIZE];
memset(tempBuff,0, sizeof(tempBuff));
int _bytes1 = 0;
switch(code){
case 'I': //
sprintf(send_buffer, "200 Switching to Binary mode\r\n");
while (1){
if(active == 0) {
_bytes1 = recv(ns, &receive_buffer[n], 1, 0);
}else{
_bytes1 = recv(ns, &receive_buffer[n], 1, 0);
}
if (_bytes1 < 0) {
sprintf(send_buffer,"226 Successfully received %d bytes\r\n",bytesRecieved1);
bytes = sendString(&ns,send_buffer);
printf("Success");
break;
bytesRecieved1 += _bytes1;
}
}
}
bytes = sendString(&ns,send_buffer);
return;
}
...
Should be able to see the image.
You are not supposed to respond to TYPE command by initiating a file transfer. That's what you should do only after you receive STOR command.
To implement TYPE command, you simply respond 200 OK and internally remember that you need to use binary (or ascii) mode for the upcoming file transfer (once you receive STOR or RETR).

How to read YUV8 data from avi file?

I have avi file that contains uncompressed gray video data. I need to extract frames from it. The size of file is 22 Gb.
How do i do that?
I have already tried ffmpeg, but it gives me "could not find codec parameters for video stream" message - because there is no codec at work, just frames.
Since Opencv just uses ffmpeg to read video, that rules out opencv as well.
The only path that seems to be left is to try and dig into the raw data, but i do not know how.
Edit: this is the code i use to read from the file with opencv. The failure occurs inside the second if. Running ffmpeg binary on the file also fails with the message above (could not find codec aprameters etc)
/* register all formats and codecs */
av_register_all();
/* open input file, and allocate format context */
if (avformat_open_input(&fmt_ctx, src_filename, NULL, NULL) < 0) {
fprintf(stderr, "Could not open source file %s\n", src_filename);
ret = 1;
goto end;
}
fmt_ctx->seek2any = true;
/* retrieve stream information */
int res = avformat_find_stream_info(fmt_ctx, NULL);
if (res < 0) {
fprintf(stderr, "Could not find stream information\n");
ret = 1;
goto end;
}
Edit:
Here is sample code i have tried to make the extraction: pastebin. The result i get is an unchanging buffer after every call to AVIStreamRead.
If you do not need cross platform functionality Video for Windows (VFW) API is a good alternative (http://msdn.microsoft.com/en-us/library/windows/desktop/dd756808(v=vs.85).aspx), i will not put an entire code block, since there's quite much to do, but you should be able to figure it out from the reference link. Basically, you do a AVIFileOpen, then get the video stream via AVIFileGetStream with streamtypeVIDEO, or alternatively do it at once with AVIStreamOpenFromFile and then read samples from the stream with AVIStreamRead. If you get to a point where you fail I can try to help, but it should be pretty straightforward.
Also, not sure why ffmpeg is failing, I have been doing raw AVI reading with ffmpeg without any codecs involved, can you post what call to ffpeg actually fails?
EDIT:
For the issue that you are seeing when the read data size is 0. The AVI file has N slots for frames in each second where N is the fps of the video. In real life the samples won't come exactly at that speed (e.g. IP surveillance cameras) so the actual data sample indexes can be non continuous like 1,5,11,... and VFW would insert empty samples between them (that is from where you read a sample with a zero size). What you have to do is call AVIStreamRead with NULL as buffer and 0 as size until the bRead is not 0 or you run past last sample. When you get an actual size, then you can again call AVIStreamRead on that sample index with the buffer pointer and size. I usually do compressed video so i don't use the suggested size, but at least according to your code snipplet I would do something like this:
...
bRead = 0;
do
{
aviOpRes = AVIStreamRead(ppavi,smpS,1,NULL,0,&bRead,&smpN);
} while (bRead == 0 && ++smpS < si.dwLength + si.dwStart);
if(smpS >= si.dwLength + si.dwStart)
break;
PUCHAR tempBuffer = new UCHAR[bRead];
aviOpRes = AVIStreamRead(ppavi,smpS,1,tempBuffer,bRead,&bRead,&smpN);
/* do whatever you need */
delete tempBuffer;
...
EDIT 2:
Since this may come in handy to someone or yourself to make a choice between VFW and FFMPEG I also updated your FFMPEG example so that it parsed the same file (sorry for the code quality since it lacks error checking but i guess you can see the logical flow):
/* register all formats and codecs */
av_register_all();
AVFormatContext* fmt_ctx = NULL;
/* open input file, and allocate format context */
const char *src_filename = "E:\\Output.avi";
if (avformat_open_input(&fmt_ctx, src_filename, NULL, NULL) < 0) {
fprintf(stderr, "Could not open source file %s\n", src_filename);
abort();
}
/* retrieve stream information */
int res = avformat_find_stream_info(fmt_ctx, NULL);
if (res < 0) {
fprintf(stderr, "Could not find stream information\n");
abort();
}
int video_stream_index = 0; /* video stream is usualy 0 but still better to lookup in case it's not present */
for(; video_stream_index < fmt_ctx->nb_streams; ++video_stream_index)
{
if(fmt_ctx->streams[video_stream_index]->codec->codec_type == AVMEDIA_TYPE_VIDEO)
break;
}
if(video_stream_index == fmt_ctx->nb_streams)
abort();
AVPacket *packet = new AVPacket;
while(av_read_frame(fmt_ctx, packet) == 0)
{
if (packet->stream_index == video_stream_index)
printf("Sample nr %d\n", packet->pts);
av_free_packet(packet);
}
Basically you open the context and read packets from it. You will get both audio and video packets so you should check if the packet belongs to the stream of interest. FFMPEG will save you the trouble with empty frames and give only those samples that have data in them.

how do i create a stereo mp3 file with latest version of ffmpeg?

I'm updating my code from the older version of ffmpeg (53) to the newer (54/55). Code that did work has now been deprecated or removed so i'm having problems updating it.
Previously I could create a stereo MP3 file using a sample format called:
SAMPLE_FMT_S16
That matched up perfectly with my source stream. This has now been replace with
AV_SAMPLE_FMT_S16
Which works fine for mono recordings but when I try to create a stereo MP3 file it bugs out at avcodec_open2 with:
"Specified sample_fmt is not supported."
Through trial and error I've found that using
AV_SAMPLE_FMT_S16P
...is accepted by avcodec_open2 but when I get through and create the MP3 file the sound is very distorted - it sounds about 2 octaves lower than usual with a massive hum in the background - here's an example recording:
http://hosting.ispyconnect.com/example.mp3
I've been told by the ffmpeg guys that this is because I now need to manually deinterleave my byte stream before calling:
avcodec_fill_audio_frame
How do I do that? I've tried using the swrescale library without success and i've tried manually feeding in L/R data into avcodec_fill_audio_frame but the results i'm getting are sounding exactly the same as without interleaving.
Here is my code for encoding:
void add_audio_sample( AudioWriterPrivateData^ data, BYTE* soundBuffer, int soundBufferSize)
{
libffmpeg::AVCodecContext* c = data->AudioStream->codec;
memcpy(data->AudioBuffer + data->AudioBufferSizeCurrent, soundBuffer, soundBufferSize);
data->AudioBufferSizeCurrent += soundBufferSize;
uint8_t* pSoundBuffer = (uint8_t *)data->AudioBuffer;
DWORD nCurrentSize = data->AudioBufferSizeCurrent;
libffmpeg::AVFrame *frame;
int got_packet;
int ret;
int size = libffmpeg::av_samples_get_buffer_size(NULL, c->channels,
data->AudioInputSampleSize,
c->sample_fmt, 1);
while( nCurrentSize >= size) {
frame=libffmpeg::avcodec_alloc_frame();
libffmpeg::avcodec_get_frame_defaults(frame);
frame->nb_samples = data->AudioInputSampleSize;
ret = libffmpeg::avcodec_fill_audio_frame(frame, c->channels, c->sample_fmt, pSoundBuffer, size, 1);
if (ret<0)
{
throw gcnew System::IO::IOException("error filling audio");
}
//audio_pts = (double)audio_st->pts.val * audio_st->time_base.num / audio_st->time_base.den;
libffmpeg::AVPacket pkt = { 0 };
libffmpeg::av_init_packet(&pkt);
ret = libffmpeg::avcodec_encode_audio2(c, &pkt, frame, &got_packet);
if (ret<0)
throw gcnew System::IO::IOException("error encoding audio");
if (got_packet) {
pkt.stream_index = data->AudioStream->index;
if (pkt.pts != AV_NOPTS_VALUE)
pkt.pts = libffmpeg::av_rescale_q(pkt.pts, c->time_base, c->time_base);
if (pkt.duration > 0)
pkt.duration = av_rescale_q(pkt.duration, c->time_base, c->time_base);
pkt.flags |= AV_PKT_FLAG_KEY;
if (libffmpeg::av_interleaved_write_frame(data->FormatContext, &pkt) != 0)
throw gcnew System::IO::IOException("unable to write audio frame.");
}
nCurrentSize -= size;
pSoundBuffer += size;
}
memcpy(data->AudioBuffer, data->AudioBuffer + data->AudioBufferSizeCurrent - nCurrentSize, nCurrentSize);
data->AudioBufferSizeCurrent = nCurrentSize;
}
Would love to hear any ideas - I've been trying to get this working for 3 days now :(
you don't want to increase pSoundBuffer if a frame hasn't been fully encoded (e.g. got_packet isn't set to true) as no memory has been written yet. Also, you are allocating a frame during each loop: there's no need for that, you can re-use the same AVFrame over an over. Your code is also leaking as you never free the AVFrame.
I wrote a code as part of MythTV that encode audio to AC3.
This also do what you were looking for: deinterleave the content.
https://github.com/MythTV/mythtv/blob/476b2a826d43fca5e658ebe787c3cb1ec2334f98/mythtv/libs/libmyth/audio/audiooutputdigitalencoder.cpp#L178
I know this question is old, but for posterity: I'm working on some audio resampling code, and after I arrived at an audio sounding very similar to the mp3 the author linked, I identified the cause as being a mismatch in audio sampling rate between the input the resampler expects and the actual data.