I'm trying to make my ALSA interface work with as low latency as possible. The ALSA documentation about all the frames / periods / buffers is very confusing, so I'm asking here.
The below code forces the ALSA to actually read 1024 times the buffer size (I set the buffer size to 1024 bytes). The writeAudio function takes 1024 * 1024 bytes before it slows down. I would like to keep this frames ? count as low as possible, so I could track the playtime in my application itself. If I try to set the period size to 2 with the snd_pcm_hw_params_set_periods (I guess it would then slow down the reading after 2 * 1024 bytes has been written to the buffer.) However this change in code doesn't change the behaviour; the player still buffers 1024 * 1024 bytes before the buffering slows down to the rate the audio is being played from the speakers.
TLDR; 1024 * 1024 bytes buffering size is way too much for me, how to lower it? Below is my code. And yes, I'm playing unsigned 8 bit with mono output.
int32_t ALSAPlayer::initPlayer(ALSAConfig cfg)
{
std::cout << "=== INITIALIZING ALSA ===" << std::endl;
if(!cfg.channels || !cfg.rate)
{
std::cout << "ERROR: player config was bad" << std::endl;
return -1;
}
m_channels = cfg.channels;
m_rate = cfg.rate;
m_frames = 1024;
uint32_t tmp;
uint32_t buff_size;
int dir = 0;
/* Open the PCM device in playback mode */
if ((pcm = snd_pcm_open(&pcm_handle, PCM_DEVICE, SND_PCM_STREAM_PLAYBACK, 0)) < 0)
{
printf("ERROR: Can't open \"%s\" PCM device. %s\n", PCM_DEVICE, snd_strerror(pcm));
}
snd_pcm_hw_params_alloca(¶ms);
snd_pcm_hw_params_any(pcm_handle, params);
if ((pcm = snd_pcm_hw_params_set_access(pcm_handle, params, SND_PCM_ACCESS_RW_INTERLEAVED)) < 0)
{
printf("ERROR: Can't set interleaved mode. %s\n", snd_strerror(pcm));
}
if ((pcm = snd_pcm_hw_params_set_format(pcm_handle, params, SND_PCM_FORMAT_S8)) < 0)
{
printf("ERROR: Can't set format. %s\n", snd_strerror(pcm));
}
if ((pcm = snd_pcm_hw_params_set_channels(pcm_handle, params, m_channels)) < 0)
{
printf("ERROR: Can't set channels number. %s\n", snd_strerror(pcm));
}
if ((pcm = snd_pcm_hw_params_set_rate_near(pcm_handle, params, &m_rate, &dir)) < 0)
{
printf("ERROR: Can't set rate. %s\n", snd_strerror(pcm));
}
// force the ALSA interface to use exactly *m_frames* number of frames
snd_pcm_hw_params_set_period_size(pcm_handle, params, m_frames, dir);
/* Write parameters */
if ((pcm = snd_pcm_hw_params(pcm_handle, params)) < 0)
{
printf("ERROR: Can't set harware parameters. %s\n", snd_strerror(pcm));
}
std::cout << "ALSA output device name: " << snd_pcm_name(pcm_handle) << std::endl;
std::cout << "ALSA output device state: " << snd_pcm_state_name(snd_pcm_state(pcm_handle)) << std::endl;
snd_pcm_hw_params_get_channels(params, &tmp);
std::cout << "ALSA output device channels: " << tmp << std::endl;
snd_pcm_hw_params_get_rate(params, &tmp, 0);
std::cout << "ALSA output device rate: " << tmp << std::endl;
snd_pcm_hw_params_get_period_size(params, &m_frames, &dir);
buff_size = m_frames * m_channels;
std::cout << "ALSA output device frames size: " << m_frames << std::endl;
std::cout << "ALSA output device buffer size: " << buff_size << "(should be 1024)" << std::endl;
return 0;
}
int ALSAPlayer::writeAudio(byte* buffer, uint32_t buffSize)
{
int pcmRetVal;
if(buffSize == 0)
{
snd_pcm_drain(pcm_handle);
snd_pcm_close(pcm_handle);
return -1;
}
if((pcmRetVal = snd_pcm_writei(pcm_handle, buffer, m_frames)) == -EPIPE)
{
snd_pcm_prepare(pcm_handle);
}
else if(pcm < 0)
{
std::cout << "ERROR: could not write to audio interface" << std::endl;
}
return 0;
}
(One frame is not necessarily one byte; please don't confuse them.)
This code does not set the buffer size.
snd_pcm_hw_params_set_periods() sets the number of periods.
snd_pcm_hw_params_set_period_size() would set the period size.
To have a 2048-frame buffer with two periods with 1024 frames each, set the number of periods to 2, and the period size to 1024.
You must check all function calls for errors, including snd_pcm_hw_params_set_period_size().
Related
I am very new to socket programming, and i am trying to send over TCP connection but getting few errors.
here is my code
FILE* File;
char* Buffer;
unsigned long Size;
File = fopen("C:\\test.zip", "rb");
if (!File)
{
printf("Error while readaing the file\n");
return;
}
// file size 1
fseek(File, 0, SEEK_END);
Size = ftell(File);
fseek(File, 0, SEEK_SET);
Buffer = new char[Size];
fread(Buffer, Size, 1, File);
char cSize[MAX_PATH];
sprintf(cSize, "%i", Size);
cout << "MAX PATH " << MAX_PATH<<endl;
cout << "cSize: " << cSize << endl;
fclose(File);
`
So this to find the size of my file. most of the code i am trying it out from other questions in here but it didnt solve my problem.
'
my send and recv:
unsigned long filechunk = 1025;
unsigned long byteSent = 0;
unsigned long bytesToSend = 0;
send(Sub, cSize, MAX_PATH, 0); // File size to client
while (byteSent < Size) {
if ((Size - byteSent) >= filechunk) {
bytesToSend = filechunk;
}
else {
bytesToSend = Size - byteSent;
}
if (send(Sub, Buffer + byteSent, bytesToSend, 0)) {
std::cout << "Sent: ";
}
byteSent += bytesToSend;
std::cout << "Size : "<<Size<<" BytesSent : "<<byteSent<<" Bytes to send: " << bytesToSend << std::endl;
system("pause");
on the client side:
int Size;
char* Filesize = new char[5000000]; // is there a better way? my sfiles size are unknown but at least 50mb
if (recv(Socket, Filesize, 5000000, 0)) // File size
{
Size = atoi((const char*)Filesize);
printf("File size: %d\n", Size);
}
char* Buffer = new char[Size];
FILE* File;
File = fopen("test.zip", "wb"); //start copying from the server, creating the file first.
std::string convert;
long conv;
std::cout << "Size: " << Size << std::endl;
int total=Size;
int byteRecv = 0;
int recvCheck;
int bytes = 1025;
//getting the file
while (byteRecv < Size ) {
recvCheck = recv(Socket, Buffer, bytes, 0);
if (recvCheck >0) // File
{
fwrite(Buffer, 1, byteRecv, File);
std::cout << "Recieved:" << byteRecv << std::endl;
Size -= byteRecv;
byteRecv += byteRecv;
std::cout << "Error: " << WSAGetLastError();
}
else {
std::cout << "Error: " << WSAGetLastError();
total += 1; // the loop often get into infinit loop so i force it in case of this error.
if (total > 3) {
break;
}
}
}
fclose(File);
So, i know it is not very efficient and i am not sure if there are similar questions as i have been digging in here for a few weeks now.
-is there a better way i can make a char*[]? as i dont know the size of the files i want to send yet.
- does ftell() and sifeof() work the same way?
-when i check for the size i recved from the server it is alays wrong. Ex: server file: 32633513, recv size: 3263
-most of the code i have taken from other problems and combined it. if you see anything that is not needed do tell me so i take notes of that.
There is a lot of wrong things but that may correct your problem at first:
On the client side replace (your are both decrementing the total count of bytes and the received ones with the wrong value):
Size -= byteRecv;
byteRecv += byteRecv;
with:
byteRecv += recvCheck; // actualizes the count of received bytes
The other problem is your buffer size. Never try to get an entire file in memory, this is nonsense in general; files are usually managed chunks by chunks. As you are reading at most 1025 bytes in each loop, then only use a buffer of size 1025, you don't need more. Same for reading and writing...
I'm trying to send a jpg file from a client to a server using TCP. When the picture arrives to the server side I can't open it, besides the size of the picture received is higher than the one sent (sent = 880 bytes , received = 894 bytes). Any one of you have an idea of how to do solve this problem ? Here is my code :
client code :
static int send_server_image(SOCKET sock){
int n = 0;
int siz = 0;
FILE *picture;
char buf[50];
char *s="";
cout << "Getting image size" << endl;
picture = fopen("C:\\Users\\n.b\\Desktop\\c++\\TCP\\tcp_client_image_pp\\test.jpg", "r");
fseek(picture, 0, SEEK_END);
siz = ftell(picture);
cout << siz << endl; // Output 880
cout << "Sending picture size to the server" << endl;
sprintf(buf, "%d", siz);
if((n = send(sock, buf, sizeof(buf), 0)) < 0)
{
perror("send_size()");
exit(errno);
}
char Sbuf[siz];
cout << "Sending the picture as byte array" << endl;
fseek(picture, 0, SEEK_END);
siz = ftell(picture);
fseek(picture, 0, SEEK_SET); //Going to the beginning of the file
while(!feof(picture)){
fread(Sbuf, sizeof(char), sizeof(Sbuf), picture);
if((n = send(sock, Sbuf, sizeof(Sbuf), 0)) < 0)
{
perror("send_size()");
exit(errno);
}
memset(Sbuf, 0, sizeof(Sbuf));
}
}
server code :
static int recv_client_image(SOCKET sock){
int n = 0;
cout << "Reading image size" << endl;
char buf[50];
int siz = 0;
if ((n = recv(sock, buf, sizeof(buf), 0) <0)){
perror("recv_size()");
exit(errno);
}
siz = atoi(buf);
cout << siz << endl; // 880 output
char Rbuffer[siz];
cout << "Reading image byte array" << endl;
n = 0;
if ((n = recv(sock, Rbuffer, sizeof(Rbuffer), 0)) < 0){
perror("recv_size()");
exit(errno);
}
cout << "Converting byte array to image" << endl;
FILE *image;
image = fopen("recu.jpg", "w");
fwrite(Rbuffer, sizeof(char), sizeof(Rbuffer), image);
fclose(image);
cout << "done" << endl;
}
Thank you.
You are using Variable Length Arrays, which is not standard C++ (ref). Even if it is accepted by your compiler, you should avoid using sizeof on that.
And you have a problem in the while(!feof(picture)). You read siz bytes from the file without any error and without setting the eof flag. On second read, you read 0 bytes and set the flag but also send another buffer.
You should write instead:
while(!feof(picture)){
n = fread(Sbuf, sizeof(char), siz, picture);
if (n > 0) { /* only send what has been read */
if((n = send(sock, Sbuf, siz, 0)) < 0) /* or (better?) send(sock, Sbuf, n, 0) */
{
perror("send_data()");
exit(errno);
}
}
/* memset(Sbuf, 0, sizeof(Sbuf)); useless for binary data */
}
Same in server part:
if ((n = recv(sock, Rbuffer, siz, 0)) < 0){
perror("recv_size()");
exit(errno);
}
cout << "Converting byte array to image" << endl;
FILE *image;
image = fopen("recu.jpg", "w");
fwrite(Rbuffer, sizeof(char), siz, image);
fclose(image);
And there is a last possibility of error, at least if you are on a platform that makes a difference between text and binary file like Windows is that you forget to open the files in binary mode, which could break the jpg image. Because on windows for a binary file, byte 0x10 is seen as the new line (\n') and written as 2 bytes 0x0d 0x10 (\r\n).
So you must open the input file in rb mode and the output file in wb mode.
Solved :
All the correction that Serge Ballesta were right. But the problem was in the way I was opening my files.
You need to open the file in binary mode ("rb" for reading, "wb" for writing), not the default text mode.
Client :
picture = fopen("C:\\Users\\n.b\\Desktop\\c++\\TCP\\tcp_client_image_pp\\test.jpg", "rb");
Server :
image = fopen("recu.jpg", "wb");
That was the main problem. Thank you.
I want to read depth frame at 640x480.
I am using windows 8.1 64bit, openni2 32bit, kinect:PSMP05000,PSCM04900(PrimeSense)
I take code reference from here:
cannot set VGA resolution
Simple Read
Combined to this code:
main.cpp
OniSampleUtilities.h
SimpleRead.vcxproj
should be compiled if you install openni2 32bit from here:
OpeniNI 2
#include "iostream"
#include "OpenNI.h"
#include "OniSampleUtilities.h"
#define SAMPLE_READ_WAIT_TIMEOUT 2000 //2000ms
using namespace openni;
using namespace std;
int main()
{
Status rc = OpenNI::initialize();
if (rc != STATUS_OK)
{
cout << "Initialize failed:" << endl << OpenNI::getExtendedError() << endl;
return 1;
}
Device device;
rc = device.open(ANY_DEVICE);
if (rc != STATUS_OK)
{
cout << "Couldn't open device" << endl << OpenNI::getExtendedError() << endl;
return 2;
}
VideoStream depth;
if (device.getSensorInfo(SENSOR_DEPTH) != NULL)
{
rc = depth.create(device, SENSOR_DEPTH);
if (rc != STATUS_OK)
{
cout << "Couldn't create depth stream" << endl << OpenNI::getExtendedError() << endl;
return 3;
}
}
rc = depth.start();
if (rc != STATUS_OK)
{
cout << "Couldn't start the depth stream" << endl << OpenNI::getExtendedError() << endl;
return 4;
}
VideoFrameRef frame;
// set resolution
// depth modes
cout << "Depth modes" << endl;
const openni::SensorInfo* sinfo = device.getSensorInfo(openni::SENSOR_DEPTH); // select index=4 640x480, 30 fps, 1mm
const openni::Array< openni::VideoMode>& modesDepth = sinfo->getSupportedVideoModes();
for (int i = 0; i<modesDepth.getSize(); i++) {
printf("%i: %ix%i, %i fps, %i format\n", i, modesDepth[i].getResolutionX(), modesDepth[i].getResolutionY(),
modesDepth[i].getFps(), modesDepth[i].getPixelFormat()); //PIXEL_FORMAT_DEPTH_1_MM = 100, PIXEL_FORMAT_DEPTH_100_UM
}
rc = depth.setVideoMode(modesDepth[0]);
if (openni::STATUS_OK != rc)
{
cout << "error: depth fromat not supprted..." << endl;
}
system("pause");
while (!wasKeyboardHit())
{
int changedStreamDummy;
VideoStream* pStream = &depth;
rc = OpenNI::waitForAnyStream(&pStream, 1, &changedStreamDummy, SAMPLE_READ_WAIT_TIMEOUT);
if (rc != STATUS_OK)
{
cout << "Wait failed! (timeout is " << SAMPLE_READ_WAIT_TIMEOUT << " ms)" << endl << OpenNI::getExtendedError() << endl;
continue;
}
rc = depth.readFrame(&frame);
if (rc != STATUS_OK)
{
cout << "Read failed!" << endl << OpenNI::getExtendedError() << endl;
continue;
}
if (frame.getVideoMode().getPixelFormat() != PIXEL_FORMAT_DEPTH_1_MM && frame.getVideoMode().getPixelFormat() != PIXEL_FORMAT_DEPTH_100_UM)
{
cout << "Unexpected frame format" << endl;
continue;
}
DepthPixel* pDepth = (DepthPixel*)frame.getData();
int middleIndex = (frame.getHeight()+1)*frame.getWidth()/2;
printf("[%08llu] %8d\n", (long long)frame.getTimestamp(), pDepth[middleIndex]);
}
depth.stop();
depth.destroy();
device.close();
OpenNI::shutdown();
return 0;
}
There is 6 mode of operation:
0: 320x240, 30 fps, 100 format
1: 320x240, 30 fps, 101 format
2: 320x240, 60 fps, 100 format
3: 320x240, 60 fps, 101 format
4: 640x480, 30 fps, 100 format
5: 640x480, 30 fps, 101 format
It can read only from modes=0-3.
At mode 4,5 i get timeout.
How i can read depth frame at 640x480?
Thanks for the help,
Tal.
====================================================
new information:
I use also this line, and i get the same results:
const openni::SensorInfo* sinfo = &(depth.getSensorInfo());
This line never execute at any mode:
cout << "error: depth fromat not supprted..." << endl;
At mode 4,5 I always get this line execute:
cout << "Wait failed! (timeout is " << SAMPLE_READ_WAIT_TIMEOUT << " ms)" << endl << OpenNI::getExtendedError() << endl;
I think maybe it a bug at openni2.
At openni1, I can read depth image at 640x480, in the same computer,os and device.
Maybe I am wrong, but I am almost sure that the problem is the order that you are doing it.
I think you should change it before depth.start() and after depth.create(device, SENSOR_DEPTH)
If I remember correctly, once it has started you may bot change the resolution of the stream.
So it should be something like this
...
if (device.getSensorInfo(SENSOR_DEPTH) != NULL)
{
rc = depth.create(device, SENSOR_DEPTH);
if (rc != STATUS_OK)
{
cout << "Couldn't create depth stream" << endl << OpenNI::getExtendedError() << endl;
return 3;
}
}
// set resolution
// depth modes
cout << "Depth modes" << endl;
const openni::SensorInfo* sinfo = device.getSensorInfo(openni::SENSOR_DEPTH);
const openni::Array< openni::VideoMode>& modesDepth = sinfo->getSupportedVideoModes();
rc = depth.setVideoMode(modesDepth[0]);
if (openni::STATUS_OK != rc)
{
cout << "error: depth fromat not supprted..." << endl;
}
rc = depth.start();
if (rc != STATUS_OK)
{
cout << "Couldn't start the depth stream" << endl << OpenNI::getExtendedError() << endl;
return 4;
}
VideoFrameRef frame;
...
I hope that this helps you, if not, please add a comment. I have a similar code working in the git repository I show you the other day, tested with a PrimeSense carmine camera.
In my case (Asus Xtion PRO in a USB 3.0 port, OpenNI2, Windows 8.1), it seems there are something wrong with OpenNI2 (or its driver) that prevents me from changing the resolution in the code. NiViewer simple hangs or has frame rates drop significantly if the color resolution is set to 640x480.
However, on Windows, I managed to change the resolution by changing the settings in PS1080.ini in OpenNI2/Tools/OpenNI2/Drivers folder. In the ini file, for Asus, make sure
UsbInterface = 2
is enabled. By default it's zero. Then set Resolution = 1 for the depth and image sections.
My Asus Xtion firmware is v5.8.22.
I've tried the method #api55 mentioned and it works. The code and result are in the following.
But there is a problem when I make the similar change to the OpenNI sample code "SampleViewer" so that I can change the resolution free. When I set the resolution to 320*240 all is well. However, when I change it to 640*480, although the program still read frames in (at a apparently slower rate), the program display just get stuck.
2015-12-27 15:15:32
Then I test the aforementioned sample viewer with a kinect 1.0 depth camera. Since the color camera has a resolution no less than 640*480, I cannot experiment the resolution of 320*240. But the program works well with kinect 1.0 at a resolution of 640*480. In conclusion, I think that there must be some problem with the ASUS Xtion camera.
#include <iostream>
#include <cstdio>
#include <vector>
#include <OpenNI.h>
#include "OniSampleUtilities.h"
#pragma comment(lib, "OpenNI2")
#define SAMPLE_READ_WAIT_TIMEOUT 2000 //2000ms
using namespace openni;
using namespace std;
int main()
{
Status rc = OpenNI::initialize();
if (rc != STATUS_OK)
{
printf("Initialize failed:\n%s\n", OpenNI::getExtendedError());
return 1;
}
Device device;
openni::Array<openni::DeviceInfo> deviceInfoList;
OpenNI::enumerateDevices(&deviceInfoList);
for (int i = 0; i < deviceInfoList.getSize(); i++)
{
printf("%d: Uri: %s\n"
"Vendor: %s\n"
"Name: %s\n", i, deviceInfoList[i].getUri(), deviceInfoList[i].getVendor(), deviceInfoList[i].getName());
}
rc = device.open(deviceInfoList[0].getUri());
if (rc != STATUS_OK)
{
printf("Counldn't open device\n%s\n", OpenNI::getExtendedError());
return 2;
}
VideoStream depth;
// set resolution
// depth modes
printf("\nDepth modes\n");
const openni::SensorInfo* sinfo = device.getSensorInfo(openni::SENSOR_DEPTH); // select index=4 640x480, 30 fps, 1mm
if (sinfo == NULL)
{
printf("Couldn't get device info\n%s\n", OpenNI::getExtendedError());
return 3;
}
rc = depth.create(device, SENSOR_DEPTH);
if (rc != STATUS_OK)
{
printf("Couldn't create depth stream\n%s\n", OpenNI::getExtendedError());
return 4;
}
const openni::Array< openni::VideoMode>& modesDepth = sinfo->getSupportedVideoModes();
vector<int> item;
for (int i = 0; i < modesDepth.getSize(); i++) {
printf("%i: %ix%i, %i fps, %i format\n", i, modesDepth[i].getResolutionX(), modesDepth[i].getResolutionY(),
modesDepth[i].getFps(), modesDepth[i].getPixelFormat()); //PIXEL_FORMAT_DEPTH_1_MM = 100, PIXEL_FORMAT_DEPTH_100_UM
if (modesDepth[i].getResolutionX() == 640 && modesDepth[i].getResolutionY() == 480)
item.push_back(i);
}
int item_idx = item[0];
printf("Choose mode %d\nWidth: %d, Height: %d\n", item_idx, modesDepth[item_idx].getResolutionX(), modesDepth[item_idx].getResolutionY());
rc = depth.setVideoMode(modesDepth[item_idx]);
if (rc != STATUS_OK)
{
printf("error: depth format not supported...\n");
return 5;
}
rc = depth.start();
if (rc != STATUS_OK)
{
printf("Couldn't start the depth stream\n%s\n", OpenNI::getExtendedError());
return 6;
}
VideoFrameRef frame;
printf("\nCurrent resolution:\n");
printf("Width: %d Height: %d\n", depth.getVideoMode().getResolutionX(), depth.getVideoMode().getResolutionY());
system("pause");
while (!wasKeyboardHit())
{
int changedStreamDummy;
VideoStream* pStream = &depth;
rc = OpenNI::waitForAnyStream(&pStream, 1, &changedStreamDummy, SAMPLE_READ_WAIT_TIMEOUT);
if (rc != STATUS_OK)
{
printf("Wait failed! (timeout is \" %d \" ms)\n%s\n", SAMPLE_READ_WAIT_TIMEOUT, OpenNI::getExtendedError());
continue;
}
rc = depth.readFrame(&frame);
if (rc != STATUS_OK)
{
printf("Read failed!\n%s\n", OpenNI::getExtendedError());
continue;
}
if (frame.getVideoMode().getPixelFormat() != PIXEL_FORMAT_DEPTH_1_MM && frame.getVideoMode().getPixelFormat() != PIXEL_FORMAT_DEPTH_100_UM)
{
printf("Unexpected frame format\n");
continue;
}
DepthPixel* pDepth = (DepthPixel*)frame.getData();
int middleIndex = (frame.getHeight() + 1)*frame.getWidth() / 2;
printf("[%08llu] %8d\n", (long long)frame.getTimestamp(), pDepth[middleIndex]);
printf("Width: %d Height: %d\n", frame.getWidth(), frame.getHeight());
}
depth.stop();
depth.destroy();
device.close();
OpenNI::shutdown();
return 0;
}
I had the same problem, but now solved it by referencing NiViewer example in OpenNI2. Apparently after you start the stream, either depth or color, you have to stop it to change the resolution and then start
const openni::SensorInfo* sinfo = device.getSensorInfo(openni::SENSOR_DEPTH);
const openni::Array< openni::VideoMode>& modesDepth = sinfo->getSupportedVideoModes();
depth.stop();
rc = depth.setVideoMode(modesDepth[4]);
depth.start();
I confirmed that this works on Asus Xtion on OpenNI2.
Hope this helps!
Final conclusion:
Actually, it is Xtion's problem itself (maybe related to hardware).
If you want just one of depth or color to be 640*480, and the other to be 320*240, it'll work. I can post my code if you want.
Details
Some of the answers above made a mistake: even the NiViewer.exe itself doesn't allow a depth 640*480 and color 640*480 at the same time.
Note: don't be misled by the visualization of NiViewer.exe, the video stream displayed is large but actually it does not mean 640*480. Actually it is initialsed with
depth: 320*240
color: 320*240
When you set either of the mode to 640*480, it is still works, which is
depth: 640*480
color: 320*240
or
depth: 320*240
color: 640*480
But when you want both of them to be the highest resolution:
depth: 640*480
color: 640*480
The viewer program starts encountering acute frame drop in the depth mode (in my case), but since the viewer retrieves the depth frame in an un-block way (the default code is written in a block way), you still see the color updates normally, while the depth updates every two seconds or even more.
To conclude
You could only set either of depth or color to be 640*480, and the other to be 320*240.
I have two functions:
a internet-socket function which gets mp3-data and writes it to file ,
a function which decodes mp3-files.
However, I would rather decode the data, which is currently written to disk, be decoded in-memory by the decode function.
My decode function looks like this, and it is all initialized via
avformat_open_input(AVCodecContext, filename, NULL, NULL)
How can I read in the AVCodecContext without a filename, and instead using only the in-memory buffer?
I thought I would post some code to illustrate how to achieve this, I have tried to comment but am pressed for time, however it should all be relatively straightforward stuff. Return values are based on interpolation of the associated message into a hex version of 1337 speak converted to decimal values, and I have tried to keep it as light as possible in tone:)
#include <iostream>
extern "C"
{
#include <libavcodec/avcodec.h>
#include <libavformat/avformat.h>
#include <libavutil/avutil.h>
};
std::string tooManyChannels = "The audio stream (and its frames) has/have too many channels to properly fit in\n to frame->data. Therefore, to access the audio data, you need to use\nframe->extended_data to access the audio data."
"It is a planar store, so\neach channel is in a different element.\n"
" E.G.: frame->extended_data[0] has the data for channel 1\n"
" frame->extended_data[1] has the data for channel 2\n"
"And so on.\n";
std::string nonPlanar = "Either the audio data is not planar, or there is not enough room in\n"
"frame->data to store all the channel data. Either use\n"
"frame->data\n or \nframe->extended_data to access the audio data\n"
"both should just point to the same data in this instance.\n";
std::string information1 = "If the frame is planar, each channel is in a separate element:\n"
"frame->data[0]/frame->extended_data[0] contains data for channel 1\n"
"frame->data[1]/frame->extended_data[1] contains data for channel 2\n";
std::string information2 = "If the frame is in packed format( and therefore not planar),\n"
"then all the data is contained within:\n"
"frame->data[0]/frame->extended_data[0] \n"
"Similar to the manner in which some image formats have RGB(A) pixel data packed together,\n"
"rather than containing separate R G B (and A) data.\n";
void printAudioFrameInfo(const AVCodecContext* codecContext, const AVFrame* frame)
{
/*
This url: http://ffmpeg.org/doxygen/trunk/samplefmt_8h.html#af9a51ca15301871723577c730b5865c5
contains information on the type you will need to utilise to access the audio data.
*/
// format the tabs etc. in this string to suit your font, they line up for mine but may not for yours:)
std::cout << "Audio frame info:\n"
<< "\tSample count:\t\t" << frame->nb_samples << '\n'
<< "\tChannel count:\t\t" << codecContext->channels << '\n'
<< "\tFormat:\t\t\t" << av_get_sample_fmt_name(codecContext->sample_fmt) << '\n'
<< "\tBytes per sample:\t" << av_get_bytes_per_sample(codecContext->sample_fmt) << '\n'
<< "\tPlanar storage format?:\t" << av_sample_fmt_is_planar(codecContext->sample_fmt) << '\n';
std::cout << "frame->linesize[0] tells you the size (in bytes) of each plane\n";
if (codecContext->channels > AV_NUM_DATA_POINTERS && av_sample_fmt_is_planar(codecContext->sample_fmt))
{
std::cout << tooManyChannels;
}
else
{
stc::cout << nonPlanar;
}
std::cout << information1 << information2;
}
int main()
{
// You can change the filename for any other filename/supported format
std::string filename = "../my file.ogg";
// Initialize FFmpeg
av_register_all();
AVFrame* frame = avcodec_alloc_frame();
if (!frame)
{
std::cout << "Error allocating the frame. Let's try again shall we?\n";
return 666; // fail at start: 66 = number of the beast
}
// you can change the file name to whatever yo need:)
AVFormatContext* formatContext = NULL;
if (avformat_open_input(&formatContext, filename, NULL, NULL) != 0)
{
av_free(frame);
std::cout << "Error opening file " << filename<< "\n";
return 800; // cant open file. 800 = Boo!
}
if (avformat_find_stream_info(formatContext, NULL) < 0)
{
av_free(frame);
avformat_close_input(&formatContext);
std::cout << "Error finding the stream information.\nCheck your paths/connections and the details you supplied!\n";
return 57005; // stream info error. 0xDEAD in hex is 57005 in decimal
}
// Find the audio stream
AVCodec* cdc = nullptr;
int streamIndex = av_find_best_stream(formatContext, AVMEDIA_TYPE_AUDIO, -1, -1, &cdc, 0);
if (streamIndex < 0)
{
av_free(frame);
avformat_close_input(&formatContext);
std::cout << "Could not find any audio stream in the file. Come on! I need data!\n";
return 165; // no(0) (a)udio s(5)tream: 0A5 in hex = 165 in decimal
}
AVStream* audioStream = formatContext->streams[streamIndex];
AVCodecContext* codecContext = audioStream->codec;
codecContext->codec = cdc;
if (avcodec_open2(codecContext, codecContext->codec, NULL) != 0)
{
av_free(frame);
avformat_close_input(&formatContext);
std::cout << "Couldn't open the context with the decoder. I can decode but I need to have something to decode.\nAs I couldn't find anything I have surmised the decoded output is 0!\n (Well can't have you thinking I am doing nothing can we?\n";
return 1057; // cant find/open context 1057 = lost
}
std::cout << "This stream has " << codecContext->channels << " channels with a sample rate of " << codecContext->sample_rate << "Hz\n";
std::cout << "The data presented in format: " << av_get_sample_fmt_name(codecContext->sample_fmt) << std::endl;
AVPacket readingPacket;
av_init_packet(&readingPacket);
// Read the packets in a loop
while (av_read_frame(formatContext, &readingPacket) == 0)
{
if (readingPacket.stream_index == audioStream->index)
{
AVPacket decodingPacket = readingPacket;
// Audio packets can have multiple audio frames in a single packet
while (decodingPacket.size > 0)
{
// Try to decode the packet into a frame(s)
// Some frames rely on multiple packets, so we have to make sure the frame is finished
// before utilising it
int gotFrame = 0;
int result = avcodec_decode_audio4(codecContext, frame, &gotFrame, &decodingPacket);
if (result >= 0 && gotFrame)
{
decodingPacket.size -= result;
decodingPacket.data += result;
// et voila! a decoded audio frame!
printAudioFrameInfo(codecContext, frame);
}
else
{
decodingPacket.size = 0;
decodingPacket.data = nullptr;
}
}
}
// You MUST call av_free_packet() after each call to av_read_frame()
// or you will leak so much memory on a large file you will need a memory-plumber!
av_free_packet(&readingPacket);
}
// Some codecs will cause frames to be buffered in the decoding process.
// If the CODEC_CAP_DELAY flag is set, there can be buffered frames that need to be flushed
// therefore flush them now....
if (codecContext->codec->capabilities & CODEC_CAP_DELAY)
{
av_init_packet(&readingPacket);
// Decode all the remaining frames in the buffer
int gotFrame = 0;
while (avcodec_decode_audio4(codecContext, frame, &gotFrame, &readingPacket) >= 0 && gotFrame)
{
// Again: a fully decoded audio frame!
printAudioFrameInfo(codecContext, frame);
}
}
// Clean up! (unless you have a quantum memory machine with infinite RAM....)
av_free(frame);
avcodec_close(codecContext);
avformat_close_input(&formatContext);
return 0; // success!!!!!!!!
}
Hope this helps. Let me know if you need more info, and I will try and help out:)
There is also some very good tutorial information available at dranger.com which you may find useful.
Preallocate the format context and set its pb field as suggested in the note of avformat_open_input() documentation.
.
Next iteration of my question:
Thank you for your inputs, it has helped me to understand a little bit more about the Frame and inputSamples utility.
I’ve done modifications to my source code with the new knowledge you’ve given me. But I still have problems, so I might not have understood fully what you meant.
Here is my OpenFile function, sorry for the name but I’ll refactor later; when it’ll work =)
//-----------------------------------------------------------------------------
/*
This Function Open a File containing the Audio, Binary, Data.
*///___________________________________________________________________________
const short* OpenFile(const char* fileName, long& fileSize, WavFormat* wav)
{
// ouvre le fichier
ifstream file;
file.open((char*)fileName, ios::binary|ios::in);
if (file.good())
{
// Read the WAV's Header
wav = CheckWavHeader(file, wav);
cout << "chunkID: " << wav->chunkID <<'\n';
cout << "chunkSize: " << wav->chunkSize <<'\n';
cout << "format: " << wav->format <<'\n';
cout << "subChunk1ID: " << wav->subChunk1ID <<'\n';
cout << "subChunk1Size: " << wav->subChunk1Size <<'\n';
cout << "audioFormat: " << wav->audioFormat <<'\n'; // audioFormat == 1, alors PCM 16bits
cout << "numChannels: " << wav->numChannels <<'\n';
cout << "sampleRate: " << wav->sampleRate <<'\n';
cout << "byteRate: " << wav->byteRate <<'\n';
cout << "blockAlign: " << wav->blockAlign <<'\n';
cout << "bitsPerSample: " << wav->bitsPerSample <<'\n';
cout << "subChunk2ID: " << wav->subChunk2ID <<'\n';
cout << "subChunk2Size: " << wav->subChunk2Size <<'\n';
// Get the file’s size
file.seekg(0L, ios::end);
fileSize = ((long)file.tellg() - DATA_POS);
file.seekg(DATA_POS, ios::beg); // back to the data.
// Read the Data into the Buffer
uint nbSamples = fileSize / sizeof(short);
short* inputArray = new short[nbSamples];
file.read((char*)inputArray, fileSize);
// Close the file and return the Data
file.close();
return (const short*)inputArray;
}
else
{
exit(-1);
}
}
I’m opening the file, checking its size, create a short buffer and read the wav’s data into the short buffer and finally I return it.
In the main, for now I commented the G711 decoder.
When I run the application, the faacEncOpen gives me 2048 for inputSamples (it’s logic since I have 2 channels in the Wav’s file for a FRAME_LEN of 1024).
So if I understood correctly, 1 Frame == 2048 samples for my application. So for each Frame I call the faacEncEncode, I give the tmpInputBuffer that is a buffer of the same size as inputSamples at the inputBuffer[i * inputSamples] index.
//-----------------------------------------------------------------------------
/*
The Main entry Point of the Application
*///_____________________________________________________________________________
int main()
{
// Get the File's Data
WavFormat* wav = new WavFormat;
long fileSize;
const short* fileInput = OpenFile("audioTest.wav", fileSize, wav);
// G711 mu-Law Decoder
//MuLawDecoder* decoder = new MuLawDecoder();
//short* inputBuffer = decoder->MuLawDecode_shortArray((byte*)fileInput, (int)nbChunk);
short* inputBuffer = (short*)fileInput;
// Info for FAAC
ulong sampleRate = wav->sampleRate;
uint numChannels = wav->numChannels;
ulong inputSamples;
ulong maxOutputBytes;
// Ouvre l'Encodeur et assigne la Configuration.
faacEncHandle hEncoder = faacEncOpen(sampleRate, numChannels, &inputSamples, &maxOutputBytes);
faacEncConfigurationPtr faacConfig = faacEncGetCurrentConfiguration(hEncoder);
faacConfig->inputFormat = FAAC_INPUT_16BIT;
faacConfig->bitRate = 64000;
int result = faacEncSetConfiguration(hEncoder, faacConfig);
/*Input Buffer and Output Buffer*/
byte* outputBuffer = new byte[maxOutputBytes];
int nbBytesWritten = 0;
Sink* sink = new Sink();
uint nbFrame = fileSize / inputSamples;
int32_t* tmpInputBuffer = new int32_t[inputSamples];
for (uint i = 0; i < nbFrame; i++)
{
strncpy((char*)tmpInputBuffer, (const char*)&inputBuffer[i * inputSamples], inputSamples);
nbBytesWritten = faacEncEncode(hEncoder, tmpInputBuffer, inputSamples, outputBuffer, maxOutputBytes);
cout << 100.0 * (float)i / nbFrame << "%\t nbBytesWritten = " << nbBytesWritten << "\n";
if (nbBytesWritten > 0)
{
sink->AddAACStream(outputBuffer, nbBytesWritten);
}
}
sink->WriteToFile("output.aac");
// Close AAC Encoder
faacEncClose(hEncoder);
// Supprimer tous les pointeurs
delete sink;
//delete decoder;
delete[] fileInput;
//delete[] inputBuffer;
delete[] outputBuffer;
delete[] tmpInputBuffer;
system("pause");
return 0;
}
When the output Data is Dumped into an .acc file (as RAW AAC), I use the application mp4muxer.exe to create an .mp4 file to listen to the final converted sound. But the sound is not good at all...
I'm wondering if there is something I'm not seeing or do not unserstand that I should.
Thank you in advance for your useful inputs.
Each call to faacEncEncode encodes inputSamples samples, not just one. Your main loop should read that many samples from the WAV file into the input buffer, then call faacEncEncode once for that buffer, and finally write the output buffer to the AAC file.
It's possible that I've misunderstood what you're doing (if so, it would be useful to know: (1) What's the OpenFile function you're calling, and does it (despite its name) actually read the file as well as opening it? (2) How is inputBuffer set up?) but:
faacEncEncode expects to be given a whole frame's worth of samples. A frame is the number of samples you got passed back in inputSamples when you called faacEncOpen. (You can give it less than a whole frame if you've reached the end of the input, of course.)
So you're getting 460 and 539 bytes for each of two frames -- not for 16 bits in each case. And it looks as if your input-data pointers are actually offset by only one sample each time, so you're handing it badly overlapping frames. (And the wrong number of them; nbChunk is not the number of frames you have.)