Convert 16 bit stereo sound to 16 bit mono sound - c++

I'm trying to convert 16 bit stereo sound from a WAVE file to 16 bit mono sound, but I'm having some struggle. I've tried to convert 8 bit stereo sound to mono and it's working great. Here's the piece of code for that:
if( bitsPerSample == 8 )
{
dataSize /= 2;
openALFormat = AL_FORMAT_MONO8;
for( SizeType i = 0; i < dataSize; i++ )
{
pData[ i ] = static_cast<Uint8>(
( static_cast<Uint16>( pData[ i * 2 ] ) +
static_cast<Uint16>( pData[ i * 2 + 1 ] ) ) / 2
);
}
But, now I'm trying to do pretty much the same with 16 bit audio, but I just can't get it to work. I can just hear some kind of weird noise. I've tried to set "monoSample" to "left"(Uint16 monoSample = left;) and the audio data from that channel works very well. The right channel as well. Can anyone of you see what I'm doing wrong?
Here's the code(pData is an array of bytes):
if( bitsPerSample == 16 )
{
dataSize /= 2;
openALFormat = AL_FORMAT_MONO16;
for( SizeType i = 0; i < dataSize / 2; i++ )
{
Uint16 left = static_cast<Uint16>( pData[ i * 4 ] ) |
( static_cast<Uint16>( pData[ i * 4 + 1 ] ) << 8 );
Uint16 right = static_cast<Uint16>( pData[ i * 4 + 2 ] ) |
( static_cast<Uint16>( pData[ i * 4 + 3 ] ) << 8 );
Uint16 monoSample = static_cast<Uint16>(
( static_cast<Uint32>( left ) +
static_cast<Uint32>( right ) ) / 2
);
// Set the new mono sample.
pData[ i * 2 ] = static_cast<Uint8>( monoSample );
pData[ i * 2 + 1 ] = static_cast<Uint8>( monoSample >> 8 );
}
}

In a 16 bit stereo WAV file, each sample is 16 bits, and the samples are interleaved. I'm not sure why you're using a bitwise OR, but you can just retrieve the data directly without having to shift. The below non-portable code (assumes sizeof(short) == 2) illustrates this.
unsigned size = header.data_size;
char *data = new char[size];
// Read the contents of the WAV file in to data
for (unsigned i = 0; i < size; i += 4)
{
short left = *(short *)&data[i];
short right = *(short *)&data[i + 2];
short monoSample = (int(left) + right) / 2;
}
Also, while 8 bit WAV files are unsigned, 16 bit WAV files are signed. To average them, make sure you store it in an appropriately sized signed type. Note that one of the samples is promoted to an int temporarily to prevent overflow.
As has been pointed out in the comments below by Stix, simple averaging may not give the best results. Your mileage may vary.
In addition, Greg Hewgill correctly noted that this assumes that the machine is little-endian.

Related

Implement a function that blends two colors encoded with RGB565 using Alpha blending

I am trying to implement a function that blends two colors encoded with RGB565 using Alpha blending
Crgb565 = (1-a)Argb565 + a*Brgb565
Where a is the alpha parameter, and the alpha blending value of 0.0-1.0 is mapped to an unsigned char value on the range 0-32.
we can choose to use a five bit representation for a instead, thus restricting it to the range of 0-31 (effectively mapping to an alpha blending value of 0.0-0.96875).
Following code I am trying to implement, can you please suggest better way wrt less temp variable , memory optimization (number of multiplications and required memory accesses ),Is my logic for alpha bending is correct? I am not getting correct result/expected output, Seems like I am missing something, please review the code, Every suggest is appreciated, have some doubt based on alpha parameter. I have put my doubts in code comment section. Is there any way to shortening the alpha blending equations(division operation)?
=====================================================
unsigned short blend_rgb565(unsigned short A, unsigned short B, unsigned char Alpha)
{
unsigned short res = 0;
// Alpha converted from [0..255] to [0..31] (8 bit to 5 bit)
/* I want the alpha parameter (0-32), do i need to add something in Alpha before right shift?? */
Alpha = Alpha >> 3;
// Split Image A into R, G, B components
/*Do I need to take it as unsigned short or uint8_t also work fine ??*/
unsigned short A_r = A >> 11;
unsigned short A_g = (A >> 5) & ((1u << 6) - 1); // ((1u << 6) - 1) --> 00000000 00111111
unsigned short A_b = A & ((1u << 5) - 1); // ((1u << 5) - 1) --> 00000000 00011111
// Split Image B into R, G, B components
unsigned short B_r = B >> 11;
unsigned short B_g = (B >> 5) & ((1u << 6) - 1);
unsigned short B_b = B & ((1u << 5) - 1);
// Alpha blend components
/*Do I need to use 255(8 bit) instead of 32(5 bit), Why we are dividing by it , I have taken the ref from internet , but need little bit more clarification ??*/
unsigned short uiC_r = (A_r * Alpha + B_r * (32 - Alpha)) / 32;
unsigned short uiC_g = (A_g * Alpha + B_g * (32 - Alpha)) / 32;
unsigned short uiC_b = (A_b * Alpha + B_b * (32 - Alpha)) / 32;
// Pack result
res= (unsigned short) ((uiC_r << 11) | (uiC_g << 5) | uiC_b);
return res;
}
=====================
EDIT:
Adding method 2 ,is this approach is correct ?
Method 2:
// rrrrrggggggbbbbb
#define RB_MASK 63519 // 0b1111100000011111 --> hex :F81F
#define G_MASK 2016 // 0b0000011111100000 --> hex :07E0
#define RB_MUL_MASK 2032608 // 0b111110000001111100000 --> hex :1F03E0
#define G_MUL_MASK 64512 // 0b000001111110000000000 --> hex :FC00
unsigned short blend_rgb565(unsigned short A,unsigned short B,unsigned char Alpha) {
// Alpha converted from [0..255] to [0..31]
Alpha = Alpha >> 3
uint8_t beta = 32 - Alpha;
// so (0..32)*Alpha + (0..32)*beta always in 0..32
return (unsigned short)
(
(
( ( Alpha * (uint32_t)( A & RB_MASK ) + beta * (uint32_t)( B & RB_MASK )) & RB_MUL_MASK )
|
( ( Alpha * ( A & G_MASK ) + beta * ( B & G_MASK )) & G_MUL_MASK )
)
>> 5 // removing the alpha component 5 bit
);
}
It's possible to reduce the multiplies from 6 to 2 if you space out the RGB values into 2 32-bit integers before multiplying:
unsigned short blend_rgb565(unsigned short A, unsigned short B, unsigned char Alpha)
{
unsigned short res = 0;
// Alpha converted from [0..255] to [0..31] (8 bit to 5 bit)
Alpha = Alpha >> 3;
// Alpha = (Alpha + (Alpha >> 5)) >> 3; // map from 0-255 to 0-32 (if Alpha is unsigned short or larger)
// Space out A and B from RRRRRGGGGGGBBBBB to 00000RRRRR00000GGGGGG00000BBBBB
// 31 = 11111 binary
// 63 = 111111 binary
unsigned int A32 = (unsigned int)A;
unsigned int A_spaced = A32 & 31; // B
A_spaced |= (A32 & (63 << 5)) << 5; // G
A_spaced |= (A32 & (31 << 11)) << 11; // R
unsigned int B32 = (unsigned int)B;
unsigned int B_spaced = B32 & 31; // B
B_spaced |= (B32 & (63 << 5)) << 5; // G
B_spaced |= (B32 & (31 << 11)) << 11; // R
// multiply and add the alpha to give a result RRRRRrrrrrGGGGGGgggggBBBBBbbbbb,
// where RGB are the most significant bits we want to keep
unsigned int C_spaced = (A_spaced * Alpha) + (B_spaced * (32 - Alpha));
// remap back to RRRRRGGGGGBBBBB
res = (unsigned short)(((C_spaced >> 5) & 31) + ((C_spaced >> 10) & (63 << 5)) + ((C_spaced >> 16) & (31 << 11)));
return res;
}
You need to profile this to see if it is faster, it assumes that multiplications you save are slower than the extra bit-manipulations you replace them with.
can you please suggest better way wrt less temp variable
There is no advantage to remove temporary variables from the implementation. When you compile with optimizations turned on (e.g. -O2 or /O2) those temp variables will get optimized away.
Two adjustments I would make to your code:
Use uint16_t instead of unsigned short. For most platforms, it won't matter since sizeof(uint16_t)==sizeof(unsigned short), but it helps to be definitive.
No point in converting alpha from an 8-bit value to a 5-bit value. You'll get better accuracy with blending if you let alpha have the full range
Some of your bit-shifting looks weird. It might work. But I use a simpler approach.
Here's an adjustment to your implementation:
#include <stdint.h>
#define MAKE_RGB565(r, g, b) ((r << 11) | (g << 5) | (b))
uint16_t blend_rgb565(uint16_t a, uint16_t b, uint8_t Alpha)
{
const uint8_t invAlpha = 255 - Alpha;
uint16_t A_r = a >> 11;
uint16_t A_g = (a >> 5) & 0x3f;
uint16_t A_b = a & 0x1f;
uint16_t B_r = b >> 11;
uint16_t B_g = (b >> 5) & 0x3f;
uint16_t B_b = b & 0x1f;
uint32_t C_r = (A_r * invAlpha + B_r * Alpha) / 255;
uint32_t C_g = (A_g * invAlpha + B_g * Alpha) / 255;
uint32_t C_b = (A_b * invAlpha + B_b * Alpha) / 255;
return MAKE_RGB565(C_r, C_g, C_b);
}
But the bigger issue is that this function works on exactly one one pair of pixel colors. If you are invoking this function across an entire image or pair of images, the overhead of using the function call is going to be a major performance issue - even with compiler optimizations and inlining. So if you are calling this function row x col times, you should probably manually inline the code into your loop that is enumerating over every pixel on an image (or pair of images).
In the same vein as #samgak's answer, you can implement more efficiently on a 64 bits architecture by "post-masking", as follows:
rrrrrggggggbbbbb
Replicate to a long long (by shifting or mapping the long long to four shorts)
---------------- rrrrrggggggbbbbb rrrrrggggggbbbbb rrrrrggggggbbbbb
Mask out the useless bits
---------------- rrrrr----------- -----gggggg----- -----------bbbbb
Multiply by α
-----------rrrrr rrrrr----------- ggggggggggg----- ------bbbbbbbbbb
Mask out the low order bits
-----------rrrrr ---------------- gggggg---------- ------bbbbb-----
Pack
rrrrrgggggbbbbb
Another saving is possible by rewriting
(1 - α) X + α Y
as
X + α (Y - X)
(or X - α (X - Y) to avoid negatives). This spares a multiply (at the expense of a comparison).
Update:
The "saving" above cannot work because the negatives should be handled component-wise.

Converting 16 bit unsigned int array to 32 bit float array

I am using directx 9 with 64bit render targets...I need to read the data on the render target surfaces. Each color component( a,r,g,b ) is encoded with 2 bytes( or 16bits x 4 = 64 ). How do I convert each 16 bit color component to a 32 bit floating point variable? Here is what I've tried:
BYTE *pData = ( BYTE* )renderTargetData;
for( UINT y = 0; y < Height; ++y )
{
for( UINT x = 0; x < width; ++x )
{
// declare 4component vector to hold 4 floats
D3DXVECTOR4 vColor;
// convert the pixel color from 16 to 32 bits
D3DXFloat16To32Array( ( FLOAT* )&vColor, ( D3DXFLOAT16* )&pData[ y + 8 * x ], 4 );
}
}
For some reason this is incorrect...In one case after conversion, where the actual renderTargetData for one pixel is ( 0, 0, 0, 65535 ), I get this result: ( 0, 0, 0, -131008.00 ).
In general, converting an integer v from integer in range [0..n] to float in range [0.0..1.0] is:
float f = v/(float)n;
So, in your case, a loop that does:
vColor.x = (pData[ y + 4 * x ])/65535.0f;
vColor.y = (pData[ y + 4 * x + 1 ])/65535.0f;
// ... etc.
should work, if we change the BYTE *pData = ( BYTE* )renderTargetData; into WORD *pData = ( WORD* )renderTargetData;
But there may be some clever way for DX to do this for you that I don't know of since I

Making a CRC table for AES3 (AES-2003)

As a bit of insight into what I am doing, I am attempting to process AES/EBU subframes for an SDI interface. That shouldn't be too important; let's abstract away from that.
Page 12 of a standards document calls for a CRC check using the polynomial: G(x) = x^8 + x^4 + x^3 + x^2 + 1 (or x^0).
The document can be found here: http://tech.ebu.ch/docs/tech/tech3250.pdf
As you can probably anticipate, I would like to generate a CRC table for the given formula. I've come across a code-snippet which uses the formula G(x) = x^8 + x^2 + x^1 + x^0.
The code snippet can be located here:
http://www.koders.com/cpp/fid9C544B36B8C41721691790197D38DAC91D2C29EF.aspx?s=crc#L8
Could the formula be modified (see the modified version below) to work with the my AES3 CRC? Will the following work?
// x^8 + x^4 + x^3 + x^2 + x^0 or (1)
void make_crc_table( void )
{
int i, j;
unsigned long poly, c;
/* terms of polynomial defining this crc (except x^8): */
static const byte p[] = {0,2,3,4};
poly = 0L;
for ( i = 0; i < sizeof( p ) / sizeof( byte ); i++ )
{
poly |= 1L << p[i];
}
for ( i = 0; i < 256; i++ )
{
c = i;
for ( j = 0; j < 8; j++ )
{
//ZeroDefect: This part has me worried.
c = ( c & 0x80 ) ? poly ^ ( c << 1 ) : ( c << 1 );
}
crctable[i] = (byte) c;
}
}
Any tips/suggestions would be much appreciated.
ZeroDefect.
As far as I can tell, this is just encoding all the possible state transitions for the CRC feedback register (see the diagrams at Wikipedia) into a lookup table.
It looks like all you should need to do is modify the p[] array to take account of your tap positions.

OpenCV Python binds incredibly slow iterations through image data

I recently took some code that tracked an object based on color in OpenCV c++ and rewrote it in the python bindings.
The overall results and method were the same minus syntax obviously. But, when I perform the below code on each frame of a video it takes almost 2-3 seconds to complete where as the c++ variant, also below, is instant in comparison and I can iterate between frames as fast as my finger can press a key.
Any ideas or comments?
cv.PyrDown(img, dsimg)
for i in range( 0, dsimg.height ):
for j in range( 0, dsimg.width):
if dsimg[i,j][1] > ( _RED_DIFF + dsimg[i,j][2] ) and dsimg[i,j][1] > ( _BLU_DIFF + dsimg[i,j][0] ):
res[i,j] = 255
else:
res[i,j] = 0
for( int i =0; i < (height); i++ )
{
for( int j = 0; j < (width); j++ )
{
if( ( (data[i * step + j * channels + 1]) > (RED_DIFF + data[i * step + j * channels + 2]) ) &&
( (data[i * step + j * channels + 1]) > (BLU_DIFF + data[i * step + j * channels]) ) )
data_r[i *step_r + j * channels_r] = 255;
else
data_r[i * step_r + j * channels_r] = 0;
}
}
Thanks
Try using numpy to do your calculation, rather than nested loops. You should get C-like performance for simple calculations like this from numpy.
For example, your nested for loops can be replaced with a couple of numpy expressions...
I'm not terribly familiar with opencv, but I think the python bindings now have a numpy array interface, so your example above should be as simple as:
cv.PyrDown(img, dsimg)
data = np.asarray(dsimg)
blue, green, red = data.T
res = (green > (_RED_DIFF + red)) & (green > (_BLU_DIFF + blue))
res = res.astype(np.uint8) * 255
res = cv.fromarray(res)
(Completely untested, of course...) Again, I'm really not terribly familar with opencv, but nested python for loops are not the way to go about modifying an image element-wise, regardless...
Hope that helps a bit, anyway!

Show RGB888 content

I have to show RGB888 content using the ShowRGBContent function.
The below function is a ShowRGBContent function for yv12->rgb565 & UYVY->RGB565
static void ShowRGBContent(UINT8 * pImageBuf, INT32 width, INT32 height)
{
LogEntry(L"%d : In %s Function \r\n",++abhineet,__WFUNCTION__);
UINT16 * temp;
BYTE rValue, gValue, bValue;
// this is to refresh the background desktop
ShowWindow(GetDesktopWindow(),SW_HIDE);
ShowWindow(GetDesktopWindow(),SW_SHOW);
for(int i=0; i<height; i++)
{
for (int j=0; j< width; j++)
{
temp = (UINT16 *) (pImageBuf+ i*width*PP_TEST_FRAME_BPP+j*PP_TEST_FRAME_BPP);
bValue = (BYTE) ((*temp & RGB_COMPONET0_MASK) >> RGB_COMPONET0_OFFSET) << (8 -RGB_COMPONET0_WIDTH);
gValue = (BYTE) ((*temp & RGB_COMPONET1_MASK) >> RGB_COMPONET1_OFFSET) << (8 -RGB_COMPONET1_WIDTH);
rValue = (BYTE) ((*temp & RGB_COMPONET2_MASK) >> RGB_COMPONET2_OFFSET) << (8 -RGB_COMPONET2_WIDTH);
SetPixel(g_hDisplay, SCREEN_OFFSET_X + j, SCREEN_OFFSET_Y+i, RGB(rValue, gValue, bValue));
}
}
Sleep(2000); //sleep here to review the result
LogEntry(L"%d :Out %s Function \r\n",++abhineet,__WFUNCTION__);
}
I have to modify this for RGB888
Here in the above function:
************************
RGB_COMPONET0_WIDTH = 5
RGB_COMPONET1_WIDTH = 6
RGB_COMPONET2_WIDTH = 5
************************
************************
RGB_COMPONET0_MASK = 0x001F //31 in decimal
RGB_COMPONET1_MASK = 0x07E0 //2016 in decimal
RGB_COMPONET2_MASK = 0xF800 //63488 in decimal
************************
************************
RGB_COMPONET0_OFFSET = 0
RGB_COMPONET1_OFFSET = 5
RGB_COMPONET2_OFFSET = 11
************************
************************
SCREEN_OFFSET_X = 100
SCREEN_OFFSET_Y = 0
************************
Here
Also PP_TEST_FRAME_BPP = 2 for yv12 -> RGB565 & UYVY -> RGB565
iOutputBytesPerFrame = iOutputStride * iOutputHeight;
// where iOutputStride = (iOutputWidth * PP_TEST_FRAME_BPP) i.e (112 * 2)
// & iOutputHeight = 160
// These are in case of RGB565
pOutputFrameVirtAddr = (UINT32 *) AllocPhysMem( iOutputBytesPerFrame,
PAGE_EXECUTE_READWRITE,
0,
0,
(ULONG *) &pOutputFramePhysAddr);
// PAGE_EXECUTE_READWRITE = 0x40 mentioned in winnt.h
// Width =112 & Height = 160 in all the formats for i/p & o/p
Now my task is for RGB888.
Please guide me what shall i do in this.
**Thanks in advance.
Conversion from yuv444 to rgb888 is pretty simple since all of the components fall on byte boundaries so no bit masking should even be needed. According to the wikipedia article nobugz referred to in the comments section, the conversion can be done in fixed point by the following
UINT8* pimg = pImageBuf;
for(int i=0; i<height; i++)
{
for (int j=0; j< width; j++)
{
INT16 Y = pimg[0];
INT16 Cb = (INT16)pimg[1] - 128;
INT16 Cr = (INT16)pimg[2] - 128;
rValue = Y + Cr + Cr >> 2 + Cr >> 3 + Cr >> 5
gValue = Y - (Cb >> 2 + Cb >> 4 + Cb >> 5) -
(Cr >> 1 + Cr >> 3 + Cr >> 4 + Cr >> 5);
bValue = Y + Cb + Cb >> 1 + Cb >> 2 + Cb >> 6;
SetPixel(g_hDisplay, SCREEN_OFFSET_X + j, SCREEN_OFFSET_Y+i, RGB(rValue,
gValue, bValue));
pimg+=3;
}
}
This assumes that your yuv444 is 8 bits per sample (24 bits per pixel). The conversion can also be done in floating point but this should be quicker if it works since your source and destinations are both fixed point. I'm also not sure the conversion to int16 is necessary, but I did it to be safe.
Note that the 444 in yuv444 is not referring to the same thing as the 888 in rgb888. The 444 refers to the subsampling that often occurs when using the TUV colorspace. For instance in YUV420, Cb and Cr are decimated by two in both directions. yuv444 just means that all three components are sampled the same (no subsampling). The 888 in rgb888 is referring to the bits per sample (8 bits for each of the three color components).
I have not actually tested this code, but it should at least give you an idea where to start.